feat: report tool failures and update Trivy to v0.69.3#21
Conversation
When a scanning tool binary is missing or fails to execute, the scanner now records a ToolError instead of silently producing zero findings. Tool errors are collected by the aggregator, displayed in CLI output, and rendered in markdown and HTML reports so users can distinguish between a clean scan and tools that didn't run.
🤖 Augment PR SummarySummary: This PR makes scanner execution failures visible to users by recording and reporting tool-level errors (e.g., missing binaries/timeouts) instead of silently returning zero findings. Changes:
Technical Notes: Tool failures are detected via 🤖 Was this summary useful? React with 👍 or 👎 |
There was a problem hiding this comment.
Pull request overview
Adds first-class reporting of scanning tool execution failures so users can distinguish “no findings” from “tools didn’t run,” and surfaces those failures in aggregated output and reports.
Changes:
- Introduces
ToolErrorand propagates tool execution failures from scanners into aggregated scan results. - Displays tool failures in CLI output and renders them in Markdown/HTML reports.
- Adds unit tests covering ToolError serialization, scanner recording behavior (subset), aggregation, and Markdown rendering.
Reviewed changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 15 comments.
Show a summary per file
| File | Description |
|---|---|
src/scanner_base.py |
Adds ToolError model and tool_errors tracking/accessor on scanners. |
src/scanners/terraform_scanner.py |
Records tool execution failures as ToolError for Terraform-related tools. |
src/scanners/python_scanner.py |
Records tool execution failures as ToolError for Python tools. |
src/scanners/secrets_scanner.py |
Records gitleaks execution failures as ToolError. |
src/scanners/npm_scanner.py |
Records npm audit / Snyk execution failures as ToolError. |
src/scanners/container_scanner.py |
Records missing/failed trivy/docker checks as ToolError. |
src/scanners/cloudformation_scanner.py |
Records missing/failed CFN tool executions as ToolError. |
src/report_aggregator.py |
Aggregates tool_errors into report output and adds metadata.tools_failed. |
src/main.py |
Collects scanner tool errors, prints warnings during scanning, and prints a consolidated “Tool Failures” section. |
src/formatters/markdown_formatter.py |
Renders a “Tool Failures” section in Markdown when errors exist. |
src/formatters/html_formatter.py |
Renders a “Tool Failures” section in HTML when errors exist. |
tests/test_tool_errors.py |
Adds tests for ToolError, scanner recording/reset (subset), aggregator inclusion, and Markdown rendering. |
build.md |
Removes trailing whitespace/blank line. |
.gitignore |
Trims trailing spaces in comments. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Update Trivy from v0.48.3 to v0.69.3 (old release deleted from GitHub) - Align all branding references to portfolio-code-scanner - Remove unused pytest import to fix flake8 F401
- Check returncode < 0 instead of == -1 to catch signal-killed processes - Use self.__class__.__name__ instead of hardcoded scanner name strings - Deduplicate npm-audit ToolError when looping over multiple package.json - Add HTML formatter test coverage for tool error rendering and escaping
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 28 out of 29 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
Dockerfile:76
- The Trivy installation step downloads a prebuilt binary from GitHub via
wgetand installs it without any checksum or signature verification, which introduces a supply chain risk. If the release artifacts or download channel were compromised, a malicious binary could be baked into this scanner image and then executed with access to repositories and CI/CD secrets. Consider validating the downloaded archive against the official checksums or signatures (or using a package manager / pinned image with integrity guarantees) before installation, and aligning this pattern for other tools installed in this Dockerfile.
ARG TRIVY_VERSION=0.69.3
RUN wget -q https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}/trivy_${TRIVY_VERSION}_Linux-64bit.tar.gz && \
tar zxvf trivy_${TRIVY_VERSION}_Linux-64bit.tar.gz && \
mv trivy /usr/local/bin/ && \
rm trivy_${TRIVY_VERSION}_Linux-64bit.tar.gz && \
trivy --version
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…scanners_run - HTML-escape tool, scanner, and error_message in markdown formatter to prevent XSS when rendered in HTML-aware markdown viewers - Include failed tool names in metadata.scanners_run so reports distinguish "ran clean" from "didn't run" - Add markdown escaping test coverage
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 28 out of 29 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…DE.md branding - terraform init non-zero exit now records a ToolError with tool name terraform-init instead of silently skipping validation - trivy --version and docker --version non-zero exits now record ToolErrors instead of logging warnings without visibility in reports - Update CLAUDE.md branding from SDLC to Portfolio Code Scanner
ghaworkflow
left a comment
There was a problem hiding this comment.
Unless the agents find Critical or High findings, I think we are good.
Summary
ToolErrorinstead of silently producing zero findings. Tool errors are collected by the aggregator, displayed in CLI output, and rendered in markdown and HTML reports so users can distinguish between a clean scan and tools that didn't run.pytestimport (flake8 F401).Test plan