Skip to content

feat: report tool failures and update Trivy to v0.69.3#21

Merged
williambrady merged 6 commits intomainfrom
feature/tool-failure-reporting
Mar 8, 2026
Merged

feat: report tool failures and update Trivy to v0.69.3#21
williambrady merged 6 commits intomainfrom
feature/tool-failure-reporting

Conversation

@williambrady
Copy link
Copy Markdown
Owner

@williambrady williambrady commented Mar 1, 2026

Summary

  • Tool failure reporting: When a scanning tool binary is missing or fails to execute, the scanner now records a ToolError instead of silently producing zero findings. Tool errors are collected by the aggregator, displayed in CLI output, and rendered in markdown and HTML reports so users can distinguish between a clean scan and tools that didn't run.
  • Trivy version update: Update from v0.48.3 to v0.69.3 — the old release was deleted from GitHub due to a security incident with the aquasecurity/trivy repository, breaking Docker builds.
  • Branding alignment: Align all references to portfolio-code-scanner across Dockerfile, action.yml, entrypoint, workflows, config, and formatters.
  • Lint fix: Remove unused pytest import (flake8 F401).

Test plan

  • All 50 unit tests passing
  • Pre-commit checks passing (trailing whitespace, end-of-file, YAML, black, flake8)
  • Docker image builds successfully with new Trivy version
  • Run scan against a test repository to verify tool error reporting works end-to-end

When a scanning tool binary is missing or fails to execute, the scanner
now records a ToolError instead of silently producing zero findings.
Tool errors are collected by the aggregator, displayed in CLI output,
and rendered in markdown and HTML reports so users can distinguish
between a clean scan and tools that didn't run.
Copilot AI review requested due to automatic review settings March 1, 2026 23:07
@augmentcode
Copy link
Copy Markdown

augmentcode Bot commented Mar 1, 2026

🤖 Augment PR Summary

Summary: This PR makes scanner execution failures visible to users by recording and reporting tool-level errors (e.g., missing binaries/timeouts) instead of silently returning zero findings.

Changes:

  • Introduced a ToolError dataclass and added tool-error tracking to ScannerBase.
  • Updated multiple scanners (Terraform/CloudFormation/Python/NPM/Secrets/Container) to capture execute_command failures (return code -1) as ToolError entries.
  • Extended ReportAggregator to collect tool errors and include them in aggregated report output and metadata (tools_failed).
  • Enhanced CLI output to warn per-scanner when tools fail and to print a summary “Tool Failures” section.
  • Updated Markdown and HTML report formatters to render a dedicated Tool Failures section/table when errors exist.
  • Added unit tests covering ToolError, scanner recording behavior, aggregator inclusion, and Markdown rendering.

Technical Notes: Tool failures are detected via ScannerBase.execute_command() returning -1 (timeouts and execution exceptions) and are surfaced alongside normal findings so users can distinguish “clean scan” from “tools didn’t run.”

🤖 Was this summary useful? React with 👍 or 👎

Copy link
Copy Markdown

@augmentcode augmentcode Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 2 suggestions posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

Comment thread src/scanners/npm_scanner.py Outdated
Comment thread src/formatters/html_formatter.py
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class reporting of scanning tool execution failures so users can distinguish “no findings” from “tools didn’t run,” and surfaces those failures in aggregated output and reports.

Changes:

  • Introduces ToolError and propagates tool execution failures from scanners into aggregated scan results.
  • Displays tool failures in CLI output and renders them in Markdown/HTML reports.
  • Adds unit tests covering ToolError serialization, scanner recording behavior (subset), aggregation, and Markdown rendering.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 15 comments.

Show a summary per file
File Description
src/scanner_base.py Adds ToolError model and tool_errors tracking/accessor on scanners.
src/scanners/terraform_scanner.py Records tool execution failures as ToolError for Terraform-related tools.
src/scanners/python_scanner.py Records tool execution failures as ToolError for Python tools.
src/scanners/secrets_scanner.py Records gitleaks execution failures as ToolError.
src/scanners/npm_scanner.py Records npm audit / Snyk execution failures as ToolError.
src/scanners/container_scanner.py Records missing/failed trivy/docker checks as ToolError.
src/scanners/cloudformation_scanner.py Records missing/failed CFN tool executions as ToolError.
src/report_aggregator.py Aggregates tool_errors into report output and adds metadata.tools_failed.
src/main.py Collects scanner tool errors, prints warnings during scanning, and prints a consolidated “Tool Failures” section.
src/formatters/markdown_formatter.py Renders a “Tool Failures” section in Markdown when errors exist.
src/formatters/html_formatter.py Renders a “Tool Failures” section in HTML when errors exist.
tests/test_tool_errors.py Adds tests for ToolError, scanner recording/reset (subset), aggregator inclusion, and Markdown rendering.
build.md Removes trailing whitespace/blank line.
.gitignore Trims trailing spaces in comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/scanners/terraform_scanner.py Outdated
Comment thread src/scanners/python_scanner.py Outdated
Comment thread src/scanners/python_scanner.py Outdated
Comment thread src/scanners/npm_scanner.py Outdated
Comment thread src/scanners/npm_scanner.py Outdated
Comment thread src/scanners/secrets_scanner.py Outdated
Comment thread src/scanners/container_scanner.py
Comment thread src/scanners/terraform_scanner.py
Comment thread src/scanners/cloudformation_scanner.py Outdated
Comment thread src/scanners/container_scanner.py
- Update Trivy from v0.48.3 to v0.69.3 (old release deleted from GitHub)
- Align all branding references to portfolio-code-scanner
- Remove unused pytest import to fix flake8 F401
@williambrady williambrady changed the title feat: report tool execution failures in scan results feat: report tool failures and update Trivy to v0.69.3 Mar 8, 2026
- Check returncode < 0 instead of == -1 to catch signal-killed processes
- Use self.__class__.__name__ instead of hardcoded scanner name strings
- Deduplicate npm-audit ToolError when looping over multiple package.json
- Add HTML formatter test coverage for tool error rendering and escaping
Copilot AI review requested due to automatic review settings March 8, 2026 21:08
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 28 out of 29 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

Dockerfile:76

  • The Trivy installation step downloads a prebuilt binary from GitHub via wget and installs it without any checksum or signature verification, which introduces a supply chain risk. If the release artifacts or download channel were compromised, a malicious binary could be baked into this scanner image and then executed with access to repositories and CI/CD secrets. Consider validating the downloaded archive against the official checksums or signatures (or using a package manager / pinned image with integrity guarantees) before installation, and aligning this pattern for other tools installed in this Dockerfile.
ARG TRIVY_VERSION=0.69.3
RUN wget -q https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}/trivy_${TRIVY_VERSION}_Linux-64bit.tar.gz && \
    tar zxvf trivy_${TRIVY_VERSION}_Linux-64bit.tar.gz && \
    mv trivy /usr/local/bin/ && \
    rm trivy_${TRIVY_VERSION}_Linux-64bit.tar.gz && \
    trivy --version

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/formatters/markdown_formatter.py
william brady added 2 commits March 8, 2026 17:50
…scanners_run

- HTML-escape tool, scanner, and error_message in markdown formatter
  to prevent XSS when rendered in HTML-aware markdown viewers
- Include failed tool names in metadata.scanners_run so reports
  distinguish "ran clean" from "didn't run"
- Add markdown escaping test coverage
Copilot AI review requested due to automatic review settings March 8, 2026 21:56
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 28 out of 29 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/scanners/terraform_scanner.py
Comment thread src/scanners/container_scanner.py Outdated
Comment thread src/scanners/container_scanner.py Outdated
Comment thread README.md
Comment thread Dockerfile
…DE.md branding

- terraform init non-zero exit now records a ToolError with tool name
  terraform-init instead of silently skipping validation
- trivy --version and docker --version non-zero exits now record
  ToolErrors instead of logging warnings without visibility in reports
- Update CLAUDE.md branding from SDLC to Portfolio Code Scanner
Copy link
Copy Markdown
Collaborator

@ghaworkflow ghaworkflow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless the agents find Critical or High findings, I think we are good.

@williambrady williambrady merged commit ea81c6b into main Mar 8, 2026
5 checks passed
@williambrady williambrady deleted the feature/tool-failure-reporting branch March 8, 2026 23:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants