Skip to content

Add diagnostic metadata verifier#9

Open
felixvippp-ai wants to merge 2 commits into
Andyyook:mainfrom
felixvippp-ai:codex/verify-diagnostics-validation
Open

Add diagnostic metadata verifier#9
felixvippp-ai wants to merge 2 commits into
Andyyook:mainfrom
felixvippp-ai:codex/verify-diagnostics-validation

Conversation

@felixvippp-ai

Copy link
Copy Markdown

Summary

Adds a robust tools/verify_diagnostics.py CLI for validating generated diagnostic metadata before submitting PRs. The verifier checks JSON structure, referenced log artifacts, subprocess/git root discovery failures, and minimum passed-module thresholds.

Changes

  • Added argparse-based tools/verify_diagnostics.py with --verbose, --json, and --threshold options.
  • Added schema validation for diagnostic JSON files and referenced .logd artifacts.
  • Wrapped subprocess usage in explicit error handling with UTF-8 decoding fallback for Windows paths.
  • Added unit coverage for valid metadata, missing required fields, threshold failures, and invalid threshold input.
  • Closes [$40 BOUNTY] [Python] Add input validation and error handling to verify_diagnostics.py #1.

Testing

  • python -m unittest tools.test_verify_diagnostics -v: passed, 4 tests.
  • python -m py_compile tools/verify_diagnostics.py tools/test_verify_diagnostics.py: passed.
  • python tools/verify_diagnostics.py diagnostic/build-e530d2f7.json --threshold 1 --verbose: passed against the generated diagnostic JSON and all referenced logd chunks.
  • python build.py: ran and committed diagnostics. Frontend passed; non-frontend language modules failed because this Windows environment does not have Rust, Go, C/C++, Java, Ruby, Lua, or Haskell toolchains installed.
  • Diagnostic artifacts committed: diagnostic/build-e530d2f7.json and diagnostic/build-e530d2f7-part001.logd through part004.logd.
  • Diagnostic password from generated metadata: f890bfaa2f8a2b29096e.

Checklist

  • Relevant modules affected by these changes build locally
  • Tests pass locally
  • Diagnostic build log is committed in this PR
  • Documentation has been updated, if applicable
  • Configuration or schema changes are documented, if applicable
  • No generated build artifacts are committed, except the required diagnostic build log
  • Changes are scoped to the PR purpose and avoid unrelated cleanup
  • Security, privacy, and error-handling implications have been considered

  • I would like to request that my diagnostic build log is removed before merging

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[$40 BOUNTY] [Python] Add input validation and error handling to verify_diagnostics.py

1 participant