refactor: port finding_verifier.py to Claude Agent SDK native tools#37
Merged
joshbouncesecurity merged 1 commit intomasterfrom Apr 19, 2026
Merged
Conversation
Replace the manual anthropic-SDK tool-dispatch loop in FindingVerifier with a single call to utilities.llm_client.run_native_verification. The SDK drives a multi-turn session with native Read/Grep/Glob/Bash tools, uses VERIFICATION_JSON_SCHEMA for structured output, and routes rate-limit handling through the centralised _run_query wrapper (sdk_errors.RateLimitError + GlobalRateLimiter notification). Changes: - Drop `client` constructor param and the whole while-MAX_ITERATIONS self.client.messages.create(tools=VERIFICATION_TOOLS, ...) loop. - Drop imports of `anthropic`, `ToolExecutor`, and VERIFICATION_TOOLS definition (now unused). - Add `output_dir` constructor param + `_save_explanation` helper (already expected by core/verifier.py, previously a no-op). - Consistency cross-check uses a dedicated single-turn AnthropicClient (via utilities.llm_client.AnthropicClient) instead of raw client.messages.create. - VERIFIER_MODEL now comes from utilities.model_config.MODEL_PRIMARY. - Preserve upstream PR #23 verify_batch API (workers, checkpoint, restored_callback) — no change to the verify_result public signature or VerificationResult return shape. - Rate-limit exceptions no longer caught locally: centralised in utilities.llm_client._run_query. Tests: - Rewrite tests/test_local_claude.py::TestVerifyWithNativeClaude to mock utilities.finding_verifier.run_native_verification instead of the old anthropic-client messages.create path. 6 new tests cover structured JSON, code-block JSON, free-text verdict fallback, unparseable text, SDK subprocess failure, and missing repo_path. - test_resume_stage2.py and test_declared_dependencies.py unchanged and passing. Part of issue #35 / SDK_MIGRATION_COMPLETION_PLAN Step 3. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
joshbouncesecurity
added a commit
that referenced
this pull request
Apr 19, 2026
Final step of the SDK migration tracked in issue #35. Removes "anthropic>=0.40.0" from pyproject.toml now that no Python code under libs/openant-core/ still imports it. Cleanup alongside the dep drop: - `utilities/context_enhancer.py`: remove the now-orphaned `import anthropic`. PR #34 took it out of `_build_error_info`; PR #36 removed `shared_client`. The import line was kept alive across both as a staging measure. - `openant/cli.py` (cmd_report_data): replace the last `anthropic.Anthropic()` instantiation — used for the HTML report's remediation-guidance LLM call — with `AnthropicClient(model=MODEL_AUXILIARY).analyze_sync(...)`. Usage tracking is now automatic via the global TokenTracker; cost display pulls from `client.get_last_call()`. Neither PR #36 nor #37 touched this site because it was outside their scope; the dep-drift test (PR #30) surfaced it when pyproject.toml's dependency list shrank. - `utilities/rate_limiter.py`: update the module docstring's example. The pre-migration example showed `except anthropic.RateLimitError as e: retry_after = e.response.headers.get(...)`. That code path no longer exists — rate-limit detection is centralised in `llm_client._run_query`, which raises `utilities.sdk_errors.RateLimitError` after notifying the global limiter. Example updated to match. Verification: - `grep -rn '^import anthropic\|^from anthropic' libs/openant-core/` returns zero hits. - `grep -rn 'anthropic\.' libs/openant-core/` returns only a historical docstring reference in `sdk_errors.py`. - `tests/test_declared_dependencies.py` passes — the regression guard from PR #30 now enforces that no undeclared imports exist with anthropic gone. - `tests/test_sdk_errors.py` (12) + `tests/test_sdk_error_surfacing.py` (9) all pass. - `import openant, core, utilities, parsers, prompts, context, report` all succeed. End state: zero `anthropic` Python dep, all LLM traffic routes through the Claude Agent SDK via `utilities.llm_client`. Step 8 (end-to-end verification with a live API key) is the only remaining non-user-action item in the plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced Apr 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ports
utilities/finding_verifier.pyfrom theanthropicSDK to the Claude Agent SDK. This is Step 3 of the SDK migration tracked in issue #35.The manual tool-dispatch loop (
search_usages,search_definitions,read_function,list_functions,finish) is replaced with a single call toutilities.llm_client.run_native_verification, which runs a multi-turn SDK session with nativeRead/Grep/Glob/Bashtools and returns structured JSON viaVERIFICATION_JSON_SCHEMA. Rate-limit handling is now centralised inutilities.llm_client._run_query(viasdk_errors.RateLimitErrorandGlobalRateLimiter), so the oldexcept anthropic.RateLimitErrorblocks are gone.This reapplies PR #25's Step 3 work, adapted for the current post-merge state (upstream PR #23 re-introduced the
anthropicpath and addedlogger/app_context/verbose/checkpoint/workers APIs that must be preserved).Key changes
libs/openant-core/utilities/finding_verifier.pyclient: anthropic.Anthropic | Noneconstructor param and the wholewhile iterations < MAX_ITERATIONS: self.client.messages.create(tools=VERIFICATION_TOOLS, ...)loop.anthropic,ToolExecutor, and theVERIFICATION_TOOLSschema.verify_resultnow callsrun_native_verificationwith the SDK-awareget_native_claude_verification_promptandVERIFICATION_JSON_SCHEMA. Process-level SDK failures return a conservative "agree" verdict; rate-limit errors surface assdk_errors.RateLimitErrorand are handled by the centralised wrapper._resolve_inconsistency) uses a dedicatedAnthropicClientsingle-turn call instead of rawclient.messages.create.output_dirconstructor param +_save_explanationhelper (already expected bycore/verifier.py).VERIFIER_MODELnow pulls fromutilities.model_config.MODEL_PRIMARY.verify_batch(..., workers=10, checkpoint=None, restored_callback=None),_verify_batch_sequential/_parallel,_verify_one,_check_consistency,_has_conclusive_exploit_path,_group_by_pattern.verify_resultpublic signature andVerificationResultreturn type are unchanged.libs/openant-core/tests/test_local_claude.pyclient=is gone) means the oldTestVerifyWithNativeClaudetests — which mockedmock_client.messages.create— cannot work as written. Rewrote the class to mockutilities.finding_verifier.run_native_verificationinstead. 6 new tests cover: structured JSON, JSON-in-code-block, free-text verdict fallback, unparseable text, SDK subprocess failure, and missingrepo_path.Files not touched
Per the task brief:
pyproject.toml,llm_client.py,agentic_enhancer/agent.py,context_enhancer.pyare all untouched.Test plan
python -c "from utilities.finding_verifier import FindingVerifier"imports cleanly.pytest tests/test_local_claude.py— 31/31 passing.pytest tests/test_resume_stage2.py— 15/15 passing (coversverify_batchcheckpoint/resume path, consistency cross-check still fires).pytest tests/test_declared_dependencies.py— 8/8 passing (no undeclared imports;anthropicstill declared inpyproject.tomland is still imported by other files — that's Steps 4-7's job).grep -n anthropic libs/openant-core/utilities/finding_verifier.py— only the historical comment at line 8 remains; no import.openant verifyon a real finding. Deferred to Step 8 of the migration plan, covered by the issue-35 E2E checklist._run_query— not re-tested here.Notes
clientconstructor keyword is removed. Any external caller that still passesFindingVerifier(client=...)will hitTypeError. All in-tree callers (experiment.py:695,core/verifier.py:138,tests/test_resume_stage2.py) already construct without aclientkwarg, so this only affected theTestVerifyWithNativeClaudetest class (updated in this PR).checkpoint_path,concurrency,run_parallel) is NOT restored here —core/verifier.pydepends on the upstreamworkers/checkpoint/restored_callbacksurface. The upstream ThreadPoolExecutor-based batch code is kept verbatim.Relates to #35.