chore: phase one cleanup and hardening

NOTES.md

@@ -153,7 +153,7 @@ English or any other language.
 
 The `eval` command (documented in
 [README.md's Evaluating accuracy section](README.md#evaluating-accuracy)) runs
-the tracer against `tests/fixtures/eval_cases.json`, a set of advisories with
+the tracer against `src/code_audit/eval_cases.json`, a set of advisories with
 known-correct answers, and reports how often the agent's output matches.
 
 **Sourcing ground truth.** Every non-null expected value in the fixture carries

README.md

@@ -105,6 +105,15 @@ Fetch an advisory and print it as JSON:
 uv run python -m code_audit advisory GHSA-jfh8-c2jp-5v3q
 ```
 
+Add `--pretty` to print a short human-readable summary (the GHSA id, summary, severity,
+and source location, one per line) instead of the full JSON:
+
+```sh
+uv run python -m code_audit advisory GHSA-jfh8-c2jp-5v3q --pretty
+```
+
+The default (no `--pretty`) output is unchanged.
+
 Trace an advisory to its introducing and fixing changes. Deterministic tracing from the
 advisory references runs first; a Claude agent then investigates only the fields that
 remain unknown, and every finding is verified against the GitHub API:
@@ -125,7 +134,7 @@ Expected output (trimmed):
   },
   "introducing_pull_request": null,
   "fixing_commit": {
-    "sha": "c77b3cb39312b83b053d23a2158b6e528f9a6ab9",
+    "sha": "c77b3cb39312b83b053d23a2158b99ac7de44dd3",
     "message": "Restrict LDAP access via JNDI (#608)",
     "files": ["..."]
   },
@@ -141,6 +150,25 @@ Expected output (trimmed):
 Fields stay `null` when no confirmed answer exists, for example an introducing change
 that predates pull requests.
 
+Add `--pretty` to print a short human-readable summary instead of the full JSON: one
+line per field (`introducing_commit`, `introducing_pull_request`, `fixing_commit`,
+`fixing_pull_request`) with a short SHA or PR number, the first line of the commit
+message or PR title, and a file or state count, or `not found` when a field is null:
+
+```sh
+uv run python -m code_audit trace GHSA-jfh8-c2jp-5v3q --pretty
+```
+
+```
+introducing_commit: f1a0cac LOG4J2-313 - Add JNDILookup (1 file)
+introducing_pull_request: not found
+fixing_commit: c77b3cb Restrict LDAP access via JNDI (#608) (1 file)
+fixing_pull_request: #608 Restrict LDAP access via JNDI (closed, merged)
+```
+
+The default (no `--pretty`) output is unchanged: full JSON, byte for byte the same as
+before, so anything scripted against it keeps working without passing `--pretty`.
+
 Add `--metrics` to print execution metrics as a second JSON document after the trace
 result. It reports timings for the deterministic and LLM phases, API request counts,
 token usage, and the estimated API cost:
@@ -188,8 +216,9 @@ uv run python -m code_audit trace-many --from-file advisories.txt
 
 `--concurrency` (default 5) bounds how many advisories are traced at once, since running
 too many in parallel would trip the GitHub and Anthropic rate limits and their retry
-backoff rather than finishing faster. `--debug` and `--metrics` work as they do for
-`trace`, writing one transcript and one metrics block per advisory. With `--metrics`,
+backoff rather than finishing faster. `--debug`, `--metrics`, and `--pretty` work as they
+do for `trace`, writing one transcript, one metrics block, or one pretty summary per
+advisory in place of its JSON block. With `--metrics`,
 each advisory's block also reports `started_at_seconds` and `ended_at_seconds` as offsets
 from the batch start, so overlapping windows show the advisories ran concurrently rather
 than one after another. The GitHub and Anthropic clients are shared across threads, so
@@ -210,7 +239,7 @@ honest `null`:
 uv run python -m code_audit eval --runs 3
 ```
 
-The cases live in `tests/fixtures/eval_cases.json`; each entry records the expected
+The cases live in `src/code_audit/eval_cases.json`; each entry records the expected
 fields and a source URL for every confirmed value. A field's expected value may be a
 list when several identifiers are acceptable (for example the same fix cherry-picked to
 different branches with different SHAs). A case marked `"disputed": true` (a

src/code_audit/agent.py

@@ -1,4 +1,6 @@
 import re
+import sys
+import threading
 from collections.abc import Callable
 from concurrent.futures import ThreadPoolExecutor
 from typing import Any, cast
@@ -17,6 +19,15 @@
 MAX_TOKENS = 16000
 MAX_TURNS = 20
 
+# MAX_TURNS bounds how many requests a run can make, but nothing bounds how long
+# a single turn's generation takes. Adaptive thinking has been observed, rarely,
+# to spend far longer than usual before landing on the same inconclusive answer
+# a quick run would have reached anyway (one observed case: 285 seconds and over
+# 16000 output tokens across just 3 turns, versus a typical 25-75 seconds for the
+# same advisory). This caps the whole run's wall-clock time well above any normal
+# case but well below that kind of outlier.
+MAX_RUN_SECONDS = 180.0
+
 # GitHub owner and repository names are limited to this ASCII set, so a
 # reported repository outside it is malformed rather than another language.
 REPOSITORY_PATTERN = re.compile(r"[A-Za-z0-9._-]+/[A-Za-z0-9._-]+")
@@ -62,6 +73,7 @@ def complete_trace(
     anthropic_client: anthropic.Anthropic | None = None,
     on_usage: Callable[[Usage], None] | None = None,
     on_turn: Callable[[dict[str, Any]], None] | None = None,
+    on_progress: Callable[[str], None] | None = None,
 ) -> TraceResult:
     """Fill the unknown fields of a deterministic trace result using one agent.
 
@@ -70,7 +82,11 @@ def complete_trace(
     client, so the returned TraceResult contains nothing the API did not
     confirm. When on_usage is given it observes the token usage of every
     model response. When on_turn is given it receives one debug record per
-    model turn.
+    model turn. When on_progress is given it receives one short human-readable
+    line per turn, unconditionally rather than only under --debug, since a run
+    can otherwise take up to MAX_RUN_SECONDS with no output at all. The run is
+    capped at MAX_RUN_SECONDS wall-clock time; exceeding it is treated like any
+    other inconclusive outcome and falls back to deterministic_result.
     """
     unknown_fields = [
         field for field in TRACE_FIELDS if getattr(deterministic_result, field) is None
@@ -80,14 +96,15 @@ def complete_trace(
 
     if anthropic_client is None:
         anthropic_client = anthropic.Anthropic()
-    findings = _run_agent(
+    findings = _run_agent_with_timeout(
         advisory,
         deterministic_result,
         unknown_fields,
         github_client,
         anthropic_client,
         on_usage,
         on_turn,
+        on_progress,
     )
     if findings is None or not _repository_is_trustworthy(findings.repository, advisory):
         # A garbled generation can yield syntactically valid JSON with a
@@ -113,6 +130,74 @@ def _repository_is_trustworthy(repository: str | None, advisory: Advisory) -> bo
     return known is None or repository.lower() == known
 
 
+class _AgentRunOutcome:
+    """Holds whatever the agent thread produced, so it can cross back to the caller."""
+
+    def __init__(self) -> None:
+        self.findings: AgentFindings | None = None
+        self.exc: BaseException | None = None
+
+
+def _run_agent_with_timeout(
+    advisory: Advisory,
+    result: TraceResult,
+    unknown_fields: list[str],
+    github_client: GitHubClient,
+    anthropic_client: anthropic.Anthropic,
+    on_usage: Callable[[Usage], None] | None,
+    on_turn: Callable[[dict[str, Any]], None] | None,
+    on_progress: Callable[[str], None] | None,
+) -> AgentFindings | None:
+    """Run the agent on a daemon thread and give up after MAX_RUN_SECONDS.
+
+    Giving up is reported to stderr and then treated exactly like any other
+    inconclusive outcome: the caller falls back to the deterministic result.
+
+    This deliberately does not use ThreadPoolExecutor. Its worker threads are
+    registered in concurrent.futures.thread's module-level _threads_queues and
+    joined by an atexit hook at interpreter shutdown, regardless of whether the
+    executor itself was shut down with wait=False; a run that times out here
+    would still block process exit until the abandoned call finally finishes or
+    the Anthropic SDK's own request timeout elapses, silently defeating the cap
+    (confirmed by timing an actual process's exit, not just when this function
+    returns: shutdown(wait=False) returned in ~1s while the process did not
+    actually exit for another ~4s, matching the abandoned call's own duration).
+    A plain daemon thread is not tracked by that registry, so the interpreter
+    does not wait for it, and the process exits as soon as its own work is
+    done.
+    """
+    outcome = _AgentRunOutcome()
+
+    def target() -> None:
+        try:
+            outcome.findings = _run_agent(
+                advisory,
+                result,
+                unknown_fields,
+                github_client,
+                anthropic_client,
+                on_usage,
+                on_turn,
+                on_progress,
+            )
+        except BaseException as exc:
+            outcome.exc = exc
+
+    thread = threading.Thread(target=target, daemon=True)
+    thread.start()
+    thread.join(timeout=MAX_RUN_SECONDS)
+    if thread.is_alive():
+        print(
+            f"Warning: tracing {advisory.ghsa_id} exceeded {MAX_RUN_SECONDS:.0f}s; "
+            "falling back to the deterministic result.",
+            file=sys.stderr,
+        )
+        return None
+    if outcome.exc is not None:
+        raise outcome.exc
+    return outcome.findings
+
+
 def _run_agent(
     advisory: Advisory,
     result: TraceResult,
@@ -121,12 +206,15 @@ def _run_agent(
     anthropic_client: anthropic.Anthropic,
     on_usage: Callable[[Usage], None] | None,
     on_turn: Callable[[dict[str, Any]], None] | None,
+    on_progress: Callable[[str], None] | None,
 ) -> AgentFindings | None:
     """Run the tool loop and return the agent's findings, or None if it stalls."""
     messages: list[MessageParam] = [
         {"role": "user", "content": build_task(advisory, result, unknown_fields)}
     ]
     for turn in range(1, MAX_TURNS + 1):
+        if on_progress is not None:
+            on_progress(f"Turn {turn}/{MAX_TURNS}: waiting for the model...")
         # The conversation only grows at the end, so caching the request
         # prefix means each turn pays full price only for the blocks added
         # since the previous turn.