Skip to content

[harness] - close four unhandled-error paths and add first tests for main()'s bind guard - #807

Merged
cgfixit merged 2 commits into
mainfrom
claude/gallant-mayer-xkxgah
Aug 5, 2026
Merged

[harness] - close four unhandled-error paths and add first tests for main()'s bind guard#807
cgfixit merged 2 commits into
mainfrom
claude/gallant-mayer-xkxgah

Conversation

@cgfixit

@cgfixit cgfixit commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Proposed changes

A CyClaw-Optimize scan of the harness console surfaced four unhandled-exception paths, each turning into a bare HTTP 500 (or a startup traceback) that static/harness.html's fetch helper cannot parse — the exact failure class the existing _timeout_err and _agentic_call except OSError guards were written to eliminate. This PR closes all four and lands the first-ever tests for harness.server.main()'s loopback bind guard.

  1. /api/harness/runs glob→mtime-sort race (harness/server.py): a run artifact deleted between the glob and os.path.getmtime sort raised OSError straight out of the route. Now guarded with with suppress(OSError): — the byte-identical pattern SessionStore.list() already carries in harness/sessions.py for the same race.
  2. Malformed usage block from an OpenAI-compatible proxy (harness/ollama.py): "usage": "none" gave AttributeError, "prompt_tokens": "abc" gave ValueError — neither was HarnessLLMError, so /api/chat's 502 handler missed them. Token tallies are cosmetic; they now degrade to 0 and the answer is delivered.
  3. CYCLAW_HARNESS_PORT failed open silently (harness/server.py main()): a non-numeric override (abc, -1, 8790x) was silently discarded and the harness bound the stored port instead of the one the operator asked for. It now fails closed with a clear message, matching the adjacent out-of-range sys.exit.
  4. Missing static/harness.html (harness/server.py create_app()): a trimmed checkout (e.g. macos/install-cyclaw.sh's --depth 1 clone gone wrong) made startup traceback with a bare FileNotFoundError. It now raises the typed HarnessConfigError with the missing path in details.

New tests/test_harness_robustness.py (14 tests, auto-discovered by pytest — no ci.yml change needed since no new source module is added): first coverage of main()'s non-loopback-host refusal and port-env guards, the runs-listing race, the missing-asset path, and four malformed-usage shapes.

Invariant / Governance Impact: none of the 6 invariants touched. harness/ is out-of-band (I6); no import into/out of the core three changed — invariant-guard passes 33/33. The loopback-only bind guard is strengthened: it gains its first regression tests, and the port override now fails closed instead of open.

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature
  • Breaking change
  • Documentation Update
  • Invariant / Governance refinement (first tests pinning main()'s loopback/port guards)

Scope note: Out-of-band harness layer only (harness/server.py, harness/ollama.py, one new test file).

Benefits / why

  • Every fix removes a "console renders an unparseable bare 500" case, completing work the existing typed-error guards started rather than opening a new front.
  • The port override failing closed removes a silent-misconfiguration trap: the operator now learns immediately that their override was rejected instead of discovering the harness on a different port.
  • main() — the layer the PEP-562 __getattr__ note identifies as the bind-address guard uvicorn harness.server:app bypasses — was previously entirely untested; the new tests pin it.

Risks to monitor

  • The malformed-usage degradation reports 0 tokens instead of erroring — a proxy that consistently sends malformed usage will under-count the console tally silently (debug-level visibility only). Judged acceptable: the alternative loses a good answer over a cosmetic counter.
  • A previously-"working" setup that relies on an invalid CYCLAW_HARNESS_PORT being ignored will now exit at startup with an explicit message. That is the intended behavior change; the message names the variable and value.
  • Trial-merged locally against open PR feat(harness): stage reviewed browser agent inputs #803 (the one open PR sharing harness/server.py): merges clean in both directions, both sides' edits survive, harness tests green on the combined tree.

Checklist

  • I have read the relevant architecture docs and SECURITY.md posture (threat model: single-operator, loopback-only)
  • This change preserves all 6 security invariants and I6 module isolation (check_invariants.py: 33 passed, 0 failed)
  • Full pytest suite green on Python 3.12 (GROK_API_KEY=dummy pytest tests/ -q — all pass, incl. 14 new tests); ruff check --select E,F,I,B,C4,UP,S clean
  • No new external network dependencies or online assumptions
  • Harness write guards untouched (guarded chain, CSRF, TrustedHost, rate limit all unchanged)
  • Architecture docs update — not needed; no topology or core behavior changed
  • Commit messages follow the prefix convention

Further comments

ELI5: four places where the harness could crash with an ugly, unexplained error now either keep working (deleted file mid-listing, weird token counts from the model server) or fail with a clear, typed message (bad port override, missing console HTML). Nothing about who may connect, what routes require auth, or how the graph routes queries changed — the loopback guard just got stricter and got its first tests. Before/after: all 33 invariant-guard checks pass identically on both sides of the diff.

🤖 Generated with Claude Code

https://claude.ai/code/session_015VHRCPLFdeWvtp21SKo5QQ


Generated by Claude Code

CyClaw Agent and others added 2 commits August 5, 2026 11:55
… guard

- /api/harness/runs: guard the glob->mtime sort against a file deleted
  mid-listing (same race SessionStore.list() already tolerates) instead
  of 500ing
- ollama: a malformed usage block from an OpenAI-compatible proxy now
  degrades token tallies to 0 instead of raising an unparseable 500
- main(): a non-numeric CYCLAW_HARNESS_PORT now fails closed like the
  adjacent range check instead of silently binding the stored port
- create_app(): missing static/harness.html raises the typed
  HarnessConfigError instead of a bare FileNotFoundError
- tests: first coverage for main()'s loopback/port guards, plus the
  race, missing-asset, and malformed-usage regressions

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VHRCPLFdeWvtp21SKo5QQ
…elper

WPS229/WPS210/WPS231/WPS429 flagged the inline try/except in
_parse_chat_response. _token_count degrades each malformed usage field
to 0 independently, so a valid sibling count now survives (test updated
to pin the per-field behavior).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VHRCPLFdeWvtp21SKo5QQ
@cgfixit
cgfixit marked this pull request as ready for review August 5, 2026 20:20
@cgfixit
cgfixit merged commit e7467f5 into main Aug 5, 2026
46 checks passed
@cgfixit
cgfixit deleted the claude/gallant-mayer-xkxgah branch August 5, 2026 20:21

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0c9cf8be36

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread harness/server.py
Comment on lines +857 to +858
with suppress(OSError):
json_files.sort(key=os.path.getmtime, reverse=True)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve mtime ordering when one run file vanishes

When any accepted artifact disappears or becomes unstatable between the glob and the getmtime key calculation, this suppresses the exception from the whole in-place sort rather than dropping only that path. The route then slices json_files[:_MAX_RUNS] in arbitrary glob order, so in a busy accepted-runs directory the console can omit the newest runs and show stale entries instead of the documented newest-first list; compute mtimes per file, skip failures, and sort the surviving pairs before slicing.

Useful? React with 👍 / 👎.

cgfixit added a commit that referenced this pull request Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant