Make analyzer runs more resilient and readable - #24
Merged
Merged
Conversation
The stage validation lane now supports concise default output and opt-in detailed diagnostics across aggregate, project-level, and single-stage entrypoints. The import-chain regression guard is included because the aggregate lane now depends on it and it prevents optional fulltext dependencies from breaking early-stage validation. Constraint: check-project.sh remains the project-level entrypoint and keeps its CLI surface unchanged Constraint: Log assertions must rely on stable text tokens, not emoji rendering Rejected: Add ANSI colored output | CI and Windows log redirection need plain readable text Rejected: Add a check-project.sh --log flag | environment passthrough avoids a second shell argument contract Confidence: high Scope-risk: moderate Directive: Keep stage log formatting centralized in scripts/test_agent/stage_logging.py Tested: python scripts/test_agent/run_contract.py Tested: python scripts/test_agent/check_project_contract.py Tested: python scripts/test_agent/run.py --log brief Tested: python scripts/test_agent/run.py --log detail Tested: CITE_ANALYZER_STAGE_LOG=detail bash ./scripts/check-project.sh Tested: invalid CITE_ANALYZER_STAGE_LOG=noisy exits non-zero with traceback Not-tested: External live smoke modes requiring network/GROBID Co-authored-by: OmX <omx@oh-my-codex.dev>
正式 analyzer 运行链路现在会输出中文阶段进度、外部 API 请求、限速、降级和最终摘要；RuntimeLogger 通过 contextvars 注入，避免污染 AnalysisState。 同时前置修复 Semantic Scholar fields 兼容和 arXiv 版本号归一化，并让 0 施引样本通过合法空产物收口到报告生成。 Constraint: Semantic Scholar 官方 Graph API 不支持 authors.name / citingPaper.authors.name 字段 Constraint: Semantic Scholar API key 按 x-api-key 发送且日志不得泄露密钥 Rejected: 将 logger 写入 AnalysisState | 会污染状态序列化和报告输入边界 Rejected: 将 2602.04144 固定为 0 施引 live gate | 外部数据库和 API 配额具有时变性 Confidence: high Scope-risk: moderate Directive: RuntimeLogger 必须通过入口 context reset；不要在业务包中缓存 logger 实例或打印原始 headers Tested: python scripts/test_agent/runtime_logging_contract.py Tested: python scripts/test_agent/run_contract.py Tested: python scripts/test_agent/check_project_contract.py Tested: python scripts/test_agent/run.py --log brief Tested: python scripts/test_agent/run.py --log detail Tested: python scripts/test_agent/e2e_real_smoke.py --target https://arxiv.org/abs/2504.19162 --max-citations 3 --log detail Tested: python -m compileall apps packages scripts/test_agent Not-tested: 将 e2e_real_smoke.py 接入默认 CI；该入口按设计保持 opt-in live smoke Co-authored-by: OmX <omx@oh-my-codex.dev>
阶段6的 LLM 定位和情感分类提示词改为中文，但保留英文结构化枚举和 evidence 机器前缀，避免破坏 JSON 数据契约与现有测试。HTML 展示层单独把 citation card 的情感标签映射为中文。 Constraint: sentiment_label 是既有 JSON/测试契约，必须继续使用 positive/neutral/critical/unknown Constraint: evidence 前缀被 stage6 contract 用于机器检索，不能整体翻译 Rejected: 全量中文化所有 evidence 常量 | 会扩大范围并破坏 fallback/no_text 等现有断言 Confidence: high Scope-risk: narrow Directive: 后续若要全报告中文化，应在展示层增加翻译，不要直接改内部枚举或机器前缀 Tested: python scripts/test_agent/llm_prompt_contract.py Tested: python scripts/test_agent/stage6.py Tested: python scripts/test_agent/stage7.py Tested: python scripts/test_agent/runtime_logging_contract.py Tested: python scripts/test_agent/run.py --log brief Tested: python scripts/test_agent/run_contract.py Tested: python scripts/test_agent/check_project_contract.py Tested: python -m compileall apps packages scripts/test_agent Tested: git diff --check Not-tested: live Semantic Scholar/GROBID smoke after prompt change; local fixture-backed e2e passed Co-authored-by: OmX <omx@oh-my-codex.dev>
报告已经有 year/source/scholar/sentiment 的结构化 payload，但 HTML 只展示列表。本次在 HTML 展示层接入 ECharts，并为单点年份、单桶情感和低基数数据保留 fallback，避免把稀疏数据伪装成趋势或满饼图。 Constraint: report.json 的 summary/charts/provenance/contexts 是既有数据契约，不能被 ECharts option 污染 Constraint: 当前 source_map 只是作者首条机构文本聚合，不能渲染成地理地图 Constraint: CDN 外部资源必须固定版本且保留无 JS fallback Rejected: 全量图表化 manual_attention_items | 它是混合诊断清单，不是干净分布数据 Rejected: 使用饼图呈现单桶 sentiment | 信息量低且容易夸大确定性 Confidence: high Scope-risk: moderate Directive: 后续若要真正地图，先做国家/地区标准化；不要直接复用 source_map 文本字段 Tested: python scripts/test_agent/stage7.py Tested: python scripts/test_agent/run_contract.py Tested: python scripts/test_agent/run.py --log brief Tested: python scripts/test_agent/check_project_contract.py Tested: python -m compileall apps packages scripts/test_agent Tested: git diff --check Tested: python scripts/test_agent/e2e_real_smoke.py --target https://arxiv.org/abs/2504.19162 --max-citations 3 --log brief Not-tested: visual browser screenshot review; Chrome DevTools MCP was unavailable in this session Co-authored-by: OmX <omx@oh-my-codex.dev>
???????????????????????????????????? retry ?????? requests / urllib ? timeout?TLS/SSL?429/5xx ? Retry-After??? detail/warn ?????? runtime ??? OpenAlex?DBLP???????????GROBID ? LLM provider ??????????Semantic Scholar ???? 1 ? 1 ????Crossref ????????????? 4 ???????????????????????????? Constraint: Semantic Scholar API ???? 1 request/second ????????????? Constraint: GROBID processFulltextDocument ???? POST????????? Rejected: ? LLM ?????? | ???????????????????? Rejected: ? service ??????????? | ?? Semantic Scholar/Crossref ?????? Confidence: high Scope-risk: moderate Directive: ????????????? packages/shared/network_retry.py??????????? Tested: python scripts/test_agent/run.py --log brief Tested: python scripts/test_agent/network_retry_contract.py Tested: python -m compileall apps packages scripts/test_agent Tested: bash ./scripts/check-docs.sh Tested: git diff --check Not-tested: opt-in live smoke against current real APIs in this final pass Co-authored-by: OmX <omx@oh-my-codex.dev>
The PR CI runs the aggregate stage validation on a clean Ubuntu runner. The stage5 and stage6 contracts require the project test dependencies to be available, so CI now installs a small CI-specific Python dependency set before running the existing repository checks. Constraint: GitHub CI is the source of truth for pull request checks Constraint: The repository does not yet maintain a full runtime lockfile Rejected: Skip stage5 when bs4 or pypdf is missing | that would hide the contract GitHub CI is expected to verify Rejected: Add an unpinned GitHub setup action | action pinning policy requires SHA-pinned workflow actions Confidence: high Scope-risk: narrow Directive: Replace requirements-ci.txt with a project lockfile once dependency management is formalized Tested: bash ./scripts/check-action-pinning.sh Tested: bash ./scripts/check-docs.sh Not-tested: Full clean-runner dependency install locally Co-authored-by: OmX <omx@oh-my-codex.dev>
GitHub PR checks should run the real aggregate validation on a clean runner, but the fixture path must not require live LLM credentials. The stage6 fixture now uses deterministic fake reference matching and sentiment classification, while requirements-ci.txt is raised to OSV-clean dependency floors. Constraint: Pull request CI runs without .env or live LLM credentials Constraint: OSV scans the CI requirements file and fails on known vulnerable ranges Rejected: Skip stage5 or stage6 in CI | GitHub should verify the aggregate validation rather than bypass it Rejected: Install langchain-openai for the fixture path | that would still require API credentials and blur offline versus live tests Confidence: high Scope-risk: narrow Directive: Keep default stage tests offline; put live LLM/API checks behind explicit opt-in smoke scripts Tested: python scripts/test_agent/stage6.py Tested: python scripts/test_agent/run.py --log brief Tested: python -m compileall apps packages scripts/test_agent Tested: bash ./scripts/check-docs.sh Tested: bash ./scripts/check-action-pinning.sh Tested: git diff --check Not-tested: GitHub clean runner after this commit Co-authored-by: OmX <omx@oh-my-codex.dev>
The pull request CI should exercise the full analyzer graph without requiring live LLM SDKs or API credentials. The E2E fixture now injects the same deterministic reference matcher and sentiment classifier used by the stage 6 validation, so repository checks can fail on orchestration regressions instead of environment-dependent LLM setup. Constraint: GitHub CI is the merge gate and must run without .env or live model credentials Rejected: Skip the E2E or stage 5 path in CI | that would reduce regression coverage for the analyzer workflow Confidence: high Scope-risk: narrow Directive: Keep default PR checks offline and deterministic; place live provider checks behind explicit opt-in scripts Tested: python scripts/test_agent/e2e_mvp.py Tested: python scripts/test_agent/run.py --log brief Tested: python -m compileall apps packages scripts/test_agent Tested: bash ./scripts/check-docs.sh Tested: bash ./scripts/check-action-pinning.sh Tested: git diff --check Not-tested: GitHub-hosted CI for this commit before push Co-authored-by: OmX <omx@oh-my-codex.dev>
Repository checks now run markdownlint across tracked documentation in GitHub Actions. A few existing planning and design documents violated list spacing, duplicate heading, and trailing-space rules, so this commit narrows the fix to those reported files without changing analyzer behavior. Constraint: GitHub CI is the authoritative merge gate for this PR Rejected: Disable markdownlint in CI | that would hide real documentation hygiene failures Confidence: high Scope-risk: narrow Directive: Keep CI lint fixes separate from analyzer behavior changes Tested: npx markdownlint-cli2 docs/design-docs/citation-analysis-maps.md docs/exec-plans/active/2026-05-05-report-frontend-upgrade.md docs/exec-plans/active/2026-05-16-runtime-log-chinese-readability.md Tested: bash ./scripts/check-docs.sh Tested: git diff --check Not-tested: GitHub-hosted CI for this commit before push Co-authored-by: OmX <omx@oh-my-codex.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test Plan