Add agent-friendly research CLI + research skill#1
Open
sherrymao wants to merge 1 commit into
Open
Conversation
Wrap the core research loop (fetch → review candidates → ingest → validate → report) into a unified `python -m scripts.cli` with `--json` machine output, plus a `cron` subcommand for scheduled fetch rounds. The CLI reuses existing functions (fetch_arxiv, ingest_file, delete_papers, validate_all, etc.) and does not replace the per-script __main__ entry points or Makefile — backward compatible. Ship a project-level `research` skill (.claude/skills/research/) so any Claude Code session in this repo auto-loads an agent-pluggable entry to the whole workflow without starting the Flask server. Covers the minimal core loop; notes/notebooks/score/briefing/notebooklm deferred to a second round via the subcommand registry. - scripts/cli.py: unified CLI (fetch/cron/candidates/ingest/papers/validate/status/report) - .claude/skills/research/SKILL.md: xhs-research-style skill with command quick-ref - tests/test_cli.py: 16 tests (dispatch, --json, candidates file r/w, cron --ingest) - Makefile: `make research CMD=...` passthrough - README.md + CLAUDE.md: document the skill + prerequisites Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
scripts/cli.py— unified agent-friendly CLI wrapping the existing research workflow. Subcommands:fetch,cron,candidates(list/update/delete),ingest,papers(list/delete),validate,status,report. Global--jsonflag for machine-readable output (in JSON mode, library progress prints are suppressed so stdout is pure JSON)..claude/skills/research/SKILL.md— project-level Claude Code skill. Auto-loads in any Claude Code session opened in this repo. Wrapspython -m scripts.cliso an agent can drive the full research loop without starting the Flask server.tests/test_cli.py— 16 hermetic tests covering dispatch,--jsonemission, candidates update/delete, and thecron --ingestround withfetch_arxivmocked.Makefile— addedmake research CMD='...'target as a passthrough to the CLI.README.md— added theresearchskill to the Claude Code Integration section.CLAUDE.md— three additions: validation vocabularies, score field layering, gitignore boundary.Why
Before:
fetch.py/ingest.pyhad argparse CLIs but only emitted human text;validate.py/report_html.pyhad no arg CLI; candidates update/delete and the rest of the write paths were only on Flask/api/*— so an agent had tomake serveand go through HTTP. The CLI + skill collapses that to a single, reusable surface.Design
fetch_arxiv/write_candidates/load_candidate_file/ingest_file/delete_papers/validate_all/generate_html_dashboard/filter_papers/load_all_papers. Old__main__blocks andmake fetch/make ingest/make validate/make htmlkeep working unchanged._pending_candidates), so they work without Flask running.cronis one round of fetch + optional auto-ingest — designed to be invoked repeatedly byScheduleWakeupor external schedulers, not a system crontab itself.--jsonis a global flag, placed before the subcommand. In JSON mode the handler runs insidecontextlib.redirect_stdout(StringIO())so library progress prints don't pollute stdout — agents can parse the output directly.Verification
python -m scripts.cli --helplists all subcommands.python -m scripts.cli --json fetch --direction post_training --days 30 --limit 5→ valid JSON withcount/file/papers.python -m scripts.cli --json candidates list/update/deletework onoutput/candidates/*.yaml.python -m scripts.cli --json cron --direction post_training --days 7 --ingest→ one fetch + ingest round, JSON summary.python -m scripts.cli --json papers list/validate/status/reportwork.make testis green for the newtests/test_cli.py(pre-existing 2test_briefing.pyfailures are unrelated; present on a clean checkout ofmain).Not in this round
Notes / notebooks / score-adjustment / briefing / notebooklm stay on the Flask web UI or direct store modules. The CLI is structured around a subcommand registry so each can be added in a follow-up without touching the dispatch layer.