Skip to content

Add GitHub Action for automatic ReEDS index update (Handled in https://github.com/Yunzhi-Chen/ReEDS-GPT-Index/actions instead)#64

Closed
Yunzhi-Chen wants to merge 1 commit into
mainfrom
yc/reeds_index
Closed

Add GitHub Action for automatic ReEDS index update (Handled in https://github.com/Yunzhi-Chen/ReEDS-GPT-Index/actions instead)#64
Yunzhi-Chen wants to merge 1 commit into
mainfrom
yc/reeds_index

Conversation

@Yunzhi-Chen
Copy link
Copy Markdown
Contributor

Summary

Add a GitHub Actions workflow that automatically rebuilds the FAISS vector index used by ReEDS-GPT whenever code changes are merged to main. This keeps the AI knowledge base in sync with the latest ReEDS codebase without manual intervention.

Technical details

The workflow builds a FAISS vector index from all text-like files in the repo (.py, .gms, .md, .csv, .yaml, etc.), using Google Gemini embeddings (gemini-embedding-001), and uploads the result to the existing GCS bucket (gs://reeds-agent-index) that powers the ReEDS-GPT ChatGPT assistant.

Implementation notes

  • Incremental updates on push to main: downloads the previous index from GCS, compares file signatures (size + mtime) via manifest.json, and only re-embeds changed/new files. Cost: <$0.10 per run.
  • Full rebuild on the 1st of each month (cron) or via manual trigger. Cost: ~$5–10 per run.
  • File timestamps are restored from git history using git-restore-mtime so the incremental skip logic works correctly in CI.
  • After uploading the updated index, the workflow restarts the Cloud Run service so it loads the fresh index immediately.
  • The build script uses lazy imports so only Gemini-related packages are installed on the CI runner (no torch or sentence-transformers).

Additional changes

None. No changes to ReEDS model code, inputs, or outputs.

Issues resolved

Resolves the manual index rebuild process described in the future development list (item: "Index ReEDS repo to improve AI chat accuracy; automate the indexing process so it updates together with ReEDS-GPT").

Validation, testing, and comparison report(s)

  • Python script passes ast.parse() syntax validation
  • YAML passes yaml.safe_load() validation
  • Workflow only triggers on push to main — pushing to feature branches has no effect
  • No impact on ReEDS model code or results
  • Two required GitHub secrets (GCP_SA_KEY, GOOGLE_API_KEY) must be configured in repo Settings → Secrets before the workflow will succeed

Checklist for author

Details to double-check

  • Charge code provided to reviewers
  • Included comparison reports for appropriate test cases
  • Documentation updated if necessary
  • If input data added/modified
  • Code formatting standardized
  • Reusable functions used where possible instead of copy/pasted code

General information to guide review

  • Zero impact on results of default case
  • No large data file(s) added/modified
  • No substantive impact on runtime for full-US reference case
  • No substantive impact on folder size for full-US reference case
  • No change to process flow (runbatch.py, d_solve_iterate.py)
  • No change to code organization
  • No change to package requirements (environment.yml or Project.toml)

Did you use LLM tools (chatbot or copilot) in the preparation of this PR? If so, describe how

Yes. GitHub Copilot (Claude) was used to write both the build script and the workflow YAML.

Tag points of contact here if you would like additional review of the relevant parts of the model

  • Github actions: @kminderm

@Yunzhi-Chen Yunzhi-Chen self-assigned this Apr 28, 2026
@Yunzhi-Chen Yunzhi-Chen closed this May 1, 2026
@Yunzhi-Chen Yunzhi-Chen changed the title Add GitHub Action for automatic ReEDS index update Add GitHub Action for automatic ReEDS index update (Handled in https://github.com/Yunzhi-Chen/ReEDS/actions instead) May 1, 2026
@Yunzhi-Chen Yunzhi-Chen changed the title Add GitHub Action for automatic ReEDS index update (Handled in https://github.com/Yunzhi-Chen/ReEDS/actions instead) Add GitHub Action for automatic ReEDS index update (Handled in https://github.com/Yunzhi-Chen/ReEDS-GPT-Index/actions instead) May 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant