ci: auto-trigger argus-test validation on PRs

[eFAILution]

Problem

Testing argus feature branches against argus-test is currently manual — someone has to go to argus-test and dispatch test-suite.yml with argus_ref=<branch>. This creates a gap where argus PRs can merge without being validated by the external test suite.

Goal

Automatically trigger argus-test's validation pipeline when an argus PR is opened/updated, and surface the results back on the argus PR — without requiring PATs.

Constraints

• No PATs — avoid long-lived tokens and the management overhead
• argus-test is private — GITHUB_TOKEN cross-repo access is limited to the huntridge-labs org context
• Reusable workflow tests (I1, I3) will always run against @main due to GitHub's static uses: requirement — this is acceptable since unit + action tests cover the core logic
• Solution should scale if more downstream test repos are added later

Options to Explore

1. repository_dispatch via GITHUB_TOKEN

Argus PR workflow sends a repository_dispatch to argus-test. Since both repos are under huntridge-labs, GITHUB_TOKEN with repo scope may work for dispatching to private repos in the same org — needs validation.

Pros: Simple, no extra tokens
Cons: GITHUB_TOKEN may not have cross-repo dispatch permissions even within the same owner. Reporting status back to the argus PR still requires a mechanism.

2. GitHub App (org-scoped)

Create a lightweight GitHub App installed on both repos. The app token can dispatch workflows cross-repo and post check statuses back.

Pros: Fine-grained permissions, no user-tied credentials, scalable
Cons: Setup overhead (app registration, private key management via secrets)

3. Reusable workflow in argus-test called from argus

Argus CI calls argus-test's test suite as a reusable workflow. Cross-repo reusable workflows work for private repos within the same org when explicitly allowed.

Pros: Native GitHub Actions, no tokens needed, results appear directly on the argus PR
Cons: Reusable workflow uses: requires a static ref — can't dynamically pass argus_ref back to itself. May require argus-test to expose a slimmed-down callable workflow.

4. Workflow artifact + polling pattern

Argus CI creates a dispatch and then polls for the argus-test run result via the GitHub API (using GITHUB_TOKEN if permitted, or actions: read on the org level).

Pros: Decoupled
Cons: Polling is fragile and slow, adds complexity

5. GitHub Actions OIDC + fine-grained token exchange

Use OIDC to mint a short-lived token scoped to argus-test. Requires configuring trust between the repos.

Pros: No long-lived secrets, modern approach
Cons: Significant setup, may be over-engineered for this use case

R&D Tasks

• Verify whether GITHUB_TOKEN can send repository_dispatch to a private repo under the same owner
• Evaluate GitHub App approach — estimate setup effort vs. long-term benefit
• Prototype the reusable workflow approach (Option 3) — test cross-repo workflow_call between argus and argus-test
• Determine how to report results back to the argus PR (commit status, check run, or PR comment)
• Consider concurrency — argus-test's cancel-in-progress: true means simultaneous argus PRs could cancel each other's validation runs

Context

• argus-test already supports argus_ref input on test-suite.yml for targeting feature branches
• Unit (U1–U5) and action (A1–A5) tests can use dynamic refs via the local checkout pattern
• Reusable workflow tests (I1, I3) are pinned to @main — this is a known limitation
• Both repos are owned by huntridge-labs

[1blt]

Thoughts

• Original thought for argus-test is that it serves as a client-focused check. Idea is that you can even delete the argus repository and the argus-test will still give you a status, kind of like is the website down pages. Also it was just to do a check with a different set of eyes.

• If the goal is to use the test suite as a conditional blocker on PR to argus itself, what are your thoughts just deleting argus-test and putting a single comprehensive test on merge to main? If test success, merge happens and we can update the website in argus. If test fails, the merge is blocked?

Pros:

• Don't need to work with PATs, everything is in one repository
• Low complexity: user's don't need to know more things on how to operate argus, block happens automatically and provides feedback and incentive to install pre-commits locally

Cons:

• Repository becomes a bit larger, and it becomes possible for users/agents to accidentally modify the tests themselves

[eFAILution]

I have integrated some reusable workflow automation as of last night. We do have the status badges on main that sort of serve the same purpose as the Argus-test dashboard? I was striving for that extremely high confidence that comes from an external consuming repo.

[1blt]

Really depends on the goals here.

For reporting

I think a website is helpful for PM types because its a one stop shop that populate information, timing, and breakdowns. The website doesn't need PATs right now because it runs on cron at the start of each Monday, but PMs don't need to care that the synchronization is immediate, just that there is a consistent movement? The status badge on argus main contains a lot of non-data ink for status, and doesn't break down by category the features that are working/ not working.

For blocking bad merges

Personally I feel that using argus-test to PR block argus feels funny with the no-pat constraint, and would move that into argus. Would like to discuss more about this.

[eFAILution]

Update: Testing Strategy for 1.0.0

After the SDK refactor (PR #88), our testing approach has evolved. Documenting the plan here.

Decision: Single repo for PR gating, argus-test for release validation

PR gating stays in-repo. The argus repo now has 403+ pytest tests, CLI E2E tests, container build/scan CI, and audit trail validation — all running on every PR. The cross-repo dispatch problem (PATs, static refs, concurrency) isn't worth solving when in-repo coverage is comprehensive.

argus-test becomes a scheduled release validator. It runs weekly (and optionally on releases) against the latest published version, validating from a real consumer's perspective. If it fails, it opens an issue on argus.

Ref Lifecycle

PR:      ./ (local checkout — tests the branch)
main:    @main (post-merge validation)  
release: @1.0.0 (release-it bumps all refs automatically)

What argus-test needs for 1.0.0

1. Update workflows to work with the restored interfaces (same inputs, argus CLI internally)
2. Add argus CLI test category (argus scan --list, argus scan container, etc.)
3. Add alerting: open an issue on argus when the scheduled run fails

Full plan

See docs/developer/testing-strategy.md in PR #88 for the complete testing strategy, ref lifecycle, and argus-test TODO.

This approach addresses the original goal — automated external validation — without the PAT/dispatch complexity. The in-repo pipeline handles PR confidence. argus-test handles release confidence.