Skip to content

Protect experiment reporting with privileged auth and bounded inputs - #38

Merged
llinsss merged 1 commit into
DogStark:mainfrom
maztah1:main
Aug 22, 2026
Merged

Protect experiment reporting with privileged auth and bounded inputs#38
llinsss merged 1 commit into
DogStark:mainfrom
maztah1:main

Conversation

@maztah1

@maztah1 maztah1 commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Summary

GET /api/v1/experiments/report and POST /api/v1/experiments/report/export had no authentication dependency, even though the README documents that every /api/v1/* route requires a bearer API key. Any unauthenticated caller could trigger a full scan of every student profile and force a file write to disk. This PR closes that gap: both routes now require a privileged (admin/researcher) account, retention_days is bounded, exports no longer leak a host filesystem path, and a new route-table test guards against a future route being added without auth.

Closes #21 — Protect experiment reporting with privileged auth and bounded inputs

Problem: get_experiment_report and export_experiment_report in api/routes.py had no Depends(...) auth dependency at all. The report scans all student profiles; the export additionally writes derived data to disk. Both were reachable by any network caller with no credentials, and retention_days was an unbounded int query parameter.

Required behavior (from the issue):

  • Missing/invalid credentials → 401; authenticated but unprivileged accounts → 403; a privileged account can retrieve/export the report.
  • retention_days bounded (1–365) with 422 on invalid values, and no profile scan/file write on an invalid value.
  • Exports return only a safe artifact identifier, never a host filesystem path.
  • The auth dependency is centralized so future experiment endpoints inherit it by default.
  • README/OpenAPI docs state the role requirement.
  • Automated tests enumerate registered /api/v1 routes and enforce the auth policy, with public endpoints explicitly allowlisted.

What changed:

  • agent/auth.py: added require_researcher, a FastAPI dependency requiring the admin or researcher role (authorize_role(account, {"admin", "researcher"})), mirroring the existing require_admin pattern used for curriculum management.
  • api/routes.py:
    • Introduced an experiments_router sub-router (prefix="/experiments") with dependencies=[Depends(require_researcher)] set at the router level, and moved both experiment endpoints onto it, then router.include_router(experiments_router). Because the dependency lives on the sub-router rather than each endpoint, any future route added under /experiments inherits the privileged-role requirement automatically.
    • retention_days changed from a bare int default to Query(default=DEFAULT_RETENTION_DAYS, ge=1, le=365) on both routes — FastAPI validates and rejects out-of-range/non-integer values with 422 before the handler body (and therefore the profile scan or file write) ever runs.
    • The export route now returns {"exported_file": os.path.basename(path)} instead of {"exported_to": path}, so only the artifact's filename is exposed, never the full host path.
    • Updated both endpoint docstrings (used as OpenAPI descriptions) to state the role requirement.
  • README.md: documented that curriculum-management and experiment-reporting routes require a privileged admin/researcher account rather than any parent/teacher account, documented the retention_days bounds, and updated the export example response to the new exported_file shape.

Tests (tests/test_api_security.py, new file):

  • TestExperimentReportAuth: missing credentials → 401; invalid credentials → 401; parent/teacher accounts → 403; admin and researcher accounts → 200 on both report and export; export response contains only a bare filename (never a path separator).
  • TestRetentionDaysValidation: out-of-range (0, -1, 366, 10000) and non-integer retention_days422; a monkeypatched compute_variant_metrics/export_experiment_report_json that raises AssertionError if called proves an invalid retention_days never reaches the profile scan or file write; boundary values 1 and 365 are accepted.
  • TestRouteAuthPolicy: walks the FastAPI dependant tree of every registered /api/v1/* route (including router-level dependencies=) and asserts require_account is present, failing the test if a future route is added with no auth dependency, except for an explicit PUBLIC_API_V1_PATHS allowlist (the two routes that are already, and intentionally, unauthenticated today — POST /api/v1/hint and GET /api/v1/neighbors/{word} — which are out of scope for this issue). A companion test also asserts the two experiment routes specifically resolve require_researcher, and another guards the allowlist itself against going stale.

Verification

Ran with Python 3.14 in a clean venv against requirements-dev.txt (the project's declared range is 3.11–3.12; CI runs those versions — see note below):

- ruff check .                                                          — passed
- mypy                                                                  — passed (no issues found in 19 source files)
- pytest --cov=agent --cov=api --cov=dashboard --cov=main --cov-branch
  --cov-report=term-missing                                            — passed (168 passed; 2 pre-existing/unrelated
                                                                           Windows-only tests excluded, see below);
                                                                           coverage 82.80% (floor: 80%)
- pip-audit -r requirements.txt -r requirements-dev.txt (with the same
  --ignore-vuln list CI uses)                                          — passed, no new findings
- API startup smoke check (uvicorn main:app + GET /)                   — passed; confirmed
  GET /api/v1/experiments/report now returns 401 with no credentials

Known Pre-existing Issues

Two pre-existing test failures are Windows-only environment artifacts, unrelated to this change (neither touches auth, experiments, or reporting code):

  • tests/test_concurrency.py::TestConcurrency::test_concurrent_attempt_same_student and test_concurrent_storage_read_writePermissionError from os.replace racing on a hardcoded /tmp/... path under concurrent writes on Windows (this repo's dev container is Linux in CI; these pass there).
  • tests/test_ai_safety.py::TestAdversarialInputCorpus::test_huge_payloads_rejected (4 parametrized cases) — ValueError: the environment variable is longer than 32767 characters, a Windows subprocess/env-var length limit hit by this test's large payload construction.

Both reproduce identically on a clean checkout of origin/main before this PR's changes, confirming they predate it.

Scope Confirmation

This PR is limited to issue #21. No unrelated bugs were fixed, no refactors were performed outside the experiment-reporting auth boundary, and no dependencies were changed.

Closes #21

GET /api/v1/experiments/report and POST /api/v1/experiments/report/export
had no authentication dependency, letting any network caller scan every
student profile and trigger a file write, contrary to the documented
"every /api/v1/* route requires an API key" policy.

- Add agent.auth.require_researcher (admin or researcher role) and gate
  both routes on it via a new experiments sub-router, so the policy is
  centralized at the router boundary and inherited by future routes added
  under /experiments.
- Bound retention_days to 1-365 via FastAPI Query validation on both routes,
  rejected with 422 before any profile scan or file write.
- Return only the exported artifact's filename instead of the host
  filesystem path.
- Document the role requirement in the README.
- Add tests/test_api_security.py: auth/role coverage for the experiment
  routes, retention_days bounds, and a route-table audit that fails if a
  future /api/v1 route is added without an auth dependency (with an
  explicit allowlist for the two routes that are intentionally public).

Closes DogStark#21
@llinsss
llinsss merged commit 7e4233c into DogStark:main Aug 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Protect experiment reporting with privileged auth and bounded inputs

2 participants