Skip to content

test(docs): an env var read through a family prefix is graded, and the dashboard auth knobs are documented - #3669

Merged
cagataycali merged 2 commits into
strands-labs:mainfrom
cagataycali:fix/env-var-grader-sees-a-prefixed-key
Sep 15, 2026
Merged

cagataycali merged 2 commits into
strands-labs:mainfrom
cagataycali:fix/env-var-grader-sees-a-prefixed-key

Conversation

@cagataycali

Copy link
Copy Markdown
Member

What

tests/test_env_vars_the_package_reads_are_documented.py now resolves a key that is built at the read site - a + chain of string literals and module constants (os.getenv(_ENV + "ENABLED")), and a resolver that wraps the parameter it reads through in such text, possibly via a local bound once (var = _ENV + name then os.getenv(var)). docs/reference/configuration.md gains a collapsed block documenting the twelve STRANDS_DASH_AUTH_* variables that shape had hidden, with each default and guard taken from dashboard/auth.py.

Why

The grader's thesis is that recognising the direct spellings alone reports a clean tree that is not one, and it had one more such shape. _key_string accepted a literal or a module constant and nothing else, and environment_resolvers accepted only a key that is a parameter - so a module that binds its family prefix once and concatenates a suffix at each read put no whole name anywhere the walk could see. Measured on main at 9c90ebd:

read shape in strands_robots/dashboard/auth.py names seen by the grader
os.getenv(_ENV + "<SUFFIX>") ENABLED, STORE, RP_NAME, RP_ID, ORIGIN, BOOTSTRAP_TOKEN, ENROLL_TOKEN_FILE no
_duration(name): var = _ENV + name; os.getenv(var) TOKEN_TTL, SESSION_MAX_AGE, HANDOFF_TTL no
_challenge_cap(name, ...): same shape CHAL_MAX, CHAL_MAX_PER_IP no

grep -rn STRANDS_DASH_AUTH README.md docs/ returns nothing, so none of the twelve was on any page - while the module's own refusals send an operator to set STRANDS_DASH_AUTH_ORIGIN and set STRANDS_DASH_AUTH_RP_ID, and the three durations and two caps are refused rather than defaulted precisely so a narrowed window is never silently widened. A knob that exists to be tightened and is named nowhere is the reassuring-direction silence the grader was written for.

The pattern is unique to this module: a scan of the package for a concatenated environment key finds no other site, so the widened walk surfaces exactly these twelve and nothing else.

Tests

  • The widened grader on main's docs: fails, naming exactly the 12 STRANDS_DASH_AUTH_* names and no other. With the new block: 27 passed.
  • Five new cells in TestTheReadShapesAreAllRecognised: a key concatenated onto a module prefix; a resolver whose local wraps the parameter in a prefix; a resolver that appends to its parameter; and two refusals - a key built from two parameters, and a prefix bound only inside the resolver - each of which names nothing a page could spell, so the boundary stays where the module docstring states it.
  • ruff check / ruff format --check / mypy clean on the test module.
  • Delta read over the graders whose input is docs/ or tests/ (the 11 modules naming configuration.md, plus the xref, ASCII, host-path, test-name, mixed-return, cast-import, sys.modules and roster graders): 433 passed; the 3 failures are json5-absent cells in test_docs_mesh_accept_permissive_acl_env_var_reference.py that fail identically on the unmodified tree in this environment.

Overlap

scripts/check_merge_base_overlap.py --paths reports no behaviour-bearing overlap; #3205 also adds rows to configuration.md, inside the mesh table, and this block is a separate <details> after the Isaac and diagnostic ones, so the two compose textually in either order.

…twelve dashboard auth knobs are documented

tests/test_env_vars_the_package_reads_are_documented.py resolves a key only
when it is a literal or a module constant, so dashboard/auth.py's
os.getenv(_ENV + "ENABLED") and its two resolvers spelling var = _ENV + name
were invisible to it by construction. Measured on main, twelve
STRANDS_DASH_AUTH_* names were read and no page under README.md or docs/
named one, while the module's own refusals tell an operator to set ORIGIN
and RP_ID.

The walk now concatenates a + chain of literals and module constants, and a
resolver may wrap the parameter it reads through in such text, possibly via
a local bound once. The grader fails on main naming exactly the twelve; the
new configuration.md block documents them with their defaults and guards.
@cagataycali
cagataycali marked this pull request as ready for review September 15, 2026 09:28
@cagataycali
cagataycali enabled auto-merge (squash) September 15, 2026 09:28

@yinsong1986 yinsong1986 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

This PR closes a blind spot in tests/test_env_vars_the_package_reads_are_documented.py: a key built at the read site from a family prefix (os.getenv(_ENV + "ENABLED"), or a resolver spelling var = _ENV + name) put no whole variable name anywhere the grader's walk could see, so the twelve STRANDS_DASH_AUTH_* variables read by strands_robots.dashboard.auth were undocumented while the module's own refusals send operators to set them. The grader now concatenates + chains of literals and module constants, models resolvers as prefix/parameter/suffix templates (following a local bound exactly once), and docs/reference/configuration.md gains a collapsed block naming all twelve. I reproduced both halves against the head: with the base tree's docs the widened grader fails naming exactly the 12 STRANDS_DASH_AUTH_* variables and nothing else, and with the new docs block the full module reports 27 passed - so the widening introduces no false positives elsewhere in the tree. I also spot-checked the documented defaults and guards against dashboard/auth.py (_DURATION_DEFAULTS = 86400/2592000/300, _challenge_cap("CHAL_MAX", 512, 2) / ("CHAL_MAX_PER_IP", 16, 1), the strictly-below refusal at import, the store path, and the ENROLL_TOKEN_FILE fallback beside the store) and every row is accurate.

What's good

  • The refusal boundary is pinned on both sides: two new negative cells (a key built from two parameters, a prefix bound only inside the resolver) keep the walk from inventing names no page could spell, alongside the three new positive shapes.
  • Docs describe existing env-var names and behaviour rather than introducing any - no new public surface, no one-way door.
  • Changelog fragment carries the PR number, new test names describe behaviour rather than provenance, added text is plain ASCII, and ruff check / ruff format --check are clean on the test module.

@cagataycali
cagataycali merged commit 8e37056 into strands-labs:main Sep 15, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants