test(docs): an env var read through a family prefix is graded, and the dashboard auth knobs are documented - #3669
Conversation
…twelve dashboard auth knobs are documented tests/test_env_vars_the_package_reads_are_documented.py resolves a key only when it is a literal or a module constant, so dashboard/auth.py's os.getenv(_ENV + "ENABLED") and its two resolvers spelling var = _ENV + name were invisible to it by construction. Measured on main, twelve STRANDS_DASH_AUTH_* names were read and no page under README.md or docs/ named one, while the module's own refusals tell an operator to set ORIGIN and RP_ID. The walk now concatenates a + chain of literals and module constants, and a resolver may wrap the parameter it reads through in such text, possibly via a local bound once. The grader fails on main naming exactly the twelve; the new configuration.md block documents them with their defaults and guards.
yinsong1986
left a comment
There was a problem hiding this comment.
Summary
This PR closes a blind spot in tests/test_env_vars_the_package_reads_are_documented.py: a key built at the read site from a family prefix (os.getenv(_ENV + "ENABLED"), or a resolver spelling var = _ENV + name) put no whole variable name anywhere the grader's walk could see, so the twelve STRANDS_DASH_AUTH_* variables read by strands_robots.dashboard.auth were undocumented while the module's own refusals send operators to set them. The grader now concatenates + chains of literals and module constants, models resolvers as prefix/parameter/suffix templates (following a local bound exactly once), and docs/reference/configuration.md gains a collapsed block naming all twelve. I reproduced both halves against the head: with the base tree's docs the widened grader fails naming exactly the 12 STRANDS_DASH_AUTH_* variables and nothing else, and with the new docs block the full module reports 27 passed - so the widening introduces no false positives elsewhere in the tree. I also spot-checked the documented defaults and guards against dashboard/auth.py (_DURATION_DEFAULTS = 86400/2592000/300, _challenge_cap("CHAL_MAX", 512, 2) / ("CHAL_MAX_PER_IP", 16, 1), the strictly-below refusal at import, the store path, and the ENROLL_TOKEN_FILE fallback beside the store) and every row is accurate.
What's good
- The refusal boundary is pinned on both sides: two new negative cells (a key built from two parameters, a prefix bound only inside the resolver) keep the walk from inventing names no page could spell, alongside the three new positive shapes.
- Docs describe existing env-var names and behaviour rather than introducing any - no new public surface, no one-way door.
- Changelog fragment carries the PR number, new test names describe behaviour rather than provenance, added text is plain ASCII, and
ruff check/ruff format --checkare clean on the test module.
What
tests/test_env_vars_the_package_reads_are_documented.pynow resolves a key that is built at the read site - a+chain of string literals and module constants (os.getenv(_ENV + "ENABLED")), and a resolver that wraps the parameter it reads through in such text, possibly via a local bound once (var = _ENV + namethenos.getenv(var)).docs/reference/configuration.mdgains a collapsed block documenting the twelveSTRANDS_DASH_AUTH_*variables that shape had hidden, with each default and guard taken fromdashboard/auth.py.Why
The grader's thesis is that recognising the direct spellings alone reports a clean tree that is not one, and it had one more such shape.
_key_stringaccepted a literal or a module constant and nothing else, andenvironment_resolversaccepted only a key that is a parameter - so a module that binds its family prefix once and concatenates a suffix at each read put no whole name anywhere the walk could see. Measured onmainat9c90ebd:strands_robots/dashboard/auth.pyos.getenv(_ENV + "<SUFFIX>")ENABLED,STORE,RP_NAME,RP_ID,ORIGIN,BOOTSTRAP_TOKEN,ENROLL_TOKEN_FILE_duration(name):var = _ENV + name; os.getenv(var)TOKEN_TTL,SESSION_MAX_AGE,HANDOFF_TTL_challenge_cap(name, ...): same shapeCHAL_MAX,CHAL_MAX_PER_IPgrep -rn STRANDS_DASH_AUTH README.md docs/returns nothing, so none of the twelve was on any page - while the module's own refusals send an operator toset STRANDS_DASH_AUTH_ORIGINandset STRANDS_DASH_AUTH_RP_ID, and the three durations and two caps are refused rather than defaulted precisely so a narrowed window is never silently widened. A knob that exists to be tightened and is named nowhere is the reassuring-direction silence the grader was written for.The pattern is unique to this module: a scan of the package for a concatenated environment key finds no other site, so the widened walk surfaces exactly these twelve and nothing else.
Tests
main's docs: fails, naming exactly the 12STRANDS_DASH_AUTH_*names and no other. With the new block: 27 passed.TestTheReadShapesAreAllRecognised: a key concatenated onto a module prefix; a resolver whose local wraps the parameter in a prefix; a resolver that appends to its parameter; and two refusals - a key built from two parameters, and a prefix bound only inside the resolver - each of which names nothing a page could spell, so the boundary stays where the module docstring states it.ruff check/ruff format --check/mypyclean on the test module.docs/ortests/(the 11 modules namingconfiguration.md, plus the xref, ASCII, host-path, test-name, mixed-return, cast-import, sys.modules and roster graders): 433 passed; the 3 failures arejson5-absent cells intest_docs_mesh_accept_permissive_acl_env_var_reference.pythat fail identically on the unmodified tree in this environment.Overlap
scripts/check_merge_base_overlap.py --pathsreports no behaviour-bearing overlap; #3205 also adds rows toconfiguration.md, inside the mesh table, and this block is a separate<details>after the Isaac and diagnostic ones, so the two compose textually in either order.