Skip to content

fix(analyzer): make E2 whitespace-tolerant and detect all os.environ read forms - #331

Open
weed33834 wants to merge 2 commits into
NVIDIA:mainfrom
weed33834:fix/e2-whitespace-tolerant-env-harvesting
Open

fix(analyzer): make E2 whitespace-tolerant and detect all os.environ read forms#331
weed33834 wants to merge 2 commits into
NVIDIA:mainfrom
weed33834:fix/e2-whitespace-tolerant-env-harvesting

Conversation

@weed33834

@weed33834 weed33834 commented Aug 2, 2026

Copy link
Copy Markdown

Summary

The E2 (Env Variable Harvesting) regex patterns are spell-checkers, not behavior detectors. os\.environ without optional whitespace between os and . means that inserting PEP8-irrelevant whitespace — os . environ . copy () — bypasses detection entirely. A skill scoring DO_NOT_INSTALL with canonical syntax can be rewritten to SAFE by adding spaces. The same blind spot lets dict(os.environ) and {**os.environ} pass clean, even though they do exactly what the rule is meant to catch.

Changes

  1. Whitespace-tolerant os.environ matching\s* between os and \. in all Python E2 patterns, so whitespace variants no longer bypass detection.

  2. New patternsdict(os.environ) and {**os.environ} (the two most common alternative forms of reading the full environment mapping) are now detected.

  3. 5 regression tests added to test_patterns.py.

Testing

$ python -m pytest tests/unit/test_patterns.py tests/nodes/analyzers/test_static_patterns.py -v
157 passed in 2.07s
$ make lint && make format
All checks passed!
157 files left unchanged

Fixes #329

rng1995
rng1995 previously approved these changes Aug 5, 2026

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Automated SkillSpector Review]

Approved. The E2 additions cover whitespace-obfuscated os.environ access plus full-environment reads through dict() and mapping unpacking without changing the rule schema. Focused E2 regressions passed (7 tests), as did Ruff lint and format checks.

@weed33834
weed33834 force-pushed the fix/e2-whitespace-tolerant-env-harvesting branch from 998d750 to 77a6ce5 Compare August 6, 2026 14:30
(r"dict\s*\(\s*os\s*\.\s*environ\s*\)", 0.6),
(r"\{\s*\*\*\s*os\s*\.\s*environ\s*\}", 0.6),
# Python: {**os.environ} — full environ read via spread
(r"\*\*\s*os\s*\.\s*environ", 0.6),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: broadening the prior mapping-literal regex to bare ** makes the malformed-Python fallback flag exponentiation. On this exact head, result = 2 ** os.environ plus a syntax error elsewhere produces an E2 finding on line 2, even though the existing valid-Python negative test correctly treats this expression as non-harvesting. Keep mapping-unpack boundaries or otherwise distinguish unpacking from exponentiation, and add a fallback-path regression.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rng1995 This has been fixed in f950c35. The dict-spread regex now requires { braces, so bare 2 ** os.environ (exponentiation) is NOT matched. Added regression tests for exponentiation-not-flagged, dict-spread-flagged, and whitespace-tolerant os.environ access patterns. mergeable=True, no conflicts. Please re-trigger CI when convenient.

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Automated SkillSpector Review]

Re-review: requesting changes on the new head. The 19 focused E2 tests and Ruff lint pass, but the conflict-resolution commit broadens the malformed-Python fallback from mapping unpacking to bare ** os.environ. Exact-head verification shows 2 ** os.environ is then reported as E2 when another syntax error triggers fallback. Restore unpacking boundaries or equivalent parsing and cover the malformed negative case. GitHub also reports merge conflicts.

@weed33834
weed33834 force-pushed the fix/e2-whitespace-tolerant-env-harvesting branch from 77a6ce5 to a92654b Compare August 11, 2026 11:28
@rng1995

rng1995 commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

@weed33834 - Please address review comments and resolve merge conflicts. Resolve any review comments that have been addressed

…read forms

The E2 regex fallback (used when Python source cannot be parsed by AST)
was missing several common os.environ access patterns and was not
whitespace-tolerant for the patterns it did cover.

Add fallback patterns for:
- os.environ['KEY'] / os.environ["KEY"] (whitespace-tolerant)
- os.environ.get('KEY') (whitespace-tolerant)

All existing patterns (items(), copy(), dict(), {**} spread) retain
whitespace tolerance. The dict-spread regex explicitly requires braces
({**os.environ}) so bare exponentiation (2 ** os.environ) is not
flagged as environment harvesting.

Add regression tests:
- Whitespace-obfuscated environ access is detected
- 2 ** os.environ (exponentiation) is NOT flagged as E2
- {**os.environ} (dict spread) IS flagged as E2

Signed-off-by: badhope <weed33834@users.noreply.github.com>
@weed33834
weed33834 force-pushed the fix/e2-whitespace-tolerant-env-harvesting branch from a92654b to f950c35 Compare August 12, 2026 18:51
weed33834 added a commit to weed33834/SkillSpector that referenced this pull request Aug 13, 2026
…sting (NVIDIA#331)

Signed-off-by: badhope <game33834@outlook.com>
@weed33834
weed33834 force-pushed the fix/e2-whitespace-tolerant-env-harvesting branch from 247e544 to 903d4c2 Compare August 13, 2026 08:23
@weed33834

Copy link
Copy Markdown
Author

Hi @rng1995 and maintainers — the DCO issue is now fixed. The branch was rewritten so all commits in this PR carry a proper Signed-off-by line (unsigned helper commits were dropped, and the merge commit with upstream main 2.9.4 is signed too).

The only remaining blocker is that CI runs on this fork PR require maintainer approval (currently action_required). Could you approve the workflow run so DCO + lint + tests can execute? In the last actual CI run (a92654b), lint, unit tests, and docker-smoke all passed — only DCO failed.

@rng1995 if possible, could you re-review the latest head (903d4c2)? The previous CHANGES_REQUESTED was against an older commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

E2 "Env Variable Harvesting" matches one spelling, so whitespace alone flips DO_NOT_INSTALL to SAFE

2 participants