feat(security,ops): add redaction, retention, CI, diagnostics#4
feat(security,ops): add redaction, retention, CI, diagnostics#4
Conversation
- Add redaction defaults with pattern-based secret detection (17 patterns) - Add retention controls with enforceable policies - Add CI workflow with quality gates (ruff, mypy, pytest) - Add diagnostic CLI for stack health verification - Add unit tests for redaction, retention, config, diagnostics - Add integration tests for retention cleanup and migrations Closes COE-230
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 564f7387d1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| [project.scripts] | ||
| stackperf = "cli:main" |
There was a problem hiding this comment.
stackperf at the packaged CLI module
In a clean install this console script will try to import cli.main, but the wheel only packages the src package (src.cli exists; cli does not). That means uv run stackperf … dies with ModuleNotFoundError before any command runs, including the new validation step in CI and the operator diagnostics this change adds.
Useful? React with 👍 / 👎.
| @click.group() | ||
| @click.version_option(version=__version__, prog_name="stackperf") | ||
| def main() -> None: | ||
| """StackPerf - Harness-agnostic benchmarking system.""" | ||
| pass |
There was a problem hiding this comment.
diagnose group on the root CLI
src/cli/diagnose.py defines diagnose health, diagnose session, and diagnose env, but main here only exposes version and never imports or adds that group. Even after the console entry point is fixed, stackperf diagnose … will still fail with No such command, so the new diagnostics feature is unreachable as shipped.
Useful? React with 👍 / 👎.
| # Check if key itself indicates sensitive data | ||
| if key.lower() in cfg.sensitive_keys: | ||
| result[key] = cfg.placeholder | ||
| else: | ||
| result[key] = redact_value(value, key, cfg) |
There was a problem hiding this comment.
This only redacts when key.lower() exactly matches one of the hard-coded names. Real config payloads often use descriptive keys like db_password or provider_api_keys; those values fall through to redact_string, so any secret that does not match one of the regexes is emitted in cleartext. That breaks the stated guarantee that logs and exports do not leak secrets.
Useful? React with 👍 / 👎.
| expiration = created_at + timedelta(days=self.retention_days) | ||
| return datetime.utcnow() > expiration |
There was a problem hiding this comment.
is_expired compares expiration against datetime.utcnow(), which is offset-naive. If created_at comes from the UTC-aware timestamps the project requires, Python raises TypeError: can't compare offset-naive and offset-aware datetimes, so retention cleanup will crash on those rows instead of expiring them.
Useful? React with 👍 / 👎.
| - name: Validate configs | ||
| run: uv run stackperf validate --all-configs | ||
| continue-on-error: true |
There was a problem hiding this comment.
continue-on-error from required CI gates
GitHub Actions' steps[*].continue-on-error allows the job to pass even when that step exits non-zero. Here the only validation step in config-validation is marked that way, and the migration smoke job does the same for its test step, so broken config validation or broken migrations still produce a green workflow and won't block merges.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 60755f9c29
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| run: uv sync --all-extras | ||
|
|
||
| - name: Validate configs | ||
| run: uv run stackperf validate --all-configs |
There was a problem hiding this comment.
I checked src/cli/__init__.py and the rest of src/cli: this PR does not add a validate subcommand or an --all-configs flag anywhere, and the documented CLI contract is bench config validate. That means this new step can never validate the config tree it is supposed to gate; once the entry-point issue is fixed it will still fail every run, and in the current workflow it only produces a green job without performing any config validation.
Useful? React with 👍 / 👎.
| run: uv sync --all-extras | ||
|
|
||
| - name: Run migration smoke test | ||
| run: uv run pytest tests/integration/test_migrations.py -v |
There was a problem hiding this comment.
This job runs tests/integration/test_migrations.py, but every test in that file is decorated with @pytest.mark.skip. Even if continue-on-error is removed, CI will still report a successful smoke test when Alembic wiring or schema migrations are broken, because pytest only records skipped tests and never exercises the migration path.
Useful? React with 👍 / 👎.
| conn = await asyncpg.connect( | ||
| host="localhost", | ||
| port=5432, | ||
| user="postgres", | ||
| password="postgres", |
There was a problem hiding this comment.
diagnose health
check_postgres_health accepts a database_url but ignores it and always dials localhost:5432 as postgres/postgres/stackperf. Any local stack that uses different credentials, port, or DB name—including the test:test@.../stackperf_test DSN wired into the new CI job—will be reported as unhealthy even when PostgreSQL is up, so the diagnostics command points operators at the wrong problem.
Useful? React with 👍 / 👎.
| re.compile(r"(?:A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}"), | ||
| ), | ||
| # Generic secret: long alphanumeric strings that look like keys | ||
| ("generic_secret", re.compile(r"\b[a-zA-Z0-9]{32,}\b")), |
There was a problem hiding this comment.
This pattern matches any 32+ character alphanumeric token, which includes every 40-character git commit SHA. Session metadata is required to preserve the exact commit for reproducibility, so if exports or logs are passed through this helper the recorded revision turns into [REDACTED], making benchmark runs impossible to trace back to the code that produced them.
Useful? React with 👍 / 👎.
1 participant
Implements COE-230: Security, Operations, and Delivery Quality.
Summary
stackperf diagnose health,stackperf diagnose session,stackperf diagnose env)Acceptance Criteria
ContentCapturePolicy.DISABLEDby defaultRetentionPolicyclass with defaults.github/workflows/ci.ymlMakefilewithcheck,test,linttargetsdiagnose healthcommandHealthCheckResult.actionfieldTest Coverage
Closes COE-230