Skip to content

fix(privacy): prevent student-data leakage through logs and observability - #39

Merged
llinsss merged 1 commit into
DogStark:mainfrom
pixels26:fix/26-log-data-leakage
Aug 22, 2026
Merged

fix(privacy): prevent student-data leakage through logs and observability#39
llinsss merged 1 commit into
DogStark:mainfrom
pixels26:fix/26-log-data-leakage

Conversation

@pixels26

Copy link
Copy Markdown
Contributor

Summary

Closes #26

Structured logs previously included raw student_id and attempted word values in messages and JSON fields. Because log sinks are shipped to third-party observability systems with retention/access rules that differ from the primary student store — and sit outside export_student_data, delete_student_data, and the documented retention sweep — they formed a shadow record of children's activity that application deletion could never purge.

This PR makes logs safe by design: identifiers are pseudonymized, learning content is reduced to bounded categorical fields, a redaction filter scrubs every emitted record as defense in depth, and the policy is documented and regression-tested.

Changes

agent/log_config.py (core)

  • Keyed pseudonymization (pseudonymize): truncated HMAC-SHA256 keyed by LOG_PSEUDONYM_KEY. The key is environment-specific and rotatable; rotating it permanently unlinks all historical correlation values. When unset, a random per-process key is used so values never survive restarts. Unsalted public hashes are never used.
  • RedactionFilter on every handler: scrubs formatted messages, plain message templates, pre-rendered exception text, Bearer/authorization headers, labeled fields (student_id=…, word=…, api_key=…, guardian_id=…), and identifier-bearing URL path segments (covers third-party access loggers such as uvicorn.access/httpx) → [REDACTED].
  • RFC 3339 UTC JSON timestamps with millisecond precision (e.g. 2026-08-21T20:11:45.935Z), validated by tests via datetime.fromisoformat + UTC-offset assertions.
  • Non-identifying JSON field allowlist with defensive transformation of legacy extras: stray student_id becomes a pseudonymous student_ref; stray word becomes word_length_bucket (short|medium|long).

main.py

  • Request-observability middleware: per-request correlation ID (context-var stamped onto every log line, returned as X-Request-ID), plus one bounded access-log line per request carrying method, route template (never raw paths/query strings), status code, latency_ms, and outcome category.
  • Unhandled exceptions are logged with redacted exception text; the profile storage-error handler no longer echoes raw exception detail (messages can embed student IDs).

api/routes.py

  • POST /profile: raw ID replaced with student_ref pseudonym + outcome.
  • POST /attempt: word/student removed from message and extras; now student_ref, word_length_bucket, success/failure outcome, response time metric.
  • POST /hint: word removed; now length bucket, attempt number, and provider_outcome (generated|fallback).

agent/hint_generator.py / agent/story_mode.py

  • Bedrock provider failures log only the exception type name plus enumerated outcomes (provider_unavailable|output_rejected|provider_error) and bounded counts — provider payloads, words, and story/hint text are never echoed.

PRIVACY.md

  • New “Logging and observability” section: data-classification table for identifiers / learning content / auth material / request bodies / provider errors / operational metadata; pseudonymization key handling and rotation; operator obligations for external log systems (explicit sink retention, access restriction, treating sinks as processors) with the explicit caveat that application deletion cannot erase independently exported or archived logs.

tests/test_log_privacy.py (new, 26 tests)

Capture-based regression tests attach to the real pipeline (both LOG_JSON=0 and LOG_JSON=1) and assert seeded API keys, student IDs, guardian IDs, attempted words, and generated story text never appear across:

  • success paths (create/attempt/hint/story/report/export/delete),
  • failure paths (401 invalid key, 403 foreign student, 404 missing, 409 duplicate),
  • Bedrock failure paths (provider exception; unsafe/malformed model output),
    plus pseudonymization properties (stability, rotation, environment separation, never an unsalted hash), timestamp validity, request-ID correlation between response header and logs, and route-template substitution for raw URLs.

One pre-existing test (TestBedrockExceptionLogging) asserted the old leaky behavior (word present in provider-error logs); it was flipped to enforce the new policy.

Acceptance criteria mapping

Criterion (issue #26) Where
No raw student/guardian ID, API key, auth header, word attempt, or story text in tested paths tests/test_log_privacy.py::TestApiPathsDoNotLeakStudentData
Correlation only via protected rotatable key or request IDs pseudonymize() + LOG_PSEUDONYM_KEY rotation tests; X-Request-ID middleware
JSON timestamps valid UTC RFC 3339 incl. sub-second TestJsonTimestamps
Log schemas and retention guidance documented PRIVACY.md “Logging and observability”
Regression tests fail on leakage in both plain-text and JSON modes Parametrized capture fixtures over both modes

Verification

  • ruff check . — clean
  • mypy — clean
  • pytest --cov=agent --cov=api --cov=dashboard --cov=main --cov-branch — 178 passed; coverage 84% (gate ≥80%)
  • uvicorn smoke start: access line renders as GET / -> 200 with correlation ID; startup sweep unaffected

Notes for reviewers

  • LOG_PSEUDONYM_KEY must be provisioned like a secret per deployment; unset it and correlation values become process-ephemeral (documented).
  • The redaction filter intentionally also covers third-party loggers routed through the root handler; operators should still verify their specific sink stack (documented obligation).

…lity

Closes DogStark#26.

Structured logs previously carried raw student_id and attempted word values
in messages and JSON fields, creating a shadow store of children's activity
outside export_student_data, delete_student_data, and the retention sweep.

- log_config: keyed HMAC-SHA256 pseudonymization (LOG_PSEUDONYM_KEY,
  environment-specific and rotatable; per-process random key fallback);
  never an unsalted public hash
- log_config: RedactionFilter scrubbing formatted messages, plain templates,
  exception text, bearer/authorization headers, labeled identifier/content
  fields, and URL path segments in both logging modes
- log_config: RFC 3339 UTC JSON timestamps and a non-identifying field
  allowlist that transforms legacy student_id/word extras defensively
- main: request/correlation-ID middleware emitting method, route template,
  status, latency, and outcome; X-Request-ID response header; storage-error
  handler no longer echoes raw exception detail
- routes/hint_generator/story_mode: logs carry only pseudonymous refs,
  length buckets, counts, error types, and enumerated outcomes
- PRIVACY.md: data-classification policy, pseudonymization key handling,
  and operator obligations for external log retention/access
- tests/test_log_privacy.py: capture-based regression tests asserting seeded
  keys/IDs/guardian IDs/words/story text never appear in either log mode,
  plus timestamp validity, correlation-ID propagation, and route-template
  redaction checks
@llinsss
llinsss merged commit e25bea1 into DogStark:main Aug 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prevent student-data leakage through logs and observability pipelines

2 participants