fix(design): lock priority-1 invariants#8
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds a foundational documentation set for Noesis Noema (architecture constitution, execution/invocation boundaries, routing determinism, error doctrine/handling, observability, session+memory lifecycle, security model, and evaluation), and updates UAT to explicitly validate determinism and observability invariants.
Changes:
- Added ADR-0000 and multiple new design specs (routing, invocation/execution flow, memory/session rules, observability, security, evaluation, error policies).
- Introduced/updated “authoritative entry point” docs (
.copilot-context.md,design/context-index.md) that define core schemas and governance pointers. - Expanded UAT checklist to include explicit determinism, trace propagation, session timeout, and log redaction checks.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 21 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/uat/UAT.md | Tightens UAT language around deterministic routing and adds a determinism/observability compliance section. |
| docs/adr/adr-0000-product-constitution.md | Adds the Human Sovereignty constitutional ADR governing subsequent design/implementation decisions. |
| .copilot-context.md | Adds an “authoritative context” doc including a normative NoemaQuestion JSON Schema. |
| design/context-index.md | Adds a design entry-point index and defines NoemaQuestion/NoemaResponse fields. |
| design/execution-flow.md | Specifies a deterministic execution flow and privacy enforcement checkpoints. |
| design/invocation-boundary.md | Defines invocation scope, lifecycle, and prohibited behaviors. |
| design/router-decision-matrix.md | Documents deterministic client-side routing rules and logging requirements. |
| design/error-doctrine.md | Defines error taxonomy, fail-fast policy, and structured error contract. |
| design/error-handling.md | Defines error classification, structured error responses, logging, and retry policy. |
| design/observability-standard.md | Defines required identifiers, event taxonomy, and production redaction policy. |
| design/security-model.md | Defines trust boundaries, threat model, and required controls/invariants. |
| design/memory-lifecycle.md | Defines session-scoped memory rules, 45-minute timeout, and forbidden persistence. |
| design/session-management.md | Defines session creation/validation/expiration rules and enforcement responsibilities. |
| design/evaluation-framework.md | Defines reproducible evaluation layers and deterministic test mode expectations. |
| design/mvp-consistency-checklist.md | Adds an MVP gate checklist for sovereignty/determinism/boundary/memory/error/observability alignment. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "required": [ | ||
| "id", | ||
| "timestamp", | ||
| "origin", | ||
| "intent", | ||
| "content", | ||
| "privacy_level" | ||
| ], |
There was a problem hiding this comment.
The normative NoemaQuestion schema here conflicts with design/context-index.md: this file makes intent required and omits session_id, while design/context-index.md lists session_id as required and intent as optional. Please reconcile these documents so there is a single consistent Question contract for implementation and validation.
| - Test: Observe system without user interaction, verify no hidden invocation | ||
|
|
||
| - [ ] **I have verified that structured error schema is respected** | ||
| - Test: Trigger errors and confirm JSON structure matches error-handling.md contract |
There was a problem hiding this comment.
This UAT step references an "error-handling.md contract", but the repo now has both design/error-handling.md and design/error-doctrine.md with different external error schemas. Please reference the single canonical spec (with correct path) so validation is checking the intended JSON contract.
| - Test: Trigger errors and confirm JSON structure matches error-handling.md contract | |
| - Test: Trigger errors and confirm JSON structure matches `design/error-doctrine.md` contract |
| ## 4.3 Input Validation | ||
|
|
||
| - All inputs must validate against NoemaQuestion schema | ||
| - Reject additionalProperties (no undeclared fields) |
There was a problem hiding this comment.
This input validation rule says to reject additionalProperties (no undeclared fields), but .copilot-context.md explicitly allows constraints.additionalProperties: true. Please clarify whether the restriction applies only to the top-level NoemaQuestion object, or tighten the schema so security and schema validation requirements are consistent.
| - Reject additionalProperties (no undeclared fields) | |
| - Reject additionalProperties (no undeclared fields) on the top-level NoemaQuestion object; nested constraint objects may explicitly opt in to additionalProperties via constraints.additionalProperties as defined in the schema |
| ### NoemaQuestion | ||
|
|
||
| The structured input object for all Invocations. | ||
|
|
||
| Required fields: | ||
| - `id` (UUID) — Question identifier | ||
| - `session_id` (UUID) — Associated session | ||
| - `content` (string) — User-provided prompt | ||
| - `privacy_level` (enum: "local" | "cloud" | "auto") — Privacy constraint | ||
| - `timestamp` (ISO-8601) — Submission time | ||
|
|
||
| Optional fields: | ||
| - `intent` (enum: "informational" | "analytical" | "retrieval") — Intent classification | ||
| - `constraints` (object) — Execution constraints | ||
|
|
||
| Schema must be validated before routing. | ||
|
|
||
| ### NoemaResponse | ||
|
|
||
| The structured output object for all successful Invocations. | ||
|
|
||
| Required fields: | ||
| - `id` (UUID) — Response identifier | ||
| - `question_id` (UUID) — Associated Question | ||
| - `session_id` (UUID) — Associated session | ||
| - `content` (string) — Generated response | ||
| - `model` (string) — Model used | ||
| - `route` (enum: "local" | "cloud") — Execution route | ||
| - `trace_id` (UUID) — Traceability identifier | ||
| - `timestamp` (ISO-8601) — Response generation time | ||
| - `fallback_used` (boolean) — Whether fallback occurred | ||
|
|
||
| Optional fields: | ||
| - `confidence` (float) — Model confidence (if available) | ||
| - `uncertainty_reason` (string) — Explanation if confidence is low | ||
|
|
||
| ## 7. Instruction to AI Agents | ||
| Before implementing any feature: | ||
| 1. Read this file. | ||
| 2. Read ADR-0000. | ||
| 3. Do not violate Constitution or contracts. No newline at end of file |
There was a problem hiding this comment.
The NoemaQuestion/NoemaResponse schemas introduced here appear to diverge from the repo’s existing locked API contract (docs/contracts/api-schema.md) which defines InvocationRequest/InvocationResponse and an ExecutionError shape. Please clarify the relationship (mapping/migration) and ensure only one canonical external contract is presented to implementers.
| ### NoemaQuestion | |
| The structured input object for all Invocations. | |
| Required fields: | |
| - `id` (UUID) — Question identifier | |
| - `session_id` (UUID) — Associated session | |
| - `content` (string) — User-provided prompt | |
| - `privacy_level` (enum: "local" | "cloud" | "auto") — Privacy constraint | |
| - `timestamp` (ISO-8601) — Submission time | |
| Optional fields: | |
| - `intent` (enum: "informational" | "analytical" | "retrieval") — Intent classification | |
| - `constraints` (object) — Execution constraints | |
| Schema must be validated before routing. | |
| ### NoemaResponse | |
| The structured output object for all successful Invocations. | |
| Required fields: | |
| - `id` (UUID) — Response identifier | |
| - `question_id` (UUID) — Associated Question | |
| - `session_id` (UUID) — Associated session | |
| - `content` (string) — Generated response | |
| - `model` (string) — Model used | |
| - `route` (enum: "local" | "cloud") — Execution route | |
| - `trace_id` (UUID) — Traceability identifier | |
| - `timestamp` (ISO-8601) — Response generation time | |
| - `fallback_used` (boolean) — Whether fallback occurred | |
| Optional fields: | |
| - `confidence` (float) — Model confidence (if available) | |
| - `uncertainty_reason` (string) — Explanation if confidence is low | |
| ## 7. Instruction to AI Agents | |
| Before implementing any feature: | |
| 1. Read this file. | |
| 2. Read ADR-0000. | |
| 3. Do not violate Constitution or contracts. | |
| The canonical external API contract for this system is defined in: | |
| - `docs/contracts/api-schema.md` | |
| That document specifies the locked wire-level schemas: | |
| - `InvocationRequest` — canonical request shape | |
| - `InvocationResponse` — canonical success response shape | |
| - `ExecutionError` — canonical error shape | |
| All external implementers **must** treat `docs/contracts/api-schema.md` as the single source of truth for: | |
| - Field names and types | |
| - Required vs. optional properties | |
| - Error envelopes and failure semantics | |
| This file uses the following conceptual names when discussing architecture: | |
| ### NoemaQuestion (conceptual) | |
| - Conceptual role: "The question being asked of the system." | |
| - **Maps to**: `InvocationRequest` as defined in `docs/contracts/api-schema.md`. | |
| - Usage: architectural and design discussions only; not a separate wire schema. | |
| ### NoemaResponse (conceptual) | |
| - Conceptual role: "The system's answer to a NoemaQuestion." | |
| - **Maps to**: `InvocationResponse` as defined in `docs/contracts/api-schema.md`. | |
| - Usage: architectural and design discussions only; not a separate wire schema. | |
| Errors, including partial failures, routing failures, or model execution problems, **must** be represented using the `ExecutionError` shape from `docs/contracts/api-schema.md`. No alternative external error envelope is permitted. | |
| If there is any discrepancy between terminology or examples in this file and `docs/contracts/api-schema.md`, the contract in `docs/contracts/api-schema.md` prevails. | |
| ## 7. Instruction to AI Agents | |
| Before implementing any feature: | |
| 1. Read this file. | |
| 2. Read ADR-0000. | |
| 3. Do not violate Constitution or contracts. | |
| 4. Treat `docs/contracts/api-schema.md` as the only canonical external API schema. |
| - docs/constitution/ | ||
| - ADR-0000 (Human Sovereignty Principle) |
There was a problem hiding this comment.
docs/constitution/ is referenced as the Product Constitution location, but that directory does not exist in the repo. Please update this reference to the actual constitution source (e.g., the ADR-0000 document) or add the missing directory so links are not broken.
| - docs/constitution/ | |
| - ADR-0000 (Human Sovereignty Principle) | |
| - ADR-0000 (Human Sovereignty Principle) — authoritative Product Constitution |
| - event_name | ||
| - timestamp (ISO 8601) | ||
| - trace_id | ||
| - session_id (if applicable) | ||
| - question_id (if applicable) |
There was a problem hiding this comment.
Requiring every event to log the full session_id embeds a secret session token into all structured logs, contradicting the requirement that session_id be treated as a secret. If logs are accessed by unauthorized parties or leaked, active session_id values can be replayed to take over sessions or correlate sensitive activity. Use trace_id or a non-replayable derivative of session_id for event correlation instead of logging the full token.
| - session_id | ||
| - route_type | ||
| - invocation_boundary_state | ||
| - model_name (if applicable) | ||
| - latency (if applicable) | ||
|
|
||
| Logs must allow full post-mortem reconstruction. |
There was a problem hiding this comment.
This logging policy requires every error record to include the full session_id, which is a secret identifier for the active session. If error logs are exposed (e.g., via log aggregation, support tooling, or breach), an attacker could reuse logged session_id values within their validity window to hijack sessions or inspect associated state. Avoid logging raw session_id in error logs; rely on trace_id or a non-replayable surrogate for correlating errors to sessions.
| - session_id | |
| - route_type | |
| - invocation_boundary_state | |
| - model_name (if applicable) | |
| - latency (if applicable) | |
| Logs must allow full post-mortem reconstruction. | |
| - session_fingerprint (non-replayable surrogate derived from session_id; never log raw session_id) | |
| - route_type | |
| - invocation_boundary_state | |
| - model_name (if applicable) | |
| - latency (if applicable) | |
| Logs must allow full post-mortem reconstruction without exposing replayable secrets such as session_id. |
| Every error must be logged with: | ||
|
|
||
| - error_code | ||
| - session_id | ||
| - question_id | ||
| - routing_decision | ||
| - model_used | ||
| - timestamp |
There was a problem hiding this comment.
Here, every error is required to be logged with the full session_id, even though session_id functions as a secret session identifier elsewhere in the design. Storing that token verbatim in logs means any unauthorized log access can yield active session_id values for replay or fine-grained tracking of user behavior. Prefer logging only trace_id or a derived, non-replayable representation of session_id so that logs remain useful for debugging without exposing bearer tokens.
| - session_id | ||
| - question_id | ||
| - route | ||
| - model | ||
| - timestamp | ||
|
|
There was a problem hiding this comment.
session_id is defined elsewhere as a secret bearer for session identity, but here each invocation is required to log the full session_id, which stores that secret in cleartext logs. Anyone with log access (or via a log breach) could replay an active session_id to hijack a session or reconstruct its interaction history. Instead, avoid logging the full session_id and rely on trace_id or a derived, non-replayable identifier (e.g., hashed/truncated form) for correlation.
| - session_id | |
| - question_id | |
| - route | |
| - model | |
| - timestamp | |
| - session_handle (non-replayable identifier derived from session_id, or trace_id) | |
| - question_id | |
| - route | |
| - model | |
| - timestamp | |
| The full session_id is a secret bearer and must never be logged in cleartext. | |
| Any derived session_handle used for logging must not be replayable and must not allow reconstruction of the original session_id. |
| All telemetry MUST include the following identifiers where applicable: | ||
|
|
||
| - trace_id (UUID) — per Invocation | ||
| - session_id (UUID) — per active session | ||
| - question_id (UUID) — per Question | ||
| - response_id (UUID) — per Response | ||
|
|
||
| Identifier rules: | ||
|
|
||
| - trace_id MUST be generated at Invocation entry | ||
| - trace_id MUST be propagated across boundaries (client ⇄ server) | ||
| - trace_id MUST appear in user-visible errors |
There was a problem hiding this comment.
This section mandates that all telemetry include the raw session_id, even though session_id is treated in the security model as a secret used to bind session state. Logging this bearer token across all telemetry surfaces means that any compromise or broad access to logs/metrics can yield reusable session_id values for session hijacking. Prefer limiting correlation to non-secret identifiers (trace_id) or to a derived, non-replayable surrogate of session_id rather than the full token.
| All telemetry MUST include the following identifiers where applicable: | |
| - trace_id (UUID) — per Invocation | |
| - session_id (UUID) — per active session | |
| - question_id (UUID) — per Question | |
| - response_id (UUID) — per Response | |
| Identifier rules: | |
| - trace_id MUST be generated at Invocation entry | |
| - trace_id MUST be propagated across boundaries (client ⇄ server) | |
| - trace_id MUST appear in user-visible errors | |
| All telemetry MUST include the following **non-secret** identifiers where applicable: | |
| - trace_id (UUID) — per Invocation | |
| - question_id (UUID) — per Question | |
| - response_id (UUID) — per Response | |
| - session_telemetry_id (string) — optional, per active session; a non-replayable surrogate derived from the secret session_id | |
| Identifier rules: | |
| - trace_id MUST be generated at Invocation entry | |
| - trace_id MUST be propagated across boundaries (client ⇄ server) | |
| - trace_id MUST appear in user-visible errors | |
| - session_id is a secret bearer token used to bind session state and MUST NOT be logged, stored in metrics, or emitted in traces | |
| - Any session_telemetry_id used in telemetry MUST be a non-replayable surrogate (for example, an HMAC or salted hash) derived from session_id that CANNOT be used to resume or hijack a session |
This pull request introduces foundational design documents that formalize the architecture, execution flow, error handling, evaluation, and memory lifecycle for the Noesis Noema system. The changes establish strict rules for determinism, human sovereignty, and compliance, ensuring that all AI operations are traceable, structured, and governed by explicit contracts. The most important changes are grouped below by theme.
Core Architecture and Schema
.copilot-context.mdanddesign/context-index.mdto define authoritative entry points, product constitution, and formal schemas forNoemaQuestionandNoemaResponse, including privacy and intent constraints. These documents enforce that all questions are structured objects and prohibit raw string prompts at invocation boundaries. [1] [2]Execution Flow and Invocation Boundary
design/execution-flow.mdanddesign/invocation-boundary.mdto specify a deterministic, traceable execution flow and strict invocation boundaries. These documents ensure every execution is triggered by explicit human action, bound to a single question, respects privacy, and prohibits hidden autonomy or side effects. [1] [2]Error Handling and Doctrine
design/error-doctrine.mdanddesign/error-handling.mdto define explicit error classification, structured error response formats, fail-fast policies, and observability requirements. The system must never hide uncertainty or silently recover, and all errors are logged and classified deterministically. [1] [2]Evaluation Framework
design/evaluation-framework.mdto establish a multi-layer evaluation process for schema compliance, routing determinism, execution integrity, and response quality. Evaluation is strictly reproducible and must not mutate system behavior or introduce autonomous optimization.Memory Lifecycle
design/memory-lifecycle.mdto define session-scoped memory rules, a fixed 45-minute timeout, and strict policies against persistent or cross-session memory. Both client and server enforce expiration, and forbidden behaviors include silent compression and cross-session carry-over.