fix(auth): resolve service-principal created_by via shared identity store

AGENTS.md

@@ -320,25 +320,30 @@ Auth model facts (the canonical statement of "which tokens are accepted" lives i
     `oid` → contributor via `entra_identities`. *Unchanged.*
   - **Service (app / managed-identity)** — `scp` absent: authorized by an Entra
     **App Role** alone — `Contributor` (write+read) or `Reader` (read-only:
-    `POST /cypher`, `GET /blobs/*`). `created_by` = `service_identities[oid]` if
-    mapped, else the stable `appid`. App-only / MI tokens (which carry `roles`,
-    not `scp`) **are now accepted** on this path.
+    `POST /cypher`, `GET /blobs/*`). `created_by` = the contributor id mapped
+    for `oid` in the **shared identity store** (same store `entra_identities`
+    uses). A service
+    `oid` with no mapping is **403** (fail-loud, names the principal) —
+    `created_by` is **never** `appid`/`azp`/`oid`/display name. App-only / MI
+    tokens (which carry `roles`, not `scp`) **are now accepted** on this path.
   - `scp` + `idtyp=="app"` → 401 (ambiguous, fail-closed).
 - The matched `created_by` surfaces on graph nodes — same provenance path as static
   mode.
 - Config fields: required for boot — `azure_client_id`, `azure_tenant_id`,
-  `entra_identities`. Optional service-path — `service_identities` (friendly
-  `created_by` map, **not** an auth gate, **no runtime CRUD** — config + redeploy),
+  `entra_identities`. Optional service-path — `service_identities` (a
+  **first-boot-only seed** into the same shared identity store; may be empty —
+  service oids are onboarded/removed at runtime via `/admin/identities`,
+  no redeploy needed),
   `service_data_role` (default `Contributor`), `reader_role` (default `Reader`).
   Env prefix `AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_`.
 - **Fail-closed:** misconfig (missing field / empty or malformed `entra_identities`)
   is a HARD startup error. `allow_unauthenticated` defaults to `false`; a server
   with no auth configured refuses to start.
 - **401** = bad/expired/wrong-audience/missing/ambiguous token; **403** = valid
-  token lacking authorization — a user whose `oid` is unmapped, or a service token
-  with no qualifying App Role (the 403 body names the principal + required roles).
-  **Behavior change (M2):** a token with no `scp` and no qualifying role now returns
-  **403** (was **401**).
+  token lacking authorization — a user or service principal whose `oid` is
+  unmapped, or a service token with no qualifying App Role (the 403 body names
+  the principal + reason/required roles). **Behavior change (M2):** a token
+  with no `scp` and no qualifying role now returns **403** (was **401**).
 
 > 🔒 **Secret hygiene — NO real identifiers in this product repo.** An `oid` is a
 > persistent personal identifier (PII). Never commit real oids, client IDs, or
@@ -357,6 +362,10 @@ Both auth modes can add/remove identities **at runtime, no restart**, via the
   resolver holds the dict **by reference**, so a `/admin` `PUT`/`DELETE` is visible
   on the next request. **No cache, no TTL** — safe because the pilot runs a
   **single replica** (the in-process map is the source of truth).
+- `entra_identities_store_path` backs **both** user and service identities —
+  one shared store, disjoint oid space. There is deliberately no
+  separate `/admin/services` endpoint; service identities are managed through
+  `PUT`/`DELETE`/`GET /admin/identities`.
 - `IdentityStore` (`identity_store.py`) commits **write-file-then-swap-memory**
   (atomic file replace first, then memory) and **fails closed** on a corrupt file
   (empty map + loud log, never a crash-loop).

README.md

@@ -265,16 +265,18 @@ resolver:
     token's `oid` maps to a contributor via `entra_identities`. *Unchanged.*
   - **Service (app / managed-identity)** — `scp` absent: authorized by an Entra
     **App Role** alone — `Contributor` (write + read) or `Reader` (read-only:
-    `POST /cypher`, `GET /blobs/*`). `created_by` is `service_identities[oid]` if
-    mapped, else the stable `appid`.
+    `POST /cypher`, `GET /blobs/*`). `created_by` is the contributor id mapped
+    for `oid` in the same shared identity store that `entra_identities` uses.
+    An unmapped `oid` is **403**, never falling back to `appid`/`azp`/`oid`/display name.
 
   See [docs/entra-auth-setup.md](docs/entra-auth-setup.md) for the canonical model.
 
 The matched contributor id is stamped onto the graph as the write-once `created_by`
 provenance field. A missing/invalid credential is a **401**; a valid credential
 that lacks the needed binding or role is a **403** — a delegated user whose `oid`
-is unmapped, or a service token with no qualifying App Role (its 403 body names the
-`appid`/`oid` and the required roles). **Behavior change (M2):** a token with no
+is unmapped, a service token with no qualifying App Role, or a service token whose
+`oid` is not in the service identity map (each 403 body names the rejected
+principal and the reason). **Behavior change (M2):** a token with no
 `scp` and no qualifying role now returns **403** (previously **401**). The server is
 headless, so there is a single fixed exempt set that never requires a token:
 `{/status, /version, /docs, /openapi.json}` — health/version plus the always-on
@@ -374,7 +376,7 @@ Values are resolved with this priority (highest first):
 | `AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_AZURE_CLIENT_ID` | `azure_client_id` | *(empty)* | App Registration (client) GUID. **Required when `auth_mode=entra`** (startup refuses otherwise). See [docs/entra-auth-setup.md](docs/entra-auth-setup.md). |
 | `AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_AZURE_TENANT_ID` | `azure_tenant_id` | *(empty)* | Azure AD tenant GUID. **Required when `auth_mode=entra`**. See [docs/entra-auth-setup.md](docs/entra-auth-setup.md). |
 | `AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_ENTRA_IDENTITIES` (JSON) | `entra_identities` | *(empty)* | Identity map `oid -> {id: <contributor>}` for the **user (delegated)** path (oids are Azure Object IDs — **PII**, never commit real values). **Required (non-empty) when `auth_mode=entra`**; the matched `id` is recorded as `created_by`. See [docs/entra-auth-setup.md](docs/entra-auth-setup.md). |
-| `AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_SERVICE_IDENTITIES` (JSON) | `service_identities` | *(empty)* | **Entra service path — optional.** `oid -> {id: <contributor>}` map giving a **friendly `created_by`** name to a service principal / managed identity. **Not an auth gate** (App Roles authorize; see below) and **never required** for boot. Unmapped services still authorize, with `created_by` = `appid`. No runtime CRUD — edit config and redeploy. See [docs/entra-auth-setup.md](docs/entra-auth-setup.md). |
+| `AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_SERVICE_IDENTITIES` (JSON) | `service_identities` | *(empty)* | **Entra service path — optional, first-boot seed only.** `oid -> {id: <contributor>}` map seeding the **friendly `created_by`** identity for a service principal / managed identity into the same shared identity store `entra_identities` uses. **Not an auth gate** (App Roles authorize; see below) and **never required** for boot — may be empty. An unmapped service `oid` is now **403** (no `appid` fallback). Runtime add/remove is via `PUT`/`DELETE /admin/identities` — no redeploy needed. See [docs/entra-auth-setup.md](docs/entra-auth-setup.md). |
 | `AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_SERVICE_DATA_ROLE` | `service_data_role` | `Contributor` | **Entra service path.** App Role name whose presence in an app token's `roles` claim grants service **write + read**. `""`/`null` disables the service write path. See [docs/entra-auth-setup.md](docs/entra-auth-setup.md). |
 | `AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_READER_ROLE` | `reader_role` | `Reader` | **Entra service path.** App Role name granting service **read-only** access (`POST /cypher`, `GET /blobs/*`). `""`/`null` disables read-only app-token gating. See [docs/entra-auth-setup.md](docs/entra-auth-setup.md). |
 | `AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_ADMIN_API_KEY` | `admin_api_key` | *(empty — admin API disabled)* | **Static-mode admin credential** — separate from the data `api_keys`; it is the only key allowed to call the `/admin/*` identity-map endpoints. Sent as a bearer token; the middleware recognizes it before the data keystore lookup. Empty → admin API returns `503`; regular data keys get `403` on `/admin/*`. Cannot be deleted/shadowed via the API. See [docs/identity-management.md](docs/identity-management.md). |

context_intelligence_server/auth.py

@@ -84,18 +84,6 @@ def __init__(self, status_code: int, reason: str) -> None:
         self.reason = reason
 
 
-def _first_nonblank(*values: Any) -> str | None:
-    """Return the first value that is a non-empty, non-whitespace str, else None.
-
-    Used to chain service created_by candidates with truthiness semantics:
-    empty/whitespace/non-string candidates fall through (B6/B8).
-    """
-    for v in values:
-        if isinstance(v, str) and v.strip():
-            return v
-    return None
-
-
 def _resolve_token(token: str, keystore: dict[str, str]) -> str | None:
     """Return the contributor id for *token*, or ``None`` if not found.
 
@@ -478,29 +466,29 @@ def resolve(
                 f"in Azure Entra, then re-request a token.",
             )
 
-        # --- created_by derivation [B6/B8]: stable claims, truthiness chaining,
-        #     NEVER app_displayname (spoofable in Entra — B8), fail-loud.
-        #     Order: service_map[oid] > appid > azp > oid.
+        # created_by comes only from the service identity map; an unmapped oid
+        # is 403, never appid/azp/oid/display_name (spoofable, not a contributor id).
         _oid_raw = claims.get("oid")
         oid_str = _oid_raw if isinstance(_oid_raw, str) and _oid_raw.strip() else ""
         oid_lower = oid_str.lower()
         mapped = self._service_identity_map.get(oid_lower) if oid_lower else None
 
-        created_by = _first_nonblank(
-            mapped,  # 1. operator-assigned contributor id
-            claims.get("appid"),  # 2. app client id (v1.0 token)
-            claims.get("azp"),  # 3. authorized party (v2.0 token)
-            oid_str,  # 4. SP object id (always present; last resort)
-        )
-        if created_by is None:
-            # Unreachable in practice (oid always present); fail-loud, never null.
+        if mapped is None:
+            _appid_raw = claims.get("appid")
+            _principal = (
+                _appid_raw
+                if isinstance(_appid_raw, str) and _appid_raw.strip()
+                else (oid_str or "(unknown)")
+            )
             raise AuthError(
-                401,
-                "Service token has no resolvable identity claim "
-                "(service map miss and appid/azp/oid all blank)",
+                403,
+                f"Service principal {_principal!r} is not authorized: oid "
+                f"{oid_lower!r} is not in the service identity map; contact "
+                f"the server administrator to add this identity "
+                f"(tenant {self._tenant_id!r})",
             )
 
-        return (created_by, roles, True)
+        return (mapped, roles, True)
 
 
 class StaticKeyResolver:

context_intelligence_server/config.py

@@ -568,8 +568,9 @@ def build_identity_map(self) -> dict[str, str]:
     # M2 non-interactive auth: service / app-token identity path
     # -------------------------------------------------------------------------
     # service_identities: the OID → contributor map for service principals /
-    # managed identities.  Same shape as entra_identities; lives in config
-    # only (no durable store — service identities don't need runtime mutation).
+    # managed identities.  Same shape as entra_identities; first-boot-only seed
+    # into the SAME shared store entra_identities uses (oids are disjoint across
+    # users and service principals, so one oid -> contributor map serves both).
     #
     # Shape: { "<oid-GUID>": {"id": "<contributor>"} }
     #
@@ -587,12 +588,12 @@ def build_identity_map(self) -> dict[str, str]:
     def _validate_service_identities(
         cls, v: dict[str, dict[str, str]] | None
     ) -> dict[str, dict[str, str]] | None:
-        """Fail-closed: same GUID-map rules as entra_identities (shared helper).
+        """Same GUID-map rules as entra_identities (shared helper); empty map allowed.
 
-        Delegates to ``_validate_identity_map()``.  See that function's docstring
-        for the full rule set.
+        Delegates to ``_validate_identity_map()`` with ``allow_empty=True`` --
+        an empty map is a supported bootstrap state, not a startup error.
         """
-        return _validate_identity_map(v, "service_identities")
+        return _validate_identity_map(v, "service_identities", allow_empty=True)
 
     def build_service_identity_map(self) -> dict[str, str]:
         """Return ``{oid_lower -> contributor_id}`` for all configured service identities.

context_intelligence_server/main.py

@@ -641,13 +641,21 @@ def create_asgi_app(
         # Build and load the entra identity store.
         entra_store = IdentityStore(Path(s.entra_identities_store_path))
         entra_store.load()
+        _config_entra_map = s.build_identity_map()
+        _config_service_map = s.build_service_identity_map()
         if not entra_store.path.exists():
-            # First boot: seed in-process map from config.  Converts the flat
-            # {oid -> contributor_id} from build_identity_map() to the rich
-            # {oid -> {"id": contributor_id}} format that IdentityStore expects.
-            config_map = s.build_identity_map()
-            if config_map:
-                rich_seed = {oid: {"id": cid} for oid, cid in config_map.items()}
+            # First boot: seed in-process map from config -- ONE shared store
+            # serves both user and service oids (disjoint key spaces make sharing
+            # safe; the disjointness check below enforces that the two config
+            # maps never name the same oid).
+            rich_seed: dict[str, dict[str, str]] = {
+                oid: {"id": cid}
+                for oid, cid in (
+                    *_config_entra_map.items(),
+                    *_config_service_map.items(),
+                )
+            }
+            if rich_seed:
                 entra_store.seed(rich_seed)
         _entra_identity_store = entra_store
         app.state.entra_identity_store = entra_store
@@ -667,16 +675,35 @@ def create_asgi_app(
                 s.entra_identities_store_path,
             )
 
+        # service_identities is a FIRST-BOOT seed only: once the store file
+        # exists, config is never re-read into it. An operator who sets
+        # SERVICE_IDENTITIES on an already-deployed server would otherwise get
+        # no signal at all — the only symptom is a 403 on the service token,
+        # whose message points at the administrator, not at the ignored config.
+        # Name the ignored oids here so the wrong lever is obvious at boot.
+        _ignored_service_oids = sorted(
+            oid for oid in _config_service_map if oid not in entra_store.flat_dict
+        )
+        if _ignored_service_oids:
+            logger.warning(
+                "service_identities config lists %d oid(s) that are NOT in the "
+                "identity store and are being IGNORED: %r. service_identities "
+                "seeds the store on FIRST BOOT ONLY and store=%s already exists, "
+                "so config changes after the first boot have no effect and these "
+                "principals will receive 403. Add them at runtime with an "
+                "IdentityAdmin-role token via PUT /admin/identities/{oid} — no "
+                "redeploy required.",
+                len(_ignored_service_oids),
+                _ignored_service_oids,
+                s.entra_identities_store_path,
+            )
+
         # Boot disjointness invariant — each oid must belong to exactly one
-        # identity source.  Building the service map here (not inline in the
-        # EntraResolver call) lets us check the overlap BEFORE construction so
-        # the server fails loudly at startup rather than silently misbehaving.
-        # Existing logic already keeps app tokens off the human map at
-        # request time; this prevents a same-oid-in-both misconfiguration.
-        _service_id_map = s.build_service_identity_map()
-        _entra_oids = set(entra_store.flat_dict.keys())
-        _service_oids = set(_service_id_map.keys())
-        _overlap = _entra_oids & _service_oids
+        # identity source.  Checked against the two CONFIG maps, not the
+        # store's flat_dict: one shared store now holds user and service oids
+        # alike, so the store itself can no longer show the overlap.  This
+        # prevents a same-oid-in-both misconfiguration at startup.
+        _overlap = set(_config_entra_map) & set(_config_service_map)
         if _overlap:
             raise RuntimeError(
                 f"Boot invariant violated: oid(s) {sorted(_overlap)!r} appear "
@@ -687,13 +714,15 @@ def create_asgi_app(
 
         # EntraResolver raises RuntimeError at construction if the JWKS
         # prefetch fails (eager fail-closed by design).
-        # Pass entra_store.flat_dict (the LIVE dict) so the resolver sees
-        # any put()/delete() made by /admin immediately, no restart required.
+        # Pass entra_store.flat_dict (the LIVE dict) as BOTH maps so the
+        # resolver sees any put()/delete() made by /admin immediately on
+        # either path, no restart required.
         resolver: StaticKeyResolver | EntraResolver = EntraResolver(
             s.azure_client_id,  # type: ignore[arg-type]  — validated non-None by config
             s.azure_tenant_id,  # type: ignore[arg-type]  — validated non-None by config
             entra_store.flat_dict,  # live reference — mutations visible immediately
-            service_identity_map=_service_id_map,  # pre-built, disjointness verified
+            # SAME live reference (one shared store, disjoint oid keyspace).
+            service_identity_map=entra_store.flat_dict,
             service_data_role=s.service_data_role,
             reader_role=s.reader_role,
             entra_admin_role=s.entra_admin_role,

docs/architecture/06-auth-flow.dot

@@ -220,12 +220,12 @@ digraph auth_flow {
         ]
 
         ServiceCreatedBy [
-            label     = "created_by derivation  (STABLE CLAIMS ONLY \u2014 D7)\nTrusted map: service_identities[oid] \u2192 friendly name\n  (optional override \u2014 NOT a gate; no unmapped\u2192403 here)\nFallback (truthiness, NOT key-presence):\n  appid  \u2192  azp  \u2192  oid\nNEVER app_displayname  (caller-spoofable, often absent)\nFail-loud 403 if all stable claims absent  (B8)"
+            label     = "created_by derivation  (SHARED IDENTITY STORE)\noid \u2192 identity store lookup (SAME store entra_identities uses;\n  oids are disjoint across users and services)\nMapped \u2192 contributor id.  Unmapped \u2192 403  (mirrors user path)\nNEVER appid / azp / oid / app_displayname as created_by"
             fillcolor = "#BBDEFB"
         ]
 
         ServiceContrib [
-            label     = "contributor_id = stable service id (str \u2014 never null)\n  map name (\"ci-pipeline\") or Azure GUID (appid/oid)\nroles = token roles claim\n\u2192 return (contributor_id, roles)"
+            label     = "contributor_id = mapped contributor id (str \u2014 never null,\n  never appid/oid)\nroles = token roles claim\n\u2192 return (contributor_id, roles)"
             fillcolor = "#C8E6C9"
             fontname  = "Helvetica-Bold"
             penwidth  = 2.5
@@ -469,7 +469,7 @@ digraph auth_flow {
     RoleGate      -> ServiceCreatedBy      [label = "role present\n(Contributor\n| Reader\n| IdentityAdmin)"]
 
     // created_by derivation: stable claims only; fail-loud if all absent
-    ServiceCreatedBy -> AuthErrCatch       [label = "all stable claims\nabsent \u2192 403  (B8)", style = dashed, color = "#BF360C", fontcolor = "#BF360C"]
+    ServiceCreatedBy -> AuthErrCatch       [label = "oid unmapped\n\u2192 403  (mirrors user path)", style = dashed, color = "#BF360C", fontcolor = "#BF360C"]
     ServiceCreatedBy -> ServiceContrib     [label = "contributor_id\nresolved"]
 
     // Service success \u2014 contributor_id str (stable id) returns to middleware