Scan precision: runtime-scope filtering, iOS lockfile support, EOL flagging (BomLens parity #15-#17)

CHANGELOG.md

@@ -7,6 +7,45 @@ and the project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.
 
 ## [Unreleased]
 
+### Added
+- **EOL operations: weekly refresh beat + admin health panel.** A weekly
+  Celery beat re-stamps the component catalog against the newest
+  endoflife.date snapshot (so release upgrades reach existing rows without
+  a re-scan, and stamps are cleared when the whitelist shrinks) and — only
+  when `EOL_REFRESH_ENABLED=true`, off by default — fetches fresh lifecycle
+  data with a sanity floor that stops a gutted sweep from displacing a good
+  dataset. The admin/health page gains an endoflife.date snapshot panel
+  (dataset age with a 180-day stale warning, flagged totals, last tick,
+  next fire) at `GET /v1/admin/eol/health`.
+- **End-of-life (EOL) component flagging.** Components matching a curated
+  endoflife.date product whitelist (Spring Boot, Express, Django, Rails,
+  Angular, Vue, Next.js, Symfony, Laravel, Spring Framework) are stamped
+  with their lifecycle verdict on the shared catalog. The Components tab
+  gains an EOL column/badge and an "EOL only" filter (`?eol=true`), the
+  drawer an End-of-life row, and the project Overview an EOL count that
+  deep-links to the filtered list. Verdicts come from a snapshot vendored
+  with the release — zero network at scan time, air-gap safe
+  (`EOL_SNAPSHOT_PATH` mounts a fresher snapshot; `EOL_ENABLED=false`
+  disables).
+- **iOS CocoaPods/SPM lockfile scanning.** A `Podfile` used to crash the
+  whole source scan (cdxgen's cocoapods cataloger throws without the `pod`
+  CLI). The scanner now excludes that cataloger and reconstructs pods —
+  components AND dependency graph, subspecs included — offline from the
+  committed `Podfile.lock`. Repos with only a committed `Package.resolved`
+  now route to the swift environment, and the sidecar executors no longer
+  re-run `swift package resolve` over a committed lockfile.
+- **Runtime-scope SBOM filtering (default ON).** Source scans now drop
+  non-deployable dependencies from the cdxgen SBOM before persist, signing and
+  vulnerability matching: Maven `test`/`provided` nodes (cdxgen scope tags
+  `optional`/`excluded`) and npm `devDependencies` (lockfile-classified `dev`).
+  CVE counts and license obligations now describe the artifact that actually
+  ships. **Component and CVE counts drop on the first re-scan of affected
+  Maven/npm projects** — the scan summary records how many components were
+  excluded, and `SCAN_SCOPE_FILTER_ENABLED=false` (or the per-ecosystem
+  `SCAN_SCOPE_FILTER_MAVEN_ENABLED` / `SCAN_SCOPE_FILTER_NODE_ENABLED`)
+  restores the full graph. SBOMs uploaded via the ingest API are never
+  filtered — an uploaded SBOM is the supplier's declared truth.
+
 ## [0.13.1] — 2026-07-07
 
 A fixes-only patch release: repairs fresh role-separated (L1) installs and

apps/backend/alembic/versions/0038_component_version_eol.py

@@ -0,0 +1,112 @@
+"""component_versions end-of-life (EOL) columns + partial index — EOL DB layer
+
+Revision ID: 0038
+Revises: 0037
+Create Date: 2026-07-11
+
+Phase: EOL (endoflife.date end-of-life flagging — DB layer, Phase M)
+PR: feat/eol-catalog
+Kind: schema (additive — expand step; no data migration)
+Forward-only: yes
+
+What:
+  - Add six columns to the existing ``component_versions`` table::
+
+      eol_state        VARCHAR(16) NULL   -- 'eol' | 'supported' | 'unknown'
+      eol_product      VARCHAR(64) NULL   -- endoflife.date product slug
+      eol_cycle        VARCHAR(32) NULL   -- derived release cycle ('3.2')
+      eol_date         DATE NULL          -- published EOL date (when dated)
+      eol_source       VARCHAR(64) NULL   -- 'endoflife.date@YYYY-MM-DD'
+      eol_evaluated_at TIMESTAMPTZ NULL   -- last stamp time
+
+  - Add a PARTIAL index on the flagged rows::
+
+        CREATE INDEX ix_component_versions_eol
+            ON component_versions (eol_state)
+            WHERE eol_state = 'eol';
+
+Why:
+  - End-of-life is a fact about *product + release cycle* — identical for
+    every scan and project observing the same ``purl_with_version``. That is
+    the KEV shape (0034: ``kev`` lives on the shared ``vulnerabilities``
+    catalog, not per-scan ``vulnerability_findings``), so EOL lives on the
+    shared ``component_versions`` catalog, not on ``scan_components``.
+    Catalog storage keeps re-evaluation possible: a newer endoflife.date
+    snapshot re-stamps existing rows (tasks/eol_catalog_refresh, Phase M
+    PR-3) exactly like kev_catalog_refresh re-stamps ``vulnerabilities``.
+  - All columns NULLable — NULL means "never evaluated / not a tracked
+    product", mirroring the enrichment's closed-whitelist philosophy (an
+    unmapped component is left untouched, never guessed). ``eol_state`` is
+    VARCHAR, not a native ENUM (same reasoning as ``kev_sync_state``'s
+    ``last_result``: a closed vocabulary enforced at the application layer
+    keeps additive vocabulary changes migration-free).
+
+Why partial (not a plain b-tree on ``eol_state``):
+  - Tracked products are a curated ~10-rule whitelist of frameworks; EOL
+    rows are a tiny minority of the cross-project component catalog. The
+    ``?eol=true`` filter and the overview/health counts read only flagged
+    rows — ``WHERE eol_state = 'eol'`` indexes exactly that set (same
+    reasoning as 0034's ``WHERE kev``).
+
+Notes:
+  - **Expand step only** (CLAUDE.md §6). Nullable ADD COLUMNs are
+    metadata-only on PG 11+ (no rewrite); the partial index covers zero
+    rows at upgrade time. The data fill happens at persist time
+    (persist_sbom_components) and via the refresh task — kept out of this
+    schema revision per the schema/data separation policy.
+  - NOT created ``CONCURRENTLY`` — Alembic migrations run in a transaction
+    and the index covers zero qualifying rows at build time (0034 note).
+  - Forward-only per CLAUDE.md §6: ``downgrade()`` raises
+    ``NotImplementedError``.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Sequence
+
+import sqlalchemy as sa
+
+from alembic import op
+
+revision: str = "0038"
+down_revision: str | None = "0037"
+branch_labels: str | Sequence[str] | None = None
+depends_on: str | Sequence[str] | None = None
+
+
+def upgrade() -> None:
+    op.add_column(
+        "component_versions",
+        sa.Column("eol_state", sa.String(16), nullable=True),
+    )
+    op.add_column(
+        "component_versions",
+        sa.Column("eol_product", sa.String(64), nullable=True),
+    )
+    op.add_column(
+        "component_versions",
+        sa.Column("eol_cycle", sa.String(32), nullable=True),
+    )
+    op.add_column(
+        "component_versions",
+        sa.Column("eol_date", sa.Date(), nullable=True),
+    )
+    op.add_column(
+        "component_versions",
+        sa.Column("eol_source", sa.String(64), nullable=True),
+    )
+    op.add_column(
+        "component_versions",
+        sa.Column("eol_evaluated_at", sa.DateTime(timezone=True), nullable=True),
+    )
+    op.create_index(
+        "ix_component_versions_eol",
+        "component_versions",
+        ["eol_state"],
+        unique=False,
+        postgresql_where=sa.text("eol_state = 'eol'"),
+    )
+
+
+def downgrade() -> None:
+    raise NotImplementedError("forward-only migration (CLAUDE.md §6)")

apps/backend/alembic/versions/0039_eol_sync_state.py

@@ -0,0 +1,94 @@
+"""eol_sync_state — single-row endoflife.date refresh status table
+
+Revision ID: 0039
+Revises: 0038
+Create Date: 2026-07-11
+
+Phase: EOL (ops closeout — refresh beat + admin/health EOL panel, Phase M)
+PR: feat/eol-ops
+Kind: schema (new empty table; no data migration)
+Forward-only: yes
+
+What:
+  - Create table ``eol_sync_state``::
+        id              BOOLEAN PK DEFAULT true   -- with CHECK (id): single-row
+        last_synced_at  TIMESTAMPTZ               -- last SUCCESSFUL feed fetch
+        last_result     VARCHAR(16)               -- synced|skipped (fetch half)
+        skipped_reason  VARCHAR(64)               -- disabled|refresh_disabled|
+                                                  -- feed_unavailable|
+                                                  -- feed_below_sanity_floor|
+                                                  -- unexpected:<ExceptionName>
+        snapshot        JSONB                     -- fetched compact dataset (few KB)
+        snapshot_date   DATE                      -- snapshot["_snapshot"], denormalised
+        products_ok     INTEGER                   -- products fetched this sync
+        products_failed INTEGER                   -- products that failed this sync
+        stamped         INTEGER                   -- catalog rows (re)stamped this tick
+        cleared         INTEGER                   -- stale stamps cleared this tick
+        duration_ms     INTEGER
+        updated_at      TIMESTAMPTZ NOT NULL DEFAULT now()  -- last attempt
+  - CHECK constraint ``ck_eol_sync_state_singleton`` — same single-row
+    construction as 0035 (``kev_sync_state``).
+
+Why:
+  - Clone of the 0035 pattern with one structural addition: the fetched
+    dataset itself rides the row (``snapshot`` JSONB). The KEV reconcile
+    writes straight into ``vulnerabilities`` and needs no dataset store; the
+    EOL weekly re-stamp pass must prefer the freshest dataset — fetched OR
+    vendored — without a network call, so the fetched one needs a durable
+    home. A few KB of JSONB on a single-row table is the cheapest one.
+  - ``snapshot_date`` is denormalised out of the JSONB so the health panel's
+    staleness computation (warn tone past 180 days) is a plain DATE read.
+  - ``stamped`` / ``cleared`` update on EVERY tick — the re-stamp half runs
+    even when the fetch is disabled (pure-local), unlike the KEV counters.
+
+Notes:
+  - Vocabularies stay VARCHAR (task-owned closed sets — 0035 reasoning).
+  - No extra indexes: single-row table, PK is the only access path.
+  - No seed row: first tick inserts; "row absent" = "never ran".
+  - Forward-only per CLAUDE.md §6: ``downgrade()`` raises
+    ``NotImplementedError``.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Sequence
+
+import sqlalchemy as sa
+from sqlalchemy.dialects.postgresql import JSONB
+
+from alembic import op
+
+revision: str = "0039"
+down_revision: str | None = "0038"
+branch_labels: str | Sequence[str] | None = None
+depends_on: str | Sequence[str] | None = None
+
+NOW = sa.text("now()")
+
+
+def upgrade() -> None:
+    op.create_table(
+        "eol_sync_state",
+        sa.Column("id", sa.Boolean(), primary_key=True, server_default=sa.text("true")),
+        sa.Column("last_synced_at", sa.DateTime(timezone=True), nullable=True),
+        sa.Column("last_result", sa.String(length=16), nullable=True),
+        sa.Column("skipped_reason", sa.String(length=64), nullable=True),
+        sa.Column("snapshot", JSONB(), nullable=True),
+        sa.Column("snapshot_date", sa.Date(), nullable=True),
+        sa.Column("products_ok", sa.Integer(), nullable=True),
+        sa.Column("products_failed", sa.Integer(), nullable=True),
+        sa.Column("stamped", sa.Integer(), nullable=True),
+        sa.Column("cleared", sa.Integer(), nullable=True),
+        sa.Column("duration_ms", sa.Integer(), nullable=True),
+        sa.Column(
+            "updated_at",
+            sa.DateTime(timezone=True),
+            nullable=False,
+            server_default=NOW,
+        ),
+        sa.CheckConstraint("id", name="ck_eol_sync_state_singleton"),
+    )
+
+
+def downgrade() -> None:
+    raise NotImplementedError("forward-only migration (CLAUDE.md §6)")

apps/backend/api/v1/admin/__init__.py

@@ -15,6 +15,7 @@
   - ``backup`` — ``/v1/admin/backup/*``  (chore PR #19: list/trigger/download/restore/delete)
   - ``trivy``  — ``/v1/admin/trivy/*``   (W6-#43e: vulnerability DB status panel)
   - ``kev``    — ``/v1/admin/kev/*``     (Phase C: CISA KEV feed sync status panel)
+  - ``eol``    — ``/v1/admin/eol/*``     (Phase M: endoflife.date snapshot status panel)
 
 W6-#43a (ADR-0001): the ``dt`` sub-router was removed when DT was replaced
 by Trivy (W6-#41); previously-issued DT audit-log rows are preserved as
@@ -30,7 +31,7 @@
 
 from core.security import require_super_admin_or_404
 
-from . import audit, backup, disk, health, kev, scans, teams, trivy, users
+from . import audit, backup, disk, eol, health, kev, scans, teams, trivy, users
 
 # Apply the super-admin gate at the parent-router level so individual route
 # signatures stay clean — each route still gets the resolved CurrentUser
@@ -51,6 +52,7 @@
 router.include_router(backup.router)
 router.include_router(trivy.router)
 router.include_router(kev.router)
+router.include_router(eol.router)
 
 
 __all__ = ["router"]

apps/backend/api/v1/admin/eol.py

@@ -0,0 +1,48 @@
+"""
+Admin EOL snapshot-health HTTP route — Phase M (PR M-3).
+
+Endpoint: ``GET /v1/admin/eol/health`` — exposes the endoflife.date dataset
+state (effective snapshot date + origin, rule/product counts, live flagged
+total, last beat tick outcome, next fire) for the admin/health EOL panel.
+Auth gated by the parent admin router (super-admin only; existence-hide for
+everyone else — the ``kev`` sub-router convention).
+
+Pure read: one PK lookup on the single-row ``eol_sync_state`` table, one
+partial-index count, plus the in-process vendored-snapshot parse. The writer
+is the weekly ``tasks/eol_catalog_refresh`` beat.
+"""
+
+from __future__ import annotations
+
+import structlog
+from fastapi import APIRouter, Depends, Request, Response, status
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from core.db import get_db
+from core.security import CurrentUser, require_super_admin_or_404
+from schemas.admin_ops import EolStatusOut
+from services.eol_health_service import get_eol_health
+
+router = APIRouter(prefix="/eol", tags=["admin"])
+log = structlog.get_logger("admin.eol.api")
+
+
+@router.get(
+    "/health",
+    response_model=EolStatusOut,
+    summary="endoflife.date snapshot status (admin) — dataset age / counters / next beat",
+)
+async def get_eol_health_endpoint(
+    request: Request,  # noqa: ARG001
+    session: AsyncSession = Depends(get_db),
+    actor: CurrentUser = Depends(require_super_admin_or_404()),  # noqa: ARG001
+) -> Response:
+    out = await get_eol_health(session)
+    return Response(
+        content=out.model_dump_json(),
+        status_code=status.HTTP_200_OK,
+        media_type="application/json",
+    )
+
+
+__all__ = ["router"]

apps/backend/api/v1/components.py

@@ -80,6 +80,14 @@ async def get_component_detail_endpoint(
         # this since W2 #31 but the hand-built response omitted it, so the
         # wire payload was always ``null``.
         dependency_scope=payload["dependency_scope"],
+        # Phase M — same hand-built-response drop class as the line above
+        # (caught by the components_eol e2e: DB and list carried the verdict,
+        # the drawer read null).
+        eol_state=payload["eol_state"],
+        eol_product=payload["eol_product"],
+        eol_cycle=payload["eol_cycle"],
+        eol_date=payload["eol_date"],
+        eol_source=payload["eol_source"],
         created_at=payload["created_at"],
         updated_at=payload["updated_at"],
     )

apps/backend/api/v1/projects.py

@@ -450,6 +450,10 @@ async def get_project_overview_endpoint(
         project_id=payload["project_id"],
         project_name=payload["project_name"],
         total_components=payload["total_components"],
+        # Phase M — hand-built-response completeness (see
+        # tests/unit/api/test_handbuilt_response_completeness.py: this
+        # construction has silently dropped fields before).
+        eol_count=payload["eol_count"],
         severity_distribution=payload["severity_distribution"],
         license_distribution=payload["license_distribution"],
         risk_score=payload["risk_score"],
@@ -506,6 +510,16 @@ async def list_project_components_endpoint(
             "values returns an empty page (not a 422). Omit to include all."
         ),
     ),
+    eol: bool | None = Query(
+        default=None,
+        description=(
+            "Phase M — end-of-life facet. ``true`` keeps only components "
+            "whose release cycle is past its published end-of-life "
+            "(endoflife.date); ``false`` keeps everything else, including "
+            "untracked components. Omit to include both. Boolean mirrors "
+            "the KEV filter UX."
+        ),
+    ),
     sort: str = Query(default="name", pattern=r"^(name|severity|license)$"),
     order: str = Query(default="asc", pattern=r"^(asc|desc)$"),
     scan_id: uuid.UUID | None = Query(
@@ -532,6 +546,7 @@ async def list_project_components_endpoint(
             license_category=license_category,
             direct=direct,
             dependency_scope=dependency_scope,
+            eol=eol,
             sort=sort,
             order=order,
             scan_id=scan_id,