Storage pool management, a beta-test fix pass, and the published docs site

.env.example

@@ -31,6 +31,12 @@ GSHARE_SESSION_URL_SCHEME=https
 # so without this the console cannot reach sessions. Leave unset to disable relaying.
 # The host must match the operator's SESSION_DOMAIN / GSHARE_SESSION_DOMAIN.
 # GSHARE_SESSION_INGRESS=10.0.0.10:30080
+# How many proxies append to X-Forwarded-For before a request reaches the API. The client
+# address recorded in login audit rows and used for the per-IP login rate limit is read that
+# many entries from the right of the header (a client can prepend values, not remove them).
+# 1 = the Compose frontend (nginx) or ingress-nginx alone; 2 = a load balancer that forwards
+# the header in front of it.
+GSHARE_TRUSTED_PROXY_HOPS=1
 
 # ── Billing knobs ──
 GSHARE_CONNECTION_TOKEN_TTL_SEC=300

.github/workflows/docs.yml

@@ -0,0 +1,81 @@
+# Publishes the documentation site to GitHub Pages.
+#
+# The built site is never committed: this workflow builds it from `docs/` and `website/` and hands
+# the result straight to Pages, so there is no gh-pages branch to keep in sync.
+#
+# Before the first run, an administrator has to set Settings → Pages → Source to "GitHub Actions"
+# on this repository. The site is then served at https://boanlab.github.io/gshare/ as a project
+# site, which is per-repository and does not occupy the organization's single boanlab.github.io.
+name: Docs site
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - 'docs/**'
+      - 'website/**'
+      - '.github/workflows/docs.yml'
+  # Lets an administrator republish without a code change, and lets a fork build a preview.
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+# One deployment at a time; a newer push waits rather than racing the one in flight.
+concurrency:
+  group: pages
+  cancel-in-progress: false
+
+env:
+  SITE_URL: https://boanlab.github.io
+  SITE_BASE_URL: /gshare/
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: website
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: '20'
+          cache: npm
+          cache-dependency-path: website/package-lock.json
+
+      - name: Install
+        run: npm ci
+
+      # A fork sits at its own owner and repository name, so it builds against its own Pages URL.
+      # Nothing here needs editing to preview a fork.
+      - name: Resolve the site address
+        run: |
+          if [ "$GITHUB_REPOSITORY" != "boanlab/gshare" ]; then
+            owner="${GITHUB_REPOSITORY%%/*}"
+            name="${GITHUB_REPOSITORY##*/}"
+            echo "SITE_URL=https://${owner}.github.io" >> "$GITHUB_ENV"
+            echo "SITE_BASE_URL=/${name}/" >> "$GITHUB_ENV"
+          fi
+
+      - name: Build
+        run: npm run build
+
+      - uses: actions/configure-pages@v5
+
+      - uses: actions/upload-pages-artifact@v3
+        with:
+          path: website/build
+
+  deploy:
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - id: deployment
+        uses: actions/deploy-pages@v4

.gitignore

@@ -71,6 +71,9 @@ hack/storage-csi-key.pub
 test/e2e/ux/out/
 test/e2e/ux/out-*/
 
+# Beta-test findings and their run logs: the same kind of point-in-time artefact.
+beta-report/
+
 # ── Local assistant/agent tooling: workstation config, not project source. ──
 .claude/
 .agents/

CONTRIBUTING.md

@@ -11,15 +11,17 @@ vulnerabilities follow [SECURITY.md](./SECURITY.md) instead of the public issue
 - **Bugs and features** — open an issue first for anything beyond a small fix. It saves
   you from building something that conflicts with work already in flight.
 - **Scope** — GShare is interactive-session only. There is deliberately no API key, CLI,
-  SDK, or batch-job surface, and MIG partitioning is out of scope. Proposals that move
-  those boundaries are welcome, but argue the case in an issue first.
+  SDK, or batch-job surface, and a user-selectable MIG session mode is out of scope
+  (`mode=mig` is rejected); MIG exists only as an admin-operated per-card pool that
+  fractional requests may land on. Proposals that move those boundaries are welcome, but
+  argue the case in an issue first.
 
 ## Development environment
 
 | Component | Runtime | Directory |
 |---|---|---|
 | `gshare-api`, `gshare-worker` | Python 3.12 + FastAPI | `backend/` |
-| `gshare-operator` | Go 1.22 + controller-runtime | `operator/` |
+| `gshare-operator` | Go 1.25 + controller-runtime | `operator/` |
 | Console | Node 20 + Vite + React | `frontend/` |
 
 You need `docker`. For deployment work you also need `kubectl` and `helm` v3.

README.md

@@ -53,7 +53,7 @@ sitting on it.
 | Path | Contents |
 |---|---|
 | `backend/` | `gshare-api` (FastAPI control plane) and `gshare-worker` (billing, queue, and reaper batch). Python 3.12 |
-| `operator/` | `gshare-operator`, the per-cluster controller that reconciles `GShareSession` resources. Go 1.22 + controller-runtime |
+| `operator/` | `gshare-operator`, the per-cluster controller that reconciles `GShareSession` resources. Go 1.25 + controller-runtime |
 | `frontend/` | The console: a React + TypeScript single-page app with live updates over SSE |
 | `charts/gshare/` | The Helm chart for the whole platform |
 | `deploy/` | Values overlays (`values/`), security baselines, monitoring, supply-chain policy, secret examples |
@@ -91,7 +91,7 @@ The custom resource is the source of truth for desired session state. The API de
 | Namespace | Purpose | Pod Security Admission |
 |---|---|---|
 | `gshare-system` | Control plane: api, worker, operator, Postgres, Redis, ingress | `baseline` |
-| `gshare-sessions` | Tenant session pods | **`restricted` (enforce)** |
+| `gshare-sessions` | Tenant session pods | **`baseline` (enforce), `restricted` (audit/warn)** — session pods keep the `restricted` shape; only a policy-granted privileged session relaxes it |
 | `gshare-infra` | Privileged DaemonSets | `privileged` |
 
 ### Reference production configuration
@@ -125,7 +125,8 @@ memberships. The top bar has an admin-mode toggle that switches between the two
 
 **Credits** start at zero for a new user. They are allocated down the hierarchy —
 super-admin to organization to group to user — and no level can hand out more than it
-received. Monthly refills are supported and are use-it-or-lose-it. A session is always
+received. Monthly refills are supported: at the start of each month a wallet is topped up to its
+monthly grant (a balance already above the grant is left alone). A session is always
 billed to the requester's own personal wallet; charging someone else's wallet or a group
 wallet is rejected. **Credits are a GPU-only concept**: CPU-class sessions and storage volumes
 are free, and are governed by resource-policy quotas instead.

backend/README.md

@@ -33,7 +33,9 @@ app/
   domain/            scheduler, credit_engine, session/volume/budget services, connection_token
   cluster/           handoff, crd (GShareSession apply), status_sync — apply only, no workload calls
   db/                base (engine and session), models
-  workers/           runner plus billing, budget_rollup, credit_refill, token_expiry, queue_ticker
+  workers/           runner plus billing, budget_rollup, credit_refill, token_expiry, queue_ticker,
+                     grace_enforcer, audit_retention, webhook_dispatcher, pool_rebalancer,
+                     node_liveness, session_liveness
 alembic/             env.py and versions/
 tests/               pytest against in-memory SQLite; real-GPU e2e lives in /test/e2e
 ```
@@ -45,12 +47,16 @@ just the API by hand:
 
 ```bash
 pip install -e ".[dev]"
-cp ../.env.example .env                    # set GSHARE_BOOTSTRAP_ADMIN_* and INTERNAL_JWT_PRIVATE_KEY
+cp ../.env.example .env                    # set GSHARE_BOOTSTRAP_ADMIN_* and GSHARE_INTERNAL_JWT_PRIVATE_KEY
 docker compose up -d postgres redis        # datastores only
 alembic upgrade head                       # apply the schema
+alembic check                              # models vs schema: must report no new operations
 uvicorn app.main:app --reload --port 8080  # docs at http://localhost:8080/api/v1/docs
-python -m app.workers.runner               # billing, budget rollup, credit refill, token expiry, queue ticker
+python -m app.workers.runner               # billing, budget rollup, credit refill, token expiry, queue ticker, and the other loops in workers/runner.py
 ```
+`downgrade` walks back one revision at a time, but **not all the way to base**: `0051_drop_boards`
+refuses to run backwards, because the notice and inquiry tables were dropped with their data by
+decision. Restore from a backup instead of downgrading past that revision.
 
 Port 8080 matches what Compose and the frontend proxy expect.
 

backend/alembic/versions/0062_volume_placement.py

@@ -0,0 +1,39 @@
+"""Record where a volume's data lives.
+
+A volume's PVC is created lazily by whichever cluster's operator first runs a session that mounts
+it, and until now nothing wrote that down: the administrator's volume list could not say which
+storage server a volume sits on, and with several pools the answer was reconstructed from mount
+history. The operator already reports every PVC each sync tick; this adds the columns that report
+fills — the provisioning cluster, the StorageClass the claim names (the pool is the pair), and
+when it was first seen. Existing rows fill in on the next tick; volumes with no PVC yet stay NULL.
+
+Revision ID: 0062_volume_placement
+Revises: 0061_org_name_unique_live
+"""
+from __future__ import annotations
+
+import sqlalchemy as sa
+from alembic import op
+
+revision = "0062_volume_placement"
+down_revision = "0061_org_name_unique_live"
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    op.add_column("storage_volume", sa.Column("cluster_id", sa.String(), nullable=True))
+    op.add_column("storage_volume", sa.Column("storage_class", sa.String(), nullable=True))
+    op.add_column("storage_volume", sa.Column("provisioned_at", sa.DateTime(timezone=True), nullable=True))
+    op.create_foreign_key(
+        "fk_storage_volume_cluster", "storage_volume", "cluster", ["cluster_id"], ["id"], ondelete="SET NULL",
+    )
+    op.create_index("ix_storage_volume_cluster_id", "storage_volume", ["cluster_id"])
+
+
+def downgrade() -> None:
+    op.drop_index("ix_storage_volume_cluster_id", table_name="storage_volume")
+    op.drop_constraint("fk_storage_volume_cluster", "storage_volume", type_="foreignkey")
+    op.drop_column("storage_volume", "provisioned_at")
+    op.drop_column("storage_volume", "storage_class")
+    op.drop_column("storage_volume", "cluster_id")

backend/alembic/versions/0063_schema_alignment.py

@@ -0,0 +1,102 @@
+"""Align the live schema with the models so `alembic check` is clean.
+
+Revision ID: 0063_schema_alignment
+Revises: 0062_volume_placement
+
+Every item here is a name, a type or a nullability that the hand-written migrations left
+different from what the models declare; none changes what the tables hold:
+
+- two CHECK constraints carried the naming convention twice
+  (ck_credit_wallet_ck_credit_wallet_wallet_sigma, ck_gpu_device_ck_gpu_device_no_overcommit);
+- the project -> group rename kept the old index names (ix_*_project_id);
+- image_build.build_args was created as JSON, the model says JSONB;
+- five tables were created with nullable created_at/updated_at (TimestampMixin is NOT NULL).
+
+Postgres only: the SQLite test harness builds the schema from the models directly.
+"""
+from __future__ import annotations
+
+import sqlalchemy as sa
+
+from alembic import op
+
+revision = "0063_schema_alignment"
+down_revision = "0062_volume_placement"
+branch_labels = None
+depends_on = None
+
+_CHECKS = {
+    # table: (name on disk, name in the models)
+    "credit_wallet": ("ck_credit_wallet_ck_credit_wallet_wallet_sigma", "ck_credit_wallet_wallet_sigma"),
+    "gpu_device": ("ck_gpu_device_ck_gpu_device_no_overcommit", "ck_gpu_device_no_overcommit"),
+}
+_INDEXES = {
+    # old name: new name
+    "ix_project_org_id": "ix_group_org_id",
+    "ix_image_build_project_id": "ix_image_build_group_id",
+    "ix_membership_project_id": "ix_membership_group_id",
+    "ix_session_project_id": "ix_session_group_id",
+}
+_TIMESTAMPED = ("node_pool", "node_pool_grant", "resource_request", "session_event", "system_setting")
+
+
+def _is_postgres() -> bool:
+    return op.get_bind().dialect.name == "postgresql"
+
+
+def _rename_constraint(table: str, old: str, new: str) -> None:
+    # Guarded: a database whose schema came from a different path may already carry the new
+    # name, and the rename must not fail the whole upgrade over it.
+    op.execute(sa.text(f"""
+        DO $$
+        BEGIN
+            IF EXISTS (SELECT 1 FROM pg_constraint WHERE conname = '{old}')
+               AND NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conname = '{new}') THEN
+                EXECUTE 'ALTER TABLE "{table}" RENAME CONSTRAINT {old} TO {new}';
+            END IF;
+        END $$;
+    """))
+
+
+def _rename_index(old: str, new: str) -> None:
+    op.execute(sa.text(f"""
+        DO $$
+        BEGIN
+            IF EXISTS (SELECT 1 FROM pg_class WHERE relname = '{old}' AND relkind = 'i')
+               AND NOT EXISTS (SELECT 1 FROM pg_class WHERE relname = '{new}' AND relkind = 'i') THEN
+                EXECUTE 'ALTER INDEX {old} RENAME TO {new}';
+            END IF;
+        END $$;
+    """))
+
+
+def upgrade() -> None:
+    if not _is_postgres():
+        return
+    for table, (old, new) in _CHECKS.items():
+        _rename_constraint(table, old, new)
+    for old, new in _INDEXES.items():
+        _rename_index(old, new)
+    op.execute(sa.text(
+        "ALTER TABLE image_build ALTER COLUMN build_args TYPE JSONB USING build_args::jsonb"
+    ))
+    for table in _TIMESTAMPED:
+        for col in ("created_at", "updated_at"):
+            # Backfill first: a NULL timestamp on an old row would otherwise fail SET NOT NULL.
+            op.execute(sa.text(f'UPDATE "{table}" SET {col} = now() WHERE {col} IS NULL'))
+            op.alter_column(table, col, existing_type=sa.DateTime(timezone=True), nullable=False)
+
+
+def downgrade() -> None:
+    if not _is_postgres():
+        return
+    for table in _TIMESTAMPED:
+        for col in ("created_at", "updated_at"):
+            op.alter_column(table, col, existing_type=sa.DateTime(timezone=True), nullable=True)
+    op.execute(sa.text(
+        "ALTER TABLE image_build ALTER COLUMN build_args TYPE JSON USING build_args::json"
+    ))
+    for old, new in _INDEXES.items():
+        _rename_index(new, old)
+    for table, (old, new) in _CHECKS.items():
+        _rename_constraint(table, new, old)

backend/app/api/credits_router.py

@@ -643,6 +643,19 @@ async def allocate(
     }
 
 
+def _credit_grant_increase(db: AsyncSession, wallet: CreditWallet, new_amount: Decimal) -> None:
+    """Raise the balance to the new grant and put the difference on the ledger.
+
+    The immediate credit used to move ``balance`` with no CreditTransaction, so the transaction
+    history no longer summed to the wallet — the one invariant the ledger exists to keep."""
+    delta = new_amount - wallet.balance
+    wallet.balance = new_amount
+    db.add(_make_txn(
+        wallet, "adjust", delta, ref="monthly_grant",
+        key=f"grant:{wallet.id}:{ids.new('transaction')}",
+    ))
+
+
 async def _grant_scope(
     db: AsyncSession, principal: Principal, wallet: CreditWallet
 ) -> tuple[CreditWallet | None, list[str]]:
@@ -711,8 +724,8 @@ async def set_monthly_grant(
     """Set a child wallet's monthly automatic refill, as the administrator one level up.
 
     The siblings' grants must sum within the parent's grant; 0 disables refills for that wallet. The
-    refill itself — resetting balance to grant at the start of each month, use-it-or-lose-it — is
-    performed by the credit_refill worker."""
+    refill itself — topping the balance up to the grant at the start of each month, never taking
+    a surplus away — is performed by the credit_refill worker."""
     wallet = await _lock_wallet(db, wallet_id)
     parent, sibling_ids = await _grant_scope(db, principal, wallet)
     new_amount = body.amount
@@ -732,7 +745,7 @@ async def set_monthly_grant(
     # An increase is credited immediately so it is usable at once; a decrease takes effect at the
     # next monthly refill. The system wallet only holds the ceiling and needs no balance.
     if wallet.owner_type != "system" and new_amount > wallet.balance:
-        wallet.balance = new_amount
+        _credit_grant_increase(db, wallet, new_amount)
     wallet.version = wallet.version + 1
     await AuditService(db).record(
         actor=principal.user_id, action="credit.set_monthly_grant",
@@ -881,7 +894,7 @@ async def bulk_monthly_grant(
     for w in wallets:
         w.monthly_grant = body.amount
         if body.amount > w.balance:
-            w.balance = body.amount
+            _credit_grant_increase(db, w, body.amount)
         w.version = w.version + 1
     await AuditService(db).record(
         actor=principal.user_id, action="credit.bulk_monthly_grant", target=parent.id,