Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions deploy/community-account-role.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
title: Cosift community account access
description: Verify shared accounts and tokens, and record token activity. No account or token creation or deletion.
stage: GA
includedPermissions:
- datastore.databases.get
- datastore.databases.getMetadata
- datastore.entities.get
- datastore.entities.list
- datastore.entities.update
41 changes: 28 additions & 13 deletions docs/COMMUNITY-ROLLOUT.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,30 @@
# Community release: operator handoff

**Shared-account integration update:** see [SHARED-ACCOUNTS.md](SHARED-ACCOUNTS.md) and [its verification record](SHARED-ACCOUNTS-VALIDATION.md). Shared mode now connects to Andrei’s auth/MCP infrastructure. Its live staging and companion-change gates must pass before rollout; earlier standalone checks do not establish shared-mode production readiness.
**2026-09-17 rollout update:** [PR #58](https://github.com/pilot-protocol/cosift/pull/58)
merged at `c8ea845`, and the authorized v0.2.7 release completed successfully in
[workflow 35255963898](https://github.com/pilot-protocol/cosift/actions/runs/35255963898).
Andrei's auth/MCP companions are merged and serving healthy production Cloud Run
origins; the installer is public at v0.4.0/v1. The signed engine is healthy with
HNSW ready. Caddy switched the public service to the shared community gateway
at 18:23:08 UTC, with login at the public entry and operational routes blocked.
The community unit uses the shared binary and its backup timer is enabled. See
[SHARED-ACCOUNTS.md](SHARED-ACCOUNTS.md) and its
[verification record](SHARED-ACCOUNTS-VALIDATION.md) for confirmed revisions,
direct URLs, verified real email/shared-account and public request/MCP checks,
and the remaining acceptance scope.

Deploy only an explicitly approved commit after its review and release gates
pass. Reviewing or testing this PR does not itself authorize a release tag,
published assets, updater activation, or production changes. Use the controlled
sequence below for an authorized rollout.

## Current production baseline (2026-09-16)
## Retained rollback baseline (2026-09-16)

Production has been restored to the original v0.2.5 engine and original Caddy
routing. The community service and its backup timer are stopped/disabled. The
engine updater timer is also disabled so another release cannot roll out without
an explicit decision. Account data and rollback backups have been retained.
Before the September 17 rollout, production was restored to the original v0.2.5
engine and original Caddy routing. The community service/backup timer and engine
updater were stopped/disabled. This records the rollback baseline, not the state
of an in-progress or completed v0.2.7 cutover. Account data and rollback backups
were retained; record each subsequent service/routing change during rollout.

The engine binary and config were compared byte-for-byte with their original
backups. Engine PID was unchanged during the public-routing rollback. The public
Expand All @@ -21,14 +33,15 @@ not be selected as a release candidate.

## Review and validation

The PR contains the exact revert of the draft ranking changes from #54, the
The merged tree contains the exact revert of the draft ranking changes from #54, the
public authentication entry and operational-route restrictions from #57, and
mode-specific limits plus contribution quality screening. The community/CLI,
local artifacts and credit ledger implementation already merged through #55 is
part of the candidate's complete tree. Review the resulting tree against v0.2.5
as well as the PR diff; reverting #54 must receive the normal owner review.
part of the release's complete tree. The shared-account integration adds the
auth/MCP connections and CLI session handoff. Compare the resulting tree and
representative query behavior against v0.2.5 when accepting the release.

Run with Go 1.26 and `GOWORK=off` when a parent workspace uses an older Go version:
Run with Go 1.26.8 and `GOWORK=off` when a parent workspace uses an older Go version:

```sh
GOWORK=off go vet ./...
Expand Down Expand Up @@ -76,7 +89,7 @@ reuse the withdrawn v0.2.6 artifacts.
strategy, or deliberately update and version-check both binaries. Run
`systemctl daemon-reload` before starting the portal. Never leave it using an
old standalone binary while upgrading the engine.
6. Check `/api/limits`, registration/login/logout, interest persistence, saved
6. Check `/api/limits`, email-code login/logout, interest persistence, saved
requests, sample CSV, and CLI guest/member requests over loopback first.
Enable the reviewed Caddy routing only after these checks pass. Confirm the
root routes to signup/login, public operational/admin/debug routes are
Expand Down Expand Up @@ -150,5 +163,7 @@ content judgments. Local embeddings are checked against server computation,
so this first version does not promise server-compute savings. New content earns
10 credits, globally deduplicated by content hash. Stripe one-time credit purchases are implemented but disabled until the secret
API key and webhook signing secret are configured. See [Stripe activation and
test-mode checks](STRIPE.md). Email verification and self-service password reset
are not enabled in this version.
test-mode checks](STRIPE.md). Shared mode verifies email codes through
`cosift-auth`; standalone local mode still lacks email verification and
self-service password reset. Article authoring and rewards for article views
remain outside this release.
70 changes: 48 additions & 22 deletions docs/SHARED-ACCOUNTS-VALIDATION.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,28 @@
# Shared-account integration verification — 2026-09-17

All changes remain proposed in Cosift PR #58. No production code, service,
organization policy, DNS, updater, Stripe charge, or Andrei repository was changed.
Companion patches were exercised in local audit checkouts only.
Cosift PR #58 merged at `c8ea845913829d5eece571c6facbb4462ab313c9`. The
auth/MCP/installer companions have also merged and released; this record
separates the original local tests from the verified deployment checks below.
The v0.2.7 engine/CLI release finished and the existing-host public cutover
completed on 2026-09-17 at 18:23:08 UTC. Real email and authenticated service
checks are recorded below, followed by successful public Search/Answer/Research,
MCP search/quota and topic-sync acceptance. No Stripe charge is claimed.

## Checks completed

| Check | Result / scope |
|---|---|
| `GOWORK=off go test -race -timeout 10m ./...` | Full Cosift regression suite, including account linking, quota isolation, saved data, moderation and CLI regressions |
| `GOWORK=off go vet ./...` | Static checks |
| `CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build ./cmd/cosift` | Production-architecture compilation; no release artifact published |
| `CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build ./cmd/cosift` | Production-architecture compilation; signed release tracked separately below |
| `node --test internal/community/webtests/*.test.cjs` | 11 passing tests, including OTP state transitions and stale shared-topic responses |
| `GOWORK=off COSIFT_SMOKE_PORT=17983 bash scripts/smoke-test.sh` | Passed: real public crawl, ingestion, health, search, contents, admin authorization; disposable local index |
| Official auth token vectors | Canonical parsing/trailing bits and HMAC over full token agree with `cosift-auth` fixtures |
| Firestore SDK protocol fixture | Real Google SDK over local gRPC: collection-group lookup, typed verified/revoked timestamps, bans, last-used writes, ambiguous/missing token and IAM-denied errors |
| Auth lifecycle / HTTP contracts | Auth start/verify/revoke, HttpOnly cookies, Google-vs-user credential headers, forwarded client IP, bounded JSON, redirects and upstream error handling |
| Additional failure regressions | Cloud Run IAM failures versus Cosift credential rejection; OTP validation and cleanup after browser cancellation; account linking rollback; CLI credential conflicts and retryable logout; per-client login and per-account MCP limits |
| Installer → real CLI handoff | Installer passes its in-memory token to the compiled CLI, which verifies it and writes a 0600 origin-bound session; a separate CLI process reads credits with no token/server/session flags. Ten installer handoff cases and CLI race regressions pass; no harness credential extraction |
| Patched MCP suite | 248 passed, 2 skipped, 5 integration tests deselected; no cloud calls or model downloads |
| Installer → real CLI handoff | Installer passes its in-memory token to the compiled CLI, which verifies it and writes a 0600 origin-bound session; a separate CLI process reads credits with no token/server/session flags. All 11 installer handoff cases and CLI race regressions pass; 255 onboarding checks and ShellCheck pass |
| MCP release suite | 266 offline tests plus 18 final deployment tests passed, with Ruff/shell checks and a local AMD64 container embedding inference (384 dimensions, UID 65532); no production search claimed by these fixtures |
| MCP → community → engine | Real MCP ASGI app and engine client against local Go gateway: concurrent accounts receive separate free allowances; repeat over-quota call does not reach the engine; `k=20`, BM25 preserved; account credential stops at the gateway |
| Community client → MCP tools | Real FastMCP protocol: follow/list/unfollow, request/idempotent repeat, missing article coverage; local fake identity/topic store and topic-resolution fixture |
| Auth proxy configuration companion | `bash -n infra/deploy.sh`; existing client-IP resolver and configuration tests pass |
Expand All @@ -33,12 +37,11 @@ must return a retryable service error, rather than declaring the user's token
revoked or account banned. The regression checks preserve both upstream services'
actual JSON authentication contracts and reject HTML/Google origin failures.

The first integration commit's Snyk check failed; the Snyk report was gated by
login. An independent official `govulncheck` scan found affected gRPC v1.82.1
An independent official `govulncheck` scan found affected gRPC v1.82.1
call paths ([GO-2026-6348](https://pkg.go.dev/vuln/GO-2026-6348)) and Go 1.26.0
standard-library paths with later security fixes. The PR now requires Go 1.26.8
and gRPC v1.83.2; CI also runs pinned `govulncheck` v1.8.0. The independent scan
is not a substitute for a successful Snyk check on the final commit. The final
and gRPC v1.83.2; CI also runs pinned `govulncheck` v1.8.0. Merged auth uses the
same Go/gRPC versions and passed its package scan and CI. The final Cosift
package-level scan reports **0 vulnerabilities in imported packages**. It still
lists GO-2026-5932 for the unused OpenPGP package within the required x/crypto
module; the application does not import that package. CI gates imported packages,
Expand All @@ -63,20 +66,43 @@ The manual fixture is test-only and never included in the shipped binary. Its
URL is disposable. See `integrations/cosift-mcp/README.md` to reproduce the
cross-repository and browser checks.

## Not established by these tests
## Deployment checks completed

These results do not certify a live shared-account deployment. The available
Google CLI credentials required reauthentication, so this sweep did not exercise
live Firestore/Secret Manager IAM, an actual Cloud Run service identity, real mail
delivery, public DNS/ingress, or credential refresh over its real lifetime.
| Component | Evidence |
|---|---|
| Cosift engine/CLI | [PR #58](https://github.com/pilot-protocol/cosift/pull/58) merged at `c8ea845`; [v0.2.7 workflow](https://github.com/pilot-protocol/cosift/actions/runs/35255963898) completed successfully. The public release contains five platform binaries with checksums/signatures and `cosift-minisign.pub` |
| Auth | [PR #1](https://github.com/pilot-protocol/cosift-auth/pull/1) merged at `e6ce919`; revision `cosift-auth-00001-k6b` serves 100%, using production Firestore `(default)` and runtime identity `cosift-auth-runtime` |
| MCP | [PR #1](https://github.com/pilot-protocol/cosift-mcp/pull/1) merged at `fa60d32`; revision `cosift-mcp-00002-lf7` serves 100%, using `(default)` and `COSIFT_ENGINE_BASE_URL=https://cosift.pilotprotocol.network` |
| Public ingress | Auth `https://cosift-auth-udik5erlkq-uw.a.run.app/health` and MCP `https://cosift-mcp-udik5erlkq-uw.a.run.app/health` both returned JSON 200 without Google credentials. Production invoker checks are disabled with no `allUsers` grant; staging checks remain enabled. No organization-policy override was made |
| Credential boundaries | Public auth revoke without a credential returned Cosift JSON 401; public MCP calls with missing or fabricated canonical tokens returned application JSON 401, not infrastructure 503. Production debug routes returned 404; staging debug routes were disabled again after attribution checks |
| Proxy attribution | The actual gateway egress was measured. Private staging kept two different forwarded client addresses in separate buckets; a direct forged forwarding chain resolved to the actual caller. Production trusts only the measured gateway `/32` and Cloud Run peer `/32` |
| Real shared account | Production email-code login succeeded. Against the gateway on host loopback, anonymous `/api/me` returned 401; repeated authenticated identity was stable, credits returned the expected shape, gateway/direct-production-MCP topic lists matched, and authenticated BM25 `k=20` search returned 20 hits. This exercised live Firestore, Secret Manager and MCP with the same real account; no topic mutation was required |
| Signed engine upgrade | Official v0.2.7 Linux ARM64 artifact passed SHA-256/minisign verification against the host trust anchor. One graceful restart completed; HNSW finished loading and the existing engine returned healthy responses. The full-corpus process was not duplicated |
| Retrieval comparison | Both BM25 baseline queries preserved all six results and their order. Hybrid TLS preserved all six URLs, with ranks 3/4 swapped. Search returned 200 in 122–265 ms, Answer in 3.506 s, Research in 2.031 s. The existing weak Go evidence remained visible: Answer declined and Research noted missing specific evidence |
| Live moderation | Useful text was allowed, garbage rejected as low quality, and harmful promotion with embedded prompt injection rejected as phishing. Anonymous moderation returned 401. These moderation fixtures were not indexed |
| Public cutover | Caddy reloaded successfully at 18:23:08 UTC; public/origin `/api/auth/config` report shared mode, root opens login, and operational routes are blocked. The community unit uses the same v0.2.7 binary path as the engine with its old standalone override removed. Community backup upload succeeded and its timer is enabled |
| Public request/account checks | Shared auth, identity, credits and topics passed through the public gateway. BM25 `k=20` returned 20 hits in 0.458 s. Authenticated Answer returned 200 with five sources and five valid cited IDs in 6.650 s; Research returned 200 with three plan steps, eight sources and five valid cited IDs in 3.926 s |
| Public MCP and shared limits | Direct MCP search returned two filtered hits in 1.395 s through the gateway. The same user's SQLite search/free quota increased, and the credits response matched the ledger. A temporary topic added through the gateway appeared in direct MCP; removing it through MCP restored both lists to their initial state |
| Local contribution | An authenticated Rust text/metadata/local-embedding artifact passed validation and reached Indexed. The account balance remained zero for this existing-page check; new-content credit fulfillment and durable receipt acceptance are separate checks. Repeating the URL through CSV reported one duplicate and zero accepted |
| Public installer | [PR #1](https://github.com/pilot-protocol/cosift-install/pull/1) merged at `d18edef`. Full Git history and all issue/PR/release content had zero Gitleaks findings; custom-token inspection found only deterministic fixtures. The repository is public. `v0.4.0` and `v1` resolve to that merge; unauthenticated downloads returned 200 and the exact tested SHA-256 `fca1e4887b98e4b18957e48932203e3f0510e249b37baeaddd9292bcc4f8945b` |

The original Google-login blocker is resolved. These checks establish deployed
services, invocation boundaries and measured proxy behavior. They do not by
themselves establish a completed authenticated web/CLI/MCP workflow.

## Acceptance still recorded separately

Status at the 18:30 UTC documentation checkpoint:

Complete the remaining revocation, saved-request and new-content reward checks.
Observe credential
refresh over its real lifetime. Retain the existing model and capacity gates;
the pre-upgrade search/Answer/Research baseline records relevance limitations,
so a healthy endpoint is not evidence of strong answer quality.

Before rollout: provision the gateway identity; confirm the exact project,
database, index, accepted Cloud Run URLs/audiences and IAM grants; verify safe
proxy attribution with two real client addresses and a spoofed direct request;
land the MCP token-forwarding and auth deployment-configuration companions;
exercise real email login, revocation and ban handling in staging. Retain the
existing load, model and Stripe acceptance gates. Article generation and the
distributed contributor network remain outside the implemented upstream scope.
Stripe purchases remain disabled until the API key and webhook signing secret
are supplied and the test-mode checks pass. Article authoring, the distributed
contributor network and rewards for article views are unfinished upstream work.

The first full test run flagged the new operator-controlled auth/MCP HTTP client
in the repository's outbound-client inventory. It now has an explicit documented
Expand Down
Loading
Loading