Skip to content

Latest commit

 

History

History
1532 lines (1022 loc) · 46.9 KB

File metadata and controls

1532 lines (1022 loc) · 46.9 KB

GATOS — SPEC v0.3 (Draft)

The key to understanding GATOS is understanding that it's just Git.

Git As The Operating Surface

You use Git for source control.
I use Git for reality control.
We are not the same.
GATOS: Git Different.

Status Draft (implementation underway)
Scope Normative specification of data model, on-disk layout, protocols, and behavioral guarantees.
Audience Implementers, auditors, integrators.

0. Conventions

The keywords MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL in this document are to be interpreted as described in RFC 2119.

graph TD
    A[RFC 2119] --> B{Keywords};
    B --> C[MUST];
    B --> D[SHOULD];
    B --> E[MAY];
Loading

Git refers to any conformant implementation supporting refs, commits, trees, blobs, notes, and atomic ref updates.

Hash defaults to BLAKE3 for content hashes and SHA-256 for policy bundle digests unless otherwise stated.


1. System Model

A GATOS node is a Git repository with a disciplined layout of refs, notes, and artifacts. A GATOS app is a set of schemas, policies, and folds that operate on append-only journals to produce deterministic state.

GATOS defines five planes:

  1. Ledger plane — append-only journals (events).
  2. State plane — deterministic folds (state roots).
  3. Policy/Trust plane — enforceable rules, grants, and multi-party consensus governance.
  4. Message plane — a commit-backed pub/sub bus.
  5. Job plane — Distributed, verifiable job execution.
graph TD
    subgraph "User / Client"
        CLI("git gatos (CLI)")
        SDK("Client SDK")
    end

    subgraph "GATOS System"
        Daemon("gatosd (Daemon)")

        subgraph "Ledger Plane"
            Ledger("gatos-ledger");
        end

        subgraph "Policy/Trust Plane"
            Policy("gatos-policy");
        end

        subgraph "State Plane"
            Echo("gatos-echo");
            KV("gatos-kv");
        end

        subgraph "Message Plane"
            Mind("gatos-mind");
        end

        subgraph "Job Plane"
            Compute("gatos-compute");
        end

        Daemon --> Policy;
        Daemon --> Echo;
        Daemon --> KV;
        Daemon --> Mind;
        Daemon --> Ledger;

        Echo --> Ledger;
        KV --> Ledger;
        Mind --> Ledger;
        Compute --> Mind;
        Compute --> Ledger;
    end

    CLI --> Daemon;
    SDK --> Daemon;

    style Policy fill:#f9f,stroke:#333,stroke-width:2px
    style Echo fill:#9cf,stroke:#333,stroke-width:2px
    style KV fill:#9cf,stroke:#333,stroke-width:2px
    style Mind fill:#9c9,stroke:#333,stroke-width:2px
    style Ledger fill:#c99,stroke:#333,stroke-width:2px
    style Compute fill:#f96,stroke:#333,stroke-width:2px
Loading

Requirements

  • Journals MUST be fast-forward-only.
  • State refs MUST be derivable from journals and policies.
  • Cache refs MUST be rebuildable and MUST NOT be authoritative.
  • Epochs MUST form a cryptographically-linked chain.

2. On-Disk Layout (Normative)

The following diagram illustrates the primary locations for GATOS artifacts within the Git repository (.git) and the working tree.

graph TD
    subgraph .git
        direction LR
        A(refs) --> A1(gatos)
        A1 --> B1(journal)
        A1 --> B2(state)
        A1 --> B3(mbus)
        A1 --> B4(jobs)
        A1 --> B5(sessions)
        A1 --> B6(audit)
        A1 --> B7(cache)
        A1 --> B8(epoch)
        A1 --> B9(policies)
        C(notes) --> C1(gatos)
    end
    subgraph Workspace
        direction LR
        D(gatos) --> D1(policies)
        D --> D2(schema)
        D --> D3(folds)
        D --> D4(trust)
        D --> D5(objects)
    end
Loading

The normative layout is as follows:

.git/
├── refs/
│   └── gatos/
│       ├── journal/
│       ├── state/
│       ├── mbus/
│       ├── mbus-ack/
│       ├── jobs/
│       │   └── <job-id>/
│       │       └── claim
│       ├── proposals/
│       ├── approvals/
│       ├── grants/
│       └── revocations/
│       ├── sessions/
│       ├── audit/
│       ├── cache/
│       └── epoch/
│       └── policies/
└── gatos/
    ├── policies/
    ├── schema/
    ├── folds/
    ├── trust/
    ├── objects/
    └── config/

Policy bundles (Normative).

  • Storage: Each policy bundle MUST be recorded as a commit on

    refs/gatos/policies/<bundle-id>, where <bundle-id> MUST be a deterministic content identifier: blake3(canonical_policy_tree) (DAG-CBOR, sorted keys; see ADR-0001). The commit’s tree contains the compiled policy artifacts; the commit trailers MUST include Policy-Root: <commit-oid> and Policy-Code-Root: sha256:<hex>.

  • Lineage: Bundle lineage MUST be represented by commit ancestry on the

    same ref (append-only). Historical bundles remain reachable.

  • Active pointer: Implementations MUST expose the effective bundle via

    refs/gatos/policies/active pointing to the commit OID of the active bundle. Updates to refs/gatos/policies/** (including active) MUST be fast-forward only. active is a plain ref (not a tag).

  • Consumers: Policy Gate evaluation MUST read the current policy from

    refs/gatos/policies/active. Verifiers and CLIs SHOULD default to the same unless explicitly overridden by a flag.

  • Mutation workflow:

    1. Fast-forward refs/gatos/policies/<bundle-id> to the candidate commit.

    2. Atomically update refs/gatos/policies/active via CAS (e.g.,

      git update-ref <old> <new>).

    3. On CAS mismatch, refetch and retry with jittered exponential backoff

      (base≈25 ms, max≈500 ms), up to 5 attempts.

    4. If retries exhaust, surface an error; the candidate bundle remains

      published but not active.


3. Identities, Actors, and Grants

3.1 Actors

Actors are strings of the form: user:<name>, agent:<name>, or service:<name>.

3.2 Capability Grants

Grants link an issuer Actor to a subject Actor, bestowing a set of capabilities (caps) that are valid until an expiration date (exp).

classDiagram
    class Grant {
        +String ulid
        +String issuer
        +String subject
        +String[] caps
        +Date exp
        +String sig
    }
    class Actor {
        <<enumeration>>
        user
        agent
        service
    }
    Grant "1" -- "1" Actor : issuer
    Grant "1" -- "1" Actor : subject
Loading

Grants MUST be committed under gatos/trust/grants/. Verifiers MUST validate the signature, issuer trust, audience, and expiry.


4. Events (Ledger Plane)

4.1 Event Envelope

All actions in GATOS are initiated via a signed Event.

classDiagram
    class EventEnvelope {
        +String type             // required; e.g., "event.append"
        +String ulid             // required; client-supplied idempotency key
        +String actor            // required; "user:<name>", "agent:<name>", ...
        +String[] caps           // optional; capabilities asserted by actor
        +Object payload          // required; event body
        +String policy_root      // required; policy commit governing evaluation
        +String sig_alg          // optional; signature algorithm id
        +String ts               // optional; RFC3339 UTC timestamp
    }
Loading

Field semantics and compatibility:

  • sig_alg — algorithm identifier (OPTIONAL). For v1, verifiers MUST

    support ed25519 (RFC 8032). Additional algorithms (e.g., ecdsa-p256, FIPS 186-4) MAY be enabled by policy. Unknown sig_alg values MUST cause rejection (or explicit policy override). When omitted, verifiers MUST infer the algorithm from signature material. If sig_alg contradicts the inferred algorithm, verifiers MUST NOT accept the envelope.

    sig_alg Required? Canonical specification references Notes
    ed25519 YES RFC 8032 Default v1 algorithm; all verifiers MUST implement and accept this identifier.
    ecdsa-p256 OPTIONAL SEC 1 (P-256 curve); RFC 6979 (deterministic ECDSA); FIPS 186-4 MAY be enabled by policy; when enabled, implementations MUST follow these specs.
    other/unknown NO Values not explicitly listed here MUST be rejected, unless a policy override exists.

    When sig_alg is omitted, verifiers MUST infer the algorithm from the signature material and key type, as described above.

  • ts — RFC 3339/ISO-8601 timestamp in UTC (YYYY-MM-DDTHH:MM:SSZ). When

    omitted, receivers MAY use the commit timestamp for display only; the envelope remains valid without ts.

  • Back/forward compatibility — Fields not present in older envelopes are simply

    absent from the canonical encoding. Receivers MUST accept envelopes with or without sig_alg/ts. Canonical bytes are computed over the envelope with sig omitted and only the fields that are present.

Canonicalization and signing (Normative):

  • Encode the envelope (with sig omitted) as DAG-CBOR per the IPLD

    DAG-CBOR spec; maps use RFC 8949 deterministic encoding (map keys MUST be sorted lexicographically by their encoded bytes; no indefinite-length items).

  • Content addressing: Event-CID = cidv1(dag-cbor, blake3(canonical_bytes))

    (BLAKE3 per spec; multihash extension as deployed).

  • Sign canonical_bytes with sig_alg (e.g., ed25519); record signature in

    commit trailers.

  • Recommended trailers: Event-CID, Sig-Alg, Sig.

  • See ADR-0001 for canonicalization test vectors.

4.2 Journal Semantics

Appending an event MUST create a new commit on an append-only ref in refs/gatos/journal/<ns>/<actor>. Ref updates MUST use atomic compare-and-swap via git update-ref <old> <new>.

CAS failure and idempotency:

  • On update-ref failure (old OID mismatch), writers MUST refetch and

    retry. Writers SHOULD implement jittered exponential backoff (e.g., base=25 ms, max≈500 ms) and SHOULD cap retries (e.g., 5 attempts) before surfacing an error to the caller.

  • Clients SHOULD supply ulid as an idempotency key; gates MAY reject

    duplicate ULIDs within a configured horizon.

  • Journals are linear; divergence is resolved by retrying the CAS against the

    latest head. Merges into journal refs are forbidden.


5. State (Deterministic Folds)

5.1 Fold Function

A fold is a pure function: $state_root = F(events_stream, policy_root)$.

graph TD
    A[Event Stream] --> B{Fold Function};
    C[Policy] --> B;
    B --> D[State Root];
Loading

For identical inputs, and the same policy_root, the byte sequence of state_root MUST be identical.

5.2 Fold Spec & Checkpoints

A Fold is defined by a .yaml spec and a compiled EchoLua program (ELC). The EchoLua source is compiled to ELC (serialized DAG-CBOR); fold_root = sha256(ELC_bytes). Its output, a State Checkpoint, is a commit on refs/gatos/state/<ns> whose tree contains the materialized state artifacts.

See also

  • Hello walkthrough (Ops): HELLO-OPS shows append → fold → verify trailers.
  • Hello walkthrough (Privacy): HELLO-PRIVACY shows pointers and deterministic public folds.

5.3 State Checkpoint Trailers (Normative)

Every state checkpoint commit under refs/gatos/state/<ns> MUST include the following trailers with canonical encodings:

State-Root: blake3:<hex>                 # lowercase hex digest of canonical state
Ledger-Head: <commit-oid>                # last ledger commit included in this fold
Policy-Root: <commit-oid>                # commit/digest that identifies the effective policy
Policy-Code-Root: sha256:<hex>           # canonical hash of policy code used (ELC/.rgc)
Fold-Engine: echo@<semver>+elc@<semver>+num=q32.32+rng=pcg32@<ver>
Fold-Root: sha256:<hex>                  # hash of EchoLua IR bytes (ELC)
Fold-Version: <schema-version>            # application/shape schema version
Fold-Math: fixed-q32.32                   # numeric model (normative in v1)
Fold-RNG: pcg32@<ver>                     # RNG algorithm id + version (if used)

Validation and compatibility:

  • Verifiers MUST require at least: State-Root, Ledger-Head,

    Policy-Root, Fold-Root, and Fold-Engine. Unknown trailers MUST be ignored.

  • When optional trailers (e.g., Fold-RNG, Fold-Math) are absent, verifiers

    MUST apply implementation defaults for the declared profile and still accept the checkpoint.

  • Trailer values MUST use the prefixed canonical encodings shown above.

These rules enable portable verification and reproducible builds of state across nodes and platforms.

Cross-reference

  • TECH-SPEC fold/verification references these trailers.
  • Walkthrough usage: HELLO-OPS.

5.4 Proof-of-Fold (PoF)

A Proof-of-Fold (PoF) binds a state checkpoint to the exact ledger window and fold definition used to derive it.

At minimum, a PoF MUST commit to:

  • Ledger-Start and Ledger-End commit OIDs (inclusive window),
  • Policy-Root (or policy digest) used for gate decisions affecting the fold,
  • Fold-Id (stable identifier or digest of the fold function/spec),
  • Fold-Root (sha256 of ELC bytes),
  • State-Root (content hash of the resulting checkpoint),
  • Policy-Code-Root (sha256 of the canonicalized policy code/ELC),
  • Signatures of the folding actor(s) when required by policy.

Implementations MAY embed PoF in commit trailers or attach a sidecar manifest. Verifiers MUST recompute the fold over the declared window and compare the resulting State-Root.


6. Policy & Decision Audit

6.1 Gate Contract

All events are evaluated by a Policy Gate before being accepted. Gates that execute policy code MUST adhere to the Deterministic Lua (EchoLua) runtime profile. Policy is authored in .rgs and compiled to .rgc/ELC; policy bundles and proofs MUST record Policy-Code-Root so the exact governing code is recoverable. $Decision = Gate.evaluate(intent, context) -&gt; {Allow | Deny(reason)}$

Note: A deterministic execution profile for Lua will be documented (see Deterministic Lua); policy engines SHOULD adhere to that profile to ensure portable verification.

sequenceDiagram
    participant Client
    participant GATOS
    participant PolicyGate

    Client->>GATOS: Propose Action (Intent)
    GATOS->>PolicyGate: Evaluate(Intent, Context)
    alt Action is Allowed
        PolicyGate-->>GATOS: Decision: Allow
        GATOS->>GATOS: Bind policy_root to event
        GATOS-->>Client: Success
    else Action is Denied
        PolicyGate-->>GATOS: Decision: Deny(reason)
        GATOS->>GATOS: Write Audit Decision
        GATOS-->>Client: Failure(reason)
    end
Loading

On DENY, the gate MUST append an audit decision to refs/gatos/audit/policy.

Important: DENY is always logged under audit. Each decision MUST include the policy rule identifier (Policy-Rule), a reproducible reason, and sufficient context (actor, target, refs) for independent verification.


7. Blob Pointers & Opaque Storage (Hybrid Privacy)

Large or sensitive data is stored out-of-band in a content-addressed store and referenced via pointers.

classDiagram
    class BlobPointer {
        +String kind: "blobptr"
        +String algo
        +String hash
        +Number size
    }
    class OpaquePointer {
        +String kind: "opaque"
        +String algo                // content cipher, e.g., "aes-256-gcm" or "chacha20poly1305"
        +String ciphertext_hash     // BLAKE3 hex of ciphertext bytes (integrity check)
        +Object encrypted_meta      // see schema below (REQUIRED for new pointers)
    }
Loading

Pointers MUST refer to bytes in gatos/objects/<algo>/<hash>. For opaque objects, no plaintext MAY be stored in Git. Public pointers in low-entropy classes MUST NOT reveal a plaintext digest; they MUST include a ciphertext digest. Pointer size SHOULD be bucketed (e.g., 1 KB/4 KB/16 KB/64 KB). If a plaintext commitment is required, use a hiding commitment and store it inside encrypted_meta.

encrypted_meta (Normative) — object fields:

  • enc: string — cipher suite id, e.g., aes-256-gcm or chacha20poly1305.
  • iv: base64url — initialization vector/nonce bytes.
  • salt: base64url — KDF salt when deriving keys (RECOMMENDED when using passphrase KDFs).
  • aad: base64url — additional authenticated data bound to encryption (OPTIONAL; empty if unused).
  • tag: base64url — AEAD tag if not appended to ciphertext (omit when suite appends tag).

ciphertext_hash is an integrity hint only. Authenticity comes from the envelope signature and repository trust rules (see Sections 3 and 6). Verifiers MUST treat ciphertext_hash as unauthenticated unless covered by a signature.

Compatibility note: previous drafts used cipher_meta. Parsers SHOULD accept inputs with cipher_meta; emit encrypted_meta going forward. A simple migration heuristic is: if cipher_meta exists and encrypted_meta is absent, rename cipher_metaencrypted_meta (see scripts/migrate_opaque_pointers.py).

Before/after example:

// legacy
{ "kind":"opaque", "algo":"aes-256-gcm", "ciphertext_hash":"blake3:…",
  "cipher_meta": {"enc":"aes-256-gcm","iv":"","salt":""} }

// normalized
{ "kind":"opaque", "algo":"aes-256-gcm", "ciphertext_hash":"blake3:…",
  "encrypted_meta": {"enc":"aes-256-gcm","iv":"","salt":""} }

8. Message Bus (Commit-Backed Pub/Sub)

The message bus provides a pub/sub system built on Git commits.

sequenceDiagram
    participant Publisher
    participant GATOS
    participant Consumer

    Publisher->>GATOS: Publish Message (QoS: at_least_once)
    GATOS-->>Consumer: Deliver Message
    Consumer->>Consumer: Process Message
    Consumer->>GATOS: Send Ack
    GATOS->>GATOS: Observe Ack Quorum
    GATOS->>GATOS: Create gmb.commit Event
Loading

Messages are appended to refs/gatos/mbus/<topic>/<shard>. Delivery is at-least-once; consumers MUST dedupe on read using the message ULID (or content hash) as an idempotency key and MAY write acks to refs/gatos/mbus-ack/. Producers SHOULD set idempotency keys.

Retention and compaction:

  • Segment topics by date and ULID (e.g., <topic>/<yyyy>/<mm>/<dd>/<segment-ulid>) and rotate when either a message or size threshold is reached (defaults: 100k messages or ~192 MB).
  • Apply TTL to old segments (default: 30 days) and write a summary commit per pruned window (counts + Merkle root of message bodies + last offsets) to preserve verifiable history.
  • Deployments SHOULD enable fetch.writeCommitGraph=true and repack.writeBitmaps=true for busy topics.

9. Sessions (Working Branches)

gatos/sessions/<actor>/<ulid> represents an ephemeral branch for interactive mutation.

graph TD
    A(main) --> B(session-1);
    B --> C(commit-a);
    C --> D(commit-b);
    subgraph "fork"
        B --> E(session-2);
    end
    subgraph "undo"
        D -- undo --> C;
    end
    subgraph "merge"
        D --> F(main);
        E --> F;
    end
Loading

10. Proofs (Commitments / ZK)

A proof envelope attests to the deterministic execution of a fold or job. Where applicable, proofs MUST record Policy-Code-Root alongside Policy-Root so the governing policy code is unambiguously identified.

classDiagram
    class ProofEnvelope {
        +String type
        +String ulid
        +String inputs_root
        +String output_root
        +String policy_root
        +String proof
        +String sig
    }
Loading

Proofs MUST be stored under refs/gatos/audit/proofs/<ns>.

10.x Proof-of-Experiment (PoX)

A PoX envelope ties together a scientific artifact’s inputs, program, policy, and outputs:

  • inputs_root — commitment to input datasets/pointers
  • program_id — canonical hash of the analysis program/container
  • policy_root — policy in effect
  • outputs_root — commitment to results
  • Links to associated PoE (jobs) and PoF (state) where applicable

PoX envelopes MUST be stored under refs/gatos/audit/proofs/experiments/<ulid>.


11. Offline Authority Protocol (OAP)

OAP governs how divergent changes from offline peers are reconciled upon reconnecting.

sequenceDiagram
    participant PeerA
    participant PeerB
    Note over PeerA, PeerB: Peers are offline and make divergent changes
    PeerA ->> PeerB: Reconnect & Exchange Envelopes
    PeerB ->> PeerB: Validate Signatures & Policy Ancestry
    alt Policies are comparable
        PeerB ->> PeerB: Prefer descendant policy
    else Policies are incomparable
        PeerB ->> PeerB: Append governance.conflict event
    end
Loading

12. Profiles

Profiles define the enforcement and operational mode of a GATOS node.

graph TD
    A(GATOS) --> B(local);
    A --> C(push-gate);
    A --> D(saas-hosted);
Loading

Nodes MUST discover the active profile via gatos/config/profile.yaml.

12.1 Research Profile (Strict)

Profile id: research.

Defaults (normative for this profile):

  • Proof-of-Fold required on state pushes: pre-receive MUST verify PoF for updates to refs/gatos/state/**.
  • Fast-forward-only refs: refs/gatos/policies/**, refs/gatos/state/**, and refs/gatos/audit/**.
  • GC anchors: refs/gatos/audit/** and the latest refs/gatos/state/** checkpoints.
  • Message bus segmentation and TTL: rotate segments at 100k messages or ~192 MB; TTL 30 days; write summary commits for pruned windows.
  • Public pointer hardening: low-entropy classes MUST NOT expose plaintext digests; public pointers MUST include a ciphertext digest; sizes SHOULD be bucketed (e.g., 1 KB, 4 KB, 16 KB, 64 KB).

Nodes advertising the research profile MUST expose diagnostics for the above and SHOULD surface violations in gatos doctor.


13. Observability & Health

Implementations SHOULD expose metrics and provide a health-check CLI command.

graph TD
    A[gatosd] -- Exposes --> B["/metrics"];
    C[Prometheus] -- Scrapes --> B;
    D[gatos doctor] -- Diagnoses --> E(Ref Invariants);
    D --> F(Epoch Continuity);
    D --> G(Cache Staleness);
Loading

14. Security Model

The security model is deny-by-default, governed by capability grants evaluated by the policy engine.

graph TD
    subgraph "Access Control"
        A[Actor] -- Requests access to --> B[Resource];
        C[Capability Grant] -- Links --> A;
        C -- Links --> D{Policy};
        D -- Evaluates request for --> B;
    end
Loading

15. Performance & GC

Epoch compaction is used to manage repository size over time.

graph TD
    A[Epoch N] --> B(Epoch N+1 Anchor);
    B -- Triggers --> C{Compaction};
    C -- Prunes --> D(Unreferenced Blobs in Epoch N);
Loading

15.1 Exports and Explorer-Root (Normative)

Exporters that materialize views MUST emit an Explorer-Root checksum.

  • For derived state exports (materialized from folds), include the fold identity

    and compute over a canonical, length-prefixed concatenation (see below).

  • For raw ledger exports (no folds applied), omit fold_root entirely (both

    the field and its length prefix) from the canonical concatenation.

Canonical serialization for Explorer-Root (Normative):

Let ledger_head, policy_root, and fold_root be lowercase hex digests. Let extractor_version be a UTF-8 string. Serialize as a concatenation of 32-bit big-endian length prefixes and raw bytes, in this exact order:

BE32(len(ledger_head_bytes)) || ledger_head_bytes ||
BE32(len(policy_root_bytes)) || policy_root_bytes ||
BE32(len(fold_root_bytes))   || fold_root_bytes   ||
BE32(len(version_bytes))     || version_bytes

Where *_bytes for digests are obtained by hex-decoding the string, and version_bytes is the UTF-8 encoding of extractor_version. Then:

Explorer-Root = blake3(serialized_bytes)

For raw ledger exports, drop the 3rd line (fold_root length and value).

Worked example (derived export):

ledger_head (hex): 1111111111111111111111111111111111111111  # 20 bytes
policy_root (hex): 2222222222222222222222222222222222222222  # 20 bytes
fold_root   (hex): aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa  # 32 bytes
version     (utf8): v1.2.3+linux-x86_64

serialized hex (BE32 prefix + fields):
00000014111111111111111111111111111111111111111100000014222222222222222222222222222222222222222200000020aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa0000001376312e322e332b6c696e75782d7838365f3634

BLAKE3(serialized) = e4cbbcc02698608bb59ed39d715ffd6e74c2865f473daf7cb861f9da33565d9d

The gatos export verify command MUST recompute Explorer-Root against the repository and report mismatches.


16. Compliance & Tests (Normative)

Implementations MUST pass a six-point certification inspection.

graph TD
    A(GATOS Implementation) --> B(Certification);
    B --> C(Deterministic Fold);
    B --> D(At-Least-Once + Idempotency);
    B --> E(Offline Reconcile);
    B --> F(Deny Audit);
    B --> G(Blob Integrity & Pointer Privacy);
    B --> H(Consensus Integrity);
    B --> I(Exclusive Job Claim);
    B --> J(Explorer-Root on Exports);
Loading

16.1 Consensus Integrity (Normative)

  • An action gated by a 2-of-3 quorum policy MUST be denied with 1 approval and MUST be allowed with 2 approvals.
  • A revoked grant MUST NOT be usable.

16.2 Additional Checks (Normative)

  • PoF required: state pushes to refs/gatos/state/** MUST include a verifiable Proof-of-Fold.
  • Policies FF-only: refs/gatos/policies/** MUST be fast-forward only.
  • Exclusive job claim: exactly one worker MUST succeed in creating refs/gatos/jobs/<job-id>/claim via compare-and-swap.
  • Pointer privacy: public pointers for low-entropy classes MUST NOT expose plaintext digests; ciphertext digest present; sizes bucketed.
  • Exports: exporters MUST emit Explorer-Root = blake3(ledger_head || policy_root || extractor_version) and gatos export verify MUST validate it.

17. CLI (Reference)

The git gatos command provides the primary user interface.

graph TD
    A(git gatos) --> B(init);
    A --> C(session);
    A --> D(event);
    A --> E(fold);
    A --> F(bus);
    A --> G(policy);
    A --> H(trust);
    A --> I(epoch);
    A --> J(verify);
    A --> K(reproduce);
    A --> L(bisect);
    A --> M(export);
    A --> N(doctor);
    A --> O(foldc);
    A --> P(policyc);
Loading

Key verbs (non-exhaustive):

  • verify — verify PoX/PoF/PoE signatures & ancestry.
  • reproduce <pox-id> — fetch pointers, run jobs (PoE), check PoF; report identical/diff.
  • bisect --state=<ref> --good <ts|commit> --bad <ts|commit> --predicate <script|jq> — binary search over checkpoints.
  • export parquet|sqlite — emit Explorer-Root; export verify <path> compares it.
  • foldc <src.lua> -o <out.elc> — compile EchoLua to ELC (records engine id).
  • policyc <src.rgs> -o <out.rgc> — compile .rgs policy DSL to deterministic IR/ELC.

See also:


18. Example Use Case: A Git-Native Work Queue

This diagram shows the data flow for enqueuing and processing a job.

sequenceDiagram
    participant Client
    participant Daemon as gatosd
    participant Ledger as gatos-ledger
    participant Bus as gatos-mind
    participant State as gatos-echo

    Client->>Daemon: 1. Enqueue Job (Event)
    Daemon->>Ledger: 2. Append `jobs.enqueue` event
    Ledger-->>Daemon: 3. Success
    Daemon->>Bus: 4. Publish `gmb.msg` to topic
    Bus-->>Daemon: 5. Success
    Daemon-->>Client: 6. Job Enqueued

    Note over Bus,State: Later, a worker consumes the job...

    participant Worker
    Worker->>Bus: 7. Subscribe to topic
    Bus->>Worker: 8. Delivers `gmb.msg`
    Worker->>Daemon: 9. Report Result (Event)
    Daemon->>Ledger: 10. Append `jobs.result` event
    Ledger-->>Daemon: 11. Success
    Daemon->>Bus: 12. Write `gmb.ack`
    Daemon-->>Worker: 13. Result Recorded

    Note over Ledger,State: A fold process runs...
    State->>Ledger: 14. Read events from journal
    State->>State: 15. Compute new state (e.g., update queue view)
    State->>Ledger: 16. Checkpoint new state
Loading

19. Job Plane (Compute)

See also: ADR-0002.

The Job Plane provides a system for scheduling, executing, and recording the results of distributed, asynchronous jobs.

19.1 Job Lifecycle

This diagram illustrates how the state of a Job transitions based on events recorded in the GATOS ledger.

stateDiagram-v2
    [*] --> pending

    pending --> claimed: Worker claims job (CAS)
    claimed --> running: Worker begins execution
    running --> succeeded: `jobs.result` (ok)
    running --> failed: `jobs.result` (fail)

    succeeded --> [*]
    failed --> [*]

    pending --> aborted: Canceled by user/policy
    claimed --> aborted: Canceled by user/policy
    aborted --> [*]
Loading

The lifecycle is represented entirely through Git objects:

  • Job: A commit whose tree contains a job.yaml manifest.
  • Claim: An atomic ref under refs/gatos/jobs/<job-id>/claims/<worker-id>, where <job-id> is the canonical BLAKE3 content_id of the job manifest (see ADR-0002 Canonical Job Identifier).
  • Result: A commit referencing the job commit, containing a Proof-Of-Execution.

19.2 Job Discovery

When a Job commit is created, a message MUST be published to a topic on the Message Plane for discovery by workers. Delivery is at-least-once; workers MUST be idempotent.

19.3 Proof-Of-Execution

The Proof-Of-Execution (PoE) MUST sign the job’s canonical content_id (BLAKE3 of the canonical unsigned job core). Trailers MUST use canonical, prefixed encodings as follows:

  • Job-Id: blake3:<hex> — canonical job identifier (content_id)
  • Proof-Of-Execution: blake3:<hex> — digest of the PoE envelope
  • Worker-Id: ed25519:<pubkey> — worker public key identifier
  • Attest-Program: blake3:<hex> — hash of runner binary or WASM module (RECOMMENDED)
  • Attest-Sig: ed25519:<sig> — signature over the attestation envelope (OPTIONAL)

Example (trailers):

Job-Id: blake3:<hex>
Worker-Id: ed25519:<pubkey>
Proof-Of-Execution: blake3:<hex>
Attest-Program: blake3:<hex>
Attest-Sig: ed25519:<sig>

See ADR-0002 for the normative PoE requirements and ADR-0001 for the definition of content_id and canonical serialization.


20. Consensus Governance (Normative)

See also: ADR-0003.

Governs gated actions via proposals, approvals, and grants. Governance artifacts are Git commits under dedicated refs (see on-disk layout). All trailers MUST use canonical, prefixed encodings (blake3:<hex>, ed25519:<pubkey>).

20.1 Workflow

Proposal → Approvals (N-of-M) → Grant. Quorum groups (e.g., @leads) MUST be defined in the trust graph (gatos/trust/graph.json).

20.2 Commit Structures (Trailers)

  • Proposal (at refs/gatos/proposals/…):

    Action: <string>
    Target: <uri>
    Proposal-Id: blake3:<hex>
    Required-Quorum: <expr>
    Expire-At: <ISO8601>
    Policy-Rule: <policy id>
    Created-By: <actor>
    

    (Note: gatos:// is the canonical URI scheme for addressing resources managed within the GATOS operating surface.)

  • Approval (at refs/gatos/approvals/…):

    Proposal-Id: blake3:<hex>
    Approval-Id: blake3:<hex>
    Signer: ed25519:<pubkey>
    Expires-At: <ISO8601>   # OPTIONAL
    
  • Grant (at refs/gatos/grants/…):

    Proposal-Id: blake3:<hex>
    Grant-Id: blake3:<hex>
    Proof-Of-Consensus: blake3:<hex>
    

20.3 Proof-Of-Consensus (PoC)

Proof-Of-Consensus is the BLAKE3 of a canonical JSON envelope containing:

  • The canonical proposal envelope (by value or Proposal-Id).
  • A sorted list (by Signer) of all valid approvals used to reach quorum (by value or Approval-Id).
  • The governance rule id (Policy-Rule) and effective quorum parameters.

PoC envelope SHOULD be stored canonically under refs/gatos/audit/proofs/governance/<proposal-id>; the Grant’s Proof-Of-Consensus trailer MUST equal blake3(envelope_bytes).

20.4 Lifecycle States

State Meaning
proposal Awaiting votes
partial Some approvals collected
granted Quorum reached; action allowed
expired Proposal timed out
revoked Grant withdrawn or superseded
stateDiagram-v2
    [*] --> proposal
    proposal --> partial: approval received
    partial --> partial: additional approvals
    proposal --> expired: ttl elapsed
    partial --> expired: ttl elapsed
    partial --> granted: quorum satisfied
    granted --> revoked: revocation committed
    expired --> [*]
    revoked --> [*]
Loading

20.5 Revocation

A grant MAY be revoked by creating a revocation commit under refs/gatos/revocations/ with trailers:

Grant-Id: blake3:<hex>
Revocation-Id: blake3:<hex>
Reason: <free-text>
Revoked-By: <actor>

20.6 Bus Topics (recommended)

gatos.governance.proposal.created, gatos.governance.approval.created, gatos.governance.grant.created, gatos.governance.grant.revoked.


Glossary

  • PoF — Proof-of-Fold. Evidence that a state root was derived deterministically from a specific ledger window under a specific fold/policy root.
  • PoE — Proof-of-Execution. Signed attestation that a worker executed a job (who/what/where/inputs/outputs).
  • PoC — Proof-of-Consensus. Evidence that a governance action met its quorum/approval rules.
  • PoX — Proof-of-Experiment. Bundle tying inputs → program → outputs for scientific reproducibility.
  • ULID — Lexicographically sortable identifier used as an idempotency key for messages.