Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ CLAUDE.local.md

# FMA body.soa bake scratch (regenerated from uploads; do not commit)
scratch-fma/
crates/osint-bake/tools/__pycache__/
__pycache__/

# big FMA body wire — lives in GitHub Releases, never in git
cockpit/public/body.soa
Expand Down
46 changes: 46 additions & 0 deletions data/osint-v3/OSINT_V3_BAKE_README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,52 @@ positions apply; it does not — every node carries a value in every dimension.)
> read these rails — wiring it (facet by GUID slot, kind by reading the 12×
> is_a per position) is the follow-up this flip clears the way for.

> **hi/lo tier-order correction (2026-07-06).** Rebaked so the stored tier
> bytes match the **declared field order**. The original bake wrote each
> `6×(8:8)` tier as a little-endian `u16`, landing the two bytes of every
> `hi:lo` pair in memory as `(lo, hi)`. But the field cards in
> `crates/cockpit-server/src/osint_classview.rs` (`OSINT_SYSTEM_FIELDS` 12,
> `OSINT_PERSON_FIELDS` 5) and the contract's position law
> (`lance-graph-contract` `ClassView::facet_rows`: **field position i = facet
> byte i**, `hi:lo` reading) read position i as facet byte i. The two disagreed
> by a byte swap within each tier pair. Decoding the pre-correction bake against
> the codebook's declared order landed only **16/611 GUID1** and **52/133
> GUID2** values in-vocabulary; swapping each tier pair's two bytes raises that
> to **609/611 GUID1** and **133/133 GUID2**. (GUID2 "before" = 52/133 counts
> the 5 McClelland fields 1-based ignoring the TWIG `_` padding byte; the
> stricter count that also requires the padding byte to be empty is 0/133 →
> 133/133, because pre-correction the *motive* value sat in the padding slot,
> not in its own position.) Per operator ruling, the **DATA was fixed (this
> rebake), not the field declarations**. Only the tier bytes changed: the
> `row_id`, the `classid` u32 (`0x0701_1000`, LE `00 10 01 07`), the header, and
> every non-GUID node field are byte-identical to the prior bake; `unswap(new) ==
> old` for all 611 rows. **This supersedes the 2026-07-04 note's claim that
> "every rail byte is byte-identical to the original bake"** — that held across
> the classid flip but not across this hi/lo correction.
>
> The remaining **2/611 GUID1** out-of-vocab after correction are genuine
> codebook↔harvest gaps, not swap artifacts: `DreamSecurity` MLType=30 (vocab
> max 29) and `Shoebox` civicUse=17 (vocab max 16) — each one past its vocab
> tail; reconcile the vocab against the harvest to close them.
>
> **McClelland indexing (GUID2 persons): 1-indexed against `mcclelland_vocab`.**
> After correction every person field's values fall in `1..=len` with the
> observed max equal to the vocab length (stage 2..5 / len 5, need 1..3 / len 3,
> receptor/rubicon/motive 1..5 / len 5), min nonzero ≥ 1, and no meaningful
> zeros — so value `v` resolves to `vocab[v-1]` (matching `airo_vocab`'s explicit
> 1-based integers). Under 0-indexing 26/133 would overflow their arrays. The
> TWIG `_` padding byte (GUID2 tier position 5) is `0` for all 133 persons after
> correction; pre-correction it wrongly carried the motive value.
>
> **Provenance.** There is no in-repo baker for this artifact (it is baked from
> the external `AdaWorldAPI/aiwar-neo4j-harvest`, which is not present and cannot
> regenerate offline). The correction was applied by the deliberate one-shot
> transform `scripts/osint_v3_rebake_hilo.py` (checked in): classid bytes 0..4
> untouched, tier bytes 4..16 swapped pairwise per GUID, `.soa` and
> `osint_v3_nodes.json` rewritten consistently and cross-verified byte-for-byte.
> The script guards against double-application (a pairwise swap is its own
> inverse) by refusing to run unless the data is in the pre-swap broken state.

## Assets
- `osint_v3.soa` — binary SoA. Header `OSINTV3\0` + `ver(u16=3) count(u32) stride(u16=36)`,
then `count` records of `row_id(u32) | GUID1(16B) | GUID2(16B)` little-endian.
Expand Down
Binary file modified data/osint-v3/osint_v3.soa
Binary file not shown.
Loading