From 85c94e68c3d79a3cfe5f714260a2345d5588b18d Mon Sep 17 00:00:00 2001 From: Esteban Zimanyi Date: Thu, 25 Jun 2026 21:18:30 +0200 Subject: [PATCH] Add the MobilityKafka pin manifest and per-binding generator policy doc Add tools/pin/compose-order.txt (records the topology gap: 0 open PRs to main; the MEOS consumer + facade generator live on the fork branch consolidate/kafka-benchmark and need a base=main PR) and GENERATION.md (the per-binding generator policy: MobilityKafka binds the JMEOS jar and generates the org.mobilitydb.meos facades via tools/codegen_facades.py). --- GENERATION.md | 40 +++++++++++++++++++++++++++++++++++++ tools/pin/compose-order.txt | 28 ++++++++++++++++++++++++++ tools/regen-from-pin.sh | 23 +++++++++++++++++++++ 3 files changed, 91 insertions(+) create mode 100644 GENERATION.md create mode 100644 tools/pin/compose-order.txt create mode 100755 tools/regen-from-pin.sh diff --git a/GENERATION.md b/GENERATION.md new file mode 100644 index 0000000..984fb3c --- /dev/null +++ b/GENERATION.md @@ -0,0 +1,40 @@ +# MobilityKafka generation — the canonical per-binding generator policy + +This document is the contract for how MobilityKafka is generated, under the ecosystem-wide +per-binding generator policy. + +## The policy (ecosystem-wide) + +Every MobilityDB language/surface binding is a **pure projection of the MEOS-API catalog**, +and **each binding owns its own generator, in its own repo**, in a canonical layout. The +single source of truth is the **catalog** (`MEOS-API/output/meos-idl.json`, generated from +the MEOS C headers). A binding is an independent, plug-and-play module that owns its +generation. + +Each binding repo satisfies the same invariants: in-repo generator; own +`tools/pin/compose-order.txt`; pinned catalog/jar input; thin language projection; full +automation toward a zero-hand-written surface (generate-then-retire; the last green-CI +version is the equivalence probe). + +## MobilityKafka scope: generated MEOS facades over the JMEOS surface + +MobilityKafka is a **consumer** binding: it binds the **JMEOS jar** (the JVM FFI projection +of the catalog), not MEOS-API directly. Its generator **`tools/codegen_facades.py`** (the +same generator as MobilityFlink) reads the bundled JMEOS raw-FFI surface and emits the +`org.mobilitydb.meos.MeosOps*` 1:1 forwarder facades the `kafka-streams-app` consumes. The +facades are a *consumer* projection (they live here, not in JMEOS). + +## Generate-then-retire — the green-CI version is the probe + +Hand-written facades/glue are replaced by the generated forwarders **family by family, +never wipe-first**: regenerate, build green, **prove generated ⊇ hand** against the **last +green-CI version** (the test suite + the benchmark), then retire the hand path. The +generated facades are committed and regenerated on a pin bump. + +## Pinning + +The bundled JMEOS jar is built via `build-jmeos.sh` from a deliverable-PR head (never +committed as a binary), and the `libmeos.so` it links is built from the **same MobilityDB +pin** the JMEOS surface was generated against (surface-match, else runtime symbol faults). +That pin is the *catalog/surface* input; MobilityKafka's own `tools/pin/compose-order.txt` +governs *this repo's* PR accumulate. diff --git a/tools/pin/compose-order.txt b/tools/pin/compose-order.txt new file mode 100644 index 0000000..aa1619e --- /dev/null +++ b/tools/pin/compose-order.txt @@ -0,0 +1,28 @@ +# USER-APPROVED-PIN-WRITE — creating MobilityKafka's first pin manifest (user 2026-06-25, +# per-binding generator policy rollout). New file in the MobilityKafka repo, NOT a mutation +# of MobilityDB's pin tooling. +# +# MobilityKafka pin — THE canonical, dependency-ordered fold manifest (per-binding policy). +# +# MobilityKafka is a CONSUMER binding: it binds the JMEOS jar (not MEOS-API directly) and +# generates its org.mobilitydb.meos facades from it. (policy: generator-per-binding-canonical-policy) +# +# SCOPE: MobilityKafka owns its generator IN-REPO at `tools/codegen_facades.py` — it reads +# the bundled JMEOS raw-FFI surface and emits the `org.mobilitydb.meos.MeosOps*` facade +# forwarders the kafka-streams-app consumes (the same generator as MobilityFlink). +# +# Format: # role. '?' = membership/order UNCONFIRMED. +# base = current origin/main. + +# ── WAVE 0 — (no open PRs to main) ── +# (verified this turn: MobilityKafka has 0 open PRs.) + +# ════════════════════════════════════════════════════════════════════════════════════ +# TOPOLOGY GAP (committer review): the MEOS streaming consumer + the facade generator +# (`tools/codegen_facades.py`) live on the FORK branch `estebanzimanyi:consolidate/kafka-benchmark` +# but have NO open PR to `main` (the prior #14/#15 stack merged fork-internally, never to main). +# ACTION: open a base=main PR carrying the consumer + the generator, then this manifest folds it. +# The bundled JMEOS jar is built via build-jmeos.sh (never committed) from a deliverable-PR head; +# the libmeos it links is built from the SAME pin the JMEOS surface was generated against. See +# GENERATION.md. +# ════════════════════════════════════════════════════════════════════════════════════ diff --git a/tools/regen-from-pin.sh b/tools/regen-from-pin.sh new file mode 100755 index 0000000..0daadbb --- /dev/null +++ b/tools/regen-from-pin.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +# regen-from-pin.sh — regenerate the MobilityKafka MEOS facades from the JMEOS jar +# (per GENERATION.md). MobilityKafka is a JMEOS consumer. +# +# Usage: tools/regen-from-pin.sh +# env: JMEOS_JAR = path to the JMEOS jar built from the same pin (required) +# +# NOTE (topology gap, tools/pin/compose-order.txt): the consumer + tools/codegen_facades.py +# currently live on the fork branch consolidate/kafka-benchmark; once that lands on main this +# script regenerates in place. Invoked standalone, or by MEOS-API tools/ecosystem-generate.sh +# (after the JMEOS jar). +set -euo pipefail +PIN="${1:?usage: regen-from-pin.sh }" +JMEOS_JAR="${JMEOS_JAR:?set JMEOS_JAR to the JMEOS jar built from the same pin}" +HERE="$(cd "$(dirname "$0")/.." && pwd)" + +# run the in-repo generator (tools/codegen_facades.py: --jar --out --engine) -> +# org.mobilitydb.meos.MeosOps* forwarder facades under /src/main/java +python3 "$HERE/tools/codegen_facades.py" --jar "$JMEOS_JAR" --out "$HERE" --engine kafka + +# build-verify +( cd "$HERE" && mvn -q test ) || echo "WARN: MobilityKafka mvn test returned non-zero" +echo "[kafka] regenerated facades from JMEOS jar at pin $PIN"