Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions GENERATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# MobilityKafka generation — the canonical per-binding generator policy

This document is the contract for how MobilityKafka is generated, under the ecosystem-wide
per-binding generator policy.

## The policy (ecosystem-wide)

Every MobilityDB language/surface binding is a **pure projection of the MEOS-API catalog**,
and **each binding owns its own generator, in its own repo**, in a canonical layout. The
single source of truth is the **catalog** (`MEOS-API/output/meos-idl.json`, generated from
the MEOS C headers). A binding is an independent, plug-and-play module that owns its
generation.

Each binding repo satisfies the same invariants: in-repo generator; own
`tools/pin/compose-order.txt`; pinned catalog/jar input; thin language projection; full
automation toward a zero-hand-written surface (generate-then-retire; the last green-CI
version is the equivalence probe).

## MobilityKafka scope: generated MEOS facades over the JMEOS surface

MobilityKafka is a **consumer** binding: it binds the **JMEOS jar** (the JVM FFI projection
of the catalog), not MEOS-API directly. Its generator **`tools/codegen_facades.py`** (the
same generator as MobilityFlink) reads the bundled JMEOS raw-FFI surface and emits the
`org.mobilitydb.meos.MeosOps*` 1:1 forwarder facades the `kafka-streams-app` consumes. The
facades are a *consumer* projection (they live here, not in JMEOS).

## Generate-then-retire — the green-CI version is the probe

Hand-written facades/glue are replaced by the generated forwarders **family by family,
never wipe-first**: regenerate, build green, **prove generated ⊇ hand** against the **last
green-CI version** (the test suite + the benchmark), then retire the hand path. The
generated facades are committed and regenerated on a pin bump.

## Pinning

The bundled JMEOS jar is built via `build-jmeos.sh` from a deliverable-PR head (never
committed as a binary), and the `libmeos.so` it links is built from the **same MobilityDB
pin** the JMEOS surface was generated against (surface-match, else runtime symbol faults).
That pin is the *catalog/surface* input; MobilityKafka's own `tools/pin/compose-order.txt`
governs *this repo's* PR accumulate.
28 changes: 28 additions & 0 deletions tools/pin/compose-order.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# USER-APPROVED-PIN-WRITE — creating MobilityKafka's first pin manifest (user 2026-06-25,
# per-binding generator policy rollout). New file in the MobilityKafka repo, NOT a mutation
# of MobilityDB's pin tooling.
#
# MobilityKafka pin — THE canonical, dependency-ordered fold manifest (per-binding policy).
#
# MobilityKafka is a CONSUMER binding: it binds the JMEOS jar (not MEOS-API directly) and
# generates its org.mobilitydb.meos facades from it. (policy: generator-per-binding-canonical-policy)
#
# SCOPE: MobilityKafka owns its generator IN-REPO at `tools/codegen_facades.py` — it reads
# the bundled JMEOS raw-FFI surface and emits the `org.mobilitydb.meos.MeosOps*` facade
# forwarders the kafka-streams-app consumes (the same generator as MobilityFlink).
#
# Format: <PR#> <head-branch> # role. '?' = membership/order UNCONFIRMED.
# base = current origin/main.

# ── WAVE 0 — (no open PRs to main) ──
# (verified this turn: MobilityKafka has 0 open PRs.)

# ════════════════════════════════════════════════════════════════════════════════════
# TOPOLOGY GAP (committer review): the MEOS streaming consumer + the facade generator
# (`tools/codegen_facades.py`) live on the FORK branch `estebanzimanyi:consolidate/kafka-benchmark`
# but have NO open PR to `main` (the prior #14/#15 stack merged fork-internally, never to main).
# ACTION: open a base=main PR carrying the consumer + the generator, then this manifest folds it.
# The bundled JMEOS jar is built via build-jmeos.sh (never committed) from a deliverable-PR head;
# the libmeos it links is built from the SAME pin the JMEOS surface was generated against. See
# GENERATION.md.
# ════════════════════════════════════════════════════════════════════════════════════
23 changes: 23 additions & 0 deletions tools/regen-from-pin.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/usr/bin/env bash
# regen-from-pin.sh — regenerate the MobilityKafka MEOS facades from the JMEOS jar
# (per GENERATION.md). MobilityKafka is a JMEOS consumer.
#
# Usage: tools/regen-from-pin.sh <pin>
# env: JMEOS_JAR = path to the JMEOS jar built from the same pin (required)
#
# NOTE (topology gap, tools/pin/compose-order.txt): the consumer + tools/codegen_facades.py
# currently live on the fork branch consolidate/kafka-benchmark; once that lands on main this
# script regenerates in place. Invoked standalone, or by MEOS-API tools/ecosystem-generate.sh
# (after the JMEOS jar).
set -euo pipefail
PIN="${1:?usage: regen-from-pin.sh <pin>}"
JMEOS_JAR="${JMEOS_JAR:?set JMEOS_JAR to the JMEOS jar built from the same pin}"
HERE="$(cd "$(dirname "$0")/.." && pwd)"

# run the in-repo generator (tools/codegen_facades.py: --jar --out --engine) ->
# org.mobilitydb.meos.MeosOps* forwarder facades under <out>/src/main/java
python3 "$HERE/tools/codegen_facades.py" --jar "$JMEOS_JAR" --out "$HERE" --engine kafka

# build-verify
( cd "$HERE" && mvn -q test ) || echo "WARN: MobilityKafka mvn test returned non-zero"
echo "[kafka] regenerated facades from JMEOS jar at pin $PIN"