Design post: Four Encodings, Three Joins (the mushaf ETL)

[omars-lab]

One new /designs post. Nothing else in the repo changes.

bytesofpurpose-blog/designs/2026-08-07-mushaf-etl.mdx · kind: backend-design · slug: design-mushaf-etl · sidebar_position: 22 · draft: true

What it is

A write-up of the data pipeline behind omars-lab/hifth, a navigation app for huffaz. The interesting problem is not that an ETL exists; it is that the same word is described four times, in four incompatible units, by four groups of people who were each solving a different problem. A printed page vendored as SVG is a few thousand anonymous outlined paths, so the whole pipeline is one job: give those paths names, and never let a name be quietly wrong.

Derived from that repo's own in-repo design doc, rewritten for a reader who has never seen it: four diagrams instead of five, no code pointers, and a Decisions section the source document does not have.

Structure

Opens on users and use cases with a <UseCaseDiagram> (hafiz / contributor / reviewer with a printed copy), then the Scope note, then <!-- truncate -->, then a <RepoPointer>.

Four diagrams: the source-to-shard spine (animated, mermaid-animated flow-dot), the four encodings and the three joins between them, the three tiers of gates, and the one-way boundary where text may enter memory but only numbers may be written.

## Key Decisions carries five calls, each with the cheaper alternative it beat: numbers cross the boundary but text does not; cache-dependent measurements are probes rather than gates; a probe writes a committed pin instead; residuals are named rather than rated; one input is checked against and never built from.

The closer is three things that are not about Arabic or SVG at all. Determinism is not correctness (a look-alike corpus was 47.8% wrong for four development cycles while every structural check passed). A pin file is how you version-control a measurement you cannot reproduce in CI. Name the residual instead of rating it, because a percentage is a budget new failures can be spent from without anyone approving the spend.

Validation

check	result
validate-post-outline	clean
validate-post-naming	clean
validate-questions	clean
validate-links	clean
validate-design-clarity	clean
validate-mermaid-complexity	clean
validate-mermaid-render	4 diagrams render clean and legible
make check	516 MDX files compiled
em-dash	0 literal, 0 —/— dodges
description length	154 chars

Also rendered on the dev server and checked in a browser: HTTP 200, 4 mermaid containers producing 4 SVGs, no syntax-error text, the use-case diagram laid out crossing-free, and the only console errors are the dev-only Cloudflare Access premium fetch. Both the spine and the gates diagram were redrawn from LR to TB after the first render showed long detour edges.

One note for the guidance files, not fixed here: author-post/homes/design.md:75 still recommends — for a dash inside a mermaid label, but author-mermaid/SKILL.md now flags the entity too, since it renders as an em-dash. I followed author-mermaid and rephrased instead.

Left as draft: true per the designs convention, so publish-site un-drafts it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EuhvbUKjGesE3uMhjCzBGt