Fine-tune a multimodal Qwen3.8-27B (28B, Apache-2.0) on Modal/GCP free credits so it beats frontier models at AIA G702/G703 pay-application review.
Each month a contractor submits a G702 (application summary) + G703 (schedule of values) with invoices, lien waivers, and change orders. Plumb ingests the documents and emits PASS or line-level exceptions with corrected figures — every math error, retainage slip, double-counted change order, unbacked waiver, and over-certified line.
Third of four parallel repos (specula, suture, plumb,
habeas) on one shared methodology (docs/methodology.md). Independent and
operable in parallel.
- Pure document-to-structured-data + deterministic reconciliation — the exact recipe that beat a frontier model at 1.7B in ReconForge, now 16× bigger. Extraction is the only uncertainty and a verifier catches it.
- Real, dollar-denominated pain. ~15% of pay apps are rejected for math errors (Siteline data); only ~5% of subs are always paid on time; payment delay is the industry's #1 complaint. Per-project, monthly, every tier.
- Competitive gap (research-verified): Siteline/Rabbet/Procore digitize and collect — none adjudicate the app against contract entitlements.
- Perfect-level target is crisp: every check is recomputable arithmetic. The verifier gives dense, deterministic RLVR rewards.
- Ingest G702 + G703 (PDF/scans or Procore/QuickBooks/Sage exports) + the contract SOV, prior approved apps, approved change orders, retainage rules, attached lien waivers.
- Extract every SOV line (% complete, this-period, cumulative, balance) and the G702 summary totals.
- Verify arithmetic; cross-check against contract/current SOV, prior approved cumulative, change-order entitlements, retainage, waivers.
- Emit
PASS | FLAG+{type, severity, line, observed, corrected, citation, correction}. No human reads documents.
See CONTRACTS.md — fixed exception taxonomy + severity weights, scoring,
contamination, benchmark rules.
forge/ seeded pay-app generator + arithmetic/entitlement verifier +
contamination monitor + golden benchmark (pkg plumb_forge)
model/ multimodal dataset builder, QLoRA SFT, GRPO RLVR, benchmark eval
cloud/ Modal app + Dockerfile + GCP spot scripts
eval/ deterministic golden harness
docs/ DECISIONS.md, BENCHMARK.md, HANDOFF.md, methodology.md
Shared stack (docs/methodology.md): QLoRA 4-bit SFT → GRPO/Dr.GRPO
(DAPO-style) RLVR against the pay-app verifier → ReST-EM → s1 curation →
benchmark. Modal your-modal-profile primary, GCP fallback. Smoke LoRA first
(DeltaNet gate).
| Metric | Target |
|---|---|
| Arithmetic-error recall | 100% |
| Severity-weighted exception recall | > 0.95 |
| Precision (no phantom exceptions) | > 95% |
| Corrected-figure exact match | > 95% |
| Parse rate | 100% |
Head-to-head vs Qwen3.8-2.4T-A95B, DeepSeek v4-flash, base Qwen3.8-27B.
P0 scaffold (current) → P1 verifier + hand-verified cases → P2 generator + golden set → P3 dataset + SFT → P4 RLVR → P5 self-play/judges → P6 head-to-head + writeup.
- 50-state retainage/prompt-pay rules → configurable rule registry.
- Scanned/hand-stamped form variance → OCR-noise augmentation.
- Financial-data security → synthetic-only; liability → human sign-off.