diff --git a/README.md b/README.md index 70a572c..430d209 100644 --- a/README.md +++ b/README.md @@ -4,22 +4,33 @@ [![IBM Bob](https://img.shields.io/badge/Built%20with-IBM%20Bob-1F70C1)](https://ibm.biz/university-bob) [![Watsonx](https://img.shields.io/badge/LLM-IBM%20Watsonx-052e56)](https://www.ibm.com/products/watsonx-ai) [![Python](https://img.shields.io/badge/Python-3.11-3776AB?logo=python&logoColor=white)](https://www.python.org/) -[![Next.js](https://img.shields.io/badge/Next.js-14-000000?logo=nextdotjs&logoColor=white)](https://nextjs.org/) +[![Next.js](https://img.shields.io/badge/Next.js-16-000000?logo=nextdotjs&logoColor=white)](https://nextjs.org/) [![FastAPI](https://img.shields.io/badge/FastAPI-0.115-009688?logo=fastapi&logoColor=white)](https://fastapi.tiangolo.com/) [![Postgres + pgvector](https://img.shields.io/badge/DB-Postgres%20%2B%20pgvector-336791?logo=postgresql&logoColor=white)](https://github.com/pgvector/pgvector) [![AI Builders Challenge July 2026](https://img.shields.io/badge/Challenge-AI%20Builders%20July%202026-FF6F00)](https://aibuilderschallenge-bob.bemyapp.com) +[![Live](https://img.shields.io/badge/Live-quebasto.com-0B7A4B)](https://quebasto.com) > AutorIA learns an author's stylistic DNA from their prior work, generates AI assistance that preserves their voice, and issues a cryptographically signed **Authorship Passport** documenting what was AI, what was human, and what sources were referenced β€” complying with **EU AI Act Article 50**. --- -## 🎬 Demo +## 🎬 Try it live -> _Coming July 31. This section will hold the YouTube video and the public deploy URL._ +| | URL | +| --------------------- | -------------------------------------------------------------- | +| **Live app** | **[https://quebasto.com](https://quebasto.com)** | +| **Passport verifier** | **[https://quebasto.com/verify](https://quebasto.com/verify)** | +| **API docs (local)** | `http://localhost:8000/docs` after `make back` | -- **Live demo**: -- **3-min video**: -- **Authorship Passport verifier**: +**Suggested 90-second walkthrough:** + +1. Open [quebasto.com](https://quebasto.com) β†’ pick **Charles Dickens**. +2. Inspect the **Style DNA** panel (radar + UMAP 2D map + distinctive vocabulary). +3. Prompt e.g. _"Write a paragraph about a foggy London evening in the 1840s"_. +4. Compare **vanilla Llama 3.3** vs **AutorIA (Dickens voice)** β€” fit scores and metrics update side by side. +5. Download the **Authorship Passport** β†’ paste it into [`/verify`](https://quebasto.com/verify) β†’ signature βœ“. + +Preloaded voices: **Jane Austen**, **Charles Dickens**, **Edgar Allan Poe**. You can also upload a new author live (`.txt` / `.md`); the three demo voices are protected from accidental deletion. --- @@ -35,12 +46,29 @@ And starting **August 2026**, [EU AI Act Article 50](https://artificialintellige AutorIA is the authorship layer for AI-assisted creators. Three pieces: -1. **Style DNA Extraction** β€” ingest an author's corpus, extract a quantifiable `StyleProfile` (lexical, syntactic, stylistic, semantic). +1. **Style DNA Extraction** β€” ingest an author's corpus, extract a quantifiable `StyleProfile` (lexical, syntactic, stylistic, semantic + distinctive vocabulary). 2. **Conditioned Generation** β€” given a prompt, generate text that preserves the author's voice (compared side-by-side with the vanilla model output). 3. **Authorship Passport** β€” every generation is bundled with a cryptographically signed JSON manifest, verifiable by anyone with the public key. --- +## βœ… What we shipped (MVP delivered) + +End-to-end product, deployed and demoable: + +| Area | Delivered | +| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| **Corpus & onboarding** | 3 Project Gutenberg authors seeded; live `.txt`/`.md` upload; async chunking + embedding + StyleProfile recompute; delete live-added authors (demo voices protected) | +| **Style DNA** | Full `StyleProfile v1.0`: lexical / syntactic / stylistic features (spaCy), Jeffreys log-odds distinctive vocab, 768-dim semantic centroid, server-side UMAP 2D (auto-recomputed on author add/remove) | +| **Generation** | Parallel vanilla vs conditioned calls on IBM Watsonx `meta-llama/llama-3-3-70b-instruct`; RAG top-k passages from pgvector; calibrated 5-component `fit_score` (0–100) | +| **Passport** | JWS ES256 signed Authorship Passport; public JWKS; download + online `/verify` screen | +| **Studio UI** | Author gallery, Style DNA (radar + scatter + vocab), side-by-side studio with comparative metrics & distinctive-vocab highlights, Passport card | +| **Infra** | Supabase (Postgres + pgvector), Railway (FastAPI + AI pipeline), Vercel (Next.js) β†’ **https://quebasto.com** | +| **Quality** | GitHub Actions CI (lint + tests), pytest + Vitest, OpenAPI contract in `docs/api_contract.yaml` | +| **IBM Bob** | 4 Custom Modes + 26+ BobShell session exports under [`bob/`](bob/) | + +--- + ## πŸ—ΊοΈ Challenge Theme Alignment > _"Reimagine Creative Industries with AI"_ β€” AI Builders Challenge, July 2026. @@ -51,7 +79,7 @@ AutorIA helps individual creators **preserve their authentic voice** when using | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | | **Technical Execution** | Full AI pipeline (spaCy + sentence-transformers + Watsonx) + real cryptographic signing (JWS ES256) + Postgres + pgvector with HNSW indexing | | **Innovation** | The "auditable authorship" layer is novel β€” almost nobody is building this in time for EU AI Act | -| **Feasibility** | Focused MVP, mainstream stack, verifiable demo, clear path to scale | +| **Feasibility** | Focused MVP, mainstream stack, live deploy at [quebasto.com](https://quebasto.com), clear path to scale | | **Challenge Fit** | Solves a concrete, named, urgent problem in a creative industry | | **Real-World Impact** | EU AI Act creates urgent demand (€14B AI-assisted creative market, August 2026 deadline) | @@ -65,8 +93,8 @@ High-level: ```mermaid graph LR - User([Creator]) --> Web[Next.js Frontend] - Web --> API[FastAPI Backend] + User([Creator]) --> Web[Next.js Frontend
quebasto.com] + Web --> API[FastAPI Backend
Railway] API --> Pipeline[AI Pipeline
spaCy + sentence-transformers] API --> Watsonx[(IBM Watsonx
Llama 3.3 70B)] API --> DB[(Postgres + pgvector
Supabase)] @@ -74,19 +102,32 @@ graph LR Signer --> Passport[Authorship Passport] ``` +**Public API surface** (full OpenAPI β†’ [`docs/api_contract.yaml`](docs/api_contract.yaml)): + +``` +GET /api/authors +GET /api/authors/{author_id}/style-profile +POST /api/authors/{author_id}/documents +POST /api/authors/{author_id}/style-profile/recompute +DELETE /api/authors/{author_id} +POST /api/generate +POST /api/passports/verify +GET /.well-known/jwks.json +``` + --- ## 🧬 How It Works β€” AI Pipeline -The `StyleProfile v1.0` captures an author's stylistic DNA across four orthogonal layers: +The `StyleProfile v1.0` captures an author's stylistic DNA across orthogonal layers: -| Layer | Examples | File | -| -------------------------- | ----------------------------------------------------------------- | ------------------------------------------------ | -| **Lexical** | Type-Token Ratio, MATTR-500, hapax ratio, avg word length | `ai_pipeline/autoria_ai/extractor/lexical.py` | -| **Syntactic** | Sentence length distribution, subordination ratio, dep-tree depth | `ai_pipeline/autoria_ai/extractor/syntactic.py` | -| **Stylistic** | Punctuation & POS distribution, discourse markers | `ai_pipeline/autoria_ai/extractor/stylistic.py` | -| **Distinctive Vocabulary** | Top-30 terms vs the other authors, ranked by log-odds-ratio | `ai_pipeline/autoria_ai/extractor/vocabulary.py` | -| **Semantic** | Author centroid (768-dim) + UMAP 2D projection | `ai_pipeline/autoria_ai/embedder.py` | +| Layer | Examples | File | +| -------------------------- | ----------------------------------------------------------------- | ---------------------------------------------------------- | +| **Lexical** | Type-Token Ratio, MATTR-500, hapax ratio, avg word length | `ai_pipeline/autoria_ai/extractor/lexical.py` | +| **Syntactic** | Sentence length distribution, subordination ratio, dep-tree depth | `ai_pipeline/autoria_ai/extractor/syntactic.py` | +| **Stylistic** | Punctuation & POS distribution, discourse markers, dialogue ratio | `ai_pipeline/autoria_ai/extractor/stylistic.py` | +| **Distinctive Vocabulary** | Top-30 terms vs the other authors (Jeffreys log-odds-ratio) | `ai_pipeline/autoria_ai/extractor/vocabulary.py` | +| **Semantic** | Author centroid (768-dim) + UMAP 2D projection | `ai_pipeline/autoria_ai/embedder.py` + `umap_projector.py` | Full feature spec β†’ **[docs/style_features.md](docs/style_features.md)**. @@ -105,7 +146,7 @@ Every generation emits a JSON manifest signed with **JWS (ES256)**, containing: - AI / human contribution percentages - `fit_score` against the target StyleProfile -The signature can be verified **publicly and offline** against the AutorIA public key at `/.well-known/jwks.json` β€” no AutorIA service required. +The signature can be verified **publicly and offline** against the AutorIA public key at `/.well-known/jwks.json` β€” no AutorIA service required. Online verification is also available at **[quebasto.com/verify](https://quebasto.com/verify)**. β†’ Full spec: **[docs/passport_schema.md](docs/passport_schema.md)**. @@ -113,9 +154,7 @@ The signature can be verified **publicly and offline** against the AutorIA publi ## πŸ€– How We Used IBM Bob -> ⚠️ **This is THE most important section for IBM judges.** It will be completed in Sprint 3 (Jul 22–28) with screenshots, metrics, and BobShell exports. - -We built AutorIA in 30 days with IBM Bob as our main copilot. Four Custom Modes β€” one per owner plus a shared crypto mode β€” orchestrated different parts of the development cycle: +We built AutorIA in ~30 days with IBM Bob as our main copilot. Four Custom Modes β€” one per technical pillar β€” orchestrated different parts of the development cycle: | Custom Mode | Purpose | Doc | | ----------------------- | ---------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | @@ -124,44 +163,53 @@ We built AutorIA in 30 days with IBM Bob as our main copilot. Four Custom Modes | **StudioComposer** | Style DNA viz, side-by-side UI, `/verify` screen, API contract alignment, i18n | [`bob/custom-modes/studio-composer.md`](bob/custom-modes/studio-composer.md) | | **PassportAuditor** | Designing and verifying the JWS-signed Passport | [`bob/custom-modes/passport-auditor.md`](bob/custom-modes/passport-auditor.md) | -Weekly BobShell session exports for each team member live in **[`bob/sessions/`](bob/sessions/)** (created on the first Friday of Sprint 1, July 10). - -The complete Bob usage report (metrics + screenshots + analysis) lives in **[`bob/usage-report.md`](bob/usage-report.md)**. +BobShell session exports live in **[`bob/sessions/`](bob/sessions/)** (Sprint 1 + Sprint 2, all three owners β€” **26+ exports**). -Our team's operational playbook for using Bob β€” prompt patterns, export workflow, anti-patterns β€” lives in **[`bob/playbook.md`](bob/playbook.md)**. +The Bob usage report lives in **[`bob/usage-report.md`](bob/usage-report.md)**. Our operational playbook (prompt patterns, export workflow, anti-patterns) is in **[`bob/playbook.md`](bob/playbook.md)**. --- ## βš™οΈ Tech Stack -| Layer | Tech | Why | -| --------------- | ------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------- | -| **Frontend** | Next.js 14 (App Router) + TypeScript + Tailwind + shadcn/ui + Recharts | Modern React, fast static + SSR, great DX for visualizations | -| **Backend** | FastAPI + Python 3.11 + Pydantic v2 + SQLAlchemy 2 + asyncpg | Async by default, type-safe, fits a Python AI pipeline natively | -| **AI Pipeline** | spaCy 3.7 (`en_core_web_lg`) + sentence-transformers (`all-mpnet-base-v2`) + scikit-learn + umap-learn | Industry-standard English NLP; strong 768-dim semantic embeddings; reproducible | -| **LLM** | IBM Watsonx (`meta-llama/llama-3-3-70b-instruct`) β€” sole model in use; `ibm/granite-4-h-small` declared as an unexercised fallback | Every generation runs on IBM Watsonx; strong creative English generation | -| **Database** | PostgreSQL 16 + pgvector (Supabase) | Single DB for relational + vector; HNSW index for fast RAG | -| **Crypto** | python-jose, ES256 (ECDSA P-256) | Standard JWS; small signatures; native browser verification | -| **Hosting** | Vercel (frontend) + Railway (backend) + Supabase (DB) | Zero-ops, free or near-free tiers, push-to-deploy | -| **Dev tools** | IBM Bob + GitHub + GitHub Projects + GitHub Actions + Docker Compose | Bob is mandatory; the rest is best-in-class CI/CD for this size | +| Layer | Tech | Why | +| --------------- | ---------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | +| **Frontend** | Next.js 16 (App Router) + React 19 + TypeScript + Tailwind v4 + shadcn/ui + Recharts | Modern React, fast SSR, strong DX for Style DNA visualizations | +| **Backend** | FastAPI + Python 3.11 + Pydantic v2 + SQLAlchemy 2 + asyncpg / supabase-py | Async by default, type-safe, fits a Python AI pipeline natively | +| **AI Pipeline** | spaCy 3.7 (`en_core_web_lg`) + sentence-transformers (`all-mpnet-base-v2`) + scikit-learn + umap-learn | Industry-standard English NLP; strong 768-dim semantic embeddings; reproducible | +| **LLM** | IBM Watsonx (`meta-llama/llama-3-3-70b-instruct`) β€” sole model in use; `ibm/granite-4-h-small` declared as an unexercised fallback | Every generation runs on IBM Watsonx; honest A/B (same model Β± style conditioning) | +| **Database** | PostgreSQL 16 + pgvector (Supabase) | Single DB for relational + vector; HNSW index for fast RAG | +| **Crypto** | python-jose, ES256 (ECDSA P-256) | Standard JWS; small signatures; native browser verification | +| **Hosting** | Vercel (frontend) + Railway (backend) + Supabase (DB) β†’ **[quebasto.com](https://quebasto.com)** | Zero-ops deploy with custom domain | +| **Dev tools** | IBM Bob + GitHub + GitHub Projects + GitHub Actions + Docker Compose | Bob is mandatory; the rest is best-in-class CI/CD for this size | --- ## πŸš€ Getting Started (Local Setup) -Prerequisites: **Python 3.11**, **Node 20+**, **Docker Desktop**. +> **Judges / reviewers:** the fastest path is the live app at **[https://quebasto.com](https://quebasto.com)**. Use the steps below only if you want to run the full stack on your machine. + +### Prerequisites + +- **Python 3.11** +- **Node 20+** +- **Docker Desktop** (local Postgres + pgvector) +- **Make** (optional but recommended) + - Windows: `winget install GnuWin32.Make`, or use WSL / run the underlying commands from the `Makefile` by hand +- An **IBM Watsonx** API key + project (required for live generation) + +### Quick start ```bash # 1. Clone -git clone https://github.com/sergi-torres/AuthorAI.git +git clone https://github.com/sergi-torres/autorIA.git cd autorIA -# 2. Copy env templates and fill in real values (Watsonx API key, etc.) +# 2. Copy env templates and fill in real values (Watsonx, Supabase / DATABASE_URL) cp .env.example .env -# Next.js reads env from frontend/, not the repo root β€” it needs its own copy: +# Next.js reads env from frontend/, not the repo root: cp frontend/.env.local.example frontend/.env.local -# 3. Install all dependencies +# 3. Install all dependencies (Python editable installs + spaCy model + npm) make install # 4. Generate Authorship Passport signing keypair (one-time) @@ -172,29 +220,47 @@ make db-up # 6. Seed the database: raw text + embeddings + style profiles make seed-full -# ⚠️ This step is slow on first run. It downloads two large ML models: +# ⚠️ Slow on first run β€” downloads ~980 MB of ML models: # β€’ all-mpnet-base-v2 (~420 MB, sentence-transformers) # β€’ en_core_web_lg (~560 MB, spaCy) # If you only need raw text without profiles/embeddings, use `make seed` instead. -# 7. Start backend + frontend (parallel) -make dev -# Frontend: http://localhost:3000 -# Backend: http://localhost:8000 (docs at /docs) +# 7. Start backend and frontend in two terminals +make back # FastAPI β†’ http://localhost:8000 (OpenAPI at /docs) +make front # Next.js β†’ http://localhost:3000 ``` -To run the AI pipeline end-to-end on the seeded corpus without the web stack: +`make dev` starts the DB and prints the same two-terminal reminder. + +### Minimal env checklist + +Fill at least these in `.env` / `frontend/.env.local` (see [`.env.example`](.env.example)): + +| Variable | Where | Purpose | +| -------------------------------------------------------- | --------------------- | ----------------------------------------------- | +| `DATABASE_URL` | root `.env` | Local Docker Postgres or Supabase pooler | +| `SUPABASE_URL` / `SUPABASE_KEY` | root `.env` | Backend DB access (`service_role` locally/prod) | +| `WATSONX_API_KEY` / `WATSONX_URL` / `WATSONX_PROJECT_ID` | root `.env` | Live generation | +| `PASSPORT_*_KEY_PATH` (or `*_PEM`) + `PASSPORT_KID` | root `.env` | Passport signing (`make keys`) | +| `NEXT_PUBLIC_API_BASE_URL` | `frontend/.env.local` | Usually `http://localhost:8000` | +| `AUTORIA_CORS_ORIGINS` | root `.env` | Include `http://localhost:3000` | + +Smoke-check secrets (booleans only): `GET http://localhost:8000/internal/env-check`. + +### Pipeline-only demo (no web stack) ```bash make demo ``` -> The dev `.venv` above is what we use day-to-day; it is **not** a -> container-parity copy of the Railway deploy image, and a few of the ways -> those two diverge are easy to trip over (e.g. a local dependency check that -> looks clean can still be 2+ GB heavier on Railway's Linux target). See -> **[docs/LOCAL_DEV.md](docs/LOCAL_DEV.md)** before trusting a local -> measurement as a stand-in for a deploy one. +### Tests & lint + +```bash +make test +make lint +``` + +> Local `.venv` day-to-day setup is **not** a byte-for-byte copy of the Railway image. Known traps (editable installs across worktrees, torch CPU vs CUDA resolution, lint pin drift) are documented in **[docs/LOCAL_DEV.md](docs/LOCAL_DEV.md)**. Production deploy notes: **[docs/DEPLOYMENT.md](docs/DEPLOYMENT.md)**. --- @@ -202,14 +268,15 @@ make demo ``` autorIA/ -β”œβ”€β”€ ai_pipeline/ # CORE β€” feature extraction, generation, passport (P2 owner) -β”œβ”€β”€ backend/ # FastAPI app + routes + DB layer (P3 owner) -β”œβ”€β”€ frontend/ # Next.js 14 app (P1 owner) +β”œβ”€β”€ ai_pipeline/ # CORE β€” feature extraction, generation, passport (P2) +β”œβ”€β”€ backend/ # FastAPI app + routes + DB layer (P3) +β”œβ”€β”€ frontend/ # Next.js 16 app (P1) β”œβ”€β”€ bob/ # IBM Bob workspace: Custom Modes + sessions + report β”œβ”€β”€ corpus/ # Demo texts (Austen, Dickens, Poe) -β”œβ”€β”€ docs/ # MVP, decision log, architecture, schemas, sprint plan +β”œβ”€β”€ docs/ # MVP, decision log, architecture, schemas, local/deploy guides β”œβ”€β”€ infra/ # Supabase SQL migrations -β”œβ”€β”€ scripts/ # seed, run_demo, generate_keys, etc. +β”œβ”€β”€ scripts/ # seed, run_demo, generate_keys, UMAP precompute, etc. +β”œβ”€β”€ keys/ # Public JWKS sample (private PEMs are gitignored) β”œβ”€β”€ .github/ # CI workflow, PR & issue templates β”œβ”€β”€ docker-compose.yml # Local Postgres + pgvector β”œβ”€β”€ Makefile # Common commands @@ -239,11 +306,11 @@ autorIA/ ## πŸ‘₯ Team -| | Name | Role | GitHub | LinkedIn | -| --- | -------------- | -------------------------------- | ------------------------------------------- | ----------------------------------------------------- | -| P1 | Sergi Torres | Frontend + Pitch + Bob Champion | [sergi-torres](https://github.com/sergi-torres) | [LinkedIn](https://www.linkedin.com/in/storres-dev/) | -| P2 | David MuΓ±oz | AI/ML Engineer | [Davisuco28](https://github.com/Davisuco28) | [LinkedIn](https://www.linkedin.com/in/dmunoz-dev/) | -| P3 | Pablo Chaume | Backend + AI Generation + Crypto | [PabloVc-77](https://github.com/PabloVc-77) | [LinkedIn](https://www.linkedin.com/in/pablo-v-chaume-magraner/) | +| | Name | Role | GitHub | LinkedIn | +| --- | ------------ | -------------------------------- | ----------------------------------------------- | ---------------------------------------------------------------- | +| P1 | Sergi Torres | Frontend + Pitch + Bob Champion | [sergi-torres](https://github.com/sergi-torres) | [LinkedIn](https://www.linkedin.com/in/storres-dev/) | +| P2 | David MuΓ±oz | AI/ML Engineer | [Davisuco28](https://github.com/Davisuco28) | [LinkedIn](https://www.linkedin.com/in/dmunoz-dev/) | +| P3 | Pablo Chaume | Backend + AI Generation + Crypto | [PabloVc-77](https://github.com/PabloVc-77) | [LinkedIn](https://www.linkedin.com/in/pablo-v-chaume-magraner/) | --- diff --git a/bob/usage-report.md b/bob/usage-report.md index fc5d8cc..a4ffc8d 100644 --- a/bob/usage-report.md +++ b/bob/usage-report.md @@ -1,7 +1,7 @@ # How We Used IBM Bob β€” AutorIA Final Report -> **Status**: skeleton. Filled in Sprint 3 (Jul 22-28). -> **Owner**: P1 (Bob Champion). +> **Status**: final (submission week, July 2026). +> **Owner**: P1 (Bob Champion), with evidence from P2 + P3 session exports. This document is the canonical artifact for the **"How we used IBM Bob"** section of the README. The README links here. @@ -9,7 +9,11 @@ This document is the canonical artifact for the **"How we used IBM Bob"** sectio ## Summary - +We treated IBM Bob as a **junior teammate with full-repo context**, not as autocomplete. From day one of Sprint 1 we imported four Custom Modes β€” one per technical pillar (`analyze β†’ generate β†’ present β†’ certify`) β€” and ran almost all non-trivial work inside BobShell with a fixed prompt pattern (CTRO: Context Β· Task Β· Restriction Β· Output). Every Friday (and at issue close) we exported sessions into `bob/sessions/` so judges can audit the trail. + +Bob accelerated the three hardest parts of AutorIA: (1) implementing a locked linguistic `StyleProfile` without drifting from `docs/style_features.md`, (2) building an honest vanilla-vs-conditioned generation path on Watsonx with a measurable `fit_score`, and (3) shipping a JWS ES256 Authorship Passport that verifies offline against JWKS. Frontend work under StudioComposer turned those outputs into a demo a non-expert can read in seconds. + +The operational discipline mattered as much as the models: PR template requires a "How IBM Bob helped" section, Custom Modes load the locked specs, and Decision Log entries record when Bob-assisted work changed a closed algorithm. That combination is what we mean by *exploiting* Bob rather than just *using* it. --- @@ -17,11 +21,13 @@ This document is the canonical artifact for the **"How we used IBM Bob"** sectio | Metric | Target | Final | |---|---|---| -| Total PRs assisted by Bob | β‰₯ 70% of PRs | _(filled Sprint 4)_ | -| Custom Modes created | 4 | _(filled Sprint 4)_ | -| BobShell sessions exported | β‰₯ 12 | _(filled Sprint 4)_ | -| Representative screenshots | β‰₯ 3 | _(filled Sprint 4)_ | -| Spans of focused Bob usage (β‰₯ 30 min) | _(track in dailies)_ | _(filled Sprint 4)_ | +| Total PRs assisted by Bob | β‰₯ 70% of PRs | **~92%** (58 / 63 merged PRs mention Bob / Custom Mode / BobShell in title or body) | +| Custom Modes created | 4 | **4** β€” StyleExtractor, GenerationConductor, StudioComposer, PassportAuditor | +| BobShell sessions exported | β‰₯ 12 | **26** JSON exports under `bob/sessions/` (+ `Sprint_1/baseline_eval.md` voice-gate evidence) | +| Sessions by owner | β€” | P1: 7 Β· P2: 9 Β· P3: 10 | +| Custom Mode usage (from export `modeId`) | β€” | style-extractor 8 Β· generation-conductor 6 Β· studio-composer 6 Β· passport-auditor 2 Β· default agent 2 | +| Representative evidence artifacts | β‰₯ 3 screenshots | **Session exports used as primary audit trail** (PNG gallery not captured; see [Session exports](#session-exports)) | +| Focused Bob spans (β‰₯ 30 min) | track in dailies | Routine: one session per issue / logical task; several multi-hour crypto and Style DNA sessions (see PassportAuditor + StudioComposer below) | --- @@ -29,85 +35,99 @@ This document is the canonical artifact for the **"How we used IBM Bob"** sectio ### StyleExtractor (owner: P2) - +Drove the entire `ai_pipeline/autoria_ai/extractor/*` stack: cleaner/chunker, lexical/syntactic/stylistic features, embeddings, distinctive vocabulary, and the composite `fit_score`. Sessions consistently forced Bob to follow `docs/style_features.md` even when GitHub issue text was stale (e.g. outdated flat weights vs the locked 5-component formula). + +**Concrete win:** implementing `fit_scorer.py` under StyleExtractor with an explicit override β€” *"issue description is OUTDATED; follow `style_features.md` Β§6"* β€” produced a testable scorer (semantic 0.35 / syntactic 0.20 / lexical 0.15 / stylistic 0.15 / vocabulary 0.15) instead of a wrong-but-green implementation of the issue text. + +Representative session: [`bob/sessions/Sprint_2/p2/fit_scorer.json`](sessions/Sprint_2/p2/fit_scorer.json) Β· also [`Sprint_1/P2/lexical.json`](sessions/Sprint_1/P2/lexical.json), [`vocabulary.json`](sessions/Sprint_1/P2/vocabulary.json), [`umap.json`](sessions/Sprint_2/p2/umap.json). ### GenerationConductor (owner: P3) - +Owned conditioned-prompt composition, Watsonx orchestration, RAG wiring, and `POST /api/generate`. The mode's bias toward *measure, don't guess* shaped how we tuned the system prompt (token budget, dialogue/subordination wording, distinctive-vocab injection) and kept the A/B honest: same `meta-llama/llama-3-3-70b-instruct` on both columns. + +**Concrete win:** the conditioner + generate path sessions produced the parallel vanilla/AutorIA response shape the UI depends on, with Passport issuance on the conditioned branch only β€” the demo hinge in under one round-trip. + +Representative session: [`bob/sessions/Sprint_2/P3/api-generate.json`](sessions/Sprint_2/P3/api-generate.json) Β· also [`Sprint_1/P2/conditioner.json`](sessions/Sprint_1/P2/conditioner.json), [`Sprint_1/P3/style_profile_retrieval.json`](sessions/Sprint_1/P3/style_profile_retrieval.json). ### StudioComposer (owner: P1) - +Built the judge-facing surface: author gallery, Style DNA panel (radar + UMAP scatter + metrics), side-by-side studio, comparative metrics, distinctive-vocab highlights, Passport download, and `/verify` layout. Prompts loaded `docs/design-system.md` + `api_contract.yaml` so components stayed contract-typed and i18n-clean (`en.ts` only). + +**Concrete 5-second clarity win:** consolidating Style DNA and generation onto a single `/author/[id]` studio (instead of a separate generate route) removed a navigation hop and put radar, scatter, prompt, and vanilla/AutorIA columns on one screen β€” the layout the demo timeline in `docs/MVP.md` Β§3 actually needs. -### PassportAuditor (owner: P3, pair with P1 on /verify) +Representative session: [`bob/sessions/Sprint_1/P1/style-dna-panel.json`](sessions/Sprint_1/P1/style-dna-panel.json) Β· also [`generate-studio.json`](sessions/Sprint_1/P1/generate-studio.json), [`Sprint_2/P1/verify-passport-screen.json`](sessions/Sprint_2/P1/verify-passport-screen.json), [`comparative-metrics-and-vocab.json`](sessions/Sprint_2/P1/comparative-metrics-and-vocab.json). - +### PassportAuditor (owner: P3, pair with P1 on `/verify`) + +Adversarial crypto mode for JWS ES256 signing, JWKS publication, verification error codes, and payload hashing rules (hashes only β€” never raw prompt/output in the Passport). Plan-first prompts ("list files and risks before writing code") caught key-management footguns early. + +**Concrete security catch:** sessions explicitly forbade generating a new ECDSA keypair on every FastAPI boot (which would silently break every verification) and required offline verification via `PASSPORT_PUBLIC_KEY_PATH` / local JWKS with typed error codes (`invalid_signature`, `unknown_kid`, `unsupported_algorithm`, …). That discipline later paid off when deploy needed PEM-content env vars because `keys/**` is gitignored. + +Representative session: [`bob/sessions/Sprint_2/P3/jwks_verify.json`](sessions/Sprint_2/P3/jwks_verify.json) Β· also [`passport_builder.json`](sessions/Sprint_2/P3/passport_builder.json). --- ## Three problems Bob solved best - - -1. **** - - +1. **Style DNA panel from a locked design system (hours β†’ one focused session)** + Issue #43 asked for radar + UMAP scatter + metric chips inside `/author/[id]`. StudioComposer was given the full design-system + contract constraints up front and delivered typed API helpers, normalization domains, empty/error/loading states, and Recharts v3 components without inventing per-chunk scatter points the contract does not expose. + Evidence: [`sessions/Sprint_1/P1/style-dna-panel.json`](sessions/Sprint_1/P1/style-dna-panel.json). -2. **** - - +2. **Passport builder that cannot leak content** + PassportAuditor planned `builder.py` against schema Β§2–6 before coding: canonical JSON hashing for StyleProfile, `sha256:` for prompt/output/snippets, reuse of existing signer/JWKS β€” and refused scope creep into `/generate` or frontend. The result is a Passport that proves provenance without storing the literary text. + Evidence: [`sessions/Sprint_2/P3/passport_builder.json`](sessions/Sprint_2/P3/passport_builder.json). -3. **** - - +3. **`fit_score` that matches the locked formula, not the stale ticket** + StyleExtractor was pointed at `docs/style_features.md` Β§6 when the GitHub issue still described wrong weights. Bob implemented the five weighted components with clamps, mocks for spaCy/embeddings in tests, and a 0–100 integer output β€” the number the side-by-side UI shows judges. + Evidence: [`sessions/Sprint_2/p2/fit_scorer.json`](sessions/Sprint_2/p2/fit_scorer.json). --- ## Where Bob struggled - +- **Stale issue text vs locked docs.** Bob will faithfully implement the prompt you give it. When a GitHub issue disagreed with `style_features.md` or `api_contract.yaml`, an uncorrected prompt produced the wrong green path. Mitigation: CTRO prompts that name the authoritative doc and say "issue text is wrong if it conflicts." +- **Heavy ML cold starts.** Sessions that touched spaCy / sentence-transformers paid download and load time; we learned to mock models in unit tests and lazy-load embeddings in the API process (see deploy/local-dev work in Sprint 2–3). +- **TF-IDF distinctive vocab looked "done" until measured.** Early extractor sessions shipped a mathematically valid TF-IDF path that collapsed with only three author-documents. Fixing it required a Decision Log ratification (Jeffreys log-odds) β€” Bob drafted candidates, but humans had to measure top-10 overlap on the real corpus before merging. +- **Screenshots as PNGs.** We prioritized BobShell JSON exports (machine-auditable, include `modeId`, costs, and full prompts) over a curated `bob/screenshots/` gallery. Judges should treat the session files below as the primary evidence. --- -## Selected screenshots +## Selected evidence (open these first) - +| # | What to look at | Why it matters | +|---|---|---| +| 1 | [`sessions/Sprint_1/P1/style-dna-panel.json`](sessions/Sprint_1/P1/style-dna-panel.json) | StudioComposer β€” full Style DNA implementation prompt + completed task list | +| 2 | [`sessions/Sprint_2/P3/jwks_verify.json`](sessions/Sprint_2/P3/jwks_verify.json) | PassportAuditor β€” plan-first crypto, offline verify, typed error codes (`modeId: passport-auditor`) | +| 3 | [`sessions/Sprint_2/p2/fit_scorer.json`](sessions/Sprint_2/p2/fit_scorer.json) | StyleExtractor β€” locked `fit_score` formula override (`modeId: style-extractor`) | +| 4 | [`sessions/Sprint_2/P3/api-generate.json`](sessions/Sprint_2/P3/api-generate.json) | GenerationConductor β€” end-to-end generate path | +| 5 | [`sessions/Sprint_1/baseline_eval.md`](sessions/Sprint_1/baseline_eval.md) | Voice-matching gate evidence (vanilla vs conditioned on fixed prompts) | -![Bob in action β€” caption 1](screenshots/bob-01.png) +Custom Mode definitions (imported into each teammate's Bob workspace): -![Bob in action β€” caption 2](screenshots/bob-02.png) +- [`custom-modes/style-extractor.md`](custom-modes/style-extractor.md) +- [`custom-modes/generation-conductor.md`](custom-modes/generation-conductor.md) +- [`custom-modes/studio-composer.md`](custom-modes/studio-composer.md) +- [`custom-modes/passport-auditor.md`](custom-modes/passport-auditor.md) -![Bob in action β€” caption 3](screenshots/bob-03.png) +Operational playbook: [`playbook.md`](playbook.md). --- ## Session exports -All weekly BobShell exports live in `bob/sessions/weekN/.md`. Highlights: +Exports are organized by sprint and owner (not the early `weekN/` sketch). Each `.json` is a raw BobShell export. + +| Sprint | Focus | P1 | P2 | P3 | +|---|---|---|---|---| +| **Sprint 1** | Foundation, extractors, Style DNA UI, API contract paths | `author-selector`, `style-dna-panel`, `generate-studio` (+ fix) | `Cleaner_and_chuncker`, `lexical`, `syntactic`, `stylistic`, `vocabulary`, `embedder`, `conditioner` | `supabase_ini`, `first_api_contract_paths`, `upload_author_docs`, `style_profile_retrieval`, `style_profile_recompute` | +| **Sprint 2** | Generation, Passport, verify UI, scoring, UMAP | `comparative-metrics-and-vocab`, `download-passport`, `verify-passport-screen` | `fit_scorer`, `umap` | `api-generate`, `jwks_verify`, `passport_builder`, `review_issue_22`, `seed_fix` | + +Also: [`Sprint_1/baseline_eval.md`](sessions/Sprint_1/baseline_eval.md) β€” recorded vanilla/conditioned generations for the R1 voice-match gate. -- **Week 1** β€” `bob/sessions/week1/` β€” onboarding + first Custom Modes (Sprint 1) -- **Week 2** β€” `bob/sessions/week2/` β€” StyleProfile pipeline (Sprint 1) -- **Week 3** β€” `bob/sessions/week3/` β€” Watsonx integration + Passport (Sprint 2) -- **Week 4** β€” `bob/sessions/week4/` β€” polish + demo prep (Sprint 3) +**Totals:** 26 BobShell JSON exports Β· 7 P1 + 9 P2 + 10 P3. --- ## Verdict - +We would use IBM Bob again on any project that has **locked specs, multiple owners, and an audit requirement**. Custom Modes were the force multiplier: they kept spaCy work linguistic, generation work empirical, UI work contract-honest, and crypto work adversarial. The habit of exporting BobShell sessions into the public repo made Bob usage *demonstrable* rather than anecdotal β€” which is exactly the bar this challenge sets.