Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ Our team's operational playbook for using Bob — prompt patterns, export workfl
| **Frontend** | Next.js 14 (App Router) + TypeScript + Tailwind + shadcn/ui + Recharts | Modern React, fast static + SSR, great DX for visualizations |
| **Backend** | FastAPI + Python 3.11 + Pydantic v2 + SQLAlchemy 2 + asyncpg | Async by default, type-safe, fits a Python AI pipeline natively |
| **AI Pipeline** | spaCy 3.7 (`en_core_web_lg`) + sentence-transformers (`all-mpnet-base-v2`) + scikit-learn + umap-learn | Industry-standard English NLP; strong 768-dim semantic embeddings; reproducible |
| **LLM** | IBM Watsonx (Llama 3.3 70B + Granite 3 8B) | End-to-end IBM stack; full-IBM points; strong creative English generation |
| **LLM** | IBM Watsonx (`meta-llama/llama-3-3-70b-instruct`) — sole model in use; `ibm/granite-4-h-small` declared as an unexercised fallback | Every generation runs on IBM Watsonx; strong creative English generation |
| **Database** | PostgreSQL 16 + pgvector (Supabase) | Single DB for relational + vector; HNSW index for fast RAG |
| **Crypto** | python-jose, ES256 (ECDSA P-256) | Standard JWS; small signatures; native browser verification |
| **Hosting** | Vercel (frontend) + Railway (backend) + Supabase (DB) | Zero-ops, free or near-free tiers, push-to-deploy |
Expand Down
8 changes: 4 additions & 4 deletions backend/tests/test_watsonx_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ def test_generate_retries_with_backoff_then_succeeds(monkeypatch):
patch("app.services.watsonx_client.Credentials"),
patch("app.services.watsonx_client.time.sleep", side_effect=sleeps.append),
):
text = generate("prompt", None, "ibm/granite-3-8b-instruct")
text = generate("prompt", None, "meta-llama/llama-3-3-70b-instruct")

assert text == "ok"
assert sleeps == [1.0]
Expand All @@ -118,7 +118,7 @@ def test_generate_exhausts_retries(monkeypatch):
patch("app.services.watsonx_client.time.sleep", side_effect=sleeps.append),
pytest.raises(WatsonxError, match="failed after"),
):
generate("prompt", None, "ibm/granite-3-8b-instruct")
generate("prompt", None, "meta-llama/llama-3-3-70b-instruct")

assert sleeps == list(_RETRY_DELAYS_SECONDS)
assert mock_model.chat.call_count == 1 + len(_RETRY_DELAYS_SECONDS)
Expand All @@ -142,7 +142,7 @@ def test_generate_hard_timeout(monkeypatch):
future = MagicMock()
future.result.side_effect = FuturesTimeoutError()
executor.submit.return_value = future
generate("prompt", None, "ibm/granite-3-8b-instruct")
generate("prompt", None, "meta-llama/llama-3-3-70b-instruct")

assert sleeps == list(_RETRY_DELAYS_SECONDS)
assert future.result.call_count == 1 + len(_RETRY_DELAYS_SECONDS)
Expand All @@ -161,7 +161,7 @@ def test_generate_live_watsonx():
text = generate(
prompt="Reply with exactly one word: pong",
system_prompt="You are a terse assistant. Answer with a single word only.",
model_id="ibm/granite-3-8b-instruct",
model_id="meta-llama/llama-3-3-70b-instruct",
params={"max_tokens": 16, "temperature": 0},
)
assert isinstance(text, str)
Expand Down
14 changes: 8 additions & 6 deletions docs/MVP.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,9 @@ These three decisions shape everything else. They are locked; changing one is a

- **LLM: IBM Watsonx, end-to-end.**
- Primary (creative writing): `meta-llama/llama-3-3-70b-instruct`
- Auxiliary (structured/classification tasks): `ibm/granite-3-8b-instruct`
- Baseline ("vanilla"): the **same** `llama-3-3-70b` but **without** style conditioning.
- Auxiliary: **none**. `ibm/granite-3-8b-instruct` was originally locked here for structured/classification tasks, but it was never wired into any code path and it is **not available to this project**: it is absent from the `eu-de` catalogue and, where it was still listed (`us-south`), its lifecycle is `withdrawn` since 2026-03-31. Revised 2026-07-28 — see `docs/decision_log.md`. Every call the product makes goes to the primary model.
- Escalation backup (R1), **not in use**: `ibm/granite-4-h-small` — confirmed `available` in `eu-de` on 2026-07-28. Nothing calls it today; it exists only as the named fallback if the Sprint 1 voice-match gate fails (see §11 R1). Adopting it would be a separate decision.
- Baseline ("vanilla"): the **same** `llama-3-3-70b` but **without** style conditioning. The escalation backup never touches this comparison.
- *Why same model on both sides:* it's an honest comparison. The only variable is our style conditioning — judges notice and respect that.

- **Language: everything in English — UI, docs, video, and the generated text.**
Expand Down Expand Up @@ -324,8 +325,9 @@ Each is justified so we don't reopen the discussion mid-sprint.

### LLM
- `ibm-watsonx-ai` SDK
- Creative: `meta-llama/llama-3-3-70b-instruct`
- Auxiliary: `ibm/granite-3-8b-instruct`
- Creative: `meta-llama/llama-3-3-70b-instruct` — the only model the product calls
- Auxiliary: none (the originally locked `ibm/granite-3-8b-instruct` is unavailable; see §2)
- Escalation backup, **not wired in**: `ibm/granite-4-h-small` (available in `eu-de`, ratified 2026-07-28). No code path, no env var, no call — only R1's named fallback
- **SPRINT 1 TASK:** validate voice-matching quality with 5 real prompts (does the conditioned output read like the target author?); if Llama < 6/10 human eval, escalate (see R1).

### Database
Expand Down Expand Up @@ -509,9 +511,9 @@ The project is "done" when **all** of these are true:

| # | Risk | Mitigation |
|---|---|---|
| **R1** | Llama-3.3-70b doesn't convincingly match a target voice | Validate Sprint 1 day 2. If <6/10, plan B: stronger conditioning / more RAG passages, `llama-3-1-405b`, or `granite-3-8b`; last resort Mistral Large via Watsonx. |
| **R1** | Llama-3.3-70b doesn't convincingly match a target voice | Validate Sprint 1 day 2. If <6/10, plan B: stronger conditioning / more RAG passages first. **Model-swap escalation: `ibm/granite-4-h-small`** — ratified 2026-07-28, confirmed `available` in `eu-de` that day. The three backups this row used to name (`llama-3-1-405b`, `granite-3-8b`, Mistral Large) are all unreachable from `eu-de` and were removed. The escalation is **declared, not exercised**: nothing calls Granite 4 today, and it never becomes the vanilla baseline — the A/B stays `llama-3-3-70b` vs itself. |
| **R2** | Stylistic metrics don't distinguish authors | Validate Sprint 1 with a blind A/B. If Austen ≈ Dickens (both 19thc British), add bigram/trigram distinctive features. (Poe is the easy separation.) |
| **R3** | Generation >15s, demo doesn't flow | P3 measures latency Sprint 2 day 1. If >10s: switch parallel→sequential with optimistic loading, and/or use `granite-8b` for the baseline (faster). |
| **R3** | Generation >15s, demo doesn't flow | P3 measures latency Sprint 2 day 1. If >10s: switch parallel→sequential with optimistic loading, shorten `max_tokens`, or fall back to the pre-recorded generations of R5. **R3 has no model-swap plan B and no replacement for one.** The old "use a faster `granite-8b` for the baseline" is void twice over: that model does not exist for us (see §2), *and* swapping only the baseline would break the A/B whatever model replaced it — the comparison is only honest while both columns run the same model. R1's `ibm/granite-4-h-small` is **not** an R3 mitigation for the same reason. The three levers above are the whole of R3. |
| **R4** | Live demo fails during recording | Sprint 3: record each step separately as an editable fallback. Final video may mix live + pre-recorded. |
| **R5** | Watsonx rate-limit / load spike | Exponential backoff (Sprint 2). For the demo, pre-record two famous generations as a local cache fallback. |
| **R6** | A teammate is sick for a week | Each role has a designated backup; pair on critical pieces; the async daily keeps the backup current. |
9 changes: 5 additions & 4 deletions docs/ONBOARDING.md
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ Lives at **`docs/decision_log.md`**. 4-column table:
Read them in `docs/decision_log.md`. The key ones to keep in your head:

- Idea = **AutorIA** (auditable authorship layer)
- LLM = **IBM Watsonx end-to-end** (Llama 3.3 70B + Granite 3 8B auxiliary)
- LLM = **IBM Watsonx end-to-end** (`meta-llama/llama-3-3-70b-instruct` is the only model we call; the Granite 3 8B auxiliary was dropped 2026-07-28 — withdrawn upstream, absent from `eu-de`. `ibm/granite-4-h-small` is the *declared, unused* R1 escalation backup, not part of the active path)
- **English everywhere** — app UI, repo, README, video, commits, and the generated text. Only the **internal team chat** stays Spanish.
- Multi-author with **3 preloaded**: Jane Austen, Charles Dickens, Edgar Allan Poe
- Stack: Next.js + FastAPI + Postgres+pgvector (Supabase) + JWS ES256
Expand Down Expand Up @@ -326,7 +326,7 @@ Full document at **`docs/MVP.md`**. Executive summary:

| Decision | Value | Why |
|---|---|---|
| LLM | **IBM Watsonx end-to-end**. Primary: `meta-llama/llama-3-3-70b-instruct`. Auxiliary: `ibm/granite-3-8b-instruct` | IBM challenge = points for full-IBM stack |
| LLM | **IBM Watsonx end-to-end**. Primary and only model in use: `meta-llama/llama-3-3-70b-instruct`. No auxiliary model (`ibm/granite-3-8b-instruct` was revised out on 2026-07-28 — unavailable in `eu-de`, withdrawn upstream). `ibm/granite-4-h-small` is the ratified R1 escalation backup — declared, never called | IBM challenge = points for running the whole product on Watsonx |
| Comparison baseline | **Same model WITHOUT style conditioning** | Honest comparison = judges notice |
| App UI + video + generated text | **English** | International IBM jury; judges can read the voice match directly |
| Authors | **Austen, Dickens, Poe** (English, public domain) | Iconic, maximally distinct voices; instantly recognizable |
Expand Down Expand Up @@ -370,7 +370,8 @@ tiktoken (approx chunking)
```
ibm-watsonx-ai SDK
Creative model: meta-llama/llama-3-3-70b-instruct
Auxiliary model: ibm/granite-3-8b-instruct
Auxiliary model: none (Granite 3 8B revised out 2026-07-28 — unavailable in eu-de)
R1 escalation backup (declared, not called): ibm/granite-4-h-small
Sprint 1 task: validate voice-matching quality with 5 prompts (gate)
```

Expand Down Expand Up @@ -961,7 +962,7 @@ On Sunday, unfinished issues either move to "Sprint N+1" or back to "Backlog" if
Your role has a backup designated (P1→P3, P2→P3, P3→P2). The daily async ensures your backup knows the state. Pair sessions ensure >1 person has touched critical pieces. Not ideal but the project doesn't collapse.

### "What if Watsonx doesn't match the author's voice well?"
Plan B in MVP: stronger conditioning / more RAG passages, then `llama-3-1-405b-instruct` or `granite-3-8b-instruct` via Watsonx. If all fail, fallback to Mistral Large via Watsonx. **But**: we validate this Sprint 1 day 2, not Sprint 4.
Plan B in MVP: stronger conditioning / more RAG passages first, then swap the model to **`ibm/granite-4-h-small`** (ratified 2026-07-28, confirmed available in `eu-de`). The escalation chain this FAQ used to name — `llama-3-1-405b-instruct`, `granite-3-8b-instruct`, Mistral Large — was fiction: none of the three is in the `eu-de` catalogue we run against, and Granite 3 8B is withdrawn upstream everywhere. Granite 4 H Small is a *declared* backup: we do not call it today, and it never replaces the vanilla baseline (that would break the A/B). **But**: we validate this Sprint 1 day 2, not Sprint 4.

### "What if a seemingly essential feature emerges that's not in the MVP?"
5-min call with all 3, 2/3 vote. If it passes → Decision Log entry + add. If not → `docs/roadmap.md` for "post-July".
Expand Down
9 changes: 6 additions & 3 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ graph TB
sys[["AutorIA<br/>Style DNA · Conditioned Generation ·<br/>Authorship Passport"]]
end

watsonx[("IBM Watsonx<br/>Llama 3.3 70B + Granite 3 8B<br/>[external LLM]")]
watsonx[("IBM Watsonx<br/>Llama 3.3 70B<br/>[external LLM]")]
gutenberg[("Project Gutenberg<br/>public-domain corpus<br/>[external source]")]

creator -->|"selects author, prompts,<br/>views side-by-side, downloads Passport"| sys
Expand All @@ -55,8 +55,11 @@ graph TB
audience). Only needs the token + public JWKS.

**External dependencies**
- **IBM Watsonx** — the LLM provider (creative + auxiliary models). The only
hard runtime external dependency for generation.
- **IBM Watsonx** — the LLM provider. A single model, `meta-llama/llama-3-3-70b-instruct`,
serves both branches of the side-by-side (conditioned and vanilla); there is no
auxiliary model. `ibm/granite-4-h-small` is the declared R1 escalation backup and is
never called on the active path (MVP §2 / §11). The only hard runtime external
dependency for generation.
- **Project Gutenberg** — source of the three preloaded corpora; used **offline**
during seeding, not at request time.

Expand Down
Loading
Loading