moodring

What emotions is this text carrying? Multi-label emotion classification over the 28-label GoEmotions taxonomy — a tuned linear model that reaches 96% of the paper's fine-tuned BERT score in 3.8 MB, served three ways: a FastAPI service, a one-file web UI, and a JS port that runs the exact same model in your browser.

▶ Live demo — in-browser inference, no server.

The task

GoEmotions (Demszky et al., ACL 2020) is 58k Reddit comments hand-labeled with 27 emotions plus neutral. It is genuinely multi-label — 1.18 labels per comment on average — and brutally imbalanced: neutral has 14,219 training examples, grief has 77.

The paper's headline baseline is a fine-tuned BERT-base at 0.46 average F1 across the taxonomy. The question this project asks: how close does a well-tuned linear model get, and what do you buy with the 400× size difference?

The model

TF-IDF (word 1–2 grams, sublinear tf, 30k features, min_df 3) into one-vs-rest logistic regression (liblinear, class_weight="balanced"), with per-label decision thresholds tuned on the dev split — a 0.05–0.95 grid maximizing each label's F1. C picked on dev (C=1.0, an interior optimum of a 0.25–4 sweep). Trained on the standard train split only.

model	macro-F1 (test)	size
predict neutral for everything	0.018	—
ours, default 0.5 thresholds	0.432	3.8 MB
ours, per-label thresholds	0.443	3.8 MB
BERT-base fine-tuned (paper, avg F1)	0.46	~440 MB

Micro-F1 is 0.530. Per-label results range from gratitude 0.91 (Reddit says thank you in very learnable ways) down to relief 0.15 (11 test examples).

What threshold tuning actually buys — and what it costs

Tuning thresholds per label is worth +0.012 macro-F1 overall, but the dumbbell below shows the fine print: big wins on mid-frequency labels (optimism 0.46→0.58, fear 0.59→0.66, joy 0.50→0.57) and a loss on the rarest ones — grief falls from 0.43 to 0.20 because its threshold was fit against just ~10 dev positives and doesn't transfer. Tuning on tiny supports is overfitting with extra steps.

The same model, three runtimes

The sklearn pipeline is deliberately JS-portable (default token pattern, sublinear tf, l2 norm — no accent stripping, no char n-grams). scripts/export_js.py writes vocab, idf, coefficients and thresholds into a single 3.8 MB binary, and web/model.js (~80 lines) reimplements the whole pipeline. Parity is tested, not assumed:

parity over 300 real test texts: max |score diff| = 7.2e-08

• FastAPI (moodring/service.py) — model warm-loaded in lifespan, ~10 ms per request
• Browser (the live Space) — a fetch shim answers /predict client-side with the exported binary; "Try a sample" pulls the held-out test split straight from the google-research repo, so no data is redistributed
• Node (web/parity_runner.mjs) — the parity harness the test suite runs

When no label clears its threshold the UI says so instead of pretending confidence — the model abstains and shows the top score in a clearly-marked fallback state:

Run it

make venv        # python3 -m venv + deps (system sklearn reused if present)
make all         # download data → train (~30 s) → figures → JS export+parity → tests
make serve       # UI at http://127.0.0.1:8012

CI runs the 24-test suite offline against a synthetic 28-label fixture model — no data download, and the Python↔JS parity test runs in CI too (GitHub runners ship node).

Layout

moodring/          package: data loading, model, JS export, FastAPI service
moodring/web/      the one-file UI (inline CSS/JS, light+dark, no CDN)
web/               model.js runtime + Node parity runner
scripts/           download / train / figures / export / Space build
tests/             24 offline tests
docs/              figures and screenshots embedded above

Honest limitations

• Single-sentence Reddit English; sarcasm and context-dependent emotion routinely fool a bag of n-grams ("It's wonderful because it's awful" reads as admiration+disgust).
• Rare-label thresholds are unstable (see the grief case above); a production system should pin rare labels to 0.5 or pool them.
• The BERT row cites the paper's number on the same splits, not a rerun; the paper reports average F1 which we compare against macro-F1.

Data: GoEmotions (Apache 2.0), downloaded at build time, never committed. MIT licensed.