Emotion Dataset

A local emotion lexicon built from two research datasets, plus a browser viewer for exploring word→emotion profiles. The goal is fast, offline sentiment-style analysis — for example, mapping words in an Unreal Engine room to lighting colors without making web requests.

What’s in this repo

Path	Description
dair/	CARER / Hugging Face six-emotion Twitter dataset (merged_training.pkl)
borealis/	CR4-NarrEmote narrative emotion annotations (CSV)
export_lexicon.py	Builds merged_emotion_lexicon.csv from both sources
merged_emotion_lexicon.csv	Primary output — word lookup table for Unreal / heuristics
emotion_lexicon.csv	Earlier dair-only lexicon (reference)
viewer/	Vanilla JS hexagon visualizer for the lexicon
commonwords.csv, word_bank.csv, word_bank_1.csv	Unreal project word lists
scripts/enrich_from_unreal.py	Proposes lexicon rows for Unreal words → unreal_lexicon_candidates.csv
scripts/merge_unreal_candidates.py	Merges approved candidates into merged_emotion_lexicon.csv
scripts/refresh_unreal_lexicon.py	Strip misclassified connecting words, NRC re-enrich, merge, apply overrides
scripts/download_nrc.py	Download NRC emotion/VAD lexicons into data/nrc/ (not redistributed)
lexicon_overrides.csv	Hand-tuned emotion profiles that override NRC/enrichment (e.g. best, blue)
documentation/dataset-processing.md	How each dataset is processed and merged
documentation/room-light-strategies.md	Room light strategies and emotion color palette
documentation/unreal-sliding-window.md	Unreal Blueprint guide for the sliding window strategy

Quick start

1. Regenerate the lexicon (optional)

Requires Python 3 and pandas:

pip install pandas
python export_lexicon.py

This reads dair/merged_training.pkl and borealis/CR4NarrEmote_t1Yes.csv, then writes merged_emotion_lexicon.csv.

1b. Enrich from Unreal word banks (optional)

Propose lexicon rows for words in commonwords.csv, word_bank.csv, and word_bank_1.csv:

python scripts/enrich_from_unreal.py

Writes unreal_lexicon_candidates.csv for review (words already in the merged lexicon are skipped).

Connecting words are limited to the explicit function-word list in export_lexicon.py (CONNECTING_WORDS, 53 words). Other entries in commonwords.csv are enriched normally — not stamped as flat neutral.

NRC lookup uses files in data/nrc/ when present (download once):

python scripts/download_nrc.py
python scripts/enrich_from_unreal.py

Or pass paths manually:

python scripts/enrich_from_unreal.py \
  --nrc-emotion path/to/NRC-Emotion-Lexicon-Wordlevel-v0.92.txt \
  --nrc-vad path/to/NRC-VAD-Lexicon.txt

Overrides in lexicon_overrides.csv take highest priority (curated VR tone). Add rows with all six emotion scores plus optional VAD columns.

Use --include-neutral to also emit sensory/domain words from neutral columns (body parts, sight, smell, etc.). Review rows where NeedsReview = yes before merging into merged_emotion_lexicon.csv.

Words in the feeling column get a dedicated sentiment pass (feeling-noun priors and lexicon-negation for dis-/un- forms like dissatisfaction). Fuzzy matching is skipped for negated forms so dissatisfaction is not copied from satisfaction.

1c. Merge approved candidates

After reviewing unreal_lexicon_candidates.csv:

python scripts/merge_unreal_candidates.py

This appends new words to merged_emotion_lexicon.csv (skipping any already present) and applies lexicon_overrides.csv. Re-running export_lexicon.py also merges candidates automatically when that file exists.

1d. Refresh after fixing connecting / NRC

To strip misclassified commonwords.csv rows (flat neutral), re-enrich with NRC, merge, and apply overrides in one step:

python scripts/refresh_unreal_lexicon.py --download-nrc

(--download-nrc only needed the first time; NRC files land in data/nrc/ and are gitignored.)

2. Run the viewer locally

The viewer loads the CSV over HTTP. Use the serve script so the lexicon is copied beside the viewer and paths resolve correctly:

.\scripts\serve-viewer.ps1

Then open: http://localhost:8765/

You should see 1685 words (53 connecting) in the footer. If counts look stale, run python scripts/refresh_unreal_lexicon.py.

Alternative (GitHub Pages layout):

.\scripts\build-pages.ps1
python -m http.server 8765 --directory pages

Then open: http://localhost:8765/

Avoid running python -m http.server from inside viewer/ without copying the CSV — Python blocks ../ paths and you may get an old or missing lexicon.

GitHub Pages

The site is published from the pages/ folder (built in CI, not docs/). On push to main, the Deploy GitHub Pages workflow copies viewer/ plus the lexicon CSVs into pages/ and deploys them.

One-time repo setup: Settings → Pages → Build and deployment → Source: GitHub Actions.

Preview the same output locally:

bash scripts/build-pages.sh
python -m http.server 8765 --directory pages

On Windows (PowerShell):

.\scripts\build-pages.ps1
python -m http.server 8765 --directory pages

Then open: http://localhost:8765/

Without a server: open viewer/index.html in a browser and use Load CSV to pick a lexicon file manually.

3. Viewer features

• Grid of hexagon radar charts — one per lexicon word
• Six vertices: Joy, Surprise, Love, Fear, Anger, Sadness
• Vertex distance from center = emotion score (0% at center, 100% at edge)
• Click a word to inspect details; polygon animates between selections
• Search, filter by primary emotion, sort by name / confidence / occurrences

Lexicon format

Each row in merged_emotion_lexicon.csv is one word:

Word,Joy,Sadness,Anger,Fear,Love,Surprise,PrimaryEmotion,Confidence,Occurrences,...
happy,0.9526,0.0207,0.0074,0.0011,0.0038,0.0144,joy,0.9526,3672,...

Import into Unreal as a DataTable, or use the six emotion floats for weighted color blending. Words with borealis data also include Valence, Arousal, and Dominance for finer lighting control.

For emotion hex colors (joy gold, sadness blue, etc.), see documentation/room-light-strategies.md.

For full processing details (regex extraction, NRC remapping, merge logic, filters), see documentation/dataset-processing.md.

Project layout

emotion_dataset/
├── dair/
│   ├── merged_training.pkl
│   └── README.md              # upstream dataset info & citation
├── borealis/
│   ├── CR4NarrEmote_t1Yes.csv # used by export script
│   ├── CR4NarrEmote_All.csv
│   └── CR4NarrEmote_ReadMe.txt
├── documentation/
│   ├── dataset-processing.md
│   ├── room-light-strategies.md
│   └── Emotions.gif           # viewer demo
├── viewer/
│   ├── index.html
│   ├── styles.css
│   └── app.js
├── export_lexicon.py
├── merged_emotion_lexicon.csv
└── emotion_lexicon.csv

Credits

• DAIR dataset: Saravia et al., CARER: Contextualized Affect Representations for Emotion Recognition — see dair/README.md
• Borealis dataset: CR4-NarrEmote — see borealis/CR4NarrEmote_ReadMe.txt

Both datasets are intended for educational and research purposes.