Add Real-World Project: Search and Summarize Your Own Journal

docs/projects/index.mdx

@@ -17,6 +17,12 @@ They're optional and ungraded. Browse them any time — each project's intro say
 
 <ProjectChooser
   projects={mergeProjectMeta([
+    {
+      id: 'journal-search-summarize',
+      title: 'Search and Summarize Your Own Journal',
+      summary:
+        'Index a folder of your own dated journal entries with local embeddings, search them semantically — "when did I last mention planning a trip?" — and summarize a date range with a free-tier LLM that cites the dates behind every claim.',
+    },
     {
       id: '2027-dependency-freshness-checker',
       title: 'Build a Dependency-Freshness Checker',

docs/projects/journal-search-summarize/_category_.json

@@ -0,0 +1,4 @@
+{
+  "label": "JournalSearchSummarizer",
+  "position": 26
+}

examples/journal-search-summarize/.env.example

@@ -0,0 +1,26 @@
+# Copy this file to .env (already gitignored) and fill in the key for
+# whichever provider you choose -- you only need ONE of the keys below.
+# Never commit a real key.
+#
+# Which provider to use: github (default), gemini, groq, mistral, cerebras,
+# or openrouter. See main.py's PROVIDERS dict for what each one needs.
+LLM_PROVIDER=github
+
+# github (default) -- a GitHub personal access token with the "models: read"
+# scope. Free, no separate signup: https://github.com/settings/tokens
+GITHUB_TOKEN=
+
+# gemini -- free-tier key from https://aistudio.google.com/
+GOOGLE_API_KEY=
+
+# groq -- free-tier key from https://console.groq.com/keys
+GROQ_API_KEY=
+
+# mistral -- free-tier key from https://console.mistral.ai/api-keys
+MISTRAL_API_KEY=
+
+# cerebras -- free-tier key from https://cloud.cerebras.ai/
+CEREBRAS_API_KEY=
+
+# openrouter -- free-tier key from https://openrouter.ai/keys
+OPENROUTER_API_KEY=

examples/journal-search-summarize/.gitignore

@@ -0,0 +1,6 @@
+.venv
+__pycache__
+*.pyc
+.env
+index.npy
+chunks.json

examples/journal-search-summarize/.python-version

@@ -0,0 +1 @@
+3.12

examples/journal-search-summarize/README.md

@@ -0,0 +1,72 @@
+# Journal Search-and-Summarize Example
+
+The local companion to the course's [Search and Summarize Your Own Journal](../../docs/projects/journal-search-summarize/index.md) project — a real CLI that indexes a folder of dated Markdown journal entries with local embeddings, lets you search them semantically, and summarizes a date range with your choice of free-tier LLM.
+
+## What's here
+
+- `data/journal/` — twelve short, realistic dated journal entries (one `YYYY-MM-DD.md` file per day, spanning July 6–22, 2026), so the tool runs out of the box with no setup.
+- `data/sample_queries.json` — six example queries with notes on which entries hold the ground truth, for the lesson's end-to-end step.
+- `main.py` — a single-file CLI with three commands:
+  - `index` — embeds every entry locally with `sentence-transformers` and saves the vectors (`data/index.npy`) and entry text (`data/chunks.json`) — no API key needed.
+  - `search "<query>"` — finds the entries most relevant to a question using NumPy cosine similarity, and prints each with a score, its date, and a snippet.
+  - `summarize START END` — sends every entry in the date range to a free-tier LLM and prints a **dated summary**: one bullet per event, each starting with the date it came from, plus an audit trail that maps every cited date back to its source file.
+- `notebook.ipynb` — a Colab/Kaggle/Binder-ready notebook that mirrors the same tool end to end, with the sample journal embedded directly in it (no local files needed). Launch it from the badges on the [lesson page](../../docs/projects/journal-search-summarize/index.md#where-to-run-this).
+
+## Running it
+
+```bash
+uv sync
+uv run python main.py index                 # embeds data/journal/ -- no API key, runs locally
+uv run python main.py search "when did I last mention planning a trip?"
+uv run python main.py search "running" --top-k 3
+```
+
+`index` and `search` are fully local and free. Only `summarize` calls a hosted language model, so it needs a free-tier API key:
+
+1. **Get a free-tier API key** from your chosen provider — see the table in the [lesson's Setup section](../../docs/projects/journal-search-summarize/index.md#get-a-free-llm-api-key) for where to get one.
+2. **Copy `.env.example` to `.env`** and fill in the key for your provider (and `LLM_PROVIDER` if you're not using the default):
+   ```bash
+   cp .env.example .env
+   # then edit .env
+   ```
+   `.env` is already gitignored — never commit a real key.
+3. **Run it**:
+   ```bash
+   uv run python main.py summarize 2026-07-06 2026-07-12
+   uv run python main.py summarize 2026-07-13 2026-07-22 --provider groq
+   ```
+
+`uv` reads `pyproject.toml`/`uv.lock` and creates an isolated environment for this project automatically on first run.
+
+### Using your own journal
+
+Replace the files in `data/journal/` with your own `YYYY-MM-DD.md` files, then re-run `uv run python main.py index` — the index only updates when you explicitly rebuild it.
+
+### Browsing the bundled sample queries
+
+```bash
+uv run python main.py search "what did I do for my mother's birthday?"
+uv run python main.py search "when are my friends and I going to Chefchaouen?"
+```
+
+Each entry in `data/sample_queries.json` has a `note` naming the entries that hold the answer, so you can check whether search actually found them.
+
+## Running it in GitHub Codespaces
+
+Click the badge above, or go to the [repo's Codespaces page](https://github.com/abderrahim-lectures/python-data-analysis-course), for a ready-to-go cloud dev environment (Node + Python + `uv` preinstalled via [`.devcontainer/devcontainer.json`](../../.devcontainer/devcontainer.json)). Once it's open:
+
+```bash
+cd examples/journal-search-summarize
+uv run python main.py index
+uv run python main.py search "what is the finance tracker project about?"
+```
+
+(add your API key as a [Codespaces secret](https://docs.github.com/en/codespaces/managing-your-codespaces/managing-encrypted-secrets-for-your-repository-and-organization#adding-secrets-for-a-repository) or `export` it for a one-off session before running `summarize`).
+
+## A note on staying current
+
+Model names and library APIs in this space change fast. `all-MiniLM-L6-v2` and the free-tier endpoints used here were both verified working while writing this example, but may have drifted by the time you read this — see the callout in the [lesson](../../docs/projects/journal-search-summarize/index.md) for what to check before relying on this code.
+
+## Built your own version?
+
+See [`examples/student-projects/`](../student-projects/) for how to share it with the class via a pull request — no git experience required, it walks through every step.

examples/journal-search-summarize/data/journal/2026-07-06.md

@@ -0,0 +1,10 @@
+# Monday, July 6
+
+Back to work after the weekend. Our data team kicked off the quarterly pipeline
+migration today -- my main task for the next few weeks.
+
+After work I started sketching a small personal project: a finance tracker CLI
+that reads my monthly expenses and prints a summary. Nothing fancy, just enough
+to stop doing the math by hand.
+
+Short run in the evening, 3 km around the neighborhood.

examples/journal-search-summarize/data/journal/2026-07-07.md

@@ -0,0 +1,7 @@
+# Tuesday, July 7
+
+Long planning meeting in the morning; nothing decided, lots of diagrams.
+
+During lunch, Yasmine and Omar brought up doing a short trip to Chefchaouen at
+the end of August. We talked about dates, the blue medina, and whether to take
+the bus or the train. No bookings yet, but the idea is on the table.

examples/journal-search-summarize/data/journal/2026-07-08.md

@@ -0,0 +1,7 @@
+# Wednesday, July 8
+
+Spent most of the day debugging an ETL bug in the staging pipeline -- turned out
+to be a timezone offset in the daily aggregation step. Fixed after lunch.
+
+Started the finance tracker project for real: created the repo and wrote the
+first module for recording a single expense (amount, category, date).

examples/journal-search-summarize/data/journal/2026-07-09.md

@@ -0,0 +1,7 @@
+# Thursday, July 9
+
+Good 5k run after work, first time this week I felt fast.
+
+Called mom in the evening. Her birthday is on July 18 and we're planning a family
+dinner at home -- she wants the usual: her cooking, all of us at the table, no
+restaurant.

examples/journal-search-summarize/data/journal/2026-07-10.md

@@ -0,0 +1,7 @@
+# Friday, July 10
+
+Wrapped up the pipeline report and sent it to the team. Feels good to close out
+the week.
+
+Trip planning: compared a few riads in Chefchaouen with Omar. Shortlist is down
+to two -- one in the medina near the main square, one quieter up the hill.

examples/journal-search-summarize/data/journal/2026-07-14.md

@@ -0,0 +1,8 @@
+# Tuesday, July 14
+
+Morning run before work, 5 km. The weather is getting hot, so early runs are the
+way to go.
+
+Finance tracker: added CSV import so I can load the bank export instead of typing
+every expense by hand. Imported June's data and the totals matched, which was
+satisfying.

examples/journal-search-summarize/data/journal/2026-07-15.md

@@ -0,0 +1,6 @@
+# Wednesday, July 15
+
+Long debugging session on the ETL again -- the same staging pipeline, a new
+failure in the deduplication step. Fixed it, but it ate the whole day.
+
+Skipped the gym. Sometimes a rest day wins.

examples/journal-search-summarize/data/journal/2026-07-16.md

@@ -0,0 +1,5 @@
+# Thursday, July 16
+
+Trip is getting real: Yasmine confirmed she and Omar can do August 22-25 for
+Chefchaouen. I checked train times from Casablanca -- the direct route is a few
+hours, and the buses are cheaper but slower. We'll decide together this weekend.

examples/journal-search-summarize/data/journal/2026-07-18.md

@@ -0,0 +1,5 @@
+# Saturday, July 18
+
+Mom's birthday. Family dinner at home as planned -- her cooking, everyone around
+the table: my parents, my two brothers, my sister and her kids. Small gift, big
+cake, a really good evening.

examples/journal-search-summarize/data/journal/2026-07-20.md

@@ -0,0 +1,8 @@
+# Monday, July 20
+
+Shipped the ETL fix to production this morning -- the staging pipeline has been
+stable all day.
+
+Started outlining the August sprint with the team: two more migration tasks and a
+monitoring dashboard. Also kicked off planning the week's meals to avoid the
+takeout spiral.

examples/journal-search-summarize/data/journal/2026-07-21.md

@@ -0,0 +1,7 @@
+# Tuesday, July 21
+
+Morning run again, 5 km, felt great.
+
+Finance tracker reached v0.2: added a monthly summary command that groups
+expenses by category and prints totals. It still can't save my money, but at
+least it shows where it goes.

examples/journal-search-summarize/data/journal/2026-07-22.md

@@ -0,0 +1,7 @@
+# Wednesday, July 22
+
+Booked the riad in Chefchaouen for August 22-24 -- the one in the medina near
+the main square. Two nights, three of us. Train tickets next, then it's just
+packing.
+
+Work was quiet: code review and a short refactor of the report module.

examples/journal-search-summarize/data/sample_queries.json

@@ -0,0 +1,29 @@
+{
+  "description": "Example queries for the sample journal in data/journal/ (2026-07-06 through 2026-07-22). Each query's 'note' names the entries that hold the ground truth, so you can check whether search -- and, for the summarizer, a dated summary -- actually found them.",
+  "queries": [
+    {
+      "query": "when did I last mention planning a trip?",
+      "note": "The Chefchaouen trip planning arc: comparing riads on 2026-07-10, confirming dates on 2026-07-16, and booking the riad on 2026-07-22."
+    },
+    {
+      "query": "how often did I go running in the past two weeks?",
+      "note": "Running appears on 2026-07-06, 2026-07-09, 2026-07-14, and 2026-07-21."
+    },
+    {
+      "query": "what did I do for my mother's birthday?",
+      "note": "Planned on 2026-07-09 and described on 2026-07-18: a family dinner at home with her cooking."
+    },
+    {
+      "query": "what is the finance tracker project about?",
+      "note": "A personal expense-tracking CLI: recording expenses (2026-07-08), CSV import (2026-07-14), and a monthly summary command (2026-07-21)."
+    },
+    {
+      "query": "when are my friends and I going to Chefchaouen?",
+      "note": "August 22-25, confirmed on 2026-07-16; the riad is booked for August 22-24 on 2026-07-22."
+    },
+    {
+      "query": "did I skip any workouts?",
+      "note": "Skipped the gym on 2026-07-15 after a long debugging day."
+    }
+  ]
+}