fix: re-derive POST day keys from record timestamps at read time (#4168)

.changelog/next/fixed-issue-4168.md

@@ -0,0 +1 @@
+- POST streak/stats/history readers now re-derive each record's day key from its own timestamp in the current timezone, so changing settings.timezone re-keys existing practice history instead of leaving it frozen in the zone it was written in

server/lib/README.md

@@ -303,7 +303,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub
 | `spacedRepetition.js` | The shared SM-2-inspired review scheduler, extracted from `services/meatspacePostMemory.js` when SongBook practice became its second consumer (#4102). One four-field schedule shape (`{ ease, intervalDays, nextReview, lastReviewed }`, no `repetitions` counter — the 0 → 1 → 6 → `round(prev * ease)` ladder derives from the previous interval, so a schedule round-trips through import/export/federation on those four fields alone). `advanceSchedule(schedule, ratio, now)` is the core (ease clamped to `MIN_EASE`..`MAX_EASE`, interval capped at `MAX_INTERVAL_DAYS` so `new Date(now + interval*DAY)` can't overflow into an Invalid Date; quality < 3 zeroes the interval to resurface the record but still applies the ease penalty). `mergeScheduleAdvance(prev, advanced, now)` gates interval GROWTH to once per review day (a shrink always applies) for UIs that submit per chunk/section; `isSameReviewDay` is that gate's own predicate, reusable by callers gating their own once-a-day progression. `scheduleOrDefault(record, schedule?)` is the read-path fallback — a record with no schedule derives one anchored to its `updatedAt`/`createdAt` (stable and in the past → due now), so "due" can't flap between two reads a millisecond apart. `ratioToQuality` / `qualityToRatio` are exact inverses over the integers, so a self-graded 0..5 review lands on the quality the user picked. `isScheduleDue` treats an absent/unparseable `nextReview` as due — a record we can't schedule is one to surface, never one to hide forever. Pure. |
 | `songPractice.js` | SongBook repertoire practice scheduling (#4102) — the pure core behind `POST /api/brain/songbook/:id/practice`. `applySongPractice(song, quality, now)` → `{ stage, practice }`, the exact partial to hand `brainStorage.updateWith`: it advances the shared `spacedRepetition` schedule from a 0..5 self-grade and moves the song's `stage` along `SONG_STAGE_ORDER` (≥ `SONG_PROMOTE_MIN_QUALITY` promotes, ≤ `SONG_REGRESS_MAX_QUALITY` regresses, a 3 holds). Promotion is gated to once per practice day (regression never is), and a practiced song floors at `learning` — `new` means "never picked up", which stops being true after one session. `songPracticeOrDefault(song)` derives the schedule on READ for every song predating the feature (nothing is backfilled to disk: a migration would restamp `updatedAt` on every record and spray federation churn, and would break the `data.reference/` seeds' byte-identity). `nextSongStage` no-ops on a stage this install doesn't recognize, so a practice log can't rewrite a value synced from a newer peer. `isSongDue(song, now)` mirrors the client's `songDueAt` in `client/src/components/songbook/constants.js`. Pure. |
 | `postTopics.js` | Pure POST practice-topic registry — `POST_TOPICS` (`{ id, label, module, surface, drillTypes }`) is the single source of truth for "what am I studying?", plus `resolveTopicForDrillType` / `isTopicEnabled` / `isMemoryItemEnabled`. Gates session composition and recommendations, including Memory's composed and dedicated practice routes plus standalone Morse. Mirrored to the client's post `constants.js` (parity test). |
-| `postStreak.js` | Pure DST-safe POST practice-streak math — the single `computePostStreaks` implementation shared by scored sessions and the training log, plus `computeUnifiedStreak` (a day is active with EITHER a session or a practice entry) and `normalizeYmd` (day-key prefix for a date that may be a full ISO timestamp). |
+| `postStreak.js` | Pure DST-safe POST practice-streak math — the single `computePostStreaks` implementation shared by scored sessions and the training log, plus `computeUnifiedStreak` (a day is active with EITHER a session or a practice entry), `normalizeYmd` (day-key prefix for a date that may be a full ISO timestamp), and `recordDayKey` / `withDerivedDayKeys` (re-derive a record's day key from its `startedAt`/`completedAt`/`timestamp` instant, so readers ignore a `date` frozen in a previously-configured timezone). |
 | `activeDays.js` | Cross-domain "days active" set math (#4120) — `unionActiveDayKeys(sources, timezone)` unions several domains' stored date values into one sorted array of user-local `YYYY-MM-DD` keys (`.length` is the honest day count; summing per-domain counts would double-count any day logged in two domains), and `toUserDayKey(value, timezone)` is its per-value normalizer. Owns the day-boundary reconciliation between POST (stamps `userLocalToday()`) and the health logs (stamp the server-local `getDateString()`): a value carrying an INSTANT (the pre-#2681 full-ISO shape some legacy training entries still store) is re-keyed into the user's timezone rather than `split('T')[0]`'d to the UTC day, while a value that is already a bare day LABEL is taken as authored — there is no instant left to re-derive from, so retro-normalizing stored health keys is out of scope by construction, not by omission. Non-day values are dropped rather than coerced. Pure. |
 | `planIds.js` | Utilities for PLAN.md `[slug]` IDs. |
 | `renderSlot.js` | Render-slot helpers for `(proof\|final)Image` per stage. |

server/lib/postStreak.js

@@ -25,6 +25,74 @@ export function normalizeYmd(value, timezone) {
   return timezone && raw.includes('T') ? toUserDayKey(raw, timezone) : raw.split('T')[0];
 }
 
+// Field order matters: `startedAt` is when the activity actually happened, and
+// `submitPostSession` deliberately preserves it across an idempotent re-submit,
+// so it is the stable anchor. `completedAt` moves on re-submit and `timestamp`
+// is what the training log / Morse rounds carry instead.
+const INSTANT_FIELDS = ['startedAt', 'completedAt', 'timestamp'];
+
+/**
+ * Day key for an INSTANT, in the user's timezone — or null when the value isn't
+ * one. A bare `YYYY-MM-DD` string is a day LABEL, not an instant: it carries no
+ * zone to re-derive from, so `toUserDayKey` takes it as authored rather than
+ * reading it as UTC midnight (which would shift it a day west of UTC).
+ */
+function instantDayKey(instant, timezone) {
+  if (!timezone) return null;
+  if (typeof instant === 'number' && Number.isFinite(instant)) {
+    const at = new Date(instant);
+    // Round-trip through ISO so `toUserDayKey` stays the ONE place that turns an
+    // instant into a user-local day (this module keeps its pure, single-import shape).
+    return Number.isNaN(at.getTime()) ? null : toUserDayKey(at.toISOString(), timezone);
+  }
+  return typeof instant === 'string' ? toUserDayKey(instant, timezone) : null;
+}
+
+/**
+ * The day key a record belongs to, RE-DERIVED from the instant it happened
+ * (`startedAt` / `completedAt` / `timestamp`) rather than read off the stored
+ * `date` (issue #4168).
+ *
+ * A stored `date` is frozen in whatever timezone was configured when it was
+ * written, so once the user changes `settings.timezone` the old keys disagree
+ * with the new-zone readers — a session saved as `2026-07-15` under one zone
+ * reads as "not today" under another. Deriving from the instant makes the
+ * stored `date` a pure cache the readers ignore, which is what the reminder
+ * path (`meatspacePostReminder.js` `isOnLocalDay`) has always done.
+ *
+ * Falls back to the stored `date` only when no usable instant survives (legacy
+ * records written before the timestamps existed) — there is nothing left to
+ * re-derive from there, so it is taken as authored.
+ *
+ * @param {object} record - an activity record (session / training entry / round)
+ * @param {string} [timezone] - user timezone; without it the stored `date` wins
+ * @returns {string|null} a `YYYY-MM-DD` day key, or null when undatable
+ */
+export function recordDayKey(record, timezone) {
+  if (!record || typeof record !== 'object') return null;
+  for (const field of INSTANT_FIELDS) {
+    const derived = instantDayKey(record[field], timezone);
+    if (derived) return derived;
+  }
+  return normalizeYmd(record.date, timezone);
+}
+
+/**
+ * Re-stamp a batch of activity records with their re-derived day key, so every
+ * downstream reader (and the client, which receives these records verbatim)
+ * sees ONE timezone-current `date`. Apply this at the read boundary — the
+ * loaders (`getPostSessions`, `getAllTrainingEntries`) — never on the write
+ * path, which must keep stamping and preserving the original stored value.
+ *
+ * @param {Array} records
+ * @param {string} [timezone]
+ * @returns {Array} records with `date` replaced by the derived key
+ */
+export function withDerivedDayKeys(records, timezone) {
+  if (!Array.isArray(records)) return [];
+  return records.map(r => (r && typeof r === 'object' ? { ...r, date: recordDayKey(r, timezone) } : r));
+}
+
 // Local-date arithmetic on `YYYY-MM-DD` strings via UTC midnight so day math
 // never drifts across DST boundaries (the activity-streak bug class).
 export function ymdToUTC(s) {
@@ -55,13 +123,13 @@ export function ymdShift(s, deltaDays) {
  * @param {string} [timezone] - user timezone for legacy ISO dates
  */
 export function computePostStreaks(records, todayStr, timezone) {
-  const dateSet = new Set((records || []).map(s => normalizeYmd(s?.date, timezone)).filter(Boolean));
+  const dateSet = new Set((records || []).map(s => recordDayKey(s, timezone)).filter(Boolean));
   const dates = Array.from(dateSet).sort();
   const completedToday = dateSet.has(todayStr);
   const lastDate = dates.length ? dates[dates.length - 1] : null;
 
   const todayScores = (records || [])
-    .filter(s => normalizeYmd(s?.date, timezone) === todayStr && typeof s?.score === 'number')
+    .filter(s => recordDayKey(s, timezone) === todayStr && typeof s?.score === 'number')
     .map(s => s.score);
   const todayScore = todayScores.length ? Math.max(...todayScores) : null;
 
@@ -98,9 +166,13 @@ export function computePostStreaks(records, todayStr, timezone) {
  * @param {string} [timezone] - user timezone for legacy ISO dates
  */
 export function computeUnifiedStreak(sessions, trainingEntries, todayStr, timezone) {
+  // Project the day key AND the instants it is re-derived from (#4168) — a bare
+  // `{ date }` projection would strip the timestamps and silently fall the whole
+  // unified streak back onto the stale stored keys.
+  const toActivity = r => ({ date: r?.date, startedAt: r?.startedAt, completedAt: r?.completedAt, timestamp: r?.timestamp });
   const activity = [
-    ...(sessions || []).map(s => ({ date: s?.date })),
-    ...(trainingEntries || []).map(e => ({ date: e?.date })),
+    ...(sessions || []).map(toActivity),
+    ...(trainingEntries || []).map(toActivity),
   ];
   const { currentStreak, longestStreak, lastDate } = computePostStreaks(activity, todayStr, timezone);
   return { current: currentStreak, longest: longestStreak, lastActiveDate: lastDate };

server/lib/postStreak.test.js

@@ -1,5 +1,5 @@
 import { describe, it, expect } from 'vitest';
-import { computePostStreaks, computeUnifiedStreak, ymdShift } from './postStreak.js';
+import { computePostStreaks, computeUnifiedStreak, recordDayKey, withDerivedDayKeys, ymdShift } from './postStreak.js';
 
 const rec = (date, score) => (score == null ? { date } : { date, score });
 
@@ -100,4 +100,66 @@ describe('computeUnifiedStreak (sessions OR training-log activity)', () => {
       current: 0, longest: 0, lastActiveDate: null,
     });
   });
+
+  it('re-derives day keys from record instants, not the stale stored `date` (#4168)', () => {
+    // Written while the user lived in Los Angeles: 2026-07-17 22:00 PDT is
+    // already 2026-07-18 in UTC. After the user moves the setting to UTC, the
+    // frozen stored key would report a gap; the instant says otherwise.
+    const sessions = [
+      { date: '2026-07-16', startedAt: '2026-07-17T05:00:00.000Z', score: 80 },
+      { date: '2026-07-17', startedAt: '2026-07-18T05:00:00.000Z', score: 90 },
+    ];
+    const training = [{ date: '2026-07-15', timestamp: '2026-07-16T05:00:00.000Z' }];
+
+    const inUtc = computeUnifiedStreak(sessions, training, '2026-07-18', 'UTC');
+    expect(inUtc).toEqual({ current: 3, longest: 3, lastActiveDate: '2026-07-18' });
+
+    // Back in the original zone the same instants still key to the stored days.
+    const inLa = computeUnifiedStreak(sessions, training, '2026-07-17', 'America/Los_Angeles');
+    expect(inLa).toEqual({ current: 3, longest: 3, lastActiveDate: '2026-07-17' });
+  });
+
+  it('scores today off the re-derived day, not the stored one (#4168)', () => {
+    const sessions = [{ date: '2026-07-17', startedAt: '2026-07-18T05:00:00.000Z', score: 91 }];
+    expect(computePostStreaks(sessions, '2026-07-18', 'UTC')).toMatchObject({
+      completedToday: true,
+      todayScore: 91,
+    });
+  });
+});
+
+describe('recordDayKey / withDerivedDayKeys (#4168)', () => {
+  it('prefers startedAt, then completedAt, then timestamp', () => {
+    expect(recordDayKey({ date: '2000-01-01', startedAt: '2026-07-18T05:00:00.000Z', completedAt: '2026-07-19T05:00:00.000Z', timestamp: '2026-07-20T05:00:00.000Z' }, 'UTC')).toBe('2026-07-18');
+    expect(recordDayKey({ date: '2000-01-01', completedAt: '2026-07-19T05:00:00.000Z', timestamp: '2026-07-20T05:00:00.000Z' }, 'UTC')).toBe('2026-07-19');
+    expect(recordDayKey({ date: '2000-01-01', timestamp: '2026-07-20T05:00:00.000Z' }, 'UTC')).toBe('2026-07-20');
+  });
+
+  it('falls back to the stored date when no instant survives', () => {
+    // Legacy records predate the timestamps — there is nothing to re-derive from,
+    // so the authored day key stands rather than becoming null.
+    expect(recordDayKey({ date: '2026-07-17' }, 'America/Los_Angeles')).toBe('2026-07-17');
+    expect(recordDayKey({ date: '2026-07-17T22:00:00.000Z' }, 'Asia/Tokyo')).toBe('2026-07-18');
+    expect(recordDayKey({ date: '2026-07-17', startedAt: 'not-a-date' }, 'UTC')).toBe('2026-07-17');
+  });
+
+  it('accepts an epoch-ms instant and rejects a non-record', () => {
+    expect(recordDayKey({ timestamp: Date.UTC(2026, 6, 18, 5) }, 'UTC')).toBe('2026-07-18');
+    expect(recordDayKey(null, 'UTC')).toBeNull();
+    expect(recordDayKey({}, 'UTC')).toBeNull();
+  });
+
+  it('leaves the stored date alone without a timezone', () => {
+    // No zone resolved ⇒ nothing to re-derive INTO. Absent must not collapse into
+    // "UTC" and silently re-key a whole history.
+    expect(recordDayKey({ date: '2026-07-17', startedAt: '2026-07-18T05:00:00.000Z' })).toBe('2026-07-17');
+  });
+
+  it('re-stamps a batch without mutating the inputs', () => {
+    const records = [{ id: 'a', date: '2026-07-17', startedAt: '2026-07-18T05:00:00.000Z', score: 70 }];
+    const derived = withDerivedDayKeys(records, 'UTC');
+    expect(derived[0]).toEqual({ id: 'a', date: '2026-07-18', startedAt: '2026-07-18T05:00:00.000Z', score: 70 });
+    expect(records[0].date).toBe('2026-07-17');
+    expect(withDerivedDayKeys(null, 'UTC')).toEqual([]);
+  });
 });