Skip to content

fix(reports): cut calendar days on the active time zone, not UTC - #2403

Open
vecchp wants to merge 2 commits into
mainfrom
DEV-warnings/report-timezones
Open

vecchp wants to merge 2 commits into
mainfrom
DEV-warnings/report-timezones

Conversation

@vecchp

@vecchp vecchp commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

This changes report numbers at month boundaries, and moves when the monthly report is sent. Anyone reconciling a new export against a previously-sent one will see small differences. Worth a heads-up to whoever owns reporting before this ships.

The bug

Calendar days were cut on UTC, in three places that have to agree and didn't.

Report ranges. Bare date objects handed to interacted_at filters were coerced to naive midnight and read with settings.TIME_ZONE, so every range ran UTC midnight to UTC midnight — Dec 31 4pm → Jan 31 4pm in Los Angeles. A note logged at 5pm on 31 January was counted in February's report, and the CSV labelled it 02/01/2025. That coercion also raised the ~40 RuntimeWarning: received a naive datetime warnings this PR set out to clear.

The schedule. ScheduledReport.hour was read on UTC's calendar, so the one active report — due on the 1st at hour 0 — fired at 00:00 UTC, which is 5pm on the last day of the month in Los Angeles. It went out before the month it reports on had ended locally.

The period. send_scheduled_report derived its month from timezone.now(), so a schedule due 1 September but retried on 2 October emailed September, with a September subject, and August was never sent at all.

The fix: let Django resolve the calendar

USE_TZ = True is what keeps storage UTC. TIME_ZONE is only the calendar to use when nobody said otherwise — we had been reading it as the former, which is why the zone looked like something we had to carry by hand.

USE_TZ = True                    # unchanged
TIME_ZONE = env("TIME_ZONE")     # was "UTC", now defaults to America/Los_Angeles

common/middleware/timezone.py was already in MIDDLEWARE, already reading a django_timezone cookie, already calling timezone.activate() — and only templates/admin/base.html ever set that cookie. createWebFetchClient now sets it too, which is the entire frontend change.

With the zone activated at the boundary, every conversion resolves itself:

call resolves against
TruncDate("interacted_at") get_current_timezone_name()
timezone.make_aware(...) the active zone
timezone.localdate() / localtime() the active zone
timezone.get_default_timezone() settings.TIME_ZONE, ignoring the active zone

SHELTER_SCHEDULE_TIME_ZONE is deleted — the one hardcoded zone in application code this touches. hmis/api_bridge.py's LOS_ANGELES_TZ stays: it is the zone the HMIS vendor emits its naive timestamps in, a property of their system rather than of our deployment, and it must not follow TIME_ZONE.

Three decisions worth checking

set_next_run ignores the request and uses get_default_timezone(). It runs from an admin save and from Celery after a send, so following the active zone would let a report set to 8am by a remote admin drift to 8am here on its first reschedule. A schedule fires once, globally — it has no viewer. Pinned by test_the_schedule_ignores_the_timezone_of_whoever_saved_it.

Shelters stay pinned, openNow and occupancy alike: a shelter is open on its own clock whoever is looking it up. That is also why the Expo app does not publish the cookie — it runs no reports queries, and this is the one mobile-facing thing that reads a calendar.

The CSV date column follows the viewer. NoteResource.dehydrate_interacted_at goes from note.interacted_at.date() to timezone.localtime(note.interacted_at) — the viewer's zone on a request, settings.TIME_ZONE in Celery. One class serves both audiences.

Why the boundary and the firing time move together

Fixing only the ranges is worse than fixing neither:

fires at 2026-09-01 00:00 UTC = 2026-08-31 17:00 Pacific
A. today, UTC boundaries:                       (2026-08-01, 2026-08-31)  right month, misses Aug 31 evening
B. local boundaries, firing unchanged:          (2026-07-01, 2026-07-31)  re-sends July, August never sent
C. local boundaries + firing at local midnight: (2026-08-01, 2026-08-31)  right, and complete

Case B skips a month, so a data migration recomputes next_run_at for active reports — set_next_run only corrects itself after a send, which is one wrong-month email too late. The period itself now comes from the run being serviced rather than the wall clock, so a late, early or retried run all produce the same month.

Verified rather than assumed

  • Storage is unaffected. connection.timezone_name returns "UTC" whenever USE_TZ (django/db/backends/base/base.py:162), confirmed under an override.
  • GraphQL output is byte-identical. Strawberry serializes with a bare .isoformat() on the UTC-aware value, never localtime.
  • Celery Beat does not move. app.conf.timezone == "UTC", enable_utc == True, no CELERY_TIMEZONE, and Celery's Django fixup never reads settings.TIME_ZONE. crontab(hour=7) stays where it was.
  • Nothing writes a naive datetime, so the flip cannot reinterpret stored values: both frontends use .toISOString(), hmis/api_bridge.py sets tzinfo explicitly, and every other strptime yields a date/time.
  • EXPLAIN: the half-open window keeps the notes_note_interacted_at index, where interacted_at__date would have cost a Seq Scan. That is why note_list_for_org builds a window instead of using the __date lookup.

Tests

1561 passed, 31 skipped, and zero RuntimeWarning: received a naive datetime. The new coverage exercises the mechanism end-to-end — a request carrying django_timezone through both the GraphQL resolver and the DRF export — rather than the selector in isolation. Every new assertion was mutation-tested: revert the behaviour, confirm it fails, restore.

Five tests outside reports/ were asserting UTC results from naive time_machine.travel(...) strings, silently riding on the process TZ. Their travel targets are now explicit UTC rather than worked around.

Known, not fixed here

  • DateCountType.date is a bare String!, so a bucketed date reaches the client with no indication of which calendar produced it — and that calendar is now per-viewer. Harmless while every org is in LA County; the fix when it matters is an optional timeZone argument the response echoes back.
  • clients/models.py computes age from timezone.now().date(), i.e. UTC's calendar — a latent off-by-one for seven hours a day. timezone.localdate() is the fix.
  • celery.py's crontab(minute=0, hour=7), # 7 AM UTC, corresponds to midnight PT is only true during daylight saving.

Summary by Sourcery

Align reporting boundaries, exports, and monthly schedules with the appropriate local calendar instead of UTC.

New Features:

  • Use the active browser or site calendar for report date ranges, day buckets, CSV dates, and monthly report periods.
  • Publish the browser time zone to the backend so web report views follow the viewer’s calendar.

Bug Fixes:

  • Correct monthly report scheduling and period selection across local month boundaries, including late or retried sends.
  • Prevent report notes near midnight from being assigned to the wrong calendar day or month and eliminate naive datetime handling in these paths.

Enhancements:

  • Centralize calendar resolution through Django’s time-zone support while keeping scheduled reports and shelter metrics pinned to the site time zone.
  • Preserve indexed note filtering with half-open, time-zone-aware datetime windows.
  • Reschedule active reports during migration so their next runs align with the site calendar.

Tests:

  • Add end-to-end coverage for time-zone-aware GraphQL summaries, DRF exports, CSV dates, scheduling, late runs, and browser time-zone synchronization.
  • Make time-travel test fixtures explicitly UTC where they previously relied on process time-zone behavior.

@sourcery-ai

sourcery-ai Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Reporting and export boundaries now follow each organization’s configured IANA timezone, using aware half-open datetime filters and timezone-aware aggregation/date rendering; month defaults and scheduled reports resolve ranges from the organization’s local calendar, with Los Angeles retained as the fallback.

Sequence diagram for organization-local report boundaries

sequenceDiagram
    participant Caller as Report caller
    participant View as Report view or resolver
    participant Resolver as organization_time_zone
    participant Selectors as Report selectors
    participant DB as Note database
    participant Export as NoteResource

    Caller->>View: Request report range
    View->>Resolver: organization_time_zone(organization)
    Resolver-->>View: Organization ZoneInfo or default
    View->>Selectors: note_list_for_org(start_date, end_date, tz)
    Selectors->>DB: Filter interacted_at >= local midnight
    Selectors->>DB: Filter interacted_at < next local midnight
    DB-->>Selectors: Matching notes
    Selectors->>DB: TruncDate(interacted_at, tz=tz)
    Selectors-->>View: Report data
    View->>Export: NoteResource(tz=tz)
    Export-->>Caller: CSV dates rendered in organization timezone
Loading

Flow diagram for local month range selection

flowchart TD
    Now[Current instant] --> LocalDate[Convert now to organization timezone]
    LocalDate --> Month[Select current or previous local calendar month]
    Month --> Dates[Inclusive start and end dates]
    Dates --> Bounds[Build aware half-open datetime range]
    Bounds --> Query[Filter organization notes]
    Query --> Buckets[Aggregate dates with TruncDate using organization timezone]
Loading

File-Level Changes

Change Details Files
Add organization-configurable reporting time zones with a Los Angeles fallback.
  • Add the IANA timezone field, migration, admin form exposure, and validation.
  • Resolve a profile timezone for configured organizations and default unconfigured organizations safely.
apps/betterangels-backend/accounts/forms.py
apps/betterangels-backend/accounts/migrations/0008_organizationprofile_timezone.py
apps/betterangels-backend/accounts/models.py
apps/betterangels-backend/accounts/tests/test_admin.py
apps/betterangels-backend/common/constants.py
Make report filters, date aggregations, defaults, and scheduled ranges use organization-local calendar days.
  • Convert inclusive date inputs into aware, half-open datetime bounds in the organization timezone.
  • Pass timezone explicitly through GraphQL, DRF, and scheduled-report paths.
  • Apply timezone-aware TruncDate grouping and derive current/previous month ranges from the organization-local date.
  • Simplify previous-month APIs to return inclusive date pairs.
apps/betterangels-backend/reports/schema.py
apps/betterangels-backend/reports/selectors.py
apps/betterangels-backend/reports/services.py
apps/betterangels-backend/reports/tasks.py
apps/betterangels-backend/reports/views.py
apps/betterangels-backend/reports/tests/test_services.py
Ensure exported interaction dates reflect the report organization's calendar rather than UTC.
  • Allow NoteResource to receive an explicit timezone for scheduled and report exports.
  • Render interacted_at labels after converting timestamps to that timezone.
  • Preserve request-local timezone behavior for unparameterized admin exports.
apps/betterangels-backend/notes/admin.py
apps/betterangels-backend/reports/services.py
apps/betterangels-backend/reports/views.py
Keep client active-status comparisons at aware datetime precision.
  • Compare interacted_at against an aware datetime threshold instead of a naive date.
apps/betterangels-backend/clients/types.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="apps/betterangels-backend/reports/tasks.py" line_range="54" />
<code_context>

     # Calculate the date range for the previous month
-    start_date, end_date = get_previous_month_range()
+    start_date, end_date = get_previous_month_range(tz=organization_time_zone(report.organization))
     month_str = start_date.strftime("%m")
     year_str = start_date.strftime("%Y")
</code_context>
<issue_to_address>
**issue (bug_risk):** A scheduled report that runs at or just after 00:00 UTC on the first day of a month can select the month before the immediately preceding calendar month for an America/Los_Angeles organization. For example, at 00:00 UTC on January 1 the organization's local date is still December 31, so the function reports November instead of December.

**Triggers:** When the Celery schedule fires before the organization's local month has rolled over.

**Suggested fix:** Base the scheduled report period on the intended scheduled run month, or schedule/guard execution using each organization's local timezone rather than deriving the period from the task's current UTC instant.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread apps/betterangels-backend/reports/tasks.py Outdated
@vecchp
vecchp force-pushed the DEV-warnings/report-timezones branch from 6511ba7 to 9f6c400 Compare August 31, 2026 20:36
@vecchp
vecchp force-pushed the DEV-warnings/attachment-save-deprecation branch from 9e6a8a2 to b3e85c7 Compare August 31, 2026 21:05
@vecchp
vecchp force-pushed the DEV-warnings/report-timezones branch from 9f6c400 to 69df125 Compare August 31, 2026 21:10
@vecchp
vecchp force-pushed the DEV-warnings/attachment-save-deprecation branch from b3e85c7 to 2b78694 Compare August 31, 2026 22:09
@vecchp
vecchp force-pushed the DEV-warnings/report-timezones branch from 69df125 to f0f5a36 Compare August 31, 2026 22:09
@vecchp
vecchp force-pushed the DEV-warnings/report-timezones branch from f0f5a36 to 642496a Compare August 31, 2026 22:32
@vecchp vecchp changed the title fix(reports): cut report boundaries on the organization's calendar days fix(reports): cut report boundaries and schedules on the operating time zone Aug 31, 2026
@vecchp
vecchp force-pushed the DEV-warnings/report-timezones branch from 642496a to 5ad17a3 Compare August 31, 2026 22:56
@vecchp
vecchp force-pushed the DEV-warnings/report-timezones branch from 5ad17a3 to d7d311d Compare August 31, 2026 23:34
@vecchp
vecchp force-pushed the DEV-warnings/report-timezones branch from d7d311d to 036347b Compare August 31, 2026 23:43
@vecchp
vecchp force-pushed the DEV-warnings/attachment-save-deprecation branch from 2b78694 to ca8f224 Compare September 1, 2026 00:13
@vecchp vecchp changed the title fix(reports): cut report boundaries and schedules on the operating time zone fix(reports): cut calendar days on the active time zone, not UTC Sep 1, 2026
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

🚀 Expo continuous deployment is ready for betterangels!

  • Project → betterangels
  • Environment → Preview
  • Platforms → android, ios
  • Scheme → betterangels
  🤖 Android 🍎 iOS
Runtime Version 4d84f3f87051ebb9c1713e7e2d4522e4b23135d9 4d84f3f87051ebb9c1713e7e2d4522e4b23135d9
Build Details Build Permalink
DetailsDistribution: INTERNAL
Build profile: preview
Runtime version: 4d84f3f87051ebb9c1713e7e2d4522e4b23135d9
App version: 1.2.11
Git commit: a1900145e6c2315da5a85930acd4fcf2f8619aa2
Build Permalink
DetailsDistribution: INTERNAL
Build profile: preview
Runtime version: 4d84f3f87051ebb9c1713e7e2d4522e4b23135d9
App version: 1.2.11
Git commit: a1900145e6c2315da5a85930acd4fcf2f8619aa2
Update Details Update Permalink
DetailsBranch: DEV-warnings-report-timezones
Runtime version: 4d84f3f87051ebb9c1713e7e2d4522e4b23135d9
Git commit: 99c92c142e25ab345635f5cbbeea0e97d2969b00
Update Permalink
DetailsBranch: DEV-warnings-report-timezones
Runtime version: 4d84f3f87051ebb9c1713e7e2d4522e4b23135d9
Git commit: 99c92c142e25ab345635f5cbbeea0e97d2969b00
Update QR

iOS Simulator Build: Simulator Build Link

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

🔍 [shelter-web] Preview available at: https://shelter.dev.betterangels.la/branches/DEV-warnings-report-timezones

Last updated: 2026-09-01T19:53:59.271Z

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

🔍 [betterangels-admin] Preview available at: https://admin.dev.betterangels.la/branches/DEV-warnings-report-timezones

Last updated: 2026-09-01T19:53:59.268Z

@vecchp
vecchp force-pushed the DEV-warnings/attachment-save-deprecation branch from ca8f224 to 4f0643e Compare September 1, 2026 15:47
paul-betterangels and others added 2 commits September 1, 2026 19:20
…me zone

Report ranges were built on UTC calendar days, so "January" ran from 4pm on
31 December to 4pm on 31 January in Los Angeles. A note logged on the last
evening of a month fell into the next month's report. In production 9 of 3,850
live notes sit in that window, all on the last evening of a month.

The monthly schedule had the same fault. `hour` was documented as UTC, so a
report due on the 1st fired at 00:00 UTC — 5pm on the last day of the month
locally — and missed everything logged that evening. `set_next_run` now places
`day_of_month`/`hour` on the operating zone's calendar, and a data migration
recomputes `next_run_at` for active reports.

Separately, when a report ran decided what it contained. `send_scheduled_report`
derived its month from `timezone.now()`, so a schedule due 1 September but
retried on 2 October emailed September — with a September subject — and August
was never sent. The period now comes from the run being serviced, and
`next_run_at` is recomputed from `day_of_month` rather than by adding a month to
a stored instant, so a late, early or retried run all produce the same month.

Exports are split by who they are for. The admin's `NoteResource` labelled rows
with `interacted_at.date()` — the UTC date, ignoring every zone — and now uses
`timezone.localtime`, so it follows the zone `TimezoneMiddleware` activates from
the browsing user's cookie, which is what someone downloading from the admin
wants. A report is a record rather than a view, so `ReportNoteResource` labels
rows on the same calendar the range was cut on; otherwise the same month exports
as two different files depending on who asked, and neither matches the emailed
copy.

The zone travels inside the values rather than beside them. `local_window`
widens an inclusive pair of dates into the aware, half-open range the ORM
filters on, and `note_list_for_org` takes that range — the same shape
`shelters.selectors.reports.daily_occupancy` already uses. Nothing passes a
`tzinfo` argument.

`SHELTER_SCHEDULE_TIME_ZONE` was already this value under a name describing one
of its callers, so it moves to `common.constants.OPERATING_TIME_ZONE` and
reports uses the same one — two hardcoded zones become one. `hmis`'s
`LOS_ANGELES_TZ` stays: it is the vendor API's convention for parsing their
naive datetimes, not our operating zone.

`TIME_ZONE` remains UTC. Storage and display are unaffected — datetimes are
stored and served in UTC and localized per viewer by each client. Only the
boundaries of an aggregate are local, because those decide which rows are
counted rather than how an instant is shown.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…a constant

Replaces the hardcoded `OPERATING_TIME_ZONE` this branch introduced with
Django's own request-scoped mechanism, fixing the same bugs with no zone named
anywhere in application code.

`TIME_ZONE` supplies the default -- `USE_TZ` is what keeps storage UTC, and
conflating the two is what made the framework's own knob look unavailable.
`TimezoneMiddleware` was already in MIDDLEWARE reading a `django_timezone`
cookie that only the Django admin ever set; the web fetch client now publishes
it too, so the chart, the totals and the downloaded CSV agree with the calendar
the reader is on. Every conversion downstream then takes no argument at all:
`TruncDate("interacted_at")`, `make_aware`, `localdate`, `localtime`.

`set_next_run` deliberately reads `get_default_timezone()` instead. It runs
from an admin save and from Celery alike, so following the request's zone would
let a report set to 8am by a remote admin drift to 8am here on its first
reschedule; a schedule fires once, globally, and has no viewer.

Shelter hours and occupancy days stay pinned to the site's zone for the same
reason -- a shelter is open on its own clock whoever is looking it up. The Expo
app deliberately does not publish the cookie: it runs no reports queries, and
`openNow` must ignore the viewer.

Deletes `OPERATING_TIME_ZONE`, `local_window`, `local_today` and
`ReportNoteResource`. `NoteResource` already serves both audiences, following
the viewer on a request and the site zone in Celery.

Five tests outside `reports/` were asserting UTC results from naive
`time_machine.travel(...)` strings, silently riding on the process TZ. Their
travel targets are now explicit UTC.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@vecchp
vecchp force-pushed the DEV-warnings/report-timezones branch from 4d0ed23 to eb2e8f0 Compare September 1, 2026 19:39
@vecchp
vecchp changed the base branch from DEV-warnings/attachment-save-deprecation to main September 1, 2026 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants