Leap-year and period handling in AUSTAL preparation

[sstromat]

Leap-year and period handling in AUSTAL preparation

Reference document. Captures what AUSTAL itself does for non-standard year lengths (365 vs 366 days, and sub-year runs) and how each layer of the OpenALAQS plugin behaves against that specification. Useful when investigating discrepancies, evaluating sub-year compliance runs, or planning further improvements.

AUSTAL specification (current)

From the AUSTAL documentation:

The "long time mean" output of AUSTAL (file name marker y00) is always the mean over the time period covered by the provided meteo time series (or file series.dmna). Hence, if the provided time series goes over 365 days the mean goes over 365, and if the provided time series goes over 366 days the mean goes over 366 days.

For short time values like the hourly mean exceeded 18 times, the number of exceedances is adjusted if the time period becomes much shorter than a year. This is logged in the file austal.log.

In other words:

AUSTAL output	Period length used
<substance>-y00a.dmna (long-time mean)	length of provided series.dmna (no hardcoded 8760)
<substance>-NNNa.dmna (per-hour grid)	one file per hour in series.dmna
<substance>-tmpa.dmna (per-receptor time series)	length of provided series.dmna
"≤N hourly exceedances per year" type limit	AUSTAL scales N proportionally for sub-year runs and logs the scaled value in austal.log

This contradicts an older convention (sometimes attributed to TA Luft 2021 or AUSTAL 3.3 internals) that held "AUSTAL divides by a hardcoded 8760 regardless of leap year". The current behaviour is the correct one to design against.

Plugin layers and their behaviour

Three layers in the plugin touch year length. All three are aligned with the current AUSTAL specification above.

Layer 1: Emissions calculator

open_alaqs/core/EmissionCalculation.py and open_alaqs/core/interfaces/SourceModule.py plus open_alaqs/core/tools/profiles_vec.py.

• profiles_vec.hours_in_year(year) returns 8784 for leap years, 8760 otherwise (via calendar.isleap).
• SourceModule._get_profile_mean (line 222 in SourceModule.py) walks the actual year length and normalises so that mean_over_year == 1.0.
• SourceModule._hours_in_year cache invalidates on year change (lines 271-276), so multi-year runs handle the leap boundary correctly.
• EmissionCalculation.check_year_coverage (lines 395-440) compares the actual run length to 8784 / 8760 and emits a warning when the period does not cover a full calendar year. The warning includes the percentage off and the suggested end_dt.

Verdict: fully leap-year aware.

Layer 2: AUSTAL writer (ALAQS-mode and existing-emissions-inventory modes)

open_alaqs/core/modules/AUSTALOutputModule.py.

• Main time iteration at line 643 uses dateutil.rrule.rrule(DAILY, dtstart=start, until=end) plus an hourly inner loop. This iterates every calendar day inclusive of February 29 in leap years.
• The time-indexed accumulator (lines 1498-1503) sets _ti_n_hours_year = hours_in_year(year), so per-source rate arrays are sized (8784, n_pol) for leap years.
• writeTimeSeriesFile (line 1894 onwards) writes hghb = len(sorted_dts) to the series.dmna header. For a Jan 1 -> Dec 31 leap-year run that is 8784; for a non-leap year, 8760. AUSTAL averages over what it receives.

Verdict: fully leap-year aware.

Layer 3: CSV-mode generator

open_alaqs/core/tools/austal_csv_generation.py.

• The function generate_austal_from_csv uses sorted_timestamps = sorted(rows_by_ts.keys()), taking whatever timestamps the user supplied in the emissions CSV.
• An optional [start_dt, end_dt] filter trims to the requested window.
• No hardcoded 8760 or 8784 anywhere.

Verdict: pass-through. The plugin correctly forwards whatever the user provided, and AUSTAL averages over that period.

Known issues

Compliance Report does not scale allowed-exceedance counts for sub-year runs

open_alaqs/core/modules/ComplianceReportOutputModule.py (_evaluate_substance, line 140 onwards):

• annual_mean metric uses np.mean(valid) over the actual data, which matches AUSTAL's y00 behaviour. Compatible with both 365 and 366 day runs.
• daily_exceedance and hourly_exceedance metrics count exceedances over the actual data, then compare to the directive's fixed allowed count (18 for PM10 daily, 3 for NO2 hourly, etc.) regardless of period length.

For sub-year runs, this is too lenient. AUSTAL itself scales the allowed count proportionally and logs the result in austal.log. The plugin does not read austal.log and does not scale.

This is acceptable for full-year compliance assessment (which is the realistic regulatory use case), but exploratory sub-year runs will get falsely lenient PASS verdicts.

Two fix options for a future release:

1. Refuse to evaluate sub-year runs. Detect period length; if much shorter than a full year, hide daily/hourly exceedance rows and show only annual mean as reference, with a banner explaining why.
2. Pro-rate the allowed count. allowed_scaled = round(allowed_directive * n_days / 365). Show the scaled count in the table so the user sees what threshold they are being judged against. This matches AUSTAL's own approach.

Option 2 is mathematically more correct. Option 1 is safer (no false PASS / FAIL verdicts on partial data).

Outdated comment in profiles_vec.py

The module-level docstring of open_alaqs/core/tools/profiles_vec.py contains a block titled "KNOWN INCONSISTENCY at the AUSTAL boundary" (lines 29-38 at the time of writing). That block claims AUSTAL divides by 8760 regardless of leap year and that feeding 8784 rates biases the reported annual mean by ~0.27%.

This is incorrect against current AUSTAL behaviour (see specification above). The plugin's current 8784-rates-for-leap-years path is exactly correct; there is no 0.27% bias.

The comment can be rewritten as:

AUSTAL boundary
---------------
AUSTAL's "long-time mean" (y00 output) is the arithmetic mean over the
period covered by the provided series.dmna, not a hardcoded 8760 hours
per year. Feeding 8784 hourly rates (one per hour of a leap year) into
series.dmna produces a mean over 366 days, which is the correct
behaviour for that calendar year. AUSTAL also adjusts short-time
statistics (e.g. the "≤18 hourly exceedances per year" limit) when the
provided period is much shorter than a year; the adjusted count is
logged in austal.log.

The plugin's emissions path uses 8784 hours for leap years to preserve
emission totals bit-for-bit, and this aligns with AUSTAL's averaging
behaviour at the writer boundary.

Compatibility matrix

Run scenario	Emissions calc	AUSTAL writer (ALAQS)	AUSTAL writer (CSV)	Compliance Report
Full leap year (Jan 1 -> Dec 31, e.g. 2024)	OK, 8784 hours, normalised	OK, 8784 rows in series.dmna	OK, pass-through	OK, y00 mean is over 366 days
Full non-leap year (Jan 1 -> Dec 31, e.g. 2025)	OK, 8760 hours, normalised	OK, 8760 rows in series.dmna	OK, pass-through	OK, y00 mean is over 365 days
Multi-year (Jan 1, Y -> Jan 1, Y+1)	OK, cache invalidates on year boundary	OK, hours iterated correctly	OK, pass-through	Marginal (treat output cautiously)
Multi-year (Y -> Y+N, N >= 2)	OK	Multi-year acknowledged in TI path comment	OK	Not designed for this; do not use
Sub-year (e.g. 90 days)	Warning emitted by check_year_coverage	OK, fewer rows in series.dmna	OK, pass-through	Too lenient on daily/hourly exceedance counts; see Known issues

Future work

In rough priority order:

1. Update the profiles_vec.py docstring block to reflect current AUSTAL behaviour (remove the false "0.27% bias" claim). One-file documentation patch.
2. Decide between Option 1 (refuse) and Option 2 (pro-rate) for the Compliance Report sub-year case. Implement and document.
3. Optionally, read the adjusted exceedance count from austal.log instead of computing it client-side. Most faithful but requires parsing AUSTAL log format, which is not a documented API.
4. Add a regression test that drives a leap-year run through both modes and asserts series.dmna has 8784 rows and the y00 mean matches the expected period.

References

• AUSTAL output structure: file-name markers y00a, NNNa, tmpa in the AUSTAL user manual.
• Plugin leap-year sanity warning: open_alaqs/core/EmissionCalculation.py::check_year_coverage, introduced in v5.1.0.
• hours_in_year utility: open_alaqs/core/tools/profiles_vec.py line 98.
• TI accumulator sizing: open_alaqs/core/modules/AUSTALOutputModule.py line 1503.
• CSV-mode time iteration: open_alaqs/core/tools/austal_csv_generation.py::generate_austal_from_csv.
• Compliance Report evaluator: open_alaqs/core/modules/ComplianceReportOutputModule.py::_evaluate_substance.