Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
6184940
feat: reports can run as background jobs
Flix6x Jul 11, 2026
79e1609
feat: reports as automations
Flix6x Jul 11, 2026
5eac468
docs: changelog entry for reports as jobs and automations
Flix6x Jul 11, 2026
791fcf4
feat: anchor default report windows to the automation's actual last run
Flix6x Jul 11, 2026
0dca8fc
docs: add an Automations concept page
Flix6x Jul 11, 2026
176d800
fix: address stack code review findings
Flix6x Jul 11, 2026
881c9bd
feat(automations): integrate reports with the CRUD stack
BelhsanHmida Aug 12, 2026
292a0fa
fix(automations): authorize reporter sensor dependencies
BelhsanHmida Aug 12, 2026
8b76815
fix(reporting): require report inputs and outputs
BelhsanHmida Aug 12, 2026
82cb3bb
fix(automations): constrain report output scope
BelhsanHmida Aug 12, 2026
3e6c2ec
fix(automations): require generators for reports
BelhsanHmida Aug 12, 2026
601f951
fix(automations): anchor reports to claimed occurrences
BelhsanHmida Aug 12, 2026
f0c3fce
fix(automations): advance report coverage monotonically
BelhsanHmida Aug 12, 2026
2350b68
test(ui): cover report automation listings
BelhsanHmida Aug 12, 2026
77c6c2d
docs(automations): document report occurrence semantics
BelhsanHmida Aug 12, 2026
95b6187
test(api): close rejected report transactions
BelhsanHmida Aug 12, 2026
2e6fe75
Merge branch 'feat/2288-automations-crud' into feat/2288-report-autom…
Flix6x Sep 1, 2026
b8b8db5
Merge remote-tracking branch 'origin/feat/2288-automations-crud' into…
Flix6x Sep 1, 2026
2701717
Merge remote-tracking branch 'origin/feat/2288-automations-crud' into…
Flix6x Sep 1, 2026
b171ac6
Merge branch 'feat/2288-automations-crud' into feat/2288-report-autom…
BelhsanHmida Sep 8, 2026
67a7e2b
Merge branch 'feat/2288-automations-crud' into feat/2288-report-autom…
BelhsanHmida Sep 11, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions documentation/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ v1.0.0 | August 25, 2026
New features
-------------

* Reports can be computed on a recurring basis by automations (``flexmeasures add automation --type reporting``), with a rolling report window expressed as Pandas offsets, or defaulting to the period since the automation's last covered window [see `PR #2297 <https://www.github.com/FlexMeasures/flexmeasures/pull/2297>`_]
* ``flexmeasures show data-sources`` now shows which organisation a data source belongs to, and can list the sensors holding data recorded by a given source [see `PR #2401 <https://www.github.com/FlexMeasures/flexmeasures/pull/2401>`_]
* The flex-context can now define multiple commodities, each specifying their own prices and grid capacities [see `PR #1946 <https://www.github.com/FlexMeasures/flexmeasures/pull/1946>`_, `PR #2172 <https://www.github.com/FlexMeasures/flexmeasures/pull/2172>`_, `PR #2235 <https://www.github.com/FlexMeasures/flexmeasures/pull/2235>`_, `PR #2271 <https://www.github.com/FlexMeasures/flexmeasures/pull/2271>`_, `PR #2355 <https://www.github.com/FlexMeasures/flexmeasures/pull/2355>`_ and `PR #2380 <https://www.github.com/FlexMeasures/flexmeasures/pull/2380>`_]
* Support multiple feeders to a shared storage [see `PR #2001 <https://www.github.com/FlexMeasures/flexmeasures/pull/2001>`_, `PR #2321 <https://www.github.com/FlexMeasures/flexmeasures/pull/2321>`_, `PR #2322 <https://www.github.com/FlexMeasures/flexmeasures/pull/2322>`_, `PR #2325 <https://www.github.com/FlexMeasures/flexmeasures/pull/2325>`_ and `PR #2431 <https://www.github.com/FlexMeasures/flexmeasures/pull/2431>`_]
Expand Down
2 changes: 1 addition & 1 deletion documentation/cli/change_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ since v1.0.0 | August 11, 2026
* Add ``flexmeasures add plan``, ``flexmeasures show plans`` and ``flexmeasures edit plan``, to manage the rate limits and quotas which apply to the accounts on a plan.
* Add ``flexmeasures edit secret`` to store an encrypted secret on an account or asset.
* Add ``flexmeasures delete secret`` to remove an encrypted secret from an account or asset.
* Add ``flexmeasures add automation``, ``flexmeasures edit automation`` and ``flexmeasures delete automation`` to manage automations (recurring tasks on an asset, with ``--type forecasting`` or ``--type scheduling`` saying which task to automate). Each automation carries its own IANA timezone (``--timezone``), in which its cron expression is interpreted.
* Add ``flexmeasures add automation``, ``flexmeasures edit automation`` and ``flexmeasures delete automation`` to manage automations (recurring tasks on an asset, with ``--type forecasting``, ``--type scheduling`` or ``--type reporting`` saying which task to automate). Each automation carries its own IANA timezone (``--timezone``), in which its cron expression is interpreted.
* ``flexmeasures add automation --type scheduling`` refuses a flex config field which fixes a moment in time, such as ``soc-at-start`` or a ``soc-targets`` entry with a ``datetime``, naming the field: a recurring schedule automation computes a fresh schedule on every run, so such a value would be stale on the next one.
* Add ``flexmeasures jobs run-automations`` to queue jobs for all automations that are due to run this minute from standard five-field cron expressions. Run this command once per minute. It makes at most one queueing attempt per automation per minute, including when an attempt fails after partially queueing jobs. Runs missed while the runner was down are caught up once, with several missed forecast runs coalesced into the latest useful forecast, and a run at a skipped or repeated daylight-saving-time hour happens exactly once.
* Add ``flexmeasures jobs run-automation --automation <id>`` to queue the jobs for a single run of one automation, now, on top of its recurring runs. This leaves the automation's cursor alone, so its next recurring run still happens as scheduled, and inactive automations can be run this way, too.
Expand Down
2 changes: 1 addition & 1 deletion documentation/cli/commands.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ of which some are referred to in this documentation.
``flexmeasures add annotation`` Add annotation to accounts, assets and/or sensors.
``flexmeasures add toy-account`` Create a toy account, for tutorials and trying things.
``flexmeasures add report`` Create a report.
``flexmeasures add automation`` Add an automation: a recurring task (computing forecasts or schedules) on an asset, with its own cron timezone.
``flexmeasures add automation`` Add an automation: a recurring task (computing forecasts, schedules or reports) on an asset, with its own cron timezone.
================================================= =======================================


Expand Down
60 changes: 50 additions & 10 deletions documentation/features/automations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,30 @@ Automations
============

An **automation** is a recurring task defined on an asset.
For now, an automation computes forecasts or schedules; automating reports is planned.
An automation computes forecasts, schedules or reports.

On each run, the automation queues jobs (so make sure a worker is processing the ``forecasting`` or ``scheduling`` queue, whichever the automation needs, see :ref:`redis-queue`).
On each run, the automation queues jobs (so make sure a worker is processing the ``forecasting``, ``scheduling`` or ``reporting`` queue, whichever the automation needs, see :ref:`redis-queue`).
The parameters of the task were stored when the automation was created, and validated with the same schema that the CLI and API use.
Timing parameters are resolved on each run — for instance, the forecast or schedule start defaults to the time the automation runs, so each run produces fresh results.

Creating an automation
----------------------
- a **type**: ``forecasts``, ``schedules`` or ``reports``;
- a **recurrence**: a cron string (e.g. ``"0 6 * * *"`` for daily at 6 AM), interpreted in the automation's own IANA timezone;
- a **data generator** (for forecasts and reports): the forecaster or reporter class and its configuration, stored on a data source.
The data source stays the same across runs, so all results the automation produces attribute to one steady source;
- **parameters**: what to compute on each run, validated by the same schema the CLI and API use for one-off runs.
Timing parameters are resolved freshly on each run, so a recurring automation always computes fresh periods
(see the type-specific sections below for the exact rules);
- an **activation status**: only active automations run.

Here is how you create an automation in the CLI, asking for daily (at 6 AM) forecasts of sensor 12:
Managing automations
--------------------

.. code-block:: bash
Automations can be managed in three ways:

flexmeasures add automation --asset 3 --name "Daily PV forecasts" --type forecasting \
--cron "0 6 * * *" --timezone Europe/Amsterdam --sensor 12

``--type`` says which task to automate (``forecasting`` or ``scheduling``, matching the queue the jobs go to), and defaults to ``forecasting``.
``--type`` says which task to automate (``forecasting``, ``scheduling`` or ``reporting``, matching the queue the jobs go to), and defaults to ``forecasting``.
The remaining options are the ones the task itself needs: a forecast automation accepts everything `flexmeasures add forecast` accepts, such as ``--forecaster`` to pick the forecaster and ``--config`` to configure it (see :ref:`forecasting`).
The forecaster and its configuration are stored on a data source, so you can also pass ``--source`` to reuse the data source of an existing forecaster, in which case ``--forecaster`` and ``--config`` (and the individual configuration options) are not needed — the data source already determines them.
That data source is required while the automation exists, so it cannot be deleted until the automation is removed.
Expand Down Expand Up @@ -68,10 +75,32 @@ For example, this automation queues a scheduling job every hour, each time sched
echo 'duration: "PT12H"' > trigger-message.yml
flexmeasures add automation --asset 3 --name "Hourly schedules" --cron "0 * * * *" --type scheduling --parameters trigger-message.yml

Automating reports
------------------

A report automation's parameters are report parameters, as ``flexmeasures add report`` accepts them, and its reporter is named with ``--reporter`` and configured with ``--config`` (see :ref:`reporting`).
As for a forecast automation, the reporter and its configuration are stored on a data source, so ``--source`` can reuse the data source of an existing reporter instead.

The report window is resolved on every run, so that each run reports on a fresh period.
Give ``start-offset`` and ``end-offset`` in the parameters for a rolling window: both take comma-separated Pandas offsets, plus ``DB`` (day begin) and ``HB`` (hour begin), applied to the run time.
For instance, ``start-offset: "-1D,DB"`` with ``end-offset: "DB"`` reports on the whole of the previous day.
Offsets are resolved in the timezone of the first output sensor, falling back to the platform timezone.

Leave the timing fields out to report on the period since the automation last covered one, falling back to the last cron period on the first run.
That coverage is recorded by the reporting job itself, once it has succeeded, so a failed report leaves no permanent gap: the next run starts where the last successful one ended.
An absolute ``start`` or ``end`` is passed through untouched, which means every run then reports on the same period.

For example, this automation queues a reporting job every night, reporting on the previous day:

.. code-block:: bash

flexmeasures add automation --asset 3 --name "Daily self-consumption report" --cron "0 1 * * *" --type reporting \
--reporter PandasReporter --config reporter-config.yml --parameters report-parameters.yml

Running automations
-------------------
--------------------

For automations to actually run, let a cron job execute the following command once per minute:
An automation is due whenever its cron string matches the current minute in its configured timezone. To actually run due automations, let a cron job execute the following command once per minute:

.. code-block:: bash

Expand All @@ -84,7 +113,8 @@ Timing parameters that default to the run time are resolved when that catch-up r
Each scheduled run receives at most one automatic queueing attempt.
If the process crashes, or queueing fails after creating some jobs, that run is not retried automatically, because a retry could duplicate partial work.

The jobs record how they were created, which is shown on the asset's status page (UI), where recent jobs are listed.
If the runner misses runs, because it was down or overloaded, it catches up when it resumes: it queues only the latest missed run of each automation, rather than replaying stale ones.
Timing parameters that default to the run time are resolved when that catch-up run is queued, so it produces a current result.

Running one automation on demand
--------------------------------
Expand All @@ -111,6 +141,16 @@ Automations defined on an asset can be viewed on the asset's *Automations* page
An automation's details show the sensors it reads from and writes to, linking to each sensor's page.
Conversely, a sensor's page lists the automations that write data to it.

Automating each feature
-----------------------

The parameters stored on an automation follow the same schemas as one-off CLI/API calls, with type-specific rules for resolving timing on each run:

- :ref:`automating_forecasts` — forecast parameters; the forecast start defaults to the run time.
- :ref:`automating_schedules` — a schedule trigger message; omit ``start`` to schedule from the run time.
- :ref:`automating_reports` — report parameters; use ``start-offset``/``end-offset`` (Pandas offsets) for a rolling window,
or omit timing fields to report on the period since the last successfully covered report window.

.. _automation_cursor:

Appendix: how the runner decides what is due
Expand Down
20 changes: 17 additions & 3 deletions documentation/features/forecasting.rst
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,20 @@ Usage:
Automating forecasts
--------------------

Instead of asking for forecasts one at a time, you can set up an *automation*: a recurring task defined on an asset, which queues forecasting jobs on a cron schedule.
See :ref:`automations`.
Schedules can be automated in the same way — see :ref:`automating_schedules`.
Instead of asking for forecasts one at a time, you can set up an *automation*: a recurring task defined on an asset (see :ref:`automations` for the full concept, including how to manage and run automations).
On each run, the automation queues forecasting jobs (so make sure a worker is processing the ``forecasting`` queue, see :ref:`redis-queue`).
When the automation was created, its forecast parameters (see above) were stored, and validated with the same schema that the CLI and API use.
Timing parameters are resolved on each run — for instance, the forecast start defaults to the time the automation runs, so each run produces fresh forecasts.
The sensor on which forecasts are saved (``sensor-to-save``, falling back to ``sensor``) must belong to the automation's asset or one of its descendants.
This relationship is checked both when the automation is created and immediately before each run.

Here is how you create a forecast automation in the CLI, asking for daily (at 6 AM) forecasts of sensor 12:

.. code-block:: bash

flexmeasures add automation --asset 3 --name "Daily PV forecasts" --type forecasting \
--cron "0 6 * * *" --timezone Europe/Amsterdam --sensor 12

A forecast automation accepts everything ``flexmeasures add forecast`` accepts, such as ``--forecaster`` to pick the forecaster and ``--config`` to configure it.
The forecaster and its configuration are stored on a data source, so you can also pass ``--source`` to reuse the data source of an existing forecaster, in which case ``--forecaster`` and ``--config`` (and the individual configuration options) are not needed — the data source already determines them.
That data source is required while the automation exists, so it cannot be deleted until the automation is removed.
25 changes: 25 additions & 0 deletions documentation/features/reporting.rst
Original file line number Diff line number Diff line change
Expand Up @@ -129,3 +129,28 @@ Here, the ``ProfitOrLossReporter`` used as source (with Id 6) is the one we conf
With the offsets, we control the timing ― we indicate that we want the new report to encompass the day of tomorrow (see Pandas offset strings).

The report sensor will now store all costs which we know will be made tomorrow by the schedule.

.. _automating_reports:

Automating reports
--------------------

Besides running a report once, a report can be computed on a recurring basis by an *automation* defined on the asset.
See :ref:`automations` for the full concept, including how to manage and run automations.

The reporter and its configuration are stored on a data source, which stays the same across runs, so all of the automation's report results attribute to one source.
The report parameters are stored on the automation itself, and their timing is resolved afresh on each run:

- Use ``start-offset`` and/or ``end-offset`` fields (comma-separated Pandas offsets, like the CLI options above) for a rolling window relative to the claimed cron occurrence, in the timezone of the first output sensor.
For instance, ``"start-offset": "-1D,DB"`` with ``"end-offset": "DB"`` reports on the whole previous day.
- Omit timing fields entirely to report from the end of the latest successfully completed report window through the claimed cron occurrence.
When no completed window is known, such as on the first run, the start falls back to the previous cron occurrence in the automation's timezone.
The completion marker only moves forward, so concurrent reporting workers that finish out of order cannot reopen an already covered period.
- Absolute ``start``/``end`` fields are also accepted, but draw a warning, as each run would then compute the same period.

For example, this automation computes a report over each past day, every morning at 1 AM:

.. code-block:: bash

flexmeasures add automation --asset 3 --name "Daily aggregation report" --cron "0 1 * * *" --type reporting \
--reporter PandasReporter --config reporter-config.yml --parameters report-parameters.yml
2 changes: 1 addition & 1 deletion flexmeasures/api/v3_0/assets.py
Original file line number Diff line number Diff line change
Expand Up @@ -1723,7 +1723,7 @@ def post_automation(self, id: int, asset: GenericAsset):
automation_type=automation_data["type"],
active=automation_data["active"],
parameters=automation_data["parameters"],
forecaster_class=automation_data["forecaster"],
generator_class=automation_data["generator"],
config=automation_data["config"],
origin="API",
check_permissions=True,
Expand Down
Loading
Loading