Fix: CEM work session - #188
Draft
Flix6x wants to merge 193 commits into
Draft
Conversation
…, consolidating all dependencies into pyproject.toml and a single uv.lock file. This simplifies and speeds up the development setup greatly. Changes: - Switch build backend from setuptools to hatchling - Remove setup.cfg, tox.ini, .isort.cfg and setup.py in favour of .flake8 and pyproject.toml - Add Poethepoet tasks - Upgrade main Python version (CI/CD, .python-version, etc.) to 3.12 - Deprecate Python 3.9 - Add .python-version for consistent Python version management - Create separate Dockerfile for flexmeasures-client - Replace pip-tools with uv in all CI/CD workflows - Remove ci/run_mypy.sh in favour of Poethepoet task - Update documentation Signed-off-by: Stijn van Houwelingen <teadrinkingprogrammer@github.io>
Signed-off-by: Stijn van Houwelingen <teadrinkingprogrammer@github.io>
Signed-off-by: Stijn van Houwelingen <teadrinkingprogrammer@github.io>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
This reverts commit d63afe5.
…connection shuts down Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
This reverts commit 4799585.
Signed-off-by: F.N. Claessen <claessen@seita.nl>
… is missing Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
This reverts commit 5328de9.
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…) and make system_description_id optional Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
This reverts commit 374e44a.
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…andlers accordingly Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…ted time The CEM now creates a dedicated 'measured-power' sensor (kW, consumption-positive) per site, separate from the StorageScheduler 'power' schedule sensor, and maps ELECTRIC.POWER.L1 measurements to it (replacing a hardcoded fallback sensor id). Measurements are binned and posted at their own SIMULATED timestamp instead of wall-clock time, and posted directly per incoming value (the RM sends one aggregated value per interval, so the old 5-minute wall-clock buffered-averaging would have suppressed nearly every post in a co-simulation). Values are posted in Watts (S2 PowerMeasurement convention) and converted to the kW sensor by FlexMeasures.
…'t crash CEM.close() calls close() on every registered control-type handler when a websocket tears down, but only FRBCTunes implemented it - FRBCSimple (and any other handler) raised AttributeError, which killed the CEM's request handler and left it unable to serve the next RM connection (houses then hang forever waiting for a schedule). Add a default no-op close() on the ControlTypeHandler base class.
Images built from FlexMeasures' uv-based Dockerfile put a pip-less /app/.venv first on PATH while bare `pip` is the system one under /usr/local, so packages installed at container start were invisible to the runtime interpreter (ModuleNotFoundError: aiohttp) on any freshly built image - notably Samuel's Windows setups - while long-cached single-Python images kept working. Use `python3 -m pip` throughout (with an ensurepip bootstrap for pip-less venvs) and exec the server with that same python3. Also set the generic SETUPTOOLS_SCM_PRETEND_VERSION alongside the client-specific one for the git-less editable install. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LQFkeGxDwErydUV5ZmHmki
An S2 PowerRange runs from start_of_range (lower bound) to end_of_range (upper bound), but the capacity derivation introduced in 9f20ab4 (March 2026) read them the other way around: production-capacity became the smallest upper bound across operation modes (e.g. 2.5 kW for a consumption-only heat pump) and consumption-capacity the largest lower bound. The scheduler consequently discharged the heat pump's thermal storage as if it were an electric battery, and capped its consumption below the device's real maximum. Derive the overall [min, max] power range instead, with production only where ranges extend below zero. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LQFkeGxDwErydUV5ZmHmki
… realizations Belief time (defect 4b): PowerMeasurements posted by the CEM now carry a simulated prior (the binned interval's end), so realized apartment power is stamped at simulation time instead of wall-clock 2026, and the UI horizon view no longer shows data "recorded in 2026". Timezone (defect 2): naive S2 measurement timestamps are now localized as Europe/Amsterdam (was UTC), matching the community orchestrator, RM and controller which all localize the same naive sim times as Europe/Amsterdam. Live apartment realizations (defect 4a): besides the dedicated measured-power sensor, the CEM now also mirrors the realized apartment aggregate onto the apartment's flex-context "aggregate-power" sensor, resolved lazily from the asset's flex_context (it is attached by the community runner only after the CEM connects, so it appears from step 1). Realized posts stay distinguishable from the StorageScheduler schedule data on that same sensor by their own source and simulated belief time. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LQFkeGxDwErydUV5ZmHmki
The RM sends one PowerMeasurement per 15-minute interval (stamped at interval start), but the posting path binned and stamped priors with the 5-minute _minimum_measurement_period, yielding belief times 10 minutes before each event's end - an ex-ante horizon on measurements. Bin and stamp at the measured-power sensor's 15-minute event resolution instead, so each realized value is believed exactly when its interval elapses (zero horizon). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LQFkeGxDwErydUV5ZmHmki
An explicit production-capacity of 0 kW gets softened by FlexMeasures' relax-constraints defaults (100 EUR/kW device breach price), so under a tight community capacity the scheduler chose to 'produce' 1.764 kW from a heat pump rather than pay the 10000 EUR/kW site breach. Setting is_strictly_non_positive on the FRBC power sensor keeps production hard-bounded at zero regardless of relaxation settings. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LQFkeGxDwErydUV5ZmHmki
Later polling steps slept for polling_interval * 2**step seconds (minutes), dozing far past the moment a schedule became available - and taking just as long to conclude it never would. Cap each sleep at 10s so both success and failure are detected promptly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: F.N. Claessen <claessen@seita.nl>
With backoff sleeps capped at 10s, 10 polling steps amounted to only ~90s of patience - less than a time-limited (120s) scheduling job - so clients gave up on schedules that were about to arrive. 30 steps puts the ceiling above polling_timeout (200s), which is the intended budget. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: F.N. Claessen <claessen@seita.nl>
The scalar soc-min/soc-max are hard bounds server-side; only the soc-minima/maxima profiles are softened by relax-soc-constraints. The FRBC fill range is a comfort band and the realized state can drift outside it (community steering keeping the heat pump off drains the buffer below the comfort floor), which made every subsequent day's schedule hard-infeasible. Clamp the hard band around the current state and leave comfort steering to the soft profiles. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: F.N. Claessen <claessen@seita.nl>
Under multi-house load a scheduling job can queue behind a full wave of solver-time-limited (120s) jobs, so 200s of polling gave up on schedules that were one wave away. 360s covers two waves plus overhead; step count raised to keep the per-sleep cap (10s) from binding first. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: F.N. Claessen <claessen@seita.nl>
Opaque orchestration metadata passed alongside the domain payload (a sibling of flex-model/flex-context, mirroring the S2 wrapper convention). Passed through verbatim to servers that accept it. Signed-off-by: F.N. Claessen <claessen@seita.nl>
Per S2, an RM only sends FRBC.* after accepting SelectControlType, so such a message proves activation. A dropped ReceptionStatus previously wedged the CEM in NO_SELECTION, bouncing every FRBC.SystemDescription with TEMPORARY_ERROR while the RM retried forever (in-vivo hang, 2026-07-26). With regression test. Signed-off-by: F.N. Claessen <claessen@seita.nl>
…tion batches, sent-instruction registry Latest-request-wins generation counters kill backlogged handler tasks (observed storm: 96 -> 2,225 instructions/burst, ack starvation, RM retry loops, 800 s control misses); content-hash dedupe of re-sent system descriptions; instruction batches carry only state CHANGES (the RM holds state, fm-s2 applies the same filter); sent-instruction registry with InstructionStatusUpdate tracking and opt-in RevokeObject on batch replacement (send_revocations, off by default for RMs without revoke handling). Six new tests. Signed-off-by: F.N. Claessen <claessen@seita.nl>
Large thermal fill levels sit far from zero; the default zero-including axis flattened all variation into a sliver. Signed-off-by: F.N. Claessen <claessen@seita.nl>
The trigger retry loop treats 'Scheduling job failed' as transient, but a deterministic server-side failure (e.g. a UniqueViolation on saving the schedule) re-fails identically on every attempt; re-triggering it five times with fresh jobs only multiplies failed jobs across the worker pool. Skip retries when the failure message carries a UniqueViolation/duplicate-key signature. Signed-off-by: F.N. Claessen <claessen@seita.nl>
The RM forwards realized power as one S2 PowerMeasurement per 15-min interval, serially; awaiting each FM post inline serialized ~200 HTTP round-trips per house per simulated day on the co-simulation critical path. Posts now run as tracked asyncio tasks with identical payloads (same event_start, prior, unit and values), and CEM.flush_measurement_posts() provides the read barrier: FRBC trigger_schedule awaits it, so FlexMeasures schedulers (and the community compliance check they feed) still see the complete realized series, exactly as the old serial awaits guaranteed. Post failures are swallowed with a debug log as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVTVfL4fQ9QSqbLXmAGD Signed-off-by: F.N. Claessen <claessen@seita.nl>
The RM re-sends the same realized-power series with every retrigger round's flexinput, so the CEM re-posted ~200 identical values per house per round. Track content keys (sensor, event start, value) and skip posts already made or in flight; a failed post releases its key so a later re-send retries it. Changed values (which realized data should never produce) still post. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVTVfL4fQ9QSqbLXmAGD Signed-off-by: F.N. Claessen <claessen@seita.nl>
The server silently substitutes the fallback scheduler's coarse charging policy when the real problem is infeasible, and the fallback saves no state-of-charge stream - which broke follow-the-schedule batteries a day later (soc-at-start resolution found nothing) and starved houses in co-simulation. Inspect scheduler_info on the returned schedule and log an ERROR naming the window, so a fallback result is never mistaken for a healthy schedule. The fallback scheduler is slated for removal in FlexMeasures v1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVTVfL4fQ9QSqbLXmAGD Signed-off-by: F.N. Claessen <claessen@seita.nl>
The CEM object is rebuilt for every RM websocket (re)connection - in co-simulation that is every replan round - so the instance-level dedupe keys and pending-post set reset each round: the dedupe never held (a profiled 1-day run re-posted each house's realized series ~3x, ~13k sensor-data calls), and a fresh instance's flush barrier could not await the previous instance's still-in-flight posts. Hoist both sets to process scope; one CEM server process serves one apartment, so that is their correct lifetime. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVTVfL4fQ9QSqbLXmAGD Signed-off-by: F.N. Claessen <claessen@seita.nl>
configure_site runs on every RM (re)connection - in co-simulation that is every replan round - and its full-catalog get_assets scan cost ~5-7s per call against a large database, about half the API time of a profiled co-simulation day. Find the site asset via an id/name listing and fetch only the match, falling back to the full scan on servers without the fields parameter (< 0.31). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVTVfL4fQ9QSqbLXmAGD Signed-off-by: F.N. Claessen <claessen@seita.nl>
Even the lean id/name listing costs seconds against a large database, and configure_site runs on every RM (re)connection - every replan round in co-simulation. Remember the resolved site-asset id at process scope (one CEM server process per apartment) and fetch it directly on reconnections, invalidating on a miss. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVTVfL4fQ9QSqbLXmAGD Signed-off-by: F.N. Claessen <claessen@seita.nl>
The RM's declared fill-level range used to be sent to FlexMeasures as hard soc-min/soc-max bounds. A hard bound at the declared range made physically-reachable states infeasible: free-floating above "full", or a time-capped incumbent left below "empty" for the next day, which is a deterministically infeasible scheduling job (the sweep-9 run-killer class). The comfort steering machinery is already soft: the fill-level target profile is posted onto soc-minima/-maxima sensors, which the scheduler enforces as breach-priced StockCommitments under relax-soc-constraints. So make the declared range the behavioral band on the soft path, and demote the hard bounds to wide safety rails: - trigger_schedule: soc-min = 0 and soc-max = 1.5x the declared range top (still widened to include the current state as a last resort). The rails only exist to keep the LP bounded (e.g. negative-price hours combined with a soc-value-at-end incentive). - send_fill_level_target_profile: clip the posted soc-minima/-maxima into the declared range (night setbacks clip up to the range bottom; ceilings clip down to the range top). Where a target band lies entirely outside the declared range, both bounds collapse to the midpoint of the clipped pair (keeping minima <= maxima pointwise) and a warning is logged. The clip lives in clip_fill_level_target_profile (FRBC/utils.py), unit-tested in tests/s2/test_frbc_soft_range.py along with the flex-model rails. FM-side needs no change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVTVfL4fQ9QSqbLXmAGD Signed-off-by: F.N. Claessen <claessen@seita.nl>
Containers that mount this repo and pip-install it at startup have no git binary, so hatch-vcs cannot derive the version and the editable install fails (observed as a worker crash-loop after a host reboot forced fresh startup installs). Configure setuptools-scm's fallback_version so such installs succeed; tagged versions still come from git wherever it is available. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVTVfL4fQ9QSqbLXmAGD Signed-off-by: F.N. Claessen <claessen@seita.nl>
FlexMeasures PR #2101 made POST .../sensors/data (and the file-upload
endpoint) return 202 Accepted with a background job id whenever the
"ingestion" RQ queue has connected workers, instead of always processing
synchronously and returning 200. The client treated 202 as done, so
callers believed data was committed when it was only queued -
post-then-trigger flows (e.g. s2/cem.py's flush_measurement_posts()
barrier before triggering a schedule) could then race ahead of ingestion.
post_sensor_data() gains await_ingestion: bool = True. On a 202 response
with a job id, it now polls GET .../jobs/<job_id> (exponential backoff
0.25s -> 2s cap, ingestion_polling_timeout seconds total, default 60s)
until the job reaches a terminal state:
- FINISHED: returns normally.
- FAILED/STOPPED/CANCELED: raises the new IngestionFailedError.
- still pending when the timeout elapses: logs an ERROR ("ingestion not
confirmed within Xs; proceeding - downstream schedules may read
incomplete data") and returns normally rather than raising - the
co-sim's watchdog/compliance checks are the intended safety net for
that case, not this call.
200 (synchronous) responses are unaffected; await_ingestion=False opts
out entirely, restoring the pre-this-change behavior.
Verified the interaction with s2/cem.py's _post_measurement_safely():
it already wraps post_sensor_data() in try/except Exception and releases
the dedupe key on failure, so IngestionFailedError (like any other post
failure) causes the RM's next re-send of the same series to retry the
post. flush_measurement_posts() needed no change - it already awaits the
post tasks to completion, which now simply takes a bit longer when
ingestion is queued.
Also fixed _post_sensor_data_file() to accept 202 (it previously treated
any non-200 status as a hard failure), since the upload endpoint is
backed by the same process_sensor_data_ingestion() as the JSON endpoint.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5EEnE2aCN7sGf1n37XuJJ
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…dler # Conflicts: # src/flexmeasures_client/client.py # src/flexmeasures_client/exceptions.py # src/flexmeasures_client/response_handling.py # tests/client/test_sensor.py
…odes FlexMeasures v1 replaced the flex-model's single signed "power-range" per operation mode with the sign-explicit "consumption-range" and "production-range", each non-negative. Sending the old field against v1 fails validation (422 Unknown field), which the CEM only surfaces as a stalled simulation: the schedule trigger raises inside a fire-and-forget task, so no instruction is ever produced and the resource manager waits forever. Split each S2 power range by sign, as v1's OperationModeSchema documents: the non-negative part becomes the consumption range, the negative part becomes the production range with its sign flipped, and a range spanning zero maps onto both, each starting at zero. Verified against that schema for an on/off heater, a battery band through zero, and a production-only band. NOT compatible with pre-v1 servers, which still require "power-range" - hence a branch of its own rather than dev/fix-handshake-handler, which the current (hybrid) simulation stack runs from. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016pU9h8MTdf4nausk3FR5gR Signed-off-by: F.N. Claessen <claessen@seita.nl>
The previous commit made the client send consumption-range and production-range unconditionally, which FlexMeasures v1.0.0 and newer expect but older servers reject, so the branch could only be used against one or the other. The shape is now chosen from the server's own reported version: the sign-explicit split from v1.0.0 onwards, the single signed power-range before that. An unknown version falls back to the older shape, which is the safe direction: a server that predates the version header also predates the split fields. This lets one branch serve both, instead of asking users to match a branch to their server. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016pU9h8MTdf4nausk3FR5gR Signed-off-by: F.N. Claessen <claessen@seita.nl>
…andler Brings the S2-to-FlexMeasures power-band mapping onto the shared branch. It is safe here because the band shape is chosen from the server's reported version, so this branch keeps working against pre-v1 servers as well as v1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016pU9h8MTdf4nausk3FR5gR
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #187.
FRBC.FillLevelTargetProfileandFRBC.UsageForecastto theFRBCSimplehandler.SimpleFRBChandler to get to the point thatFRBCInstructionsare sent back.FRBC.LeakageBehaviourto theFRBCSimplehandler.