Tariffs: add temperature type and OpenMeteo#27780
Tariffs: add temperature type and OpenMeteo#27780daniel309 wants to merge 24 commits intoevcc-io:masterfrom
Conversation
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 1 issue, and left some high level feedback:
- In
applyTemperatureCorrectionyou do a nested loop over all forecast rates for every 15‑minute slot; since the rates are time-ordered, consider advancing an index or precomputing a per-slot slice to avoid O(n*m) behavior on longer horizons. - Using
0as a sentinel forHeatingThresholdandHeatingCoefficientmeans users cannot explicitly configure these to zero; if that should be allowed, consider using pointers or a separate*Enabledflag instead of overloading0as "use defaults". - The generated enum files (
tarifftype_enumer.go,tariffusage_enumer.go) have been hand-edited (including the// Made with Bobcomments); this can easily diverge from the code generator, so it would be safer to adjust the source for generation and regenerate these instead of modifying them manually.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In `applyTemperatureCorrection` you do a nested loop over all forecast rates for every 15‑minute slot; since the rates are time-ordered, consider advancing an index or precomputing a per-slot slice to avoid O(n*m) behavior on longer horizons.
- Using `0` as a sentinel for `HeatingThreshold` and `HeatingCoefficient` means users cannot explicitly configure these to zero; if that should be allowed, consider using pointers or a separate `*Enabled` flag instead of overloading `0` as "use defaults".
- The generated enum files (`tarifftype_enumer.go`, `tariffusage_enumer.go`) have been hand-edited (including the `// Made with Bob` comments); this can easily diverge from the code generator, so it would be safer to adjust the source for generation and regenerate these instead of modifying them manually.
## Individual Comments
### Comment 1
<location path="tariff/weather.go" line_range="94-99" />
<code_context>
+ }
+
+ data := make(api.Rates, 0, len(res.Hourly.Time))
+ for i, tsStr := range res.Hourly.Time {
+ if i >= len(res.Hourly.Temperature2m) {
+ break
+ }
+
+ // Open-Meteo returns ISO 8601 strings like "2024-01-15T14:00"
+ ts, err := time.ParseInLocation("2006-01-02T15:04", tsStr, time.Local)
+ if err != nil {
</code_context>
<issue_to_address>
**issue (bug_risk):** Parsing timestamps with time.Local is likely incorrect for Open-Meteo’s UTC defaults and may misalign temperatures with slots.
Open-Meteo timestamps are UTC by default unless a timezone is requested. Parsing them with `time.ParseInLocation(..., time.Local)` will reinterpret UTC timestamps as local time and shift all slots by the local offset. Please either: (a) request `&timezone=UTC` and parse with `time.UTC`, or (b) request `&timezone=auto` and parse using that timezone, so `Rate.Start`/`End` stay aligned with the forecast data.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
|
You've hit an actual problem spot-on, but I'm not sure that the solution is the right one, conceptually and technically: Imho the "right" approach here would be to measure and collect heating demands separately and have a different "heating" optimizer model for those. |
|
The weather tariff is a nice touch (really: it's temperature, not weather?) and might be a separate PR on its own that will become useful sooner or later. |
If you dont have a heatpump (or more general, heating that uses electricity), you dont configure/add the
Yes, that would be most clean. It would require the heating meter and a bunch other things (separate optimizer model) likely making config and implementation more complicated. And I am really wondering if all that additional code, effort and complication (to the user) is really worth it. Assume a single-family home household like ours: when heating (heatpump in our case) is active, daily energy consumption is dominated by that one energy consumer. In numbers:
Happy to split this out (and rename to temperature) or do it all at once together with the heating code that uses it. Let me know how to proceed. |
Yes :)
Yes. And we'd need to figure out how meters (not even necessarily) chargers would be handed to the optimizer. Collecting arbitrary metrics is already "almost" done: #23185 (btw, would love help with this PR).
That's already in the backlog, examples exist.
Happy to discuss how to do this, maybe in separate issue. Or join the |
|
hmm, ok, not sure how to proceed. It sounded like you would like to break this down into re-useable parts, and maybe get some use early to test how well it works now that heating is still on for a few weeks. Otherwise next opportunity is October 2026. so, I could split this work into 3 pieces:
it would also serve as a testbed for debugging etc.
does that sound feasible? (btw. not sure how much time I can dedicate to this, 3. for sure is not a quick change. I do think 1. and 2. are possible relatively short-term though). |
|
@andig instead of a separate optimizer, my first thought on this would be to make the household load prediction more similar to how we handle tariffs. That would add flexibility for users to configure it with just the "historic" strategy that we have today, or create a more complex calculation with temperature adjustment factors. I guess some users would be able to remove the heating energy from their basic prediction (set a different meter?), make a separate time series just for heating, adjust that one based on temperature, and then combine those two time series to generate the input for the optimizer. If you agree this would be helpful I could probably write that? Then combined with the weather tariff and adjustment strategy that @daniel309 made you have full flexibility. The only thing is... It won't be easy to make a visual UX for configuration of these kinds of structures. |
most heatpump "chargers" already provide example: |
|
I would really appreciate if we could move this brain storming out or PRs ;)
Yes, that's absolutely obvious
Yes, still needs to tracking PR to complete. Who/ when?
Yes and yes- but I don't have any good idea how to do that right now (but also didn't try). We've really always tried to not shoot ourselves into the foot and do things properly and in steps. |
daniel309
force-pushed
the
feature/weather-dependent-household-load
branch
2 times, most recently
from
481dacf to
5db4fac
Compare
There was a problem hiding this comment.
Hey - I've found 1 issue, and left some high level feedback:
- The
runloop usestime.Tick(time.Hour)which never gets GC’d and also delays the first fetch by an hour; consider switching totime.NewTickerwith an immediate initial fetch (and properStop) to avoid leaks and to populate data as soon as the tariff starts. - The configuration check
if cc.Latitude == 0 && cc.Longitude == 0forbids the valid (0,0) coordinate; if you only want to guard against missing config, consider using a separate boolean or range validation instead of treating 0,0 as an error.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The `run` loop uses `time.Tick(time.Hour)` which never gets GC’d and also delays the first fetch by an hour; consider switching to `time.NewTicker` with an immediate initial fetch (and proper `Stop`) to avoid leaks and to populate data as soon as the tariff starts.
- The configuration check `if cc.Latitude == 0 && cc.Longitude == 0` forbids the valid (0,0) coordinate; if you only want to guard against missing config, consider using a separate boolean or range validation instead of treating 0,0 as an error.
## Individual Comments
### Comment 1
<location path="tariff/temperature.go" line_range="79" />
<code_context>
+
+ client := request.NewHelper(t.log)
+
+ for tick := time.Tick(time.Hour); ; <-tick {
+ uri := fmt.Sprintf(
+ "https://api.open-meteo.com/v1/forecast?latitude=%f&longitude=%f&hourly=temperature_2m&past_days=7&forecast_days=3&timezone=UTC",
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Hourly ticker delays the first fetch by one hour, which can leave the tariff empty for a long time.
Because `for tick := time.Tick(time.Hour); ; <-tick {` only enters the body after one hour, no data is fetched on startup. Given the `runOrError` pattern, callers likely expect an initial fetch (or error) soon after start. Consider fetching once before starting a ticker:
```go
func (t *Temperature) run(done chan error) {
client := request.NewHelper(t.log)
fetch := func() {
// existing body of the loop
}
fetch() // initial fetch
ticker := time.NewTicker(time.Hour)
defer ticker.Stop()
for range ticker.C {
fetch()
}
}
```
This also avoids the unbounded lifetime of `time.Tick` and makes startup behavior predictable.
Suggested implementation:
```golang
func (t *Temperature) run(done chan error) {
var once sync.Once
client := request.NewHelper(t.log)
ticker := time.NewTicker(time.Hour)
defer ticker.Stop()
for range ticker.C {
```
To fully implement your suggestion (initial fetch + avoiding `time.Tick`):
1. Extract the *entire* body of the original `for` loop into a `fetch` closure that captures `client`, `once`, `done`, and `t`:
```go
fetch := func() {
uri := fmt.Sprintf(
"https://api.open-meteo.com/v1/forecast?latitude=%f&longitude=%f&hourly=temperature_2m&past_days=7&forecast_days=3&timezone=UTC",
t.latitude, t.longitude,
)
var res openMeteoResponse
if err := backoff.Retry(func() error {
return backoffPermanentError(client.GetJSON(uri, &res))
}, bo()); err != nil {
once.Do(func() { done <- err })
t.log.ERROR.Println(err)
return
}
// keep the rest of the loop body here (updating tariff, logging, etc.)
}
```
2. Place `fetch()` once before the ticker loop so that an initial fetch happens immediately:
```go
fetch()
ticker := time.NewTicker(time.Hour)
defer ticker.Stop()
for range ticker.C {
fetch()
}
```
3. Remove the now-obsolete `ticker := time.NewTicker(...)` / `for range ticker.C` from inside the loop (replaced by the code above).
You’ll need to adjust the exact contents of `fetch` to match the full original loop body, which is not completely visible in the provided snippet.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
update: discussion on temperature adjustment logic: #27873 |
Adds a new tariff type that fetches hourly outdoor temperatures from
Open-Meteo (free, no API key) and exposes them as api.Rates where
Rate.Value is the temperature in °C.
The tariff fetches 7 days of past actuals + 3 days of forecast in a
single API call (past_days=7&forecast_days=3&timezone=UTC), so consumers
have access to both historical and future temperature data.
Files changed:
- tariff/temperature.go: new Temperature tariff type, registered as
'open-meteo' in the tariff registry
- api/tariff.go: TariffTypeTemperature, TariffUsageTemperature
- api/globalconfig/types.go: Temperature config.Typed in Tariffs struct
- tariff/tariffs.go: Temperature api.Tariff field + Get() case
- cmd/setup.go: wire up conf.Temperature in configureTariffs()
Configuration:
tariffs:
temperature:
type: open-meteo
latitude: 48.137
longitude: 11.576
- INFO on startup: configured lat/lon - DEBUG on each hourly fetch: slot count and time range - WARN on timestamp parse errors (was already present) - ERROR on fetch failures (was already present) - Remove stray comment at end of file
- Use 'for ; true; <-time.Tick(interval)' pattern (same as solcast.go) so the first fetch happens immediately on startup instead of after 1h - Replace (0,0) guard with proper range validation: lat in [-90,90], lon in [-180,180], allowing the valid Gulf of Guinea coordinate
- Add Temperature field to TariffRefs struct - Update configureTariff call to use new signature (conf, deviceName, target) - Move Temperature configuration to 'resolve tariff roles' section
daniel309
force-pushed
the
feature/weather-dependent-household-load
branch
from
f56dc54 to
80710fc
Compare
|
ok, build is green. can you have another look @andig ? |
andig
changed the title
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- In
Temperature.runyou usetime.Tick(time.Hour), which creates a leaking ticker; consider switching to atime.NewTickerwith a properdefer ticker.Stop()and afor range ticker.Cloop to avoid goroutine and resource leaks. - The hard-coded
pastDays := 7andforecastDays := 3inTemperature.runcould be made configurable via the tariff config to allow tuning for different climates or use cases instead of recompiling for different horizons.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In `Temperature.run` you use `time.Tick(time.Hour)`, which creates a leaking ticker; consider switching to a `time.NewTicker` with a proper `defer ticker.Stop()` and a `for range ticker.C` loop to avoid goroutine and resource leaks.
- The hard-coded `pastDays := 7` and `forecastDays := 3` in `Temperature.run` could be made configurable via the tariff config to allow tuning for different climates or use cases instead of recompiling for different horizons.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
- Delete tariff/temperature.go (custom Go implementation) - Add templates/definition/tariff/temperature.yaml (template-based) - Add special handling in tariff.go to keep 7 days of historical data for TariffTypeTemperature - Uses same pattern as other forecast tariffs with jq transformation - Simpler, more maintainable, and consistent with project conventions Addresses code review feedback: the temperature tariff can be implemented using the existing template system with yaml/jq instead of custom Go code.
The template system only supports 'price', 'solar', and 'co2' device groups. Temperature forecasting is conceptually similar to solar/CO2 forecasting (all provide forecast data for optimization), so 'solar' group is appropriate. Fixes CI/CD error: 'could not find devicegroup definition: temperature'
- Renamed file: temperature.yaml → open-meteo-temperature.yaml - Updated template name to match filename - Changed brand to 'Open-Meteo Temperature' for uniqueness Fixes CI/CD error: 'product titles must be unique' (conflict with open-meteo.yaml)
|
@andig yaml/jq solution ready to merge. what kind of UI changes you have in mind? |
- Changed from hourly to minutely_15 (15-minute intervals) - Updated jq transformation: 3600s → 900s (15 minutes) - Provides 960 data points (10 days × 96 intervals/day) - Better alignment with evcc's 15-minute optimization slots - Updated descriptions and comments to mention 15-minute intervals This provides more accurate temperature data that matches evcc's optimization granularity.
|
@andig I would suggest to split UI changes into separate PR and open a tracking issue (epic) to tie everything together. After all, this is the base work for the temp correction algorithm for temperature-dependent household loads (heating) and ultimately for optimizer to deal properly with those loads. So the todo list is
|
- Separate heater load from base household consumption - Apply temperature correction only to heating devices (heat pumps, electric heaters) - Base loads (lighting, appliances) remain unchanged - Backward compatible: falls back to old behavior if no heating devices Changes: - Extended metrics DB to track per-loadpoint consumption - Added loadpoint energy tracking infrastructure in Site - Implemented profile extraction and aggregation functions - Modified homeProfile() to separate, correct, and merge profiles Addresses feedback on PR evcc-io#27780 that temperature adjustment should only apply to heating devices, not entire household consumption.
|
@andig @ekkea @t0mas I worked on the full implementation on my fork as a PR just to feel out the changes. Doesnt look too bad or complicated. see: daniel309#1 |
|
@andig @ekkea @t0mas I worked on the full implementation just to feel out the changes and test how it works. I am testing with a build from this branch and for the first time, optimizer now creates good estimates for household consumption including the heating. I am continuing to test, but I feel this is close to ready now. btw. PR #28232 looks heavy, but it is based on the temp tariff. once this is merged it will be less than 200 lines across 3 files. |
- Separate heater load from base household consumption - Apply temperature correction only to heating devices (heat pumps, electric heaters) - Base loads (lighting, appliances) remain unchanged - Backward compatible: falls back to old behavior if no heating devices Changes: - Extended metrics DB to track per-loadpoint consumption - Added loadpoint energy tracking infrastructure in Site - Implemented profile extraction and aggregation functions - Modified homeProfile() to separate, correct, and merge profiles Addresses feedback on PR evcc-io#27780 that temperature adjustment should only apply to heating devices, not entire household consumption.
|
Was it planned that the temperature-forecast could be optionally calibrated using a real temperature sensor (e.g., one that is read into evcc via Modbus TCP from the heating control system), similar to how PV production forecasts are handled (for now, as an experimental feature)? |
no, I didnt plan that. The open meteo data has pretty accurate (for my case at least) past temperature readings. If this is not accurate for your case, this could be a nice follow-on PR. |
4 participants
Update: restricted this PR to the new temperature tariff. The household load profile adjustments based on temperature will be handled separately in PR #28232
TODO
--
leaving the full PR description below for history and context of the discussion.
Problem
The evopt energy optimizer uses a household load profile (
gt) to predict how much energy the home will consume in each future 15-minute slot. This profile is currently computed as a 30-day historical average — the same flat pattern is repeated regardless of weather conditions.This is a significant blind spot: heating and cooling loads are strongly temperature-dependent. On a cold winter day, a home with a heat pump or electric heating can consume 30–80% more energy than on a mild day. When the optimizer doesn't know this, it:
Solution
This PR adds two things:
A new
open-meteoweather tariff that fetches 10 days of hourly outdoor temperatures from Open-Meteo (free, no API key, no account required) in a single call: 7 days of past actuals + 3 days of forecast. The data is exposed asapi.RateswhereRate.Valueis the temperature in °C.Temperature correction of the household load profile in
homeProfile(). The historical lookback is reduced from 30 days to 7 days as outdoor temperatures can significantly change within such a long period of time. Household life usually happens within 1 week rhythms so this seemed a better default. Before the profile is passed to the optimizer, each slot's load is adjusted based on how the forecast temperature at that slot deviates from the 7-day historical average at the same hour-of-day:where:
T_past_avg[h]= average temperature at hour-of-dayhover the past 7 daysT_forecast[i]= forecast temperature at the wall-clock time of slotiheatingCoefficient= fractional load sensitivity per °C (default 0.05 = 5%/°C)The correction is gated on the 24h average actual temperature of the past 24 hours. If that average is at or above
heatingThreshold(default 12°C), heating is considered off and no correction is applied to any slot. Using past actuals (not forecast) for the gate is more reliable: if it was cold yesterday, heating is likely still running today.Example: With default settings (
heatingThreshold=12°C,heatingCoefficient=0.05) and a 7-day historical average of 8°C at a given hour:The correction is relative to the historical average that produced the baseline load, so daily patterns (e.g. higher consumption in the evening) are preserved — only the magnitude is adjusted. This is consistent with standard heating degree-day methodology.
If no weather tariff is configured,
applyTemperatureCorrection()is a no-op and behaviour is identical to before.Files Changed
tariff/weather.goWeathertariff type polling Open-Meteo hourly (past_days=7&forecast_days=3)api/tariff.goTariffTypeWeatherandTariffUsageWeatherconstantsapi/globalconfig/types.goWeather config.TypedtoTariffsstructtariff/tariffs.goWeather api.Tarifffield;Get()handlesTariffUsageWeathercmd/setup.goconfigureTariffs()core/site.goHeatingThresholdandHeatingCoefficientconfig fieldscore/site_optimizer.gohomeProfile()calls newapplyTemperatureCorrection()andnearestRate()helperConfiguration
The feature is opt-in — no changes needed for existing setups.
To enable, add to
evcc.yaml:Tuning guidance
heatingThreshold: Set to the 24h average outdoor temperature above which your heating system is fully off. Typical values: 10–15°C depending on building insulation and personal preference. The default of 12°C suits an average insulated house.heatingCoefficient: Represents how sensitive your home's energy consumption is to temperature, as a fraction of the average load per °C. A well-insulated passive house might use 0.02; a poorly insulated older building might use 0.08 or higher. You can estimate it by comparing your historical energy consumption on cold vs. mild days.Notes
past_days=7&forecast_days=3&timezone=UTC.heatingThreshold) is based on past 24h actual temperatures (not forecast), preventing the correction from running in summer. Using actuals is more reliable: if it was warm yesterday, heating is off today regardless of what the forecast says.coolingThreshold/coolingCoefficientcould be added similarly in a follow-up.open-meteotariff type name follows the existing tariff registry naming convention (provider name, lowercase).TODO