smartmeter-fetch

Logs into grid operator smart meter web portals (starting with Netz NÖ) and fetches quarter-hourly consumption and production readings, stores them locally, and serves them over a small versioned HTTP API for other tools to consume. Companion to hass-smartmeter, a Home Assistant integration that reads from that API — this tool only fetches and stores, it doesn't know or care about Home Assistant.

Disclaimer: Built for personal, educational use only. This is an independent, unofficial tool, not affiliated with, endorsed by, or supported by Netz NÖ or any other grid operator. It drives the operator's web portal using the same requests your browser makes, which may not be permitted under that portal's Terms of Service — check those before using it. Use is entirely at your own risk; see Disclaimer below.

Install

Each release ships a .deb package alongside plain binaries. On a Debian/Ubuntu server:

curl -LO https://github.com/welworx/smartmeter-fetch/releases/latest/download/smartmeter-fetch_<version>_linux_amd64.deb
sudo apt install ./smartmeter-fetch_<version>_linux_amd64.deb

(use arm64 instead of amd64 on ARM hosts). No apt repository is hosted — this installs the downloaded file directly and won't auto-update; re-run the above with a new version to upgrade.

Status

The EVN/Netz NÖ provider, the JSON file store, and a CLI to drive them (list-points, fetch, get, profile, upgrade) are implemented. fetch persists readings to -data-dir without printing them by default (pass -json for that). get ensures a date range is fetched, then reads it back — as a table, JSON, or CSV, optionally aggregated (-sample raw|hour|day|week|month|quarter, default raw) and optionally exported to templated file(s) via -out. Each file -out renders is replaced wholesale on every run (never appended to), so an export is only ever as complete as the range of the run that wrote it — -data-dir stays the source of truth. A day already present in -data-dir is skipped rather than re-fetched, unless -force is passed. serve runs the /v1 query API (GET /v1/points, GET /v1/readings) against -data-dir — see Container below for running it as a long-lived podman container.

Each reading carries a quality code when the portal reports one: L2 (substitute value, final) or L3 (substitute value, still provisional — may change on a later fetch). Measured values leave quality empty for now — the portal's own UI calls these L1, but that code isn't present anywhere in the API response, only inferred from context, so it isn't hardcoded; see internal/provider/evn/evn.go's dayRecord comment.

export SMARTMETER_USER=you@example.com
export SMARTMETER_PASSWORD=hunter2

smartmeter-fetch list-points
smartmeter-fetch fetch -point <id> -day 2024-01-15
smartmeter-fetch fetch -point <id> -from 2024-01-01 -to 2024-01-31  # date range
smartmeter-fetch fetch -point <id> -from -30 -to -20                # or days before today; -to defaults to today
smartmeter-fetch fetch -since-latest                      # resume every point from its last stored day
smartmeter-fetch fetch -point <id> -day 2024-01-15 -log-level debug
smartmeter-fetch fetch -point <id> -day 2024-01-15 -verbose         # shorthand for -log-level debug
smartmeter-fetch get -point <id> -day 2024-01-15                          # read back a day (fetches first if needed)
smartmeter-fetch get -point <id> -from -365 -sample day -format csv       # a year of daily totals as CSV
smartmeter-fetch get -sample day -out "data/<profile>/<zaehlerpunkt_id>/<yyyy>.csv"  # export every point/year, no real IDs in paths

Credentials can also be stored once, encrypted under a master passphrase (argon2id + AES-256-GCM, in credentials.enc under the OS config dir), via smartmeter-fetch profile add <name> — verified live against the portal before saving, and recheckable later with smartmeter-fetch profile verify. See smartmeter-fetch help for the full profile command reference.

smartmeter-fetch upgrade checks GitHub for a newer release and can download, verify, and install it in place:

smartmeter-fetch upgrade -check   # check only, don't install
smartmeter-fetch upgrade -y       # install the latest release, no confirmation prompt

Environment variables

Variable	Purpose
SMARTMETER_USER	Portal username. Same as -user; -user wins if both are set.
SMARTMETER_PASSWORD	Portal password. Same as -password; -password wins if both are set.
SMARTMETER_PASSPHRASE	credentials.enc master passphrase, skips the interactive prompt (used by profile commands and, as a fallback, by list-points/fetch/get when reading a stored -profile).
SMARTMETER_CONFIG_DIR	Directory holding credentials.enc (default: OS config dir, e.g. ~/Library/Application Support/smartmeter-fetch on macOS).
SMARTMETER_DATA_DIR	Directory fetch/get persist readings under (default: ./data). Same as -data-dir; -data-dir wins if both are set.
SMARTMETER_ADDR	Address serve listens on. Same as -addr; -addr wins if both are set. (default: :8790)

Architecture

Two small interfaces are the whole design:

type Provider interface {
    Name() string
    ListPoints(ctx context.Context) ([]Point, error)
    FetchDay(ctx context.Context, pointID string, day time.Time) ([]Reading, error)
    Location() *time.Location // day-boundary timezone, e.g. FetchDay's "day"
}

type Store interface {
    Put(ctx context.Context, provider, pointID string, readings []Reading, loc *time.Location) error
    Get(ctx context.Context, provider, pointID string, since time.Time) ([]Reading, error)
    Latest(ctx context.Context, provider, pointID string, loc *time.Location) (day time.Time, found bool, err error)
    Has(ctx context.Context, provider, pointID string, day time.Time, loc *time.Location) (bool, error)
    ListPoints(ctx context.Context) ([]PointRef, error)
}

• internal/provider/evn — the first Provider implementation, for smartmeter.netz-noe.at. More grid operators can be added as siblings here without touching storage or the API.
• internal/store/jsonfile — the first Store implementation: one JSON file per provider/metering point/day (data/<provider>/<point>/<date>.json, default -data-dir), written atomically so a delayed/backfilled day never gets read half-written. Other backends (Postgres, MariaDB, SQLite) can be added later behind the same interface if a single user ever needs them — not built speculatively.
• internal/api — a versioned (/v1) HTTP query API. This is the only thing consumers like hass-smartmeter ever talk to; they never see which provider or storage backend is behind it.

Query API (v1)

• GET /v1/points — discover known metering points across all configured providers.
• GET /v1/readings?point=<id>&since=<RFC3339> — cursor-based read, so consumers can resume from their own last-seen timestamp instead of re-fetching everything.

Container

A multi-arch (linux/amd64, linux/arm64) image is published to ghcr.io/welworx/smartmeter-fetch on every release. The container's main process is smartmeter-fetch serve, listening on :8790 — it only reads -data-dir, so it needs no portal credentials. New data is pulled in by running fetch -since-latest inside the same running container on a schedule, from the host's crontab:

podman run -d \
  --name smartmeter-fetch \
  -e SMARTMETER_USER=you@example.com \
  -e SMARTMETER_PASSWORD=hunter2 \
  -v ./data:/data:U \
  -e SMARTMETER_DATA_DIR=/data \
  -p 8790:8790 \
  --user 65532:65532 \
  ghcr.io/welworx/smartmeter-fetch:latest

# host crontab (crontab -e) — fetch new readings every 15 minutes
*/15 * * * * podman exec smartmeter-fetch /smartmeter-fetch fetch -since-latest

The image runs as the distroless nonroot user (UID/GID 65532), so the mounted data directory has to be writable by that user. The :U on the -v mount tells podman to chown the source directory to whatever host UID/GID the container's 65532 actually maps to — which is the point: a plain chown -R 65532:65532 ./data on the host is only correct for rootful podman/docker. Under rootless podman (the default) container UID 65532 maps into your subuid range, not to host UID 65532, so the container still can't write and fetch fails on its first write while the API happily keeps serving []. The rootless equivalent of a manual chown is podman unshare chown -R 65532:65532 ./data; :U does it for you and is correct in both modes.

On SELinux hosts (Fedora, RHEL, CoreOS) the mount also needs a relabel option or the container is denied access outright: use -v ./data:/data:U,Z (Z = private label for this container; z = shared label, if several containers mount the same directory).

See docs/podman-compose.example.yml for the same setup via podman-compose. The API's OpenAPI 3.0 description is served at GET /openapi.json.

The first release pushes the GHCR package as private — flip it to public once, by hand, under the repo's Packages settings. The org.opencontainers.image.source label links the package to this repo but does not change its visibility.

Disclaimer

This project is not affiliated with, endorsed by, or supported by Netz NÖ, EVN, or any other grid operator. It is provided "as is", without warranty of any kind. Scraping a provider's web portal may violate its Terms of Service — you are responsible for checking and complying with those terms. Use at your own risk.