Vantage Publisher

vantage-publisher.py reads live data from a Davis Vantage Pro2 console, optionally stores CSV rows locally, optionally publishes MQTT packets, and optionally sends Signal K deltas via websocket.

Documentation

The technical and operational manual provides a detailed account of the implementation, its data semantics, and its operational limits:

Features

Continuous station stream with reconnect
Parameter filtering via parameters.json
Optional local CSV storage
Optional MQTT publishing with offline queue
Optional direct Signal K websocket publishing
Dry run mode for configuration/debug checks
Built-in HTTP server for browsing stored CSV files (optional basic auth)

Requirements

Python 3.8+
Vantage Pro2 reachable as tcp:127.0.0.1:<usbPort> (typically through ser2net)

Install dependencies:

python3 -m pip install -r requirements.txt

Configuration (`config.json`)

{
  "uuid": "it.uniparthenope.meteo.ws1",
  "name": "Centro Direzionale",
  "airlinkId": "",
  "airlinkApiKey": "",
  "airlinkApiSecret": "",
  "lon": 14.2845,
  "lat": 40.8569,

  "storage": true,
  "mqtt": false,
  "signalk": false,

  "usbPort": 22222,
  "usbPollInterval": 1.0,
  "delay": 10,
  "timeout": 60,

  "pathStorage": "/storage/vantage-pro/",

  "mqttBroker": "mqtt-broker.local",
  "mqttPort": 1883,
  "mqttUser": "",
  "mqttPass": "",
  "mqttQos": 1,
  "mqttFormat": "flat",

  "signalkServerUrl": "ws://signalk.local:3000/signalk/v1/stream",
  "signalkToken": "",
  "signalkContext": "meteo.it.uniparthenope.meteo.ws1",
  "signalkPathMap": {
    "BarTrend": "environment.outside.pressureTrend",
    "Barometer": "environment.outside.pressure",
    "TempIn": "environment.inside.temperature",
    "HumIn": "environment.inside.humidity",
    "TempOut": "environment.outside.temperature",
    "WindSpeed": "environment.wind.speedApparent",
    "WindSpeed10Min": "environment.wind.speedAverage",
    "WindDir": "environment.wind.angleApparent",
    "HumOut": "environment.outside.humidity",
    "RainRate": "environment.rain.rate",
    "SolarRad": "environment.solar.radiation",
    "RainStorm": "environment.rain.storm.total",
    "StormStartDate": "environment.rain.storm.startDate",
    "RainDay": "environment.rain.day",
    "RainMonth": "environment.rain.month",
    "RainYear": "environment.rain.year",
    "ETDay": "environment.outside.evapoTranspiration.day",
    "ETMonth": "environment.outside.evapoTranspiration.month",
    "ETYear": "environment.outside.evapoTranspiration.year",
    "BatteryStatus": "electrical.batteries.sensor.status",
    "BatteryVolts": "electrical.batteries.sensor.voltage",
    "ForecastIcon": "environment.weather.forecast.icon",
    "ForecastRuleNo": "environment.weather.forecast.ruleNumber",
    "SunRise": "environment.sun.rise",
    "SunSet": "environment.sun.set"
  },

  "httpEnabled": false,
  "httpHost": "0.0.0.0",
  "httpPort": 8080,
  "httpUser": "",
  "httpPass": "",
  "httpRoot": "/storage/vantage-pro/",

  "offlineMaxMessages": 200000,
  "offlineMaxAgeSec": 604800,
  "airlinkIntervalSec": 300
}

Key runtime booleans

storage: enable/disable local CSV storage (default: true)
mqtt: enable/disable MQTT publishing (default: false)
signalk: enable/disable Signal K websocket publishing (default: false)

mqttFormat supports:

flat (default/fallback)
geojson

AirLink credentials:

airlinkApiKey: WeatherLink API key for current conditions API
airlinkApiSecret: WeatherLink secret used in X-Api-Secret

Parameters file (`parameters.json`)

Boolean map of station fields:

true: include field
false: exclude field

If the file is missing, all fields are included.

Command line options

--config <path> config file path (default config.json)
--parameters <path> parameters file path (default parameters.json)
--signalk true|false override config signalk
--mqtt true|false override config mqtt
--storage true|false override config storage
--dry dry mode (no storage, no MQTT/Signal K/http connections; packets/rows logged only)

Usage examples

# Use config defaults
python3 vantage-publisher.py

# Enable MQTT and storage explicitly
python3 vantage-publisher.py --mqtt true --storage true

# Enable Signal K direct websocket together with MQTT and storage
python3 vantage-publisher.py --signalk true --mqtt true --storage true

# Dry mode validation (live console reads; no output publishing or CSV storage)
python3 vantage-publisher.py --dry

# Custom config + parameters
python3 vantage-publisher.py \
  --config /etc/vantage/config.json \
  --parameters /etc/vantage/parameters.json

Dry mode behavior

When --dry is active:

MQTT connection/publish is disabled
Signal K websocket connection/publish is disabled
local CSV writes are disabled
HTTP storage server is disabled
station reads still connect to ser2net; --dry is not an offline configuration validator
the configured delay applies between output cycles
generated outputs are logged (including datetime-valued station fields):
- CSV_ROW;...
- MQTT_PACKET;...
- SIGNALK_UPDATE;...

HTTP server for local storage

If httpEnabled is true, the app starts an HTTP server exposing httpRoot.

Configuration keys:

httpEnabled (true|false)
httpHost (default 0.0.0.0)
httpPort (default 8080)
httpRoot directory to serve (default pathStorage)
httpUser optional basic auth username
httpPass optional basic auth password

Authentication behavior:

if httpUser is empty, no authentication is required
if httpUser is set, HTTP Basic Auth is required

MQTT payloads

`flat`

{
  "Datetime": "2026-02-24T10:15:40Z",
  "TempOut": 12.7,
  "WindSpeed": 3,
  "position": { "latitude": 40.8569, "longitude": 14.2845 },
  "name": "Centro Direzionale"
}

`geojson`

{
  "type": "Feature",
  "geometry": {
    "type": "Point",
    "coordinates": [14.2845, 40.8569]
  },
  "properties": {
    "Datetime": "2026-02-24T10:15:40Z",
    "TempOut": 12.7,
    "uuid": "it.uniparthenope.meteo.ws1",
    "name": "Centro Direzionale"
  }
}

MQTT topic is always uuid.

All MQTT packets are written to the SQLite queue before publishing. A bounded batch is advanced each main-loop cycle, even when the station has no new readings. Records are removed only after Paho reports publish completion (broker acknowledgment for QoS 1/2; transmission for QoS 0). Queue age and size limits still apply, including before replay after a restart. A crash between delivery and queue deletion can cause duplicate delivery; consumers should tolerate duplicates. QoS 0 does not provide a broker acknowledgment. See Paho publish completion documentation.

Signal K deltas

When Signal K is enabled (signalk=true or --signalk true), the publisher sends deltas with:

context: signalkContext (default meteo.<uuid>)
navigation.position: station lat/lon
remaining fields:
- from signalkPathMap if present
- otherwise standard mappings for common weather keys
- otherwise, fallback to environment.<field>.

Direct Signal K configuration

To publish directly to a Signal K server, set:

signalk: true
signalkServerUrl: websocket stream endpoint (ws://... or wss://...)
signalkToken: API token (if your Signal K server requires authentication)
signalkContext: target context (usually meteo.<uuid>)

Runtime behavior when Signal K is enabled:

The publisher checks whether Signal K security is enabled.
If security is enabled and signalkToken is missing or invalid, it automatically submits an access request.
While waiting for token approval, the main loop continues other enabled operations (CSV storage and/or MQTT).
The publisher periodically re-checks access request status and token validity.
As soon as a valid token is available, it is saved into config.json (signalkToken) and direct websocket publishing starts automatically.

Note:

Datetime, DatetimeWS, position, and name are handled internally by the publisher and are not required in signalkPathMap.

For the Signal K server https://signalk.meteo.uniparthenope.it, use the websocket stream URL: wss://signalk.meteo.uniparthenope.it/signalk/v1/stream

Example `config.json` (Signal K direct mode)

{
  "uuid": "it.uniparthenope.meteo.ws1",
  "name": "Centro Direzionale",
  "airlinkId": "",
  "airlinkApiKey": "",
  "airlinkApiSecret": "",
  "lon": 14.2845,
  "lat": 40.8569,

  "storage": true,
  "mqtt": false,
  "signalk": true,

  "usbPort": 22222,
  "usbPollInterval": 1.0,
  "delay": 10,
  "timeout": 60,

  "pathStorage": "/storage/vantage-pro/",

  "mqttBroker": "",
  "mqttPort": 1883,
  "mqttUser": "",
  "mqttPass": "",
  "mqttQos": 1,
  "mqttFormat": "flat",

  "signalkServerUrl": "wss://signalk.meteo.uniparthenope.it/signalk/v1/stream",
  "signalkToken": "REPLACE_WITH_SIGNAL_K_TOKEN",
  "signalkContext": "meteo.it.uniparthenope.meteo.ws1",
  "signalkPathMap": {
    "BarTrend": "environment.outside.pressureTrend",
    "Barometer": "environment.outside.pressure",
    "TempIn": "environment.inside.temperature",
    "HumIn": "environment.inside.humidity",
    "TempOut": "environment.outside.temperature",
    "WindSpeed": "environment.wind.speedApparent",
    "WindSpeed10Min": "environment.wind.speedAverage",
    "WindDir": "environment.wind.angleApparent",
    "HumOut": "environment.outside.humidity",
    "RainRate": "environment.rain.rate",
    "SolarRad": "environment.solar.radiation",
    "RainStorm": "environment.rain.storm.total",
    "StormStartDate": "environment.rain.storm.startDate",
    "RainDay": "environment.rain.day",
    "RainMonth": "environment.rain.month",
    "RainYear": "environment.rain.year",
    "ETDay": "environment.outside.evapoTranspiration.day",
    "ETMonth": "environment.outside.evapoTranspiration.month",
    "ETYear": "environment.outside.evapoTranspiration.year",
    "BatteryStatus": "electrical.batteries.sensor.status",
    "BatteryVolts": "electrical.batteries.sensor.voltage",
    "ForecastIcon": "environment.weather.forecast.icon",
    "ForecastRuleNo": "environment.weather.forecast.ruleNumber",
    "SunRise": "environment.sun.rise",
    "SunSet": "environment.sun.set"
  },

  "httpEnabled": false,
  "httpHost": "0.0.0.0",
  "httpPort": 8080,
  "httpUser": "",
  "httpPass": "",
  "httpRoot": "/storage/vantage-pro/",

  "offlineMaxMessages": 200000,
  "offlineMaxAgeSec": 604800,
  "airlinkIntervalSec": 300
}

Step-by-step

Create a local config from the sample:
- cp config.json.sample config.json
Edit config.json and set:
- signalk to true
- signalkServerUrl to wss://signalk.meteo.uniparthenope.it/signalk/v1/stream
- signalkToken to a valid token from your Signal K server
Keep mqtt as false if you only want direct Signal K publishing.
Start the publisher:
- python3 vantage-publisher.py --config config.json --signalk true
If Signal K security is enabled, approve the pending access request on the Signal K server UI/API.
Wait for the periodic token check; the publisher will save the approved token into config.json and begin websocket publishing automatically.
Verify updates on Signal K:
- check that context meteo.it.uniparthenope.meteo.ws1 receives navigation.position and weather paths.
Optional validation mode:
- run python3 vantage-publisher.py --dry to inspect generated SIGNALK_UPDATE logs without network publish.

Archive collection (`collect-history.py`)

collect-history.py downloads archive records from the station using PyVantagePro normalized JSON rows (get_archives_as_json), so values are exported in SI-oriented units already provided by the library.

Behavior:

Uses logging (no print)
Supports start/stop date range from command line
Applies parameters.json filtering (if provided)
Writes CSV only when --output is provided
If --output is omitted, rows are logged to console

Options

--url <station-url> station connection URL (default: tcp:127.0.0.1:22222)
--timeout <seconds> read timeout (default: 10)
--start <ISO-datetime> archive start datetime
--stop <ISO-datetime> archive stop datetime (optional)
--parameters <file> parameters map file (default: parameters.json)
--output <file.csv> output file path (optional)
--log-level <LEVEL> logger level (default: INFO)

Examples

# Collect from 2026-03-01 to 2026-03-08 and write CSV
python3 collect-history.py \
  --start 2026-03-01T00:00:00 \
  --stop 2026-03-08T00:00:00 \
  --output /tmp/history.csv

# Collect from a date and log rows to console only
python3 collect-history.py --start 2026-03-07T00:00:00

Storage layout

CSV files (hourly rotation): <pathStorage>/<uuid>/<YYYY>/<MM>/<DD>/<uuid>_<YYYYMMDD>Z<HH>00.csv
- example: /storage/vantage-pro/it.uniparthenope.meteo.ws1/2026/02/26/it.uniparthenope.meteo.ws1_20260226Z1400.csv
MQTT offline queue DB: <pathStorage>/mqtt_offline_queue.sqlite (or mqttSpoolFile)

CSV schema expansion uses an atomic file replacement so a failed rewrite preserves the existing CSV. If expansion fails, the new row is skipped and the error is logged. An empty pathStorage skips CSV storage, as reported at startup.

CI/CD

GitHub Actions checks Python 3.8, 3.12, and 3.13, builds the runtime image, and smoke-tests its real dependencies. Successful pushes to main and v* tags publish the tested linux/amd64 image to ghcr.io/ccmmma/vantage-publisher. Pull requests and manual runs validate without publishing. Station rollout remains operator-controlled.

See CI/CD documentation for permissions, image tags, installation, and validation limits.

Development validation

python3 -m unittest discover -s tests -v
python3 -m py_compile vantage-publisher.py airlink.py collect-history.py tests/test_publisher.py

Regression tests use temporary storage and mocked station/network clients; they do not require a live station or installed network dependencies. Live station, MQTT, and Signal K integration should be checked in the deployment environment.

make run mounts configuration at the runtime's default paths under /vantage-publisher. Use Docker Compose for the supplied persistent storage mount.

License

Apache-2.0

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
docs		docs
tests		tests
util		util
.dockerignore		.dockerignore
.gitignore		.gitignore
AGENTS.md		AGENTS.md
Dockerfile		Dockerfile
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
airlink.py		airlink.py
collect-history.py		collect-history.py
config.json.sample		config.json.sample
docker-compose.yml		docker-compose.yml
get-docker.sh		get-docker.sh
parameters.json.sample		parameters.json.sample
requirements.txt		requirements.txt
updater.sh		updater.sh
vantage-publisher.py		vantage-publisher.py

Navigation Menu

Uh oh!

Repository files navigation

Vantage Publisher

Documentation

Features

Requirements

Configuration (`config.json`)

Key runtime booleans

Parameters file (`parameters.json`)

Command line options

Usage examples

Dry mode behavior

HTTP server for local storage

MQTT payloads

`flat`

`geojson`

Signal K deltas

Direct Signal K configuration

Example `config.json` (Signal K direct mode)

Step-by-step

Archive collection (`collect-history.py`)

Options

Examples

Storage layout

CI/CD

Development validation

License

About

Releases

Packages

Contributors

Languages

Uh oh!

Latest commit

History

Folders and files

Repository files navigation

Vantage Publisher

Documentation

Features

Requirements

Configuration (config.json)

Key runtime booleans

Parameters file (parameters.json)

Command line options

Usage examples

Dry mode behavior

HTTP server for local storage

MQTT payloads

flat

geojson

Signal K deltas

Direct Signal K configuration

Example config.json (Signal K direct mode)

Step-by-step

Archive collection (collect-history.py)

Options

Examples

Storage layout

CI/CD

Development validation

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Configuration (`config.json`)

Parameters file (`parameters.json`)

`flat`

`geojson`

Example `config.json` (Signal K direct mode)

Archive collection (`collect-history.py`)