Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ roots.
`/home/<user>/Sensorius/` on Linux or `/Users/<user>/Sensorius/` on macOS.
- Inside pytest, relative roots remain relative for test isolation.
- Absolute paths are used unchanged.
- The 'source of truth' for the device toml files is the device toml files

Canonical runtime state:

Expand Down
13 changes: 12 additions & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -554,11 +554,22 @@ Barometric pressure gauges use the altitude entered in General Settings
(`[Astral].ALTITUDE`, metres) to center raw/absolute pressure ranges on standard
atmospheric pressure at that elevation, with at least 50 hPa of weather margin
on either side. Metric bounds round outward to 10 hPa; Imperial bounds round
outward to 0.1 inHg. Refresh the dashboard after changing altitude. Blank or
outward to 0.1 inHg. Dashboard refreshes update the range after altitude changes. Blank or
invalid altitude retains the broad default range for raw pressure.

Sea-level-corrected readings (Ecowitt relative pressure, WeeWX barometer, and
BME/VPD/AQI sensors with altitude calibration) instead use a sea-level range:
960–1070 hPa or 28.4–31.4 inHg. Explicit absolute-pressure metrics always use
site altitude. Gauge scaling does not change measurements, history, MQTT, or
calibration values.

For Nodus sensors, calibration altitude comes from the device's TOML via
retained MQTT metadata (`sensor.calibration.Device.ALTITUDE_METERS`, or the
corresponding child in `sensors`). Sensorius mirrors explicit values, including
zero, into its sensor settings shadow. Missing fields in older firmware do not
overwrite an existing mirrored altitude. Sensorius does not infer calibration
from pressure readings or substitute the hub altitude for device calibration.
An open dashboard receives updated calibration context with its regular JSON
refresh and updates gauge limits, tick labels, and color bands without a page
reload. The calibration form's "System Altitude" comes from General Settings;
it is not confirmation that the remote device's altitude has been mirrored.
116 changes: 116 additions & 0 deletions docs/nodus_metadata_gaps.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# Nodus metadata gaps and firmware handoff

Audit date: 2026-09-20. Compared the local cPyNodus_II and cPyNodus_III
payload builders with Sensorius MQTT ingest and settings materialization.
This is a source-code audit, not a complete hardware validation.

The device TOML files are authoritative. Sensorius shadows are caches of
explicitly advertised device values, not replacement configuration sources.

## Confirmed gaps

| Area | Current firmware behavior | Consequence / proposed addition |
| --- | --- | --- |
| Pressure calibration | Both versions omit calibration altitude from retained startup `meta`. | Publish `sensor.calibration.Device.ALTITUDE_METERS`; repeat in each `sensors[]` child on III. This fixes the pressure range mismatch. |
| Other calibration values | Both startup snapshots omit `Calibration.System` and `Calibration.Device`; calibration-status builders contain status/identity/time and optional extra fields, not a complete calibration snapshot. | Temperature, RH, CO2, AQI/gas, light/PPFD, APVPD and soil offsets can remain factory defaults in a new shadow, or stale in an existing shadow. Publish supported TOML calibration sections and persisted calibration status. |
| Single-sensor identity | II's sensor block lacks logical `device` and `config_file`. III supplies those fields for multi-sensor entries but omits them from its primary compatibility block. | Always include both fields in `sensor` and every `sensors[]` entry. Avoid inferring device family from the sensor ID or guessing a configuration filename. |
| Time and Home Assistant configuration | Both startup builders omit these sections, although Sensorius accepts their `meta/patch` deltas and has shadow sections for them. | A new subscriber cannot reconstruct the device's time/HA configuration from retained startup metadata. Add explicit snapshots if these sections are intended to be mirrored. |
| Switch configuration completeness | Both retained `meta/switch` builders publish channel identities, labels, state and topics, but omit `enable_pin`, physical `pin`, and `override_script`. | Remote switching has enough information to work, but the resulting shadow is not an exact physical configuration mirror. Publish these fields if the shadow must expose device configuration; keep physical pins separate from remote channel identity. |

For calibration, include the keys actually supported by each sensor, preserving
explicit zeros and false values. Known keys include:

- `Calibration.Device`: `ALTITUDE_METERS`, `TEMP_OFFSET`, `RH_OFFSET`,
`CO2_OFFSET`, `AQI_OFFSET`, `GAS_OFFSET`, `LUX_OFFSET`, `PPFD_OFFSET`,
`APVPD_TEMP_CAL_VAL`, `APVPD_RH_CAL_VAL`, `SOIL_TEMP_CAL_VAL`,
`SOIL_MOIST_CAL_VAL`, `SOIL_PH_CAL_VAL`, `SOIL_EC_CAL_VAL`.
- `Calibration.System`: applicable offsets and persisted reference fields
such as `REF_SENSOR_ID`, `REF_RANGE_HOURS`, `REF_START_TS`, `REF_END_TS`,
and `REF_NOTE`, where supported by the firmware's TOML schema.
- `Calibration`: persisted `CALIBRATED` and `CALIB_STATUS`, where supported.
Keep persisted calibration state distinct from transient session status.

Suggested Time fields: `AUTO_TIMEZONE`, `TZ`, `TZ_OFFSET`, `TZ_NAME`,
`NTP_SERVER`, `NTP_SERVER_IP`. Suggested HomeAssistant fields:
`DISCOVERY_PREFIX`, `BASE_TOPIC`, `PUBLISH_DISCOVERY_RETAIN`,
`PUBLISH_STATE_RETAIN`, `PUBLISH_LEGACY_SENSOR_TOPIC`.
These broader additions require agreed casing and Sensorius full-snapshot
materialization; the pressure branch does not implement all of them.

## Pressure payload supported by this branch

Add this to the existing sensor object; preserve all existing fields:

```json
{
"sensor_id": "avpd-1jm5s1",
"device": "avpd",
"config_file": "sensor_i2c.toml",
"calibration": {
"Device": {
"ALTITUDE_METERS": 1719.0
}
}
}
```

Use the value loaded from that sensor's device TOML. Numeric strings such as
`"1719.0"` are accepted by Sensorius as well. Zero explicitly means zero;
omission means no information, and must not reset the hub's existing shadow.
For multi-sensor devices, use each child's own altitude and keep the primary
`sensor` compatibility view consistent with its corresponding child.

Sensorius now mirrors this field into the sensor shadow through its settings
manager. The existing dashboard then uses the mirrored device calibration to
select 960–1070 hPa / 28.4–31.4 inHg for corrected pressure. No reading-based
inference or device calibration writes are required. Regular dashboard JSON
refreshes pick up new calibration context and update existing gauge ranges.

## Delivery and replay requirements

Keep correlated `meta/patch` updates for accepted changes. Those patches are
non-retained, so also ensure the corresponding retained snapshot becomes
current after a persisted configuration change. Otherwise, a hub that was
offline during a patch can later receive the old startup value. Current
command handlers publish non-retained deltas; startup/recovery builders are
separate paths. Verify snapshot freshness for MQTT edits, local web edits,
and device restarts.

For larger calibration/configuration snapshots, a separately advertised,
retained topic can keep the main discovery packet small. Such a split is a
proposal and is not yet implemented in this branch; it requires coordinated
subscriber support. The pressure field above can be added to existing `meta`
without a new topic.

Verify firmware changes with:

1. Startup from TOML with nonzero altitude and explicit zero altitude.
2. Different calibration values for two children on one device.
3. An accepted patch followed by a newly connected hub reading retained data.
4. Offline hub during a device configuration edit, then hub reconnect.
5. Repeated snapshots: no redundant shadow writes.
6. Partial/older metadata: preserve previously mirrored fields.

## Other differences and limits

III uses `password_configured` flags in discovery; II still publishes
obfuscated password fields. This is a compatibility difference, not a reason
to add credentials to metadata. Align on configured flags when reconciling the
contracts. Current Sensorius contract prose still describes older credential
fields and needs a separate reconciliation with the firmware.

Sensor bus wiring and III-specific web display, weather, notification and
Ecowitt configuration are also absent from the compact discovery snapshot.
They are outside the current Sensorius remote-sensor discovery contract;
include them only if a complete device-configuration mirror is intended.

Source locations inspected:

- `/Users/twfarley/Projects/cPyNodus_II/cpynodus_ii/features/payloads.py`
- `/Users/twfarley/Projects/cPyNodus_III/cpynodus_iii/features/payloads.py`
- `/Users/twfarley/Projects/cPyNodus_III/cpynodus_iii/features/command_handlers.py`
- `/Users/twfarley/Projects/cPyNodus_III/cpynodus_iii/features/publish_cycle.py`
- `/Users/twfarley/Projects/cPyNodus_III/cpynodus_iii/features/web_handlers.py`
- `/Users/twfarley/Projects/Sensorius_AI/sensorius/saiMQTTIngest.py`

No firmware or live device configuration was modified during this audit.
11 changes: 9 additions & 2 deletions docs/sensorius_contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,10 @@ The retained status payload is:
```

`channels` lists canonical Nodus channel IDs and their enabled Advanced-rule
names. An empty list explicitly clears prior ownership. The retained
display names from `script_json.name`, falling back to the rule ID only when
the name is missing or blank. Nodus displays these strings directly; renaming
a rule updates the retained status on the next publisher scan.
An empty list explicitly clears prior ownership. The retained
availability payload uses schema `nodus-automation-availability/v1`, the same
controller fields, `updated_at`, and `status` set to `online` or `offline`.

Expand Down Expand Up @@ -301,9 +304,13 @@ The payload must include:
`sensor.data_topic`,
`sensor.event_topic`, `sensor.availability_topic`,
`sensor.display_metrics`, `sensor.display_styles`
- optional `sensor.calibration.Device.ALTITUDE_METERS`, reflecting the device
TOML calibration altitude in metres (number or numeric string); Sensorius
mirrors explicit values, including zero, and preserves existing shadow
calibration when the field is omitted by older firmware
- optional `sensors` array for a multi-sensor Nodus; every entry carries its
own `sensor_id`, logical device, hardware, `config_file`, location, display
settings, data topic, event topic, and availability topic
settings, calibration altitude, data topic, event topic, and availability topic
- `switch.device_id`, `switch.channel_count`, and `switch.meta_topic` when
switch capability is present

Expand Down
2 changes: 1 addition & 1 deletion sensorius/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@
release notes, and supporting tooling can report a consistent build identity.
"""

__version__ = "v0.26.262.2"
__version__ = "v0.26.263.3"
16 changes: 13 additions & 3 deletions sensorius/saiHtml.py
Original file line number Diff line number Diff line change
Expand Up @@ -2325,8 +2325,8 @@ def _format_stat_value(stat_value):
yield "<script type='module'>"
yield "\"use strict\";"
yield f"import {{ pressureGaugeConfig }} from '/ui_static/js/pressure_gauges.js?v={APP_VERSION}';"
yield f"const pressureAltitude = {json.dumps(pressure_altitude)};"
yield f"const pressureSensorContext = {json.dumps(pressure_sensor_context or {})};"
yield f"let pressureAltitude = {json.dumps(pressure_altitude)};"
yield f"let pressureSensorContext = {json.dumps(pressure_sensor_context or {})};"
yield f"import {{ startRainGauges }} from '/ui_static/js/rain_gauges.js?v={APP_VERSION}';"

yield "let stepCount = 0;"
Expand Down Expand Up @@ -4884,7 +4884,7 @@ def _format_stat_value(stat_value):
yield " const gaugeValue = convertForDisplay(value, config);"
yield " const existingGauge = window[`${safe}_gauge`];"
yield " if (existingGauge && existingGauge.__sensoriusCanvas === canvas && typeof existingGauge.set === 'function') {"
yield " if (config.rain_period && existingGauge.maxValue !== config.max) {"
yield " if (existingGauge.maxValue !== config.max || existingGauge.minValue !== config.min) {"
yield " existingGauge.maxValue = config.max;"
yield " existingGauge.setMinValue(config.min);"
yield " existingGauge.setOptions({...existingGauge.options, staticZones: config.zones, staticLabels: {...existingGauge.options.staticLabels, labels: config.ticks}});"
Expand Down Expand Up @@ -5272,6 +5272,16 @@ def _format_stat_value(stat_value):
yield " });"
yield " if (!d || typeof d !== 'object' || !d.values || typeof d.values !== 'object' || Array.isArray(d.values)) throw new Error('Invalid dashboard response');"
yield " for (const [sid, latest] of Object.entries(d.values || {})) currentValues[sid] = Object.assign({}, latest);"
yield " let pressureContextChanged = false;"
yield " if (Object.prototype.hasOwnProperty.call(d, 'pressure_altitude') && d.pressure_altitude !== pressureAltitude) {"
yield " pressureAltitude = d.pressure_altitude;"
yield " pressureContextChanged = true;"
yield " }"
yield " if (d.pressure_sensor_context && JSON.stringify(d.pressure_sensor_context) !== JSON.stringify(pressureSensorContext)) {"
yield " pressureSensorContext = d.pressure_sensor_context;"
yield " pressureContextChanged = true;"
yield " }"
yield " if (pressureContextChanged) initGauge();"
yield " __lastJsonOnly = d;"
yield " __lastJsonOnlyAtMs = Date.now();"
yield " try {"
Expand Down
37 changes: 37 additions & 0 deletions sensorius/saiMQTTIngest.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,31 @@
MQTT_CALLBACK_SLOW_LOG_INTERVAL_SEC = 60.0

# module helpers
def _merge_nodus_pressure_calibration(doc: dict, calibration: object) -> bool:
"""Mirror an explicitly advertised device altitude without inventing defaults."""
if not isinstance(calibration, dict):
return False
device = calibration.get("Device")
if not isinstance(device, dict):
return False
raw = device.get("ALTITUDE_METERS")
if raw is None or isinstance(raw, bool):
return False
try:
altitude = float(raw)
except (TypeError, ValueError):
return False
if not -500 <= altitude <= 10000:
return False
for name in ("Calibration", "Device"):
if not isinstance(doc.get(name), dict):
doc[name] = OrderedDict()
doc = doc[name]
if doc.get("ALTITUDE_METERS") == altitude:
return False
doc["ALTITUDE_METERS"] = altitude
return True

def _slugify(text: str) -> str:
return (text or "").strip().lower().replace(" ", "_")

Expand Down Expand Up @@ -3243,6 +3268,14 @@ def _sensor_targets() -> list[dict]:
child_name = section.split(".", 1)[1].strip()
if not child_name:
return False
if (section_key == "calibration.device" and key_upper == "ALTITUDE_METERS"
and not (update.get("sensor_id") or update.get("name"))):
# Legacy untargeted patches apply to the primary sensor. Keep
# its retained snapshot consistent with the shadow write below.
primary = meta.get("sensor") or {}
primary_id = primary.get("sensor_id") if isinstance(primary, dict) else None
if primary_id:
self._apply_nodus_meta_patch_update(meta, {**update, "sensor_id": primary_id})
targets = _sensor_targets()
if update.get("sensor_id") or update.get("name"):
for sensor in targets:
Expand Down Expand Up @@ -4091,6 +4124,7 @@ def _coerce_switch_state(raw_state) -> bool | None:
"config_file": config_file,
"display_metrics": display_metrics,
"display_styles": display_styles,
"calibration": sensor_blob.get("calibration"),
})
if sensor_ids_for_host:
self.nodus_host_sensors[base] = list(sensor_ids_for_host)
Expand Down Expand Up @@ -6034,6 +6068,7 @@ def _canonical_location(value: str | None) -> str:
style_block[style_key] = style_val
changed = True

changed = _merge_nodus_pressure_calibration(data, s.get("calibration")) or changed
if changed:
sensor_mgr.save(sensor_id, data)
if DEBUG:
Expand Down Expand Up @@ -6084,6 +6119,7 @@ def _canonical_location(value: str | None) -> str:
for idx in range(6):
style_block[f"METRIC_{idx + 1}"] = remote_display_styles[idx] if idx < len(remote_display_styles) else "Graph24hr"

_merge_nodus_pressure_calibration(data, s.get("calibration"))
sensor_mgr.save(sensor_id, data)
else:
data = OrderedDict()
Expand Down Expand Up @@ -6119,6 +6155,7 @@ def _canonical_location(value: str | None) -> str:
for idx in range(6):
style_block[f"METRIC_{idx + 1}"] = remote_display_styles[idx] if idx < len(remote_display_styles) else "Graph24hr"

_merge_nodus_pressure_calibration(data, s.get("calibration"))
sensor_mgr.save(sensor_id, data)
if DEBUG:
printDM(f"[itaot-settings] seeded sensor settings for {sensor_id}", location=MODULE)
Expand Down
5 changes: 3 additions & 2 deletions sensorius/saiNodusAutomationStatus.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ def _device_inventory(ingest) -> dict[str, dict]:


def build_status_channels(rules: dict, switch_id: str, channel_ids) -> dict[str, list[str]]:
"""Map enabled Advanced rules to known channels for one Nodus switch."""
"""Map enabled Advanced rule display names to known Nodus switch channels."""
known = {str(value or "").strip() for value in (channel_ids or [])}
controlled: dict[str, set[str]] = {}
switch_prefix = f"{str(switch_id or '').strip()}::"
Expand All @@ -80,6 +80,7 @@ def build_status_channels(rules: dict, switch_id: str, channel_ids) -> dict[str,
script = rule.get("script_json")
if not isinstance(script, dict) or not _enabled(script.get("enabled", True)):
continue
name = str(script.get("name") or "").strip() or str(rule_id)
for action in script.get("actions") or []:
if not isinstance(action, dict):
continue
Expand All @@ -88,7 +89,7 @@ def build_status_channels(rules: dict, switch_id: str, channel_ids) -> dict[str,
continue
channel_id = switch_key[len(switch_prefix):].strip()
if channel_id in known:
controlled.setdefault(channel_id, set()).add(str(rule_id))
controlled.setdefault(channel_id, set()).add(name)
return {
channel_id: sorted(names)
for channel_id, names in sorted(controlled.items())
Expand Down
2 changes: 2 additions & 0 deletions sensorius/saiWebRoutes.py
Original file line number Diff line number Diff line change
Expand Up @@ -3540,6 +3540,8 @@ def _resolve_meas_status_for_sid(sid: str) -> str:
"locations": sensor_locations,
"expected_gauge_map": expected_gauge_map,
"expected_display_style_map": expected_display_style_map,
"pressure_altitude": display_settings.get("pressure_altitude"),
"pressure_sensor_context": pressure_sensor_context,
"available_switches": available_switches,
"renderable_switches": renderable_switches,
"renderable_switches_view": renderable_switches_view,
Expand Down
Loading
Loading