MQTT Interoperability: cmd/result JSON format and 'ramses_rf' compatibility

[PWhite-Eng]

Hi Peter / @IndaloTech,

I am currently working on the ramses_cc Home Assistant integration (specifically ramses-rf/ramses_cc#483), adding native support for Home Assistant's MQTT service as a transport layer.

During the code review of the ramses_esp MQTT implementation (specifically in ramses-mqtt.c), we identified an interoperability hurdle that affects how standard libraries like ramses_rf interact with the firmware via MQTT.

The Observation

In components/ramses-mqtt/ramses-mqtt.c, the mqtt_publish_cmd_result function formats the JSON payload using an integer for the return code:

// Current implementation
sprintf( data, "{ \"cmd\":\"%s\", \"err\":\"%s\", \"return\":%d} ",cmd, esp_err_to_name(err),retVal );

This results in a payload like: {"cmd": "!V", "return": 0}.

The Issue

Most RAMSES client libraries (including ramses_rf) are designed to interact with a serial port, where they expect the response to a command (like !V) to be a specific string (e.g., # evofw3 0.7.1).

When ramses_esp returns an integer 0 via MQTT:

1. Type Mismatch: Clients expecting a string response may crash or fail to parse the integer.
2. Handshake Failure: The !V command is critical for identifying the gateway. If the client receives 0 instead of a string containing # evofw3..., it assumes the gateway is invalid and aborts the connection.

Suggestion

To improve compatibility with the broader ecosystem, would you consider updating the MQTT response to include the actual console output or version string in the JSON payload?

For example, returning the string value would allow the MQTT transport to act as a transparent bridge for existing libraries:

JSON

{
  "cmd": "!V",
  "return": "# evofw3 0.7.1"
}

We have implemented a workaround in the ramses_cc client side for now (synthesizing a fake handshake when we see return: 0), but a fix at the firmware level would make ramses_esp plug-and-play compatible with more Home Assistant integrations out of the box.

Thanks

Phil / @PWhite-Eng

[ghoti57]

Not trivial.

The command string received via mqtt is just passed directly to the CLI code and the output is just generated directly to the serial (USB) interface.

The output is never available to the MQTT code.
The output could be several lines of text.

!v is not implemented in ramses_esp (or any other ! command)

I implemented the cmd interface so I could send things like reset.

The version of ramses_esp is published in
RAMSES\GATEWAY<id>\info

I could probably make a special case in mqtt for !v
Are any of the other ! commands from evofw3 used/required?

[PWhite-Eng]

Hi @ghoti57,

Thanks for the feedback and the insight into the serial-to-MQTT pipeline. That explanation makes perfect sense regarding why the serial output (and thus the command response) isn't naturally available to the MQTT publisher.

To answer your specific question:

Are any of the other! commands from evofw3 used/required?

No.

I have reviewed the ramses_rf / ramses_cc codebases, and they do not require any of the other evofw3 "bang" commands (!C, !F, !B, !T, !E, !I) for operation via MQTT.

Technical Context:

1. HGI80 Compatibility: ramses_rf is designed to be compatible with the official Honeywell HGI80, which does not support any ! commands. As a result, the core logic (discovery, packet handling, heating control) relies purely on standard RAMSES II packets (RQ, RP, I, W).
2. MqttTransport: The MQTT transport layer in ramses_rf functions as a transparent pipe. It does not attempt to tune the radio (!F) or configure hardware parameters (!C) remotely; it assumes the ESP/firmware handles the hardware abstraction.
3. Liveness Checks: ramses_rf uses the 7FFF (Puzzle) packet to detect the gateway's presence and MAC address, rather than querying it via !I or !V.

Regarding !V (Version):

While ramses_rf doesn't strictly require it to function, !V is often used by various tools and scripts to identify the gateway firmware.

• If you are willing to make a special case for !V to return the version string (e.g., in the JSON err field or a specific val field), that would help maintain broader compatibility with generic console tools.
• However, for the specific purpose of the Home Assistant integration, we can adapt to read the version from your proposed RAMSES GATEWAY/info topic (or status payload) instead of polling !V.

Summary:

We only need the MQTT transport to reliably pass standard RAMSES II packets (.../rx and .../tx). We do not need support for !C, !F, !T, or !B via MQTT.

Thanks again for looking into this
Phil