Skip to content

Voice asks which device you mean when two fit the name - #22

Merged
AboveColin merged 9 commits into
feature/1.16from
voice-ask-back
Sep 24, 2026
Merged

AboveColin merged 9 commits into
feature/1.16from
voice-ask-back

Conversation

@AboveColin

@AboveColin AboveColin commented Sep 23, 2026 •

Copy link
Copy Markdown
Owner

When the action answer is sure but the entity answer is split between two exposed devices, the voice agent now asks "Do you mean A or B?" and does not fall back. The result has continue_conversation=True, so a satellite opens the microphone for the reply.

How it decides to ask (interpret._two_that_fit):

  • Exactly two exposed devices each hold at least ASK_BACK_FLOOR (0.2) of the entity answer. A third device above the floor means that one question does not settle it, so the agent falls back as before.
  • The two together reach the confidence floor the agent acts on.
  • Their names differ, or the names are the same and the rooms differ. Then the question names the room: "Kitchen light (Kitchen) or Kitchen light (Office)". When nothing tells them apart, the agent falls back.

0.2 is not measured on a real instance. The comment says so.

How the reply is handled:

  • The agent keeps the command, the first response and the snapshot, keyed by conversation id, until CONVERSATION_TIMEOUT (5 minutes, Home Assistant's chat session lifetime).
  • The reply gets one Choice question over the two devices and "neither". Its state is {"command": ..., "reply": ...}. It goes through the same budget check, reservation and error handling as a command.
  • A confident pick runs the first command with the entity answer settled. interpret runs again on the first sentence, so a brightness that the sentence named still comes from the text.
  • A reply that picks neither, or picks without confidence, is handled as a new command.
  • The question goes into the chat log as assistant content, so an LLM fallback agent sees it in the history.

The handler is split into _ask (the budget check and the call), _trace, _act (everything after interpret, unchanged), _ask_which and _resolve. The new line which_device is in strings.json and all 13 translations.

Tests: 12 new tests in test_conversation.py. They cover the question, the pick, neither, an unsure reply, another conversation, expiry, a third device, same name and same room, same name and two rooms, a Dutch pipeline, and a reply past the budget.

Changes after live testing on a development instance:

  • A shared name is asked about when the model put most of its answer on "none". Measured: none 0.55, one Lamp 0.44, the other 0.01. The agent now reads the top device in that case.
  • The reply's trace shows which device the reply picked, and no longer fails with a missing action key.
  • The reply gets a second question, new_request: does it ask for something of its own? Pure picks scored 0.08 to 0.26, new commands 0.91 to 0.97. At 0.5 or more the reply is handled as a new command, so "never mind, turn off the lamp in the bedroom" turns that lamp off.
  • The room of the satellite that heard the command settles a shared name, as it does for Home Assistant's own agent. The lookup is the same: the satellite entity's area, then its device's area. Live: "turn on the lamp" with two lights called Lamp turned on the one in the device's room, 4 of 4. A device in a third room, or in no room, still got the question.

Local gate: ruff, ruff format, mypy --strict, pytest (433 passed, 12 skipped), config_flow 100%, mkdocs --strict, hassfest. EXIT 0.

Part 3 of the 1.16 work. Stacked on #21.

When the action is clear and the entity answer is split between two exposed
devices, the agent asks "Do you mean A or B?" and keeps the microphone open.
The reply gets one small question about those two devices, and the first
command then runs on the one it picked. A reply that picks neither is handled
as a new command. The question lasts as long as the chat session.
# Conflicts:
#	tests/test_conversation.py
# Conflicts:
#	custom_components/jev/conversation.py
On a development instance with two lights both called "Lamp", "turn on
the lamp" came back as one of them at 1.00, every time. The entity
answer never split, so the ask-back never asked, and the agent acted on
a guess that looked sure.

The candidates now come from the exposed names of the chosen device's
kind. The whole name said beats a name that only shares a word with the
command, so "the lamp" is Lamp beside a Desk lamp, and "the desk lamp" is
the Desk lamp. Two names that fit equally well get the question. More
than two fall back. A room the model is sure of narrows the list first,
so "the lamp in the office" acts. When no name fits the words at all, the
model's answer stands as before.

A reply that picks a device reads the command again without the check,
so it does not ask the same question twice.
Home Assistant's own agent prefers the satellite's area when a name fits
devices in several rooms. Jev asked instead. It now reads the satellite
entity's area, then its device's area, the same lookup, and asks only
when that room does not hold exactly one of the devices.
@AboveColin
AboveColin changed the base branch from voice-trace to feature/1.16 September 24, 2026 09:49
@AboveColin
AboveColin merged commit 28ef3fc into feature/1.16 Sep 24, 2026
17 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant