Add Community-trained wake words to your Voice setup

[jlpouffier]

Problem Statement

Users of VPES voice assistants are locked into a fixed set of wake words with no way to customise them. For users who want something more personal, more accessible, or simply not on the default list, there is no path forward. This makes the platform feel rigid at a moment when flexibility in voice interaction is increasingly expected, and it is a gap that is straightforward to close. At the same time, community contributors have started training wake words and publishing them, which could help us ease this problem without getting deep into personally-trained wake words.

Scope

• A new "Custom Wake Words" instructions and category under HACs
• Selected wake words are saved to the user's HACs account and appear in the wake word dropdown on the device information page
• Basic validation of the GitHub repo URL and repo format

Not in Scope

• Hosting, curating, or moderating wake word repositories
• Building or uploading custom wake word audio/model files directly within the product
• Wake word training or personalisation (voice matching, accent adaptation)
• Support for non-GitHub sources (e.g. GitLab, direct file upload) in this iteration
• Managing or removing previously added custom wake words (future iteration)

Community signals

We have contributors experimenting with training wake words, with various levels of success. One in particular built one for Hey Morgan and its published. . It makes sense to foster these kind of contributions in Voice, same way we do with Custom cards in Dashboards or blueprints in Automations. And at the same time, it makes sense to encourage users to test new wake words that the community provides, with an easy way to include them.

Foreseen Solution

1. Users navigate to HACs and find the wake word repo they want.
2. Users install the wake word
3. Those wake words appear under the device dropdown

Decision log

15/07 - We will use a simple 1 wake word - 1 repo implementation and HACs for installing them at first. This will allow us to have the feature enable, and the work on discoverability later down the road.

[laupalombi]

Adding feedback on the design right now.

[mbb365]

Adding current mockup / higher fi version with the changes

• Added table column for 'where used'. This shows on what devices (or how many devices) you use that wake word
• Removal barriers. If in use, a warning appears and directions to remove it from the device information page
• Unable to remove repo from table if a wake word is in use dialogue when removing

[laupalombi]

@mbb365 I believe that as per @synesthesiam last feedback we cannot just "add repo" from there, the options are:

• an ofical repo is pre-filled - is already there, with all toggles disabled
• we need to link people to "if you want experiment with untested more wake words, find them in HACS here (link to HACS)" (for example trademarked things we cannot put on the official repo but people might want to add).

Also super minor feedback - we need language of each wake word to understand how it was trained, it should show as a column in the table.

[mbb365]

Updated mockup here

Five minor questions

1. What should the title of the dialogue that directs to HACS be? I am currently running with 'Add more via HACS'
2. Are we going to enable them to remove the official repo?
3. Once they have landed on the HACS page, how do they find more providers? Or is that up to them
4. I assumed we no longer need the 'refresh' so I removed it.
5. I assumed we no longer needed the delete table, so I removed it. This would be handled with HACS, no?

[laupalombi]

I like this @mbb365

What should the title of the dialogue that directs to HACS be? I am currently running with 'Add more via HACS'
I like it, sounds good.

Are we going to enable them to remove the official repo?

No, or at least not in this first iteration - we'll probably add it in the future depending on how well perceived this is

Once they have landed on the HACS page, how do they find more providers? Or is that up to them

We can send them to HACS filtered by Wake word providers when we have them. Depending on when we end up doing this feature, we'll see how the landscape is. If not, the main page of HACs is fine.

I assumed we no longer need the 'refresh' so I removed it.

Perfect! Agreed.

I assumed we no longer needed the delete table, so I removed it. This would be handled with HACS, no?

Exactly - this table is good for disabling it.

[laupalombi]

I believe this is good enough for consider it Ready for approval - moving it

[laupalombi]

@mbb365 We got leaadership feedback. Apparently the ad-hoc repo is not an option we want - we have HACs for that, and if they are really good we should just pre-ship them into the firmware. So can you please do some version where a user goes to HACs and just has this screen to activate them?

I'd argue you can even remove the Wake word from the device page and just direct people to this one, so they have more context. The device page is not a good place to have a list of custom wake words and get all the context you need.

[mbb365]

I have talked about the changes with @laupalombi and we identified 3 new concept directions and issues.

1. Controlling the Wake Words per device & from the repo table.

2. Linking to the device page from the repo table.

3. A combination of these two.

We need to think about how it will develop after this, and what knock on effects it might have.

I'll be doing these next week. (post 01-07)

[synesthesiam]

When thinking about this, make sure to consider that wake words link a device and a pipeline. So a device may have one wake word that runs pipeline A, and another (different) wake word that runs pipeline B.

We've hard-coded only 2 wake words and pipelines for now to keep complexity down, so the UI will need to accommodate this.