WIP: Microphone in car

ARCHITECTURE.md

@@ -102,3 +102,78 @@ Callback storage lives in a `WeakMap`, which keeps resolve and reject handlers a
 The 30 second timeout is a safety net. It rejects with `AirConsoleUserMediaError.timeout` if the platform never answers.
 
 `mediaPermissionPending_` blocks duplicate requests and ignores stale platform messages after cleanup.
+
+# Audio Input Device Selection
+
+## Why
+
+A phone controller connected to a car often captures through the car's Bluetooth hands-free microphone. Opening that
+stream activates the hands-free profile, the car's media session loses audio focus, and the platform pauses the
+session. The player can not act on that pause, so instead the platform offers them the other audio inputs of their
+phone and asks the game to capture from the one they pick.
+
+The platform decides on the controller side: it can not observe the car's audio focus itself, so it correlates an
+audio-focus-loss pause with recent microphone activity on that controller.
+
+## Actors
+
+**Game** reports its audio inputs and implements `onAudioInputDeviceChange` to swap streams.
+
+**API** forwards the reported devices to the platform and delivers the platform's selection to the game.
+
+**Platform** decides whether a pause was caused by the microphone and, if so, shows the audio input selection instead
+of the pause overlay.
+
+## Sequence
+
+```mermaid
+sequenceDiagram
+    autonumber
+    actor Game
+    participant API
+    participant Platform
+    participant Browser
+
+    Game->>API: getUserMedia({ audio: true })
+    API->>Platform: sendEvent_('requestUserMediaPermission', { constraints })
+    API->>Platform: sendEvent_('microphoneRequested', {})
+    Note over Platform: arm the audio focus loss window
+    Note over API,Browser: media permission flow (see above)
+    Browser-->>API: stream
+    API->>Platform: sendEvent_('userMediaPermissionGranted', { constraints })
+    Note over Platform: re-arm the window, the microphone is now engaged
+    API-->>Game: Promise resolves with stream
+
+    Game->>Browser: enumerateDevices()
+    Browser-->>Game: MediaDeviceInfo[]
+    Game->>API: setAudioInputDevices(devices, activeDeviceId)
+    Note over API: keep audioinput entries with a deviceId,<br/>map to { deviceId, label, groupId }
+    API->>Platform: sendEvent_('audioInputDevicesReported', { devices, activeDeviceId })
+
+    Platform->>Platform: audio focus lost inside the window
+    Platform->>Platform: show audio input selection instead of the pause overlay
+    Platform->>API: event setAudioInputDevice { deviceId }
+    API-->>Game: onAudioInputDeviceChange(deviceId)
+    Game->>Browser: stop tracks, getUserMedia({ audio: { deviceId: { exact } } })
+    Browser-->>Game: new stream
+    Game->>API: setAudioInputDevices(devices, deviceId)
+    API->>Platform: sendEvent_('audioInputDevicesReported', ...)
+```
+
+## Key Design Decisions
+
+`setAudioInputDevices` accepts the result of `enumerateDevices()` unchanged. It keeps entries whose `kind` is
+`audioinput` (or that carry no `kind` at all, so plain objects can be reported too) and that have a `deviceId`, and
+reduces each to `{ deviceId, label, groupId }`. Nothing else about a `MediaDeviceInfo` is useful to the platform, and
+`toJSON` output would not survive `postMessage` cloning in every browser.
+
+`deviceId` values are scoped to the origin that enumerated them, so the platform treats them as opaque: it renders the
+`label` and hands the `deviceId` back unchanged. It never enumerates devices itself, because the ids it would get do
+not match the game's.
+
+`setAudioInputDevice` is handled before the `mediaPermissionPending_` guard in the inbound event handler. That guard
+exists to drop stale messages of a permission request that already settled, and audio input selection happens long
+after that, with no request pending.
+
+Reporting an empty list is how a game says the microphone is no longer in use. The platform then has nothing to offer
+and falls back to its normal pause handling.

CHANGELOG.md

@@ -12,6 +12,15 @@ Release notes follow the [keep a changelog](https://keepachangelog.com/en/1.0.0/
 
 ### Added
 
+- New `AirConsole.setAudioInputDevices(devices, activeDeviceId)` for controllers to report their audio inputs and the
+  device the current microphone stream uses.
+- New `AirConsole.onAudioInputDeviceChange(device_id)` callback, called when the player picked a different microphone on
+  the platform. The game closes its current stream and opens a new one on the given device.
+  - Together these let the platform offer a different microphone instead of pausing when opening the microphone takes
+    the audio focus away, e.g. a phone controller connected to a car capturing through the car's Bluetooth microphone.
+- `getUserMedia` now also informs the platform that a microphone was requested, which the platform correlates with the
+  audio focus it loses.
+
 ## [1.11.0] - 2026-07-07
 
 ### Added 

airconsole-1.11.0.js

@@ -875,6 +875,13 @@ AirConsole.prototype.getUserMedia = function getUserMedia(constraints) {
     // Send the request to the platform to decide where and how the user media request needs to take place based on
     //  browser or controller environment.
     me.sendEvent_('requestUserMediaPermission', { constraints: constraints });
+    // Let the platform know a microphone is being requested. The platform uses this to recognize situations where
+    //  opening the microphone is what takes the audio focus away from the platform (E.g. a phone controller capturing
+    //  through a car's Bluetooth microphone), so it can offer the player another audio input instead of pausing.
+    if (constraints.audio) {
+      console.log('DRG:airconsole-1.11.0.js: 882:getUserMedia:microphoneRequested:', constraints);
+      me.sendEvent_('microphoneRequested', {});
+    }
   });
 };
 
@@ -899,6 +906,89 @@ AirConsole.prototype.rejectMediaPermission_ = function rejectMediaPermission_(er
   if (cb) { cb.reject(error); }
 }
 
+/**
+ * @typedef {Object} AirConsole~AudioInputDevice
+ * @property {string} deviceId - Identifier of the audio input, as reported by
+ *   {@link https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/enumerateDevices enumerateDevices}.
+ * @property {string} label - Human readable name of the audio input. Empty until a media permission was granted.
+ * @property {string} [groupId] - Identifier of the group the device belongs to.
+ */
+
+/**
+ * Reports the audio inputs available on this controller and the one the current stream uses.
+ * Can only be called by a controller (not the screen).
+ *
+ * The platform needs this to offer the player a different microphone when opening one takes the audio focus away, for
+ * example when a phone controller connected to a car captures through the car's Bluetooth microphone and the platform
+ * would otherwise pause. When the player picks a different audio input,
+ * {@link AirConsole.prototype.onAudioInputDeviceChange} is called with its `deviceId`.
+ *
+ * Call this whenever the situation changes: after a stream was opened, after switching to another device, and when
+ * `navigator.mediaDevices` fires `devicechange`. Report an empty list once the microphone is no longer in use, so the
+ * platform does not offer devices for a stream that does not exist anymore.
+ *
+ * Note: `deviceId` values are scoped to the origin that enumerated them. The platform only displays the `label` and
+ * hands the `deviceId` back unchanged.
+ *
+ * @param {Array<MediaDeviceInfo|AirConsole~AudioInputDevice>} devices - Devices to report. Entries of a kind other
+ *   than `audioinput` and entries without a `deviceId` are ignored, so the result of `enumerateDevices()` can be
+ *   passed as is.
+ * @param {string} [activeDeviceId] - The `deviceId` the current stream uses, e.g.
+ *   `stream.getAudioTracks()[0].getSettings().deviceId`.
+ *
+ * @example
+ * const devices = await navigator.mediaDevices.enumerateDevices();
+ * airconsole.setAudioInputDevices(devices, stream.getAudioTracks()[0].getSettings().deviceId);
+ *
+ * @see AirConsole.prototype.onAudioInputDeviceChange
+ * @see AirConsole.prototype.getUserMedia
+ */
+AirConsole.prototype.setAudioInputDevices = function setAudioInputDevices(devices, activeDeviceId) {
+  if (this.device_id === AirConsole.SCREEN) {
+    throw "Only controllers can call setAudioInputDevices!";
+  }
+
+  var audioInputs = (devices || []).filter(function isReportableAudioInput(device) {
+    return !!device && !!device.deviceId && (!device.kind || device.kind === 'audioinput');
+  }).map(function toAudioInputDevice(device) {
+    return {
+      deviceId: device.deviceId,
+      label: device.label || '',
+      groupId: device.groupId || ''
+    };
+  });
+
+  this.audioInputDevices_ = audioInputs;
+  this.activeAudioInputDeviceId_ = activeDeviceId || '';
+
+  console.log('DRG:airconsole-1.11.0.js: 964:setAudioInputDevices:audioInputs:', audioInputs,
+    this.activeAudioInputDeviceId_);
+
+  this.sendEvent_('audioInputDevicesReported', {
+    devices: audioInputs,
+    activeDeviceId: this.activeAudioInputDeviceId_
+  });
+};
+
+/**
+ * Gets called when the platform asks the game to capture from a different audio input, because the player picked one.
+ * Close the current stream and open a new one on the given device, then report the new situation with
+ * {@link AirConsole.prototype.setAudioInputDevices}.
+ * @abstract
+ * @param {string} device_id - The `deviceId` of the audio input to use, as previously reported by the game.
+ *
+ * @example
+ * airconsole.onAudioInputDeviceChange = async function (device_id) {
+ *   stream.getTracks().forEach(function (t) { t.stop(); });
+ *   stream = await navigator.mediaDevices.getUserMedia({ audio: { deviceId: { exact: device_id } } });
+ *   const devices = await navigator.mediaDevices.enumerateDevices();
+ *   airconsole.setAudioInputDevices(devices, device_id);
+ * };
+ *
+ * @see AirConsole.prototype.setAudioInputDevices
+ */
+AirConsole.prototype.onAudioInputDeviceChange = function(device_id) {};
+
 /**
  * Releases resources held by this AirConsole instance.
  * Call this when the instance is no longer needed — for example, when navigating
@@ -1693,6 +1783,14 @@ AirConsole.prototype.onPostMessage_ = function(event) {
   } else if (data.action === 'event') {
     const { type } = data;
 
+    // Audio input selection happens while a stream is already open, so it is handled before the media permission
+    // guard below, which only applies to the events of a pending permission request.
+    if (type === 'setAudioInputDevice') {
+      console.log('DRG:airconsole-1.11.0.js: 1789:onPostMessage_:setAudioInputDevice:', data.data);
+      me.onAudioInputDeviceChange(data.data ? data.data.deviceId : undefined);
+      return;
+    }
+
     // Guard: ignore stale platform messages that arrive after state has been cleaned up
     // (e.g. after the 30-second timeout has already resolved the pending Promise).
     if (!me.mediaPermissionPending_) {
@@ -1722,6 +1820,8 @@ AirConsole.prototype.onPostMessage_ = function(event) {
           // Note: 'userMediaPermissionGranted' is both sent upward (controller → platform) and
           // received downward (platform → controller for native controllers). The direction is
           // determined by context: outbound is sent here; inbound is handled by this event branch.
+          console.log('DRG:airconsole-1.11.0.js: 1823:onPostMessage_:userMediaPermissionGranted:',
+            me.mediaPermissionConstraints_);
           me.sendEvent_('userMediaPermissionGranted', {
             constraints: me.mediaPermissionConstraints_,
           });

beta/airconsole-1.11.1.js

@@ -875,6 +875,13 @@ AirConsole.prototype.getUserMedia = function getUserMedia(constraints) {
     // Send the request to the platform to decide where and how the user media request needs to take place based on
     //  browser or controller environment.
     me.sendEvent_('requestUserMediaPermission', { constraints: constraints });
+    // Let the platform know a microphone is being requested. The platform uses this to recognize situations where
+    //  opening the microphone is what takes the audio focus away from the platform (E.g. a phone controller capturing
+    //  through a car's Bluetooth microphone), so it can offer the player another audio input instead of pausing.
+    if (constraints.audio) {
+      console.log('DRG:airconsole-1.11.1.js: 882:getUserMedia:microphoneRequested:', constraints);
+      me.sendEvent_('microphoneRequested', {});
+    }
   });
 };
 
@@ -899,6 +906,89 @@ AirConsole.prototype.rejectMediaPermission_ = function rejectMediaPermission_(er
   if (cb) { cb.reject(error); }
 }
 
+/**
+ * @typedef {Object} AirConsole~AudioInputDevice
+ * @property {string} deviceId - Identifier of the audio input, as reported by
+ *   {@link https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/enumerateDevices enumerateDevices}.
+ * @property {string} label - Human readable name of the audio input. Empty until a media permission was granted.
+ * @property {string} [groupId] - Identifier of the group the device belongs to.
+ */
+
+/**
+ * Reports the audio inputs available on this controller and the one the current stream uses.
+ * Can only be called by a controller (not the screen).
+ *
+ * The platform needs this to offer the player a different microphone when opening one takes the audio focus away, for
+ * example when a phone controller connected to a car captures through the car's Bluetooth microphone and the platform
+ * would otherwise pause. When the player picks a different audio input,
+ * {@link AirConsole.prototype.onAudioInputDeviceChange} is called with its `deviceId`.
+ *
+ * Call this whenever the situation changes: after a stream was opened, after switching to another device, and when
+ * `navigator.mediaDevices` fires `devicechange`. Report an empty list once the microphone is no longer in use, so the
+ * platform does not offer devices for a stream that does not exist anymore.
+ *
+ * Note: `deviceId` values are scoped to the origin that enumerated them. The platform only displays the `label` and
+ * hands the `deviceId` back unchanged.
+ *
+ * @param {Array<MediaDeviceInfo|AirConsole~AudioInputDevice>} devices - Devices to report. Entries of a kind other
+ *   than `audioinput` and entries without a `deviceId` are ignored, so the result of `enumerateDevices()` can be
+ *   passed as is.
+ * @param {string} [activeDeviceId] - The `deviceId` the current stream uses, e.g.
+ *   `stream.getAudioTracks()[0].getSettings().deviceId`.
+ *
+ * @example
+ * const devices = await navigator.mediaDevices.enumerateDevices();
+ * airconsole.setAudioInputDevices(devices, stream.getAudioTracks()[0].getSettings().deviceId);
+ *
+ * @see AirConsole.prototype.onAudioInputDeviceChange
+ * @see AirConsole.prototype.getUserMedia
+ */
+AirConsole.prototype.setAudioInputDevices = function setAudioInputDevices(devices, activeDeviceId) {
+  if (this.device_id === AirConsole.SCREEN) {
+    throw "Only controllers can call setAudioInputDevices!";
+  }
+
+  var audioInputs = (devices || []).filter(function isReportableAudioInput(device) {
+    return !!device && !!device.deviceId && (!device.kind || device.kind === 'audioinput');
+  }).map(function toAudioInputDevice(device) {
+    return {
+      deviceId: device.deviceId,
+      label: device.label || '',
+      groupId: device.groupId || ''
+    };
+  });
+
+  this.audioInputDevices_ = audioInputs;
+  this.activeAudioInputDeviceId_ = activeDeviceId || '';
+
+  console.log('DRG:airconsole-1.11.1.js: 964:setAudioInputDevices:audioInputs:', audioInputs,
+    this.activeAudioInputDeviceId_);
+
+  this.sendEvent_('audioInputDevicesReported', {
+    devices: audioInputs,
+    activeDeviceId: this.activeAudioInputDeviceId_
+  });
+};
+
+/**
+ * Gets called when the platform asks the game to capture from a different audio input, because the player picked one.
+ * Close the current stream and open a new one on the given device, then report the new situation with
+ * {@link AirConsole.prototype.setAudioInputDevices}.
+ * @abstract
+ * @param {string} device_id - The `deviceId` of the audio input to use, as previously reported by the game.
+ *
+ * @example
+ * airconsole.onAudioInputDeviceChange = async function (device_id) {
+ *   stream.getTracks().forEach(function (t) { t.stop(); });
+ *   stream = await navigator.mediaDevices.getUserMedia({ audio: { deviceId: { exact: device_id } } });
+ *   const devices = await navigator.mediaDevices.enumerateDevices();
+ *   airconsole.setAudioInputDevices(devices, device_id);
+ * };
+ *
+ * @see AirConsole.prototype.setAudioInputDevices
+ */
+AirConsole.prototype.onAudioInputDeviceChange = function(device_id) {};
+
 /**
  * Releases resources held by this AirConsole instance.
  * Call this when the instance is no longer needed — for example, when navigating
@@ -1693,6 +1783,14 @@ AirConsole.prototype.onPostMessage_ = function(event) {
   } else if (data.action === 'event') {
     const { type } = data;
 
+    // Audio input selection happens while a stream is already open, so it is handled before the media permission
+    // guard below, which only applies to the events of a pending permission request.
+    if (type === 'setAudioInputDevice') {
+      console.log('DRG:airconsole-1.11.1.js: 1789:onPostMessage_:setAudioInputDevice:', data.data);
+      me.onAudioInputDeviceChange(data.data ? data.data.deviceId : undefined);
+      return;
+    }
+
     // Guard: ignore stale platform messages that arrive after state has been cleaned up
     // (e.g. after the 30-second timeout has already resolved the pending Promise).
     if (!me.mediaPermissionPending_) {
@@ -1722,6 +1820,8 @@ AirConsole.prototype.onPostMessage_ = function(event) {
           // Note: 'userMediaPermissionGranted' is both sent upward (controller → platform) and
           // received downward (platform → controller for native controllers). The direction is
           // determined by context: outbound is sent here; inbound is handled by this event branch.
+          console.log('DRG:airconsole-1.11.1.js: 1823:onPostMessage_:userMediaPermissionGranted:',
+            me.mediaPermissionConstraints_);
           me.sendEvent_('userMediaPermissionGranted', {
             constraints: me.mediaPermissionConstraints_,
           });

tests/airconsole-1.11.0-spec.html

@@ -31,6 +31,7 @@
     <script src="spec/methods/spec-immersive.js"></script>
     <script src="spec/methods/spec-player-silencing.js"></script>
      <script src="spec/methods/spec-usermedia-permissions.js"></script>
+     <script src="spec/methods/spec-audio-input-devices.js"></script>
      <script src="spec/methods/spec-game-configuration.js"></script>
 
      <!-- include spec files here... -->

tests/airconsole-1.11.1-spec.html

@@ -31,6 +31,7 @@
     <script src="spec/methods/spec-immersive.js"></script>
     <script src="spec/methods/spec-player-silencing.js"></script>
      <script src="spec/methods/spec-usermedia-permissions.js"></script>
+     <script src="spec/methods/spec-audio-input-devices.js"></script>
      <script src="spec/methods/spec-game-configuration.js"></script>
 
      <!-- include spec files here... -->

tests/spec/airconsole-1.11.0-spec.js

@@ -398,6 +398,19 @@ describe("AirConsole 1.11.0", function () {
         testUserMediaPermissions();
     });
 
+    /**
+     ======================================================================================
+     TEST AUDIO INPUT DEVICE SELECTION FUNCTIONALITY
+     */
+
+    describe("Audio Input Devices", function () {
+        afterEach(function () {
+            tearDown();
+        });
+
+        testAudioInputDevices();
+    });
+
     /**
      ======================================================================================
      TEST CONFIGURATION FUNCTIONALITY