feat: media playback controls

.agents/media-control-feature-overview.md

@@ -0,0 +1,121 @@
+# Media Control Feature Overview
+
+## Status
+**Complete** — all phases implemented and integrated. _(2026-03-28)_
+
+## Goal
+Enable AnotherGlass to mirror the currently playing media session from the Android phone to Glass and allow playback control from Glass back to the phone, while `GlassService` is running.
+
+## Scope
+1. Mobile-side `MediaExtension` that listens to OS media sessions and forwards state + routes commands.
+2. Shared media protocol DTOs and API constants in `shared`.
+3. Shared `MediaController` in `glass-shared` used by both Glass apps.
+4. Mobile app UI showing now-playing state and local controls.
+5. Glass Enterprise Edition (EE) timeline media card with gesture controls.
+6. Glass Explorer Edition (XE) media live card + `MediaPlaybackActivity` control screen.
+7. Python debug client bidirectional support (receives and handles media commands from Glass).
+
+## Non-goals (this iteration)
+- Album art transfer over RPC.
+- In-app media browsing / queue management.
+- Per-app allow-list filtering.
+- Settings toggle to enable/disable the feature independently.
+
+---
+
+## Architecture
+
+### Data Flow
+
+#### Phone → Glass (state updates)
+1. Android OS media session fires a callback.
+2. `MediaExtension` normalises it into `MediaStateData` and emits via `GlassService`.
+3. `RPCMessage(MediaAPI.ID, MediaStateData)` is sent over the transport.
+4. Host service on Glass (`HostService`) routes it to `MediaController.instance`.
+5. UI observes `MediaController.getState()` and re-renders.
+
+#### Glass → Phone (control commands)
+1. User triggers a card action (tap / swipe).
+2. UI calls `MediaController.instance.sendCommand(MediaCommandData)`.
+3. `MediaController` forwards via `IRPCSender` set by the host service.
+4. `RPCMessage(MediaAPI.ID, MediaCommandData)` is sent over the transport.
+5. `GlassService` on the phone routes it to `MediaExtension.onCommand(...)`.
+6. `MediaExtension` calls `TransportControls` on the active media session.
+
+---
+
+## Key Files
+
+### Shared protocol (`shared`)
+| File | Purpose |
+|------|---------|
+| `shared/media/MediaAPI.kt` | Service ID constant: `"Media"` |
+| `shared/media/MediaStateData.kt` | Playback state DTO: title, artist, album, sourceApp, sourcePackage, positionMs, durationMs, actionsMask, lastUpdatedMs, playbackState |
+| `shared/media/MediaCommandData.kt` | Command DTO: `command` enum + `seekToMs`. Commands: `Play`, `Pause`, `TogglePlayPause`, `Next`, `Previous`, `SeekTo` |
+
+### Mobile (`mobile`)
+| File | Purpose |
+|------|---------|
+| `extensions/media/MediaExtension.kt` | Discovers active sessions via `MediaSessionManager`; selects best controller (prefers `STATE_PLAYING`); throttles position-only updates to 1 s; executes inbound `MediaCommandData` via `TransportControls` |
+| `core/GlassService.kt` | Owns `MediaExtension` lifecycle; exposes `mediaState: StateFlow<MediaStateData?>`; routes inbound `MediaAPI` messages to extension |
+| `ui/mainscreen/MediaStatusCard.kt` | Compose card: hidden when extension inactive; shows app/title/artist/position; play/pause/prev/next buttons gated by `actionsMask` |
+| `ui/mainscreen/MainScreen.kt` | Conditionally renders `MediaStatusCard` |
+
+### Shared Glass layer (`glass-shared`)
+| File | Purpose |
+|------|---------|
+| `glass/shared/media/MediaController.kt` | Singleton state holder; `getState(): StateFlow<MediaStateData?>`; `setService`/`clearService`/`sendCommand` lifecycle; clears state on reconnect |
+
+### Glass Enterprise Edition (`glass-ee`)
+| File | Purpose |
+|------|---------|
+| `core/HostService.kt` | Routes `MediaAPI` messages to `MediaController`; registers/clears `IRPCSender` |
+| `ui/MainActivityEx.kt` | Adds/removes `MediaCard` module on the timeline based on state presence |
+| `ui/cards/MediaCard.kt` | Observes state; displays app/title/artist; gesture controls: single tap → `TogglePlayPause`, timed double tap → `Next`, two-finger tap → `Pause` |
+| `res/layout/layout_card_media.xml` | Card layout |
+
+### Glass Explorer Edition (`glass-xe`)
+| File | Purpose |
+|------|---------|
+| `host/HostService.java` | Routes `MediaAPI` messages to `MediaController`; registers/clears `IRPCSender` |
+| `host/media/MediaCardController.java` | Publishes/unpublishes media live card; renders `RemoteViews` using `CardBuilder.Layout.AUTHOR`; opens `MediaPlaybackActivity` on tap |
+| `host/media/MediaPlaybackActivity.kt` | 3-card `CardScrollView`: Prev (`CAPTION`), Center (`AUTHOR`), Next (`CAPTION`). Center: single tap → `TogglePlayPause`, timed double tap → `Next`. All commands via `MediaController.sendCommand(...)` |
+
+### Python debug client (`python`)
+| File | Purpose |
+|------|---------|
+| `client.py` | `MediaDebugController` simulates a 5-track playlist with full state machine; UI mode handles inbound `MediaCommandData` messages from Glass (`TogglePlayPause`, `Play`, `Pause`, `Next`, `Previous`) and updates the UI; console mode provides a manual menu |
+
+---
+
+## Card Layout Decisions
+
+### XE Live Card & Center playback card — `CardBuilder.Layout.AUTHOR`
+Fields mapped:
+- `setHeading(title)` — track title (or source app name if no title)
+- `setSubheading(artist ?? sourceApp)` — artist, falling back to source app
+- `setFootnote(sourceApp)` — source app name (live card) / interaction hint (playback center card)
+- `setTimestamp(playbackState.name)` — e.g. "Playing", "Paused"
+
+### XE Prev / Next cards — `CardBuilder.Layout.CAPTION`
+Short action label + footnote hint; avoids text cropping seen with `LAYOUT_MENU`.
+
+### EE Media Card — custom XML layout (`layout_card_media.xml`)
+Full-bleed card with title, artist, state, and gesture hint footer.
+
+---
+
+## Platform Notes
+- `MediaSessionManager` requires notification-listener permission to enumerate sessions — reuses the existing `NotificationService` component name.
+- Session churn (ads, app handoffs) is handled by re-selecting the best controller on every `OnActiveSessionsChangedListener` callback and `onSessionDestroyed`.
+- Position-tick updates are throttled to 1 Hz via a state fingerprint; play/pause/track-change events are sent immediately.
+- `actionsMask` bits from the media session are forwarded as-is; UI disables controls for unsupported actions.
+- Commands no-op safely when no active session exists.
+
+## Expansion Ideas
+- **Album art** — transfer as a Base64 or compressed byte array in `MediaStateData`; render as `setIcon` on `AUTHOR` cards.
+- **Seek bar** — add a position/duration progress view; use `MediaCommandData.Command.SeekTo` + `seekToMs`.
+- **Per-app filter** — settings toggle or allow-list stored in `SharedPreferences`; `MediaExtension` skips disallowed packages.
+- **Settings toggle** — add `Settings.MEDIA_ENABLED` flag; start/stop `MediaExtension` dynamically.
+- **Voice commands** — EE voice grammar entry to trigger play/pause/next without tapping the card.
+- **EE queue card** — swipe-right from the media card to show upcoming tracks if the session exposes a queue.

README.md

@@ -7,7 +7,8 @@ Companion application to handle **Google Glass Explorer Edition** and **Google G
 
 Currently, the application can:
  * pass GPS data from the phone to the Glass
- * forwarding ongoing and one-shot notifications to the Glass
+ * forward ongoing and one-shot notifications to the Glass
+ * control media playback on the phone (play/pause, next, previous) directly from Glass
  * perform tilt to wake up on Enterprise Edition like on Explorer Edition (be aware of battery usage)
  * pass WiFi network information (SSID and password) from the phone to the Glass (Explorer Edition only)
 
@@ -78,8 +79,26 @@ Originally, Glass application was serving as a host, and mobile was supposed to
 
 Can use Java object stream or JSON Lines to send data to Google Glass, since I don't want to mess with protocol buffers yet.
 
+## Media Playback Controls
+
+While the Host Service is running, AnotherGlass mirrors the currently playing media session from the phone to Glass in near real-time. Any app that integrates with the Android media system (Spotify, YouTube, YouTube Music, Podcast apps, etc.) is supported automatically — no extra setup is required.
+
+**What you see:**
+- The current track title, artist, and playback state are shown on the phone app UI.
+- **Explorer Edition:** a dedicated media live card appears in the Glass timeline whenever something is playing. Tap it to open the playback controls screen.
+- **Enterprise Edition:** a media card appears in the Glass timeline when a session is active.
+
+**Controls from Glass:**
+- **Tap** — play / pause toggle.
+- **Double-tap** — skip to next track.
+- **Explorer Edition playback screen:** swipe left/right to reach dedicated Previous and Next track cards; tap either to send that command.
+
+Controls that the source app does not support are automatically disabled based on what the app reports as available actions.
+
+The media card disappears automatically when there is nothing playing.
+
 ## Debug Python client
-There is a simple Python client in `python` folder to test the Glass Enterprise application without the mobile application. It can send fake GPS coordinates and notifications to the Glass.
+There is a simple Python client in `python` folder to test the Glass Enterprise application without the mobile application. It can send fake GPS coordinates, notifications, and simulated media playback state to the Glass, and also receives and handles media control commands sent back from the Glass.
 Make sure to change communication protocol to JSON Lines by setting `SerializerProvider.currentSerializer` to `JSON`.
 
 ## AnotherGlass Plans

glass-ee/src/main/java/com/damn/anotherglass/glass/ee/host/core/HostService.kt

@@ -10,13 +10,17 @@ import androidx.lifecycle.LifecycleService
 import com.damn.anotherglass.shared.device.DeviceAPI
 import com.damn.anotherglass.shared.gps.GPSServiceAPI
 import com.damn.anotherglass.shared.gps.Location
+import com.damn.anotherglass.shared.media.MediaAPI
+import com.damn.anotherglass.shared.media.MediaStateData
 import com.damn.anotherglass.shared.notifications.NotificationData
 import com.damn.anotherglass.shared.notifications.NotificationsAPI
 import com.damn.anotherglass.shared.rpc.IRPCClient
 import com.damn.anotherglass.shared.rpc.RPCMessage
 import com.damn.anotherglass.shared.rpc.RPCMessageListener
 import com.damn.glass.shared.rpc.WiFiClient
 import com.damn.glass.shared.gps.MockGPS
+import com.damn.glass.shared.media.IRPCSender
+import com.damn.glass.shared.media.MediaController
 import com.damn.glass.shared.notifications.NotificationController
 import org.greenrobot.eventbus.EventBus
 
@@ -66,7 +70,6 @@ class HostService : LifecycleService(), IService {
         return START_STICKY
     }
 
-    @Override
     override fun onCreate() {
         super.onCreate()
         gps = MockGPS(this)
@@ -75,12 +78,13 @@ class HostService : LifecycleService(), IService {
         batteryStatus.observe(this) {
             client?.send(RPCMessage(DeviceAPI.SERVICE_NAME, it))
         }
+        MediaController.instance.setService { client?.send(it) }
     }
 
-    @Override
     override fun onDestroy() {
         super.onDestroy()
         Log.i(TAG, "HostService stopped")
+        MediaController.instance.clearService()
         gps.remove()
         client?.stop()
         sounds.release()
@@ -99,6 +103,7 @@ class HostService : LifecycleService(), IService {
                 Log.d(TAG, "Connected to $device")
                 state = IService.ServiceState.CONNECTED
                 NotificationController.instance.onServiceConnected()
+                MediaController.instance.onServiceConnected()
 
                 batteryStatus.value?.let {
                     client?.send(RPCMessage(DeviceAPI.SERVICE_NAME, it))
@@ -120,6 +125,12 @@ class HostService : LifecycleService(), IService {
                         notificationNotifier.notify(notificationData)
                     }
 
+                    MediaAPI.ID -> {
+                        Log.d(TAG, "Media data received")
+                        val mediaState = data.payload as? MediaStateData ?: return
+                        MediaController.instance.onMediaStateUpdate(mediaState)
+                    }
+
                     else -> Log.e(TAG, "Unknown service: ${data.service}")
                 }
             }

glass-ee/src/main/java/com/damn/anotherglass/glass/ee/host/ui/MainActivity.kt

@@ -24,6 +24,7 @@ import com.damn.anotherglass.glass.ee.host.core.Settings
 import com.damn.anotherglass.glass.ee.host.core.tiltawake.TiltToWakeService
 import com.damn.anotherglass.glass.ee.host.debug.DebugManager
 import com.damn.anotherglass.glass.ee.host.ui.MainActivityEx.addNotificationsModule
+import com.damn.anotherglass.glass.ee.host.ui.MainActivityEx.addMediaModule
 import com.damn.anotherglass.glass.ee.host.ui.cards.BaseFragment
 import com.damn.anotherglass.glass.ee.host.ui.cards.MapCard
 import com.damn.anotherglass.glass.ee.host.ui.cards.ServiceStateCard
@@ -111,6 +112,7 @@ class MainActivity : BaseActivity() {
         tabLayout.setupWithViewPager(viewPager, true)
 
         addNotificationsModule(timeLine)
+        addMediaModule(timeLine)
 
         timeLine.setCurrent(1, false)
 
@@ -244,9 +246,7 @@ class MainActivity : BaseActivity() {
 
         fun bindGlassService() {
             try {
-                if (bound) {
-                    unbindService(connection)
-                }
+                if (bound) unbindService(connection)
             } catch (e: Exception) {
                 e.printStackTrace()
             }

glass-ee/src/main/java/com/damn/anotherglass/glass/ee/host/ui/MainActivityEx.kt

@@ -1,7 +1,9 @@
 package com.damn.anotherglass.glass.ee.host.ui
 
 import androidx.lifecycle.lifecycleScope
+import com.damn.glass.shared.media.MediaController
 import com.damn.glass.shared.notifications.NotificationController
+import com.damn.anotherglass.glass.ee.host.ui.cards.MediaCard
 import com.damn.anotherglass.glass.ee.host.ui.cards.NotificationsCard
 import kotlinx.coroutines.flow.launchIn
 import kotlinx.coroutines.flow.onEach
@@ -25,4 +27,20 @@ object MainActivityEx {
         }.launchIn(this.lifecycleScope) // Assuming MainActivity is a LifecycleOwner
     }
 
+    fun MainActivity.addMediaModule(timeLine: ITimeline) {
+        val mediaFlow = MediaController.instance.getState()
+        // Initial check — show card if there is already a media state
+        if (mediaFlow.value != null) {
+            timeLine.addFragment(MediaCard.newInstance(), 0)
+        }
+
+        mediaFlow.onEach { state ->
+            if (state == null)
+                timeLine.removeByType(MediaCard::class.java)
+            else when (timeLine.indexOfFirst(MediaCard::class.java)) {
+                -1 -> timeLine.addFragment(MediaCard.newInstance(), 0, true)
+                // card already present — it updates itself via its own flow observer
+            }
+        }.launchIn(this.lifecycleScope)
+    }
 }

glass-ee/src/main/java/com/damn/anotherglass/glass/ee/host/ui/cards/MediaCard.kt

@@ -0,0 +1,93 @@
+package com.damn.anotherglass.glass.ee.host.ui.cards
+
+import android.os.Bundle
+import android.view.LayoutInflater
+import android.view.View
+import android.view.ViewGroup
+import androidx.lifecycle.lifecycleScope
+import com.damn.anotherglass.glass.ee.host.databinding.LayoutCardMediaBinding
+import com.damn.anotherglass.shared.media.MediaCommandData
+import com.damn.anotherglass.shared.media.MediaStateData
+import com.damn.glass.shared.media.MediaController
+import kotlinx.coroutines.Job
+import kotlinx.coroutines.delay
+import kotlinx.coroutines.flow.launchIn
+import kotlinx.coroutines.flow.onEach
+import kotlinx.coroutines.launch
+
+class MediaCard : BaseFragment() {
+
+    private var binding: LayoutCardMediaBinding? = null
+    private var pendingSingleTapJob: Job? = null
+
+    override fun onCreateView(
+        inflater: LayoutInflater,
+        container: ViewGroup?,
+        savedInstanceState: Bundle?
+    ): View? = LayoutCardMediaBinding.inflate(inflater).also { binding = it }.root
+
+    override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
+        super.onViewCreated(view, savedInstanceState)
+        MediaController.instance.getState()
+            .onEach { state ->
+                if (state == null) return@onEach // card is about to be removed
+                binding?.update(state)
+            }
+            .launchIn(viewLifecycleOwner.lifecycleScope)
+    }
+
+    override fun onDestroyView() {
+        pendingSingleTapJob?.cancel()
+        pendingSingleTapJob = null
+        super.onDestroyView()
+        binding = null
+    }
+
+    // Single tap toggles play/pause unless a second tap arrives quickly.
+    override fun onSingleTapUp() {
+        if (pendingSingleTapJob != null) {
+            pendingSingleTapJob?.cancel()
+            pendingSingleTapJob = null
+            sendCommand(MediaCommandData.Command.Next)
+            return
+        }
+        pendingSingleTapJob = viewLifecycleOwner.lifecycleScope.launch {
+            delay(DOUBLE_TAP_TIMEOUT_MS)
+            sendCommand(MediaCommandData.Command.TogglePlayPause)
+            pendingSingleTapJob = null
+        }
+    }
+
+    // Two-finger tap → stop
+    override fun onTwoFingerTap() {
+        pendingSingleTapJob?.cancel()
+        pendingSingleTapJob = null
+        sendCommand(MediaCommandData.Command.Pause)
+    }
+
+    private fun sendCommand(command: MediaCommandData.Command) {
+        MediaController.instance.sendCommand(MediaCommandData(command))
+    }
+
+    private fun LayoutCardMediaBinding.update(state: MediaStateData) {
+        mediaApp.text = state.sourceApp ?: state.sourcePackage ?: ""
+        mediaTitle.text = state.title ?: ""
+        mediaArtist.text = state.artist ?: ""
+        mediaState.text = when (state.playbackState) {
+            MediaStateData.PlaybackStateValue.Playing    -> "▶ Playing"
+            MediaStateData.PlaybackStateValue.Paused     -> "⏸ Paused"
+            MediaStateData.PlaybackStateValue.Buffering  -> "⏳ Buffering"
+            MediaStateData.PlaybackStateValue.Stopped    -> "⏹ Stopped"
+            MediaStateData.PlaybackStateValue.None       -> ""
+        }
+        footer.text = "Tap: play/pause  •  Double tap: next  •  ✌ tap: stop"
+    }
+
+    companion object {
+        private const val DOUBLE_TAP_TIMEOUT_MS = 280L
+
+        @JvmStatic
+        fun newInstance(): MediaCard = MediaCard()
+    }
+}
+