feat(sdk): add active-tool audio support - #119
Conversation
brunoro
changed the title
|
Okay, I think this is an awesome foundation. Few things I'd like to note from a practicality and end-user ux standpoint (ignoring the foreground service stuff, which you folks are already working on lol): This PR would introduce an internal ExoPlayer instance managed entirely by the SDK. Which is great for 99.9% of use cases, but music streaming is a tricky business 😭 The nice thing is that media3 already has a pretty decent streaming pipeline. That's what the TIDAL half of phono uses (since TIDAL serves raw streaming URLs). I can definitely rework the Spotify half towards that. Currently we have a fully custom sink, since Spotify serves encrypted chunks. But it would absolutely be possible to decrypt, cache, etc., then feed directly to media3 as a custom source. The problem at the moment is that media3's proper streaming capabilities live in ExoPlayer.Builder, which is walled off. While it would be possible to just feed through some of these params as Light-denoted APIs, I think that's probably not the best long-term solution since devs may need more over time, and since it increases surface area for breaking changes from exoplayer itself I would ask that the SDK allow handoff of pre-configured ExoPlayer instances, instead of LightOS always making its own. Even if LightOS were to copy the config into a new ExoPlayer build, the cache and priority setup we would be using to properly stream decrypted ogg wouldn't come along. AudioAttributes, AudioSink, focus listening, pause/play/etc., would remain gated, but the tool can configure the input half (source factory, cache, priority, load control). SDK can own and reassert sink, attributes, focus, lifecycle after it adopts the player. I'm happy to prototype the handoff here. The TIDAL backend runs a fully configured media3 player already, so would prob be a low lift to work and post some proto-code here |
|
As an aside, it's really exciting to collaborate on the sdk with you guys. Community has been proposing more backends for phono, which could mean a mostly unified streaming client, which is pretty sick. lmk if you'd like me to make a PR w/ a rough prototype of how this could all work EDIT: realizing that the above two posts sound a little AI-ish. I wrote them myself I promise lol |
|
@jonathancaudill At the highest level, you're saying you could get what you need by allowing devs to inject an exoplayer instance/factory instead of using our default? I'd definitely be open to that as long as it doesn't make things any more complex for the 99.9% of cases. I'm gonna stack a PR on top of this with some kotlin'y stuff in just a bit here but after that, feel free to open your own. (we're not gonna wait to merge this but it should still be safe to work off of after my changes go in) Edit: my changes are in #123 - want to test them in the morning before I merge. |
PR lightphone#119 landed LightAudioPlayer, which already tracks everything the lock screen needs. Rather than make tools mirror that state into the session by hand, add an attach(lightContext, player) overload that derives it: transport callbacks route into the player, and its flows are combined into the LightNowPlaying/LightPlaybackStatus pushed on every change. release() cancels the collector before tearing the session down. The player kept queue metadata only inside ExoPlayer, so expose it as nowPlaying/hasNext/hasPrevious StateFlows. Those also save any caller mapping currentMediaItemIndex back onto its own queue, which is what the audio-demo player screen does today. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… rework PR lightphone#119 rebuilt the client audio surface around LightAudio/LightAudioPlayer, but in doing so it dropped the MediaStore reader this branch had carried: tools lost LightAudioLibrary, the SealedLightContext.audioLibrary accessor, and useMediaVolumeKeys. Nothing replaced them, and the Gradle plugin blocks tools from reaching a ContentResolver themselves, so after lightphone#119 a tool had no sanctioned way to enumerate the on-device music library at all. Restores the three pieces verbatim from 751ac3e, with the LightActivity volume handling merged into the key overrides that have since grown a screen/server routing chain rather than added alongside them. onKeyMultiple is left alone, as it was before. Playback itself stays on lightphone#119's LightAudioPlayer — this only returns the read-side library API and the volume HUD. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3 participants
Adds foreground audio support to the SDK client through a small, opinionated API built on Android's audio libraries:
LightAudiocreates the different audio components.LightAudioPlayerprovides queue-based playback for files, assets, and URLs usingandroidx.media3.LightAudioRecorderrecords microphone input to MPEG-4 files with AAC audio usingandroid.media.MediaRecorderLightAudioCaptureexposes real-time microphone input as aFlowof PCM buffers usingandroid.media.AudioRecordLightAudioVoiceplays short PCM buffers for synthesized sounds and samples usingandroid.media.AudioTrackandroid.media.AudioManager.android.permission.RECORD_AUDIOto tool permission allowlist.Persistent audio (that continues playing after the tool is left) support is not included in this first public surface. Since it requires a bit of additional work on LightOS, we'll first focus on getting the active-tool API stable.
The
audio-demoexample showcases the usage of the library:Other minor changes:
orientationfield to TOML config to define tool screen orientation