Skip to content

Music sources: build the library from added folders and songs - #2

Open
4bit33 wants to merge 7 commits into
masterfrom
feature/music-sources
Open

4bit33 wants to merge 7 commits into
masterfrom
feature/music-sources

Conversation

@4bit33

@4bit33 4bit33 commented Sep 19, 2026

Copy link
Copy Markdown
Owner

Summary

The library was built by scanning the whole device through MediaStore: it needed READ_MEDIA_AUDIO and re-scanned on every cold start and every foreground. It is now built from what the user adds: folders (system folder picker) and single songs (system file picker). Nothing scans by itself; a new file in an added folder shows up after Refresh library. No storage/audio permission any more.

What changed

  • Sources (Storage Access Framework). Folders via OpenDocumentTree, songs via OpenMultipleDocuments, both with persisted READ grants. RoomSourceRepository is the only place that takes or releases a grant. Adding or removing a source cancels a running scan first (the scanner is single-flight, so a scan in progress would swallow the follow-up request) and rescans afterwards.
  • Scanner. SafLibraryScanner replaces the MediaStore scanner: one DocumentsContract child query per directory, and the metadata/artwork extractors, normalizer and artwork cache are reused. Reconciliation is source-scoped (SourceReconciler): a stored song is deleted only when its source was listed COMPLETE, so a source that lost access, is unmounted or was listed partially never deletes anything.
  • Identity. Song.id stays a Long but is now a stable 63-bit hash of provider authority + documentId (pinned by a golden-value test because it is a persisted format), so the saved queue, favorites and playlists survive removing and re-adding a folder. Folder listings skip dot-files (._x.mp3, .trashed-*); a file the user picks explicitly is trusted.
  • Database v3. New sources table; songs get a sourceId FK with ON DELETE CASCADE; the MediaStore columns are gone. MIGRATION_2_3 is a deliberate clean start (old songs dropped; playlist items, favorites and history emptied; playlist names kept), with the DDL copied verbatim from the Room-generated code. Source inserts use IGNORE, never REPLACE (REPLACE deletes the parent row and cascades every song).
  • UI. Pickers are created once in the app shell and reached through LocalMusicActions. Settings gets a "Music sources" group: a row per folder (song count, remove with confirmation), one aggregate "Added songs" row, Add folder / Add songs, and a hint about the folders Android refuses. A folder that lost access stays listed as "Access lost" and tapping it reopens the picker there. The empty state on Home and on every Library tab offers both actions. "Rescan library" is now "Refresh library"; Home no longer flashes the empty state before Room's first emission; playlist counts only count items whose song is still in the library.
  • Removed. The MediaStore scanner, data source, candidate mapping and old reconciler; the app-start and resume scans; and the whole audio-permission stack (gates, manager, status model, Settings row, manifest permissions, perm_* strings, the CheckingPermission / PermissionRequired scan states). FOREGROUND_SERVICE* and POST_NOTIFICATIONS stay.
  • Docs. ADR-010 in ARCHITECTURE.md (supersedes ADR-005) plus the pipeline, triggers and access sections.

Verification

  • ./gradlew testDebugUnitTest: 108 tests, 0 failures. New: stable-id golden values, audio-doc filter, relative paths, unique names, source-scoped reconcile (including "never delete for an unavailable or incomplete source") and the ignore-short prune.
  • Migration on a real v2 database pulled from the phone: user_version 3, empty foreign_key_check, integrity_check ok, and the schema is identical to a fresh install (diffed against the Room-generated DDL).
  • Tested on a device (Nothing A059, Android 16) over adb with real folders, no crashes in logcat:
    • Empty state with both actions; the system picker refuses the storage root and Download (as the Settings hint says)
    • Add Music/Telegram (1 song) and Download/YTDLnis (nested Audio/, Video/, Backups/): 47 songs / 169 MB, exactly what the old MediaStore library held for that folder; the video folder is not imported; the Folders tab shows Telegram/ and YTDLnis/Audio/; the scan took 2.2 s for 48 files
    • Play from a SAF URI, also after force-stop (persisted grants); cold start runs no scan; sources and songs survive a reinstall
    • Add songs (2 files) shows "Added songs". This found a bug, fixed in 3188ce7: a picked .trashed-* file was silently dropped by the folder dot-file filter
    • Copy a file into an added folder + Refresh: added=1; delete it + Refresh: removed=1
    • Ignore-short off + Refresh adds the short file; on + Refresh prunes it
    • Remove a folder and the "Added songs" row: their songs leave, the others stay, and dumpsys activity permissions shows the persisted grants released (4 to 0)
    • Access lost, simulated by pointing a source at a URI with no grant (a real revoke cannot be triggered over adb): the row shows "Access lost", Refresh removes nothing, re-adding moves the songs to the new source without duplicates, and removing the leftover row does not touch them
  • Not verified by me: a genuine grant revoke, the scan progress banner by eye (scans finish in about 2 s), and very large libraries (about 46 ms per file here, so a few thousand songs should take a couple of minutes on the first import).

Notes for the reviewer

  • The clean start is destructive by decision. On the test phone it emptied a hand-made playlist; I restored that one by relinking titles from a backup. Nothing in the app relinks old favorites or playlists (explicitly out of scope).
  • Known limits (also in ADR-010): Android caps persisted grants (512, or 128 before API 30; every picked file costs one) and grants do not survive backup/restore (allowBackup=true), so access is derived from persistedUriPermissions and re-adding the same folder repairs it. The folder picker refuses the storage root and Download on Android 11+. .nomedia / IS_MUSIC are no longer honoured. A moved or renamed file is a new song. The same file reached through two providers (the picker's Audio tab and a folder) can appear twice.
  • With "Ignore short files" on, short files are re-read on every refresh (marked with a comment in the scanner; a persisted "filtered" marker is the upgrade if it ever hurts).
  • The SAF glue (SafAudioDataSource, RoomSourceRepository) has no JVM tests because it needs a ContentResolver; the device run above covers it.
  • Out of scope, follow-up PR: gestures (swipe down to close Now Playing, mini-player swipes, long-press song menu with "Remove from library").

🤖 Generated with Claude Code

4bit33 and others added 7 commits September 19, 2026 23:03
…oncile

Groundwork for replacing the device-wide MediaStore scan with user-added
folders and files (additive, nothing wired yet):

- StableIds.songId: 63-bit hash of provider authority + documentId. A
  persisted format (favorites/playlists key on it), pinned by golden values
  computed with an independent MD5 implementation.
- isAudioDoc / joinRelPath / audioCandidateOf: audio filter (mime, then
  extension for generic types; dot-files skipped), root-relative folder
  paths, SAF row -> candidate mapping.
- SourceReconciler: incremental diff where a stored row may only be
  deleted when its source was enumerated completely, so an unavailable or
  partially listed source can never wipe the library.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
One DocumentsContract child query per directory (no DocumentFile), an
explicit stack with a visited set, cancellation checked per directory and
row. Reports WALK status (complete / incomplete / unavailable) so a scan
can tell "file is gone" from "provider or grant is gone". Not wired yet.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
DB v3: new `sources` table (folder = TREE, single song = FILE) and songs now
carry a `sourceId` FK with ON DELETE CASCADE; MediaStore columns are gone and
`Song.id` is the stable SAF hash. MIGRATION_2_3 is a clean start (old songs
dropped, playlist items / favorites / history cleared, playlist names kept),
with DDL copied from the Room-generated code. Verified on a phone against a
real v2 DB: user_version 3, foreign_key_check empty, schema identical to a
fresh install.

SafLibraryScanner replaces the MediaStore scanner: single-flight, walks every
source that still has a read grant, source-scoped reconcile (only a COMPLETE
source can delete), first-seen dateAdded, ignore-short applied after
extraction. RoomSourceRepository takes/releases the persistable grant and
cancels a running scan first; Add/Remove use cases rescan afterwards.

Nothing scans automatically any more: onAppStarted / onForegrounded are gone.
Removes the MediaStore pipeline (scanner, data source, candidate mapping,
old Reconciler) and the dead SongDao/SongMapper helpers. PermissionDenied now
means "lost access to this file's folder".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The two system pickers (folder tree, multiple audio files) are created once
in the app shell and exposed to screens through LocalMusicActions; a pick
takes the grant and scans right away. Settings gets a "Music sources" group
(a row per folder with song count and remove, one aggregate "Added songs"
row, Add folder / Add songs, and a hint about the folders Android refuses).
A folder whose access was lost stays listed and reopens the picker there.
The empty state on Home and every Library tab now offers both actions.

Also: "Rescan library" becomes "Refresh library"; Home no longer flashes the
empty state before Room's first emission; playlist counts only count items
whose song is still in the library; Settings stats refresh when a scan
completes; sources are removed in one batch with a single rescan.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The library no longer needs READ_MEDIA_AUDIO / READ_EXTERNAL_STORAGE, so the
permission gates, manager, status model, the Settings "Audio access" row,
the manifest permissions, the perm_* strings, the permissionAsked pref and
the CheckingPermission / PermissionRequired scan states go. Home and Library
render their content directly. FOREGROUND_SERVICE* and POST_NOTIFICATIONS
stay for playback.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The dot-file filter (AppleDouble `._x.mp3`, `.trashed-*`) and the audio
name/type check exist for folder listings. Applied to a file the user picked
with the system picker they silently dropped it: found on a phone, where a
picked `.trashed-...mp3` never showed up under "Added songs".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant