features: tags blacklisting, download queue and smart downloads - #92
Conversation
…sting-and-download-queue # Conflicts: # app/src/main/kotlin/io/github/landwarderer/futon/core/db/migrations/Migration28To29.kt
|
Debug APK built successfully! Download artifact: debug-apk Artifact will be available for 7 days |
|
Debug APK built successfully! Download artifact: debug-apk Artifact will be available for 7 days |
There was a problem hiding this comment.
Pull request overview
This PR adds user-configurable content filtering (tag blacklisting) and significantly expands the download system with a persistent queue, scheduling constraints (charging/off-peak/quota), and “smart downloads” behavior tied to reading progress. It also includes a local-library rescan flow to address imported downloads/local indexing issues.
Changes:
- Add tag-blacklisting UI + settings key, and apply blacklist filtering across Explore/Search/list pipelines.
- Introduce a Room-backed download queue with a scheduler worker (WorkManager) supporting charging/Wi‑Fi/off‑peak conditions and queue reordering UI.
- Add smart-download/cleanup + storage quota enforcement hooks into download/local cleanup and reading/history flows; add supporting DB migration/converters.
Reviewed changes
Copilot reviewed 74 out of 74 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| app/src/test/kotlin/io/github/landwarderer/futon/download/ui/worker/DownloadTaskTest.kt | Unit test coverage for DownloadTask serialization defaults. |
| app/src/test/kotlin/io/github/landwarderer/futon/download/ui/worker/DownloadSchedulerTest.kt | Unit tests for off-peak window calculations. |
| app/src/main/res/xml/pref_sources.xml | Adds “Blacklisted tags” preference entry. |
| app/src/main/res/xml/pref_downloads.xml | Adds scheduling/limits preferences (charging, off-peak, quota, smart downloads). |
| app/src/main/res/values/strings.xml | Adds strings for queue/scheduling/quota/rescan/blacklist. |
| app/src/main/res/values/dimens.xml | Adds FAB bottom padding dimen for queue screen. |
| app/src/main/res/menu/opt_local.xml | Adds “Rescan” menu action for local storage. |
| app/src/main/res/menu/opt_downloads.xml | Adds entry point to the download queue screen. |
| app/src/main/res/menu/opt_download_queue.xml | New queue screen actions (queue favorites, clear queue). |
| app/src/main/res/layout/item_download_queue.xml | New RecyclerView item layout for queued downloads. |
| app/src/main/res/layout/activity_tags_blacklist.xml | New activity layout for managing blacklisted tags. |
| app/src/main/res/layout/activity_downloads.xml | Adds quota reached + storage usage cards to Downloads screen. |
| app/src/main/res/layout/activity_download_queue.xml | New queue screen layout with list + start FAB. |
| app/src/main/kotlin/io/github/landwarderer/futon/tracker/work/TrackWorker.kt | Propagates new requiresCharging field into scheduled download tasks. |
| app/src/main/kotlin/io/github/landwarderer/futon/suggestions/domain/TagsBlacklist.kt | Adds logging and refactors blacklist checks. |
| app/src/main/kotlin/io/github/landwarderer/futon/settings/work/WorkScheduleManager.kt | Schedules download scheduler updates on relevant preference changes. |
| app/src/main/kotlin/io/github/landwarderer/futon/settings/utils/TimePreference.kt | New custom Preference for selecting times via MaterialTimePicker. |
| app/src/main/kotlin/io/github/landwarderer/futon/settings/utils/TagsAutoCompleteProvider.kt | Updates tag search query path/limit. |
| app/src/main/kotlin/io/github/landwarderer/futon/settings/storage/directories/MangaDirectoriesViewModel.kt | Adjusts directory accessibility + StatFs exception handling. |
| app/src/main/kotlin/io/github/landwarderer/futon/settings/storage/directories/DirectoryConfigAD.kt | Updates inaccessible-directory message. |
| app/src/main/kotlin/io/github/landwarderer/futon/settings/sources/TagsBlacklistViewModel.kt | New ViewModel to fetch tags + persist blacklist selection. |
| app/src/main/kotlin/io/github/landwarderer/futon/settings/sources/TagsBlacklistActivity.kt | New activity to manage blacklist chips + search. |
| app/src/main/kotlin/io/github/landwarderer/futon/settings/sources/SourcesSettingsFragment.kt | Navigates to TagsBlacklistActivity from preferences. |
| app/src/main/kotlin/io/github/landwarderer/futon/settings/DownloadsSettingsFragment.kt | Validates quota changes; triggers scheduler updates on off-peak changes. |
| app/src/main/kotlin/io/github/landwarderer/futon/search/ui/multi/SearchViewModel.kt | Filters search results by blacklisted tags (with DB tag backfill). |
| app/src/main/kotlin/io/github/landwarderer/futon/reader/ui/ReaderViewModel.kt | Wires queue/smart-download dependencies; saves state on chapter change. |
| app/src/main/kotlin/io/github/landwarderer/futon/local/ui/LocalStorageCleanupWorker.kt | Enforces storage quota during cleanup. |
| app/src/main/kotlin/io/github/landwarderer/futon/local/ui/LocalListMenuProvider.kt | Adds “Rescan” action to trigger index update service. |
| app/src/main/kotlin/io/github/landwarderer/futon/local/ui/LocalListFragment.kt | Updates permission request flow for Android R+. |
| app/src/main/kotlin/io/github/landwarderer/futon/local/domain/EnforceStorageQuotaUseCase.kt | New use case to delete oldest downloaded chapters until under quota. |
| app/src/main/kotlin/io/github/landwarderer/futon/local/domain/DeleteReadChaptersUseCase.kt | Adds favorite-skipping + oldestOnly deletion support + extra logging. |
| app/src/main/kotlin/io/github/landwarderer/futon/local/data/LocalStorageManager.kt | Adds total-download-bytes computation helper. |
| app/src/main/kotlin/io/github/landwarderer/futon/local/data/LocalMangaRepository.kt | Exposes APIs needed for quota enforcement (getRawList/getAllFiles). |
| app/src/main/kotlin/io/github/landwarderer/futon/local/data/index/LocalMangaIndex.kt | Wraps update in runCatchingCancellable; adds scan debug output. |
| app/src/main/kotlin/io/github/landwarderer/futon/list/ui/MangaListViewModel.kt | Filters lists by blacklisted tags. |
| app/src/main/kotlin/io/github/landwarderer/futon/history/domain/HistoryUpdateUseCase.kt | Triggers smart download on chapter change. |
| app/src/main/kotlin/io/github/landwarderer/futon/explore/ui/ExploreViewModel.kt | Filters recommendation list by blacklisted tags. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/ui/worker/DownloadWorker.kt | Adds charging constraint propagation, bandwidth limiting, smart cleanup/quota enforcement hooks. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/ui/worker/DownloadTask.kt | Adds requiresCharging to task serialization/equality. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/ui/worker/DownloadSchedulerWorker.kt | New worker to process the queue respecting conditions and off-peak scheduling. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/ui/list/DownloadsViewModel.kt | Adds observable storage usage for downloads screen. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/ui/list/DownloadsMenuProvider.kt | Routes to queue screen. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/ui/list/DownloadsActivity.kt | Displays storage/quota cards and updates padding for overlays. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/ui/DownloadQueueViewModel.kt | New ViewModel to observe queue, mutate items, and trigger scheduler. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/ui/DownloadQueueAdapter.kt | New adapter for queue list items + drag handle integration. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/ui/DownloadQueueActivity.kt | New activity for viewing/reordering queue and starting processing. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/ui/dialog/DownloadDialogViewModel.kt | Adds “queue vs start now” behavior and charging propagation. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/ui/dialog/ChaptersSelectMacro.kt | Changes “WholeManga” macro to return all chapter IDs. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/domain/usecase/SmartDownloadUseCase.kt | New use case for auto-enqueueing next chapter during reading. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/domain/usecase/QueueAllUnreadFromFavoritesUseCase.kt | New use case to enqueue unread chapters for all favorites. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/domain/usecase/AddUnreadToQueueUseCase.kt | New use case to enqueue unread chapters for a manga. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/data/repository/SmartDownloadRepository.kt | New repository for smart-download state persistence. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/data/repository/DownloadQueueRepository.kt | New repository managing queue insert/update/reorder and scheduler triggering. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/data/entity/SmartDownloadEntity.kt | New Room entity for smart-download state. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/data/entity/DownloadQueueEntity.kt | New Room entity for queued downloads. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/data/dao/SmartDownloadDao.kt | New DAO for smart-download entity access. |
| app/src/main/kotlin/io/github/landwarderer/futon/download/data/dao/DownloadQueueDao.kt | New DAO for queue access + priority reorder transaction. |
| app/src/main/kotlin/io/github/landwarderer/futon/details/ui/pager/ChaptersPagesViewModel.kt | Adds queue scheduling path and safer local update handling. |
| app/src/main/kotlin/io/github/landwarderer/futon/details/ui/DetailsViewModel.kt | Wires in queue/scheduler dependencies through base viewmodel. |
| app/src/main/kotlin/io/github/landwarderer/futon/details/ui/DetailsMenuProvider.kt | Fixes delete visibility for local items. |
| app/src/main/kotlin/io/github/landwarderer/futon/core/util/ext/IO.kt | Adds optional bandwidth-limited writing for Okio sinks. |
| app/src/main/kotlin/io/github/landwarderer/futon/core/util/BandwidthLimiter.kt | New bandwidth limiter for download throttling. |
| app/src/main/kotlin/io/github/landwarderer/futon/core/prefs/AppSettings.kt | Adds keys/settings for blacklist, off-peak, quota, smart downloads, charging-only, bandwidth. |
| app/src/main/kotlin/io/github/landwarderer/futon/core/nav/AppRouter.kt | Adds navigation to tags blacklist and download queue screens. |
| app/src/main/kotlin/io/github/landwarderer/futon/core/db/Tables.kt | Adds table constants for queue and smart downloads. |
| app/src/main/kotlin/io/github/landwarderer/futon/core/db/migrations/Migration29To30.kt | New migration creating queue + smart download tables. |
| app/src/main/kotlin/io/github/landwarderer/futon/core/db/MangaDatabase.kt | Bumps DB version, registers entities/DAOs/migration, adds type converters. |
| app/src/main/kotlin/io/github/landwarderer/futon/core/db/entity/TagEntity.kt | Adds index to speed tag title queries. |
| app/src/main/kotlin/io/github/landwarderer/futon/core/db/dao/TagsDao.kt | Adds query to search all tags by title. |
| app/src/main/kotlin/io/github/landwarderer/futon/core/db/converters/DataConverters.kt | New type converters for int/long arrays to text columns. |
| app/src/main/kotlin/io/github/landwarderer/futon/core/AppModule.kt | Provides new DAOs via Hilt (queue + manga DAO). |
| app/src/main/kotlin/io/github/landwarderer/futon/backups/ui/restore/RestoreService.kt | Triggers local index update service after restoring settings. |
| app/src/main/AndroidManifest.xml | Registers TagsBlacklistActivity and DownloadQueueActivity. |
| app/src/androidTest/kotlin/io/github/landwarderer/futon/download/ui/worker/DownloadWorkerIntegrationTest.kt | Adds instrumented test for constraint propagation (currently broken). |
| if (wifiOnly != other.wifiOnly) return false | ||
| if (charging_only != other.charging_only) return false | ||
| if (offPeakOnly != other.offPeakOnly) return false | ||
|
|
||
| return true | ||
| } | ||
|
|
||
| override fun hashCode(): Int { | ||
| var result = id.hashCode() | ||
| result = 31 * result + mangaId.hashCode() | ||
| result = 31 * result + chaptersIds.contentHashCode() | ||
| result = 31 * result + priority | ||
| result = 31 * result + createdAt.hashCode() | ||
| result = 31 * result + wifiOnly.hashCode() | ||
| result = 31 * result + charging_only.hashCode() | ||
| result = 31 * result + offPeakOnly.hashCode() | ||
| return result | ||
| } |
| private fun fetchTags() { | ||
| launchJob(Dispatchers.IO) { | ||
| try { | ||
| val request = Request.Builder() | ||
| .url("https://raw.githubusercontent.com/AppFuton/filters/refs/heads/main/data/tags.json") | ||
| .build() | ||
| okHttpClient.newCall(request).execute().use { response -> | ||
| if (response.isSuccessful) { | ||
| val tags = response.body.byteStream().use { stream -> | ||
| Json.decodeFromStream<List<String>>(stream) | ||
| } | ||
| _allTags.value = tags.sorted() | ||
| } | ||
| } | ||
| } finally { | ||
| loadingCounter.decrement() | ||
| } | ||
| } | ||
| } |
| @WorkerThread | ||
| fun getTotalBytesUsedByDownloads(): Long { | ||
| return runBlocking { | ||
| getConfiguredStorageDirs().sumOf { it.computeSize() } | ||
| } | ||
| } |
| fun bind(item: DownloadQueueItem) { | ||
| binding.textViewTitle.text = item.manga?.title ?: "Unknown" | ||
| binding.textViewDetails.text = "${item.entity.chaptersIds.size} chapters" | ||
|
|
||
| binding.imageViewCover.setImageAsync(item.manga?.coverUrl, item.manga) | ||
|
|
||
| binding.buttonRemove.setOnClickListener { onRemove(item) } | ||
| binding.buttonPause.setOnClickListener { onPauseToggle(item) } | ||
| binding.buttonPause.setImageResource( | ||
| if (item.entity.isPaused) R.drawable.ic_action_resume else R.drawable.ic_action_pause | ||
| ) | ||
| binding.imageViewHandle.setOnTouchListener { _, event -> | ||
| if (event.actionMasked == MotionEvent.ACTION_DOWN) { | ||
| onStartDrag(this) | ||
| } | ||
| false | ||
| } | ||
| } |
| fun openDownloadQueue() { | ||
| startActivity(Intent(contextOrNull() ?: return, Class.forName("io.github.landwarderer.futon.download.ui.DownloadQueueActivity"))) | ||
| } |
| var tagsBlacklist: Set<String> | ||
| get() { | ||
| val string = prefs.getString(KEY_TAGS_BLACKLIST, null)?.trimEnd(' ', ',') | ||
| if (string.isNullOrEmpty()) { | ||
| return emptySet() | ||
| } | ||
| return string.split(',').mapToSet { it.trim() } | ||
| } | ||
| set(value) = prefs.edit { putString(KEY_TAGS_BLACKLIST, value.joinToString(", ")) } |
| if (activeWorks.any { it.state == WorkInfo.State.RUNNING }) { | ||
| Log.d("DownloadScheduler", "A download is already running, scheduling check in 30s") | ||
| scheduleNextCheck(30, TimeUnit.SECONDS) | ||
| return@runCatchingCancellable Result.success() | ||
| } |
| val constructor = schedulerClass.declaredConstructors[0] | ||
| constructor.isAccessible = true | ||
| val args = arrayOfNulls<Any>(constructor.parameterCount) | ||
| val schedulerInstance = constructor.newInstance(*args) | ||
|
|
| if (delayMs > 0) { | ||
| runBlocking { | ||
| kotlinx.coroutines.delay(delayMs) | ||
| } | ||
| synchronized(this) { | ||
| lastTime = SystemClock.elapsedRealtime() | ||
| } | ||
| } |
| println("LocalMangaIndex: Starting update") | ||
| runCatchingCancellable { | ||
| db.withTransaction { | ||
| val dao = db.getLocalMangaIndexDao() | ||
| dao.clear() |
|
@onlyashd are any of the things the AI listed something we should worry about? |
|
@onlyashd I my issue #82 I had 2 things I wanted (should probably have been split into 2 sepparate issue),
I think that smart download option kind of solves the first issue, But my other issue was that since you need to download the manga to read it, and if you want to have it saved for later, you might as well save it to device storage right when you read it. |
|
@enkvadrat i think we can do that in the next version. |
|
The functionality is not that hard, but how would the ui for the options look like? |
I think the smart downloads can have another option called "Delete downloaded chapters after reading" so if turned off the chapters would be saved. |
|
Does smart download also download the chapter you are currently reading? |
I'll make it do that |
|
I think it would be better to have separate options:
|
|
Sounds good, these can fall under smart downloads |
This PR is bigger than planned, sorry.
Features
Add Tag Filtering in Content #26
hide items from search and explorer based on their tags. Manage it in Settings > Manga sources > Blacklisted tags.
Download Scheduler - Schedule manga downloads for optimal times #74
items can be added to a queue that will download them in the specified order.
Order can be changed by drag-&-drop. Secondary options:
"Charging only","Off-peak"and"Storage quota".Download pages as the manga is read #82
if enabled, the next chapter will be downloaded and the oldest read one will be deleted while you read. Example: you have 5 chapters downloaded, when you start the 4th, the 6th will be downloaded and the 1st will be deleted.
Fixes:
Screenshots
New download options
Download quota limitation
Smart downloads
Tags blacklisting
Tip
Logcat filters for tests:
Note
If the user disables Smart downloads, continues reading and then re-enable it the "skiped" chapters won't be downloaded. The deletion process will work the same, removing the oldest downloaded chapter.