fix(nzbdav): register migrated files in file_health on symlink match#597
Closed
javi11 wants to merge 3 commits into
Closed
fix(nzbdav): register migrated files in file_health on symlink match#597javi11 wants to merge 3 commits into
javi11 wants to merge 3 commits into
Conversation
POST /import/nzbdav/migrate-symlinks rewrote library symlinks on disk but never registered the resulting files in file_health (the table the VFS and health checker rely on), and never advanced import_migrations.status to symlinks_migrated. So the migration_stats counter stayed at zero and the health system had no record of the migrated files. Reshape SymlinkLookup into Resolve + CommitRewrites: the walker collects every successfully rewritten item and calls CommitRewrites once after the walk (skipped in dry-run). DBSymlinkLookup now holds HealthRepository, the queue Repository (for source_nzb_path resolution via queue_item_id), and a ConfigGetter; CommitRewrites bulk-advances migration status via the existing MarkSymlinksMigrated and upserts file_health rows via the same AddFileToHealthCheck call the regular post-processor uses. Per-item file_health failures are joined, not aborted.
The previous commit registered file_health rows directly from DBSymlinkLookup.CommitRewrites, passing the mount-relative final_path as library_path and skipping .meta enrichment. Switch to a scoped sync that reuses the library worker's existing per-file path: read each rewritten symlink's .meta, build an AutomaticHealthCheckRecord with correct LibraryPath / SourceNzbPath / ReleaseDate, and batch-upsert. - RewrittenItem now carries the on-disk library symlink path the walker just rewrote; RewriteReport.RewrittenItems exposes them to the handler. - New public LibrarySyncWorker.SyncFiles wraps the private processMetadataForSync + BatchAddAutomaticHealthChecks flow for an explicit file list. Missing .meta skips silently; unreadable .meta is registered as corrupted via the existing helper. - nzbdav handler swaps the TriggerManualSync call for SyncFiles. Response gains library_files_attempted / library_files_registered counts so the operator gets a synchronous confirmation without polling /library-sync/status. - DBSymlinkLookup shrinks back to a single-arg constructor; CommitRewrites is now solely responsible for advancing import_migrations status. Tradeoffs vs full TriggerManualSync: no global library walk, no findFilesToDelete invocation (so Phase-1-only file_health rows are no longer at risk of being pruned by the cutover step), and the API response includes the registered count without an async wait. Tests: extend symlinks_test to verify LibraryPath is populated on committed items and mirrored in RewriteReport.RewrittenItems. Add four table cases in library_sync_test covering SyncFiles empty input, happy path with correct library_path/source_nzb_path, missing-meta silent skip, and corrupt-meta corrupted registration.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
POST /import/nzbdav/migrate-symlinksrewrote library symlinks on disk but never registered the resulting files infile_health— the table the VFS and health checker rely on — so the health system had no record of migrated files.import_migrations.statusfromimported→symlinks_migrated, so the UI'smigration_stats.symlinks_migratedcounter stayed at zero.What changed
SymlinkLookupis nowResolve+CommitRewrites. The walker inRewriteLibrarySymlinkscollects every successfully rewritten item and callsCommitRewritesonce at the end of the walk (skipped in dry-run). Commit errors are reported in the report but not propagated as a fatal walk error — the filesystem rewrites already happened.DBSymlinkLookupnow holds*HealthRepository, the queue*Repository(used to resolvesource_nzb_pathviaqueue_item_id), and aConfigGetter.CommitRewritesbulk-advances migration status via the existingMarkSymlinksMigrated, then upserts afile_healthrow per item via the sameAddFileToHealthCheckcall the regular post-processor uses (health_scheduler.go). IndividualAddFileToHealthCheckfailures are joined viaerrors.Join, never aborted.handleMigrateNzbdavSymlinkspassess.healthRepo,s.queueRepo, ands.configManager.GetConfigGetter()toNewDBSymlinkLookup, plus a nil-check forhealthRepo.Reviewer notes
file_health(upsert viaON CONFLICT(file_path)) andimport_migrations(MarkSymlinksMigratedis a bulkWHERE id IN (...)) already supported this; re-runningmigrate-symlinksis idempotent.processBatchand queue processing) is untouched. Files imported via Phase 1 already getfile_healthrows from the post-processor'sScheduleHealthCheck— no regression risk there.dry_runremains a pure preview:CommitRewritesis not invoked.Test plan
go build ./...greengo vet ./...cleango test -race ./internal/importer/migration/... ./internal/database/... ./internal/api/... ./internal/importer/...all greenCommitRewritesis called once with the expectedRowIDset on success and not at all in dry-runPOST /import/nzbdav→ wait forcompletedPOST /import/nzbdav/migrate-symlinksagainst a library containing symlinks targeting the nzbdav.ids/directorySELECT status, COUNT(*) FROM import_migrations GROUP BY statusshows non-zerosymlinks_migratedSELECT COUNT(*) FROM file_health WHERE file_path IN (<rewritten finalPaths>)matchesreport.rewrittenmigrate-symlinks; second pass should be a no-op (no duplicatefile_healthrows, status alreadysymlinks_migrated)dry_run: true; no new rows in either table