Skip to content

fix(nzbdav): register migrated files in file_health on symlink match#597

Closed
javi11 wants to merge 3 commits into
mainfrom
session/unruffled-khorana-da5f6b
Closed

fix(nzbdav): register migrated files in file_health on symlink match#597
javi11 wants to merge 3 commits into
mainfrom
session/unruffled-khorana-da5f6b

Conversation

@javi11
Copy link
Copy Markdown
Owner

@javi11 javi11 commented May 20, 2026

Summary

  • POST /import/nzbdav/migrate-symlinks rewrote library symlinks on disk but never registered the resulting files in file_health — the table the VFS and health checker rely on — so the health system had no record of migrated files.
  • Same call site also never advanced import_migrations.status from importedsymlinks_migrated, so the UI's migration_stats.symlinks_migrated counter stayed at zero.

What changed

  • SymlinkLookup is now Resolve + CommitRewrites. The walker in RewriteLibrarySymlinks collects every successfully rewritten item and calls CommitRewrites once at the end of the walk (skipped in dry-run). Commit errors are reported in the report but not propagated as a fatal walk error — the filesystem rewrites already happened.
  • DBSymlinkLookup now holds *HealthRepository, the queue *Repository (used to resolve source_nzb_path via queue_item_id), and a ConfigGetter. CommitRewrites bulk-advances migration status via the existing MarkSymlinksMigrated, then upserts a file_health row per item via the same AddFileToHealthCheck call the regular post-processor uses (health_scheduler.go). Individual AddFileToHealthCheck failures are joined via errors.Join, never aborted.
  • handleMigrateNzbdavSymlinks passes s.healthRepo, s.queueRepo, and s.configManager.GetConfigGetter() to NewDBSymlinkLookup, plus a nil-check for healthRepo.

Reviewer notes

  • No schema changes — both file_health (upsert via ON CONFLICT(file_path)) and import_migrations (MarkSymlinksMigrated is a bulk WHERE id IN (...)) already supported this; re-running migrate-symlinks is idempotent.
  • Phase 1 import path (processBatch and queue processing) is untouched. Files imported via Phase 1 already get file_health rows from the post-processor's ScheduleHealthCheck — no regression risk there.
  • dry_run remains a pure preview: CommitRewrites is not invoked.

Test plan

  • go build ./... green
  • go vet ./... clean
  • go test -race ./internal/importer/migration/... ./internal/database/... ./internal/api/... ./internal/importer/... all green
  • Unit test updated to assert CommitRewrites is called once with the expected RowID set on success and not at all in dry-run
  • Manual end-to-end:
    • Run POST /import/nzbdav → wait for completed
    • Run POST /import/nzbdav/migrate-symlinks against a library containing symlinks targeting the nzbdav .ids/ directory
    • Confirm SELECT status, COUNT(*) FROM import_migrations GROUP BY status shows non-zero symlinks_migrated
    • Confirm SELECT COUNT(*) FROM file_health WHERE file_path IN (<rewritten finalPaths>) matches report.rewritten
    • Re-run migrate-symlinks; second pass should be a no-op (no duplicate file_health rows, status already symlinks_migrated)
    • Re-run with dry_run: true; no new rows in either table

javi11 added 3 commits May 20, 2026 19:47
POST /import/nzbdav/migrate-symlinks rewrote library symlinks on disk but
never registered the resulting files in file_health (the table the VFS and
health checker rely on), and never advanced import_migrations.status to
symlinks_migrated. So the migration_stats counter stayed at zero and the
health system had no record of the migrated files.

Reshape SymlinkLookup into Resolve + CommitRewrites: the walker collects
every successfully rewritten item and calls CommitRewrites once after the
walk (skipped in dry-run). DBSymlinkLookup now holds HealthRepository, the
queue Repository (for source_nzb_path resolution via queue_item_id), and a
ConfigGetter; CommitRewrites bulk-advances migration status via the
existing MarkSymlinksMigrated and upserts file_health rows via the same
AddFileToHealthCheck call the regular post-processor uses. Per-item
file_health failures are joined, not aborted.
The previous commit registered file_health rows directly from
DBSymlinkLookup.CommitRewrites, passing the mount-relative final_path as
library_path and skipping .meta enrichment. Switch to a scoped sync that
reuses the library worker's existing per-file path: read each rewritten
symlink's .meta, build an AutomaticHealthCheckRecord with correct
LibraryPath / SourceNzbPath / ReleaseDate, and batch-upsert.

- RewrittenItem now carries the on-disk library symlink path the walker
  just rewrote; RewriteReport.RewrittenItems exposes them to the handler.
- New public LibrarySyncWorker.SyncFiles wraps the private
  processMetadataForSync + BatchAddAutomaticHealthChecks flow for an
  explicit file list. Missing .meta skips silently; unreadable .meta is
  registered as corrupted via the existing helper.
- nzbdav handler swaps the TriggerManualSync call for SyncFiles. Response
  gains library_files_attempted / library_files_registered counts so the
  operator gets a synchronous confirmation without polling
  /library-sync/status.
- DBSymlinkLookup shrinks back to a single-arg constructor; CommitRewrites
  is now solely responsible for advancing import_migrations status.

Tradeoffs vs full TriggerManualSync: no global library walk, no
findFilesToDelete invocation (so Phase-1-only file_health rows are no
longer at risk of being pruned by the cutover step), and the API
response includes the registered count without an async wait.

Tests: extend symlinks_test to verify LibraryPath is populated on
committed items and mirrored in RewriteReport.RewrittenItems. Add four
table cases in library_sync_test covering SyncFiles empty input, happy
path with correct library_path/source_nzb_path, missing-meta silent
skip, and corrupt-meta corrupted registration.
@javi11 javi11 closed this May 30, 2026
@javi11 javi11 deleted the session/unruffled-khorana-da5f6b branch May 30, 2026 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant