Skip to content

[stable34] fix(filecache): announce every removed entry so metadata is cleaned up - #64497

Open
backportbot[bot] wants to merge 4 commits into
stable34from
backport/63998/stable34
Open

backportbot[bot] wants to merge 4 commits into
stable34from
backport/63998/stable34

Conversation

@backportbot

@backportbot backportbot Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

Backport of #63998

Warning, This backport's changes differ from the original and might be incomplete ⚠️

Todo

  • Review and resolve any conflicts
  • Review and verify the backported changes
  • Amend HEAD commit to remove the line stating to skip CI

Learn more about backports at https://docs.nextcloud.com/server/stable/go.php?to=developer-backports.

@backportbot backportbot Bot added bug 3. to review Waiting for reviews labels Sep 17, 2026
@backportbot backportbot Bot added this to the Nextcloud 34.0.5 milestone Sep 17, 2026
Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>
clear() no longer dispatches removal events for every entry of the
storage. With admin_audit enabled that wrote one log line per file of a
deleted user. It now removes the filecache_extended and file metadata
rows batch by batch, before deleting the filecache rows themselves.

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>
cleanByMountId() deleted the filecache rows of an unmounted storage but
left their filecache_extended and file metadata rows behind. It now goes
through Storage::removeFileCacheEntries(), the same path clear() uses.

files:cleanup also removes filecache_extended, files_metadata and
files_metadata_index rows whose file is no longer in the filecache, so
rows orphaned before this fix are cleaned up too.

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>
@solracsf
solracsf force-pushed the backport/63998/stable34 branch from dc9cc79 to 6df9950 Compare September 18, 2026 05:42
@solracsf
solracsf marked this pull request as ready for review September 18, 2026 05:43
@solracsf
solracsf requested a review from a team as a code owner September 18, 2026 05:43
@solracsf
solracsf requested review from come-nc, leftybournes and salmart-dev and removed request for a team September 18, 2026 05:43
Oracle returns numeric columns as strings, so the fallback lookup in
MetadataRequestService::getStorageId() passed a string to
FilesMetadata::setStorageId(int) and failed with a TypeError. On master
the cast arrived with fcc63b9, which was not backported.

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>
@f0rthelulz

Copy link
Copy Markdown

A production data point from 34.0.2 (MariaDB, Redis file locking), in support of getting this into 34.x. 34.0.2 still ships the unfixed Cache::removeChildren.

What happened. A folder subtree of about 462,000 cache entries had been removed from disk, and occ files:scan --shallow of its parent was run: one Cache::remove() into removeChildren(). It ran about 100 minutes inside a single transaction and did not finish.

  • The processlist showed a continuous stream of DELETE FROM oc_files_metadata[_index] WHERE file_id IN (0, 1, 2, 3, …) at about 2 statements/s.
  • Handler_delete did not move, so each statement deleted 0 rows. The ids are array_chunk positions, because keys are not preserved, and every chunk re-sends the accumulated $cacheEntryRemovedEvents.
  • By the arithmetic of the unfixed loop, 462 chunks means about Σ2k ≈ 214,000 metadata DELETEs with IN lists growing towards 462k values, plus about 107M per-entry CacheEntryRemovedEvent dispatches. That is days of work.
  • The open transaction held row locks the whole time, so other writes, cron included, failed with 1205 Lock wait timeout exceeded.
  • A background cron job (Text's Cleanup, going through the change watcher) independently started the same removal on the same missing tree and held the locks too.

The workaround. SIGKILL rolled both back cleanly. We then recreated the empty folder skeleton, let a recursive files:scan drop the file rows one at a time, and removed the folders leaves-first, so that no removal ever had cached descendants. The two lines in this PR (a fresh $cacheEntryRemovedEvents per chunk, and array_chunk(…, true)) are exactly what we had tried to patch in locally.

The metadata half, drop*ForFiles binding $fileIds instead of $chunk, is also in 34.0.2, but it is already backported (#62771, #64092).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3. to review Waiting for reviews bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants