feat: reap stale/failed upload orphans (scoped #65)#101
Merged
Conversation
ravirajsinh45
added a commit
that referenced
this pull request
Jul 7, 2026
ravirajsinh45
force-pushed
the
feat/reap-stale-uploads
branch
from
7f1545e to
d9f4484
Compare
ravirajsinh45
force-pushed
the
chore/storage-usage-tests-and-alignment
branch
from
9fac2a6 to
ee28017
Compare
ravirajsinh45
added a commit
that referenced
this pull request
Jul 7, 2026
ravirajsinh45
added a commit
that referenced
this pull request
Jul 7, 2026
…destroys active uploads) (review #101)
ravirajsinh45
force-pushed
the
feat/reap-stale-uploads
branch
from
d9f4484 to
9916b6b
Compare
ravirajsinh45
force-pushed
the
chore/storage-usage-tests-and-alignment
branch
from
ee28017 to
f5e87a5
Compare
ravirajsinh45
added a commit
that referenced
this pull request
Jul 7, 2026
ravirajsinh45
added a commit
that referenced
this pull request
Jul 7, 2026
…destroys active uploads) (review #101)
ravirajsinh45
force-pushed
the
feat/reap-stale-uploads
branch
from
9916b6b to
b304e70
Compare
ravirajsinh45
force-pushed
the
chore/storage-usage-tests-and-alignment
branch
from
f5e87a5 to
43e578c
Compare
ravirajsinh45
changed the base branch from
chore/storage-usage-tests-and-alignment
to
main
…eclaim orphan versions) - _reap_stale_uploads(db) soft-deletes uploading/failed versions older than the configurable cutoff and best-effort deletes their S3 objects; reap_stale_uploads() task wraps session lifecycle + commit. Registered hourly in beat_schedule. - Shared real_db fixture moved to conftest.py (used by storage-usage + reaper tests).
…destroys active uploads) (review #101)
ravirajsinh45
force-pushed
the
feat/reap-stale-uploads
branch
from
b304e70 to
872907e
Compare
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on the storage-usage branch. A scoped slice of #65: reclaim storage from stuck/failed
uploads — the leak the committed-only storage cap (#98) can't see.
What it does
Hourly Celery beat
reap_stale_uploads:ListMultipartUploads,Initiatedolder thanSTALE_UPLOAD_TIMEOUT_HOURS, default 24) — the only way to reach stuck multiparts, since theUploadId is not stored in the DB.
uploading/failedversions older than the cutoff and best-effort deletes theirS3 objects (raw + processed/HLS prefix + thumbnail). DB rows stay for No garbage collection for soft-deleted assets / orphaned S3 objects / expired share links #65's retention GC.
New:
settings.stale_upload_timeout_hours;s3_service.list_stale_multipart_uploads+delete_prefix.Logic split into a testable
_reap_stale_uploads(db)(mutates, no commit) + a task wrapper (session +commit).
Out of scope (stays in #65): soft-deleted-asset cascade GC,
ShareLink.expires_atenforcement, ageneral orphan sweeper, retention window, manual purge endpoint.
Testing
S3 deletes), and a real-DB selection test (old
uploading/failedreaped; recent/readyuntouched).Stacked on
chore/storage-usage-tests-and-alignment; retarget as the stack merges.