Skip to content

fix(proofs): reclaim orphaned IPFS uploads and dedup identical photo content - #98

Merged
cybermax4200 merged 1 commit into
ecotask-network:mainfrom
Christopherdominic:fix/ipfs-orphan-cleanup-dedup-69
Aug 24, 2026
Merged

fix(proofs): reclaim orphaned IPFS uploads and dedup identical photo content#98
cybermax4200 merged 1 commit into
ecotask-network:mainfrom
Christopherdominic:fix/ipfs-orphan-cleanup-dedup-69

Conversation

@Christopherdominic

Copy link
Copy Markdown
Contributor

Closes #69

Problem

In submitProof, photos are hashed and uploaded to IPFS before the DB transaction that creates the proof. Three paths could leave the uploaded CID permanently orphaned (never referenced by any DB row, no cleanup):

  1. A multi-photo batch partially uploads, then a later file's upload fails — the earlier successful uploads are never reclaimed.
  2. The commit transaction re-checks eligibility (task/claim state can race between preflight and commit) and rejects the submission — already-uploaded CIDs are left behind.
  3. The proof commits successfully, but the subsequent enqueueVerification call fails. The code already rolls back the Proof/ProofPhoto rows in that case, but never touched the IPFS content, so it kept the CID orphaned even though nothing in the DB pointed to it anymore.

There was also no reuse of an already-stored CID for identical content: resubmitting a photo whose bytes already exist under a committed ProofPhoto row would re-upload to IPFS, and the DB insert would then throw on the unique sha256 constraint — crashing with a 500 and orphaning the newly-uploaded (redundant) CID.

Fix

  • ipfsService.removeFromIPFS(cid): best-effort, failure-swallowing removal (drops the upload from the web3.storage account in real mode; no-op in mock mode). Cleanup is a courtesy — it logs and swallows its own errors rather than masking whatever error triggered it.
  • submitProof now tracks every CID it newly uploads during the request and reclaims all of them on every path that doesn't end in a durably committed proof: a failed photo batch, a rejected commit transaction, and a failed post-commit verification enqueue.
  • Content-address dedup: before uploading a photo, look up the DB for an existing proof_photos row with the same sha256. If found, reuse its CID instead of re-uploading. Since sha256 is globally unique, only the row that already owns a hash keeps it — a reused row stores sha256: null so the constraint isn't violated (Postgres treats multiple NULLs as distinct under a UNIQUE index).

Testing

  • Extended tests/routes/proofs.test.ts:
    • removes the committed proof when verification enqueue fails now also asserts removeFromIPFS is called with the orphaned CID.
    • New: reclaims the uploaded CID when the commit transaction rejects the submission — injects a commit-time eligibility failure after a successful upload and asserts cleanup.
    • New: reuses an existing CID for identical photo content instead of re-uploading — asserts uploadToIPFS is skipped and the new row is created with the existing CID and sha256: null.
  • Full suite run locally against a real Postgres (matching the CI workflow exactly: npm ci, prisma generate, prisma migrate deploy, npm run build, npm run check:jwt-secret, npm test): 41 suites / 350 tests passing.
  • npm run lint clean.

🦊 Contribution from the GrantFox Third Campaign.

…content

submitProof uploaded photos to IPFS before the DB transaction that
references them. If the commit transaction rejected the submission (e.g. a
race on task/claim eligibility), a partial upload batch failed partway
through, or the post-commit enqueueVerification step failed, the CIDs
already sitting on IPFS were never cleaned up — a permanent, unreferenced
leak with no reclamation path.

- Add ipfsService.removeFromIPFS: a best-effort, failure-swallowing removal
  used to reclaim CIDs uploaded during a request that never ends up
  committed. No-ops in mock mode.
- Track every CID newly uploaded during a submitProof call and reclaim them
  on every path that doesn't end in a durably committed proof: a failed
  photo batch, a rejected commit transaction, and a failed verification
  enqueue (which already rolled back the DB rows but left IPFS content
  behind).
- Dedup identical content: before uploading, check for an existing
  proof_photos row with the same sha256 and reuse its CID instead of
  re-uploading. Since sha256 is globally unique, only the row that already
  owns a hash keeps it; a reused row stores sha256: null so the DB
  constraint isn't violated.

Closes ecotask-network#69
@cybermax4200
cybermax4200 merged commit 2a06c03 into ecotask-network:main Aug 24, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

IPFS uploads are orphaned when the proof commit fails after upload; no content-address dedup

2 participants