persist OneDrive thumbnails to ImageKit CDN to fix expiring URLs#165
Open
RsbhThakur wants to merge 1 commit intodevfrom
Open
persist OneDrive thumbnails to ImageKit CDN to fix expiring URLs#165RsbhThakur wants to merge 1 commit intodevfrom
RsbhThakur wants to merge 1 commit intodevfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changelog
All changes in this PR are documented here.
Summary
Two independent concerns are addressed in this PR:
Permanent thumbnail storage — OneDrive / Microsoft Graph thumbnail URLs expire after ~30 minutes, causing every page load to re-hit the Graph API. Thumbnails are now uploaded to ImageKit CDN and stored permanently in MongoDB at two points:
<img onError>refresh path uploads to ImageKit and permanently updates MongoDB, so the file is never fetched from Graph API again.Deleting a file also deletes its ImageKit thumbnail.
Node.js v25 compatibility —
jsonwebtokenv8 used the long-removedSlowBufferAPI and crashed on startup under Node 25. Upgraded to v9, which resolves this without any breaking API changes. The underlyingbuffer-equal-constant-timepackage is also patched to guard againstSlowBufferbeing undefined.New dependency —
server/@imagekit/nodejsjsonwebtokenSlowBuffercrash on Node 25New files
server/services/imagekit.jsLazy-initialised ImageKit singleton (deferred until first use so env vars are loaded).
uploadThumbnail(oneDriveFileId, imageBuffer)— wraps the buffer withtoFile(), uploads to ImageKit under/thumbnails/{fileId}.webpwithuseUniqueFileName: false(overwrites on re-upload), applies WebP conversion at the CDN edge. Returns{ url, fileId }wherefileIdis ImageKit's internal ID needed for deletion.deleteThumbnail(imagekitFileId)— deletes the thumbnail from ImageKit by its internal file ID.isImageKitUrl(url)— returnstruewhen a URL already points at the configured ImageKit endpoint; used to skip re-uploading on already-migrated files.server/scripts/thumbnail_migration.mongodbmongosh queries for one-time migration of legacy files:
files.fileId— speeds upFileModel.findOne({ fileId })lookups.files.thumbnail— enables fast scans for unmigrated files.imagekit_permanent/onedrive_expiring/no_thumbnail.updateMany— clears legacy OneDrive URLs so each file self-migrates lazily on next client request.5–7. Verification and progress-monitoring queries.
Modified files
server/modules/course/course.model.jsAdded
imagekitFileId: { type: String }toFileSchema.Stores ImageKit's internal file ID so thumbnails can be deleted from ImageKit when a file is removed.
server/services/UploadFile.jsUploadFile()— After a successful OneDrive upload,createLinkand the thumbnail fetch from Graph API now run in parallel viaPromise.all(previously sequential — saves one full round-trip). The file is saved to MongoDB immediately with the temporary OneDrive URL, then a fire-and-forget background task:uploadThumbnail).imagekitFileId.The client response is not blocked by the ImageKit upload at any point.
DeleteFile()— Looks upimagekitFileIdfrom MongoDB and callsdeleteThumbnail()in a fire-and-forget task before proceeding with the OneDrive deletion. The client response is not blocked.server/modules/onedrive/onedrive.controller.jsthumbnail()controller — rewritten. Handles legacy files whose stored OneDrive URLs have expired. New behaviour implements a 2-tier lookup:Legacy files hit Tier 2 exactly once, then permanently move to Tier 1.
server/package.json@imagekit/nodejsadded.jsonwebtokenupgraded^8.5.1→^9.0.3.server/.envThree new environment variables required:
Get these from the ImageKit dashboard under Developer Options.
server/jsconfig.json(new)Added to prevent VS Code from scanning into
node_modulesand surfacing TypeScript warnings from third-party SDK source files.Migration path for existing deployments
IMAGEKIT_*variables to the server environment.server/scripts/thumbnail_migration.mongodbin mongosh. This strips old OneDrive URLs so files self-migrate on next client browse.No data loss occurs. Files without a thumbnail show no preview until a client triggers the refresh endpoint, at which point they are permanently stored.
What does NOT change
thumbnailfield onFileModelcontinues to hold a string URL — now a permanent ImageKit URL instead of an expiring OneDrive one.