Move image storage from Postgres bytea to Supabase Storage by ThatXliner · Pull Request #69 · billion-app/billion

ThatXliner · 2026-04-04T05:10:12Z

Summary

Adds a storage-agnostic uploadImage/deleteImage API in packages/db/storage backed by Supabase Storage, keeping Supabase as a db-package implementation detail
Scraper now uploads DALL-E images to object storage and stores the public URL in Video.imageUrl instead of writing raw JPEG bytes to a bytea column
API serves image URLs directly instead of base64-encoding blobs (no more data URIs)
Renames imageUri → imageUrl across frontend, API, and social-media-agent
Includes a migration script to move existing blobs to Supabase Storage

Deployment steps

Create Supabase Storage bucket: Go to Supabase dashboard → Storage → New bucket → name it images → set to Public

Add env vars to your deployment environment:

SUPABASE_URL=https://<project-ref>.supabase.co
SUPABASE_SERVICE_ROLE_KEY=<your-service-role-key>
SUPABASE_STORAGE_BUCKET=images

Push schema: pnpm db:push to add the image_url column to the video table

Migrate existing blobs:

cd packages/db
pnpm with-env tsx migrate-images-to-storage.ts --dry-run  # preview
pnpm with-env tsx migrate-images-to-storage.ts            # run for real

Drop deprecated columns (after verifying migration):

ALTER TABLE video DROP COLUMN image_data;
ALTER TABLE video DROP COLUMN image_mime_type;
ALTER TABLE video DROP COLUMN image_width;
ALTER TABLE video DROP COLUMN image_height;

Remove the deprecated bytea/imageData fields from packages/db/src/schema.ts and the custom bytea type if no longer used

Test plan

Verify pnpm db:push succeeds with the new image_url column
Run migration script with --dry-run and confirm it finds existing blobs
Run migration and verify images are accessible via Supabase Storage URLs
Run scraper and confirm new videos get imageUrl populated (not imageData)
Verify feed loads images from URLs in the Expo app
Confirm API no longer returns base64 data URIs

🤖 Generated with Claude Code

Adds a storage-agnostic uploadImage/deleteImage API in packages/db/storage backed by Supabase Storage, so consumers never import Supabase directly. Adds imageUrl text column to Video schema (imageData kept temporarily for migration). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Scraper now uploads DALL-E images via @acme/db/storage and stores the public URL in Video.imageUrl. API serves the URL directly instead of base64-encoding blobs — eliminates the data URI overhead. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Renames imageUri → imageUrl across frontend and social-media-agent. Adds Supabase Storage env vars to .env.example. Adds migration script to move existing bytea blobs to Supabase Storage. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

vercel · 2026-04-04T05:10:18Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
billion-nextjs	Ready	Preview, Comment	Apr 4, 2026 7:51am

Copilot

Pull request overview

This PR introduces a DB-package storage abstraction for image uploads (backed by Supabase Storage) and transitions the Video model from storing binary image blobs to storing an imageUrl, updating upstream generators and downstream consumers accordingly.

Changes:

Add @acme/db/storage with uploadImage/deleteImage backed by Supabase Storage.
Extend the Video schema with imageUrl (keeping imageData temporarily) and add a one-off migration script to move blobs to object storage.
Update scraper generation, API payloads, and Expo/social-media-agent types/UI to use imageUrl instead of imageUri.

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
social-media-agent/src/agent.ts	Renames `ContentItem.imageUri` → `imageUrl` to match new API model.
pnpm-lock.yaml	Adds Supabase JS and related transitive deps to the lockfile.
packages/db/src/storage.ts	New storage abstraction for uploading/deleting images via Supabase Storage.
packages/db/src/schema.ts	Adds `Video.imageUrl`, repositions `thumbnailUrl`, and marks blob columns as deprecated.
packages/db/package.json	Exposes `./storage` export and adds `@supabase/supabase-js` dependency.
packages/db/migrate-images-to-storage.ts	New script to migrate `video.image_data` blobs into storage and backfill `image_url`.
packages/api/src/router/video.ts	Switches video feed payload from `imageUri` (data URI) to `imageUrl`.
packages/api/src/router/content.ts	Updates content card schema from `imageUri` to `imageUrl`.
apps/scraper/src/utils/db/video-operations.ts	Generates JPEG, uploads to storage, and persists `imageUrl` on `Video` upsert.
apps/expo/src/app/(tabs)/index.tsx	Uses `imageUrl` when rendering thumbnails in browse list.
apps/expo/src/app/(tabs)/feed.tsx	Uses `imageUrl` for feed image rendering (fallback to `thumbnailUrl`).
.env.example	Documents Supabase Storage env vars needed for uploads.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-04T05:15:09Z

+          articlePreview: video.description,
+          imageUrl: video.imageUrl ?? undefined,
+          thumbnailUrl: video.thumbnailUrl ?? undefined,
+          originalContentId: video.contentId,


The feed response now only returns imageUrl; if there are existing video rows that still have imageData populated but haven’t been migrated to imageUrl yet (as implied by keeping imageData in the schema and adding a migration script), those posts will lose images in clients. Consider temporarily keeping the previous imageData → data-URI fallback (or gating this change behind completing the migration) to avoid a rollout regression.

Copilot · 2026-04-04T05:15:10Z


+# Supabase Storage (for image uploads)
+# Project URL: https://supabase.com/dashboard/project/_/settings/api
+SUPABASE_URL=https://your-project-ref.supabase.co


SUPABASE_SERVICE_ROLE_KEY is a highly-privileged secret; since this file is a template that often gets copied into multiple environments, it would help to explicitly note that this key must only ever be used server-side (never in Expo/browser code) and should be kept out of any client-exposed env var mechanisms.

Suggested change

SUPABASE_URL=https://your-project-ref.supabase.co

SUPABASE_URL=https://your-project-ref.supabase.co

# WARNING: `SUPABASE_SERVICE_ROLE_KEY` is highly privileged and must only be used server-side.

# Never use it in Expo/browser code and never expose it via `EXPO_PUBLIC_*` or any other client-exposed env var mechanism.

ThatXliner · 2026-04-04T05:31:09Z

Code review

Found 1 issue:

deleteImage cleanup on DB update failure deletes the previously-working image — In video-operations.ts, when content is being regenerated, uploadImage overwrites the existing file at videos/${contentType}/${contentId}.jpg (via upsert: true). If the subsequent db.update to write imageUrl fails, the catch block calls deleteImage(storagePath), which deletes the file that was just uploaded. However, since the upload already overwrote the previous file at the same deterministic path, the old imageUrl still stored in the DB now points to a deleted file. This means a transient DB failure during regeneration corrupts the previously-working image URL. The fix would be to only call deleteImage when the upload itself failed (not when the DB update failed), or to skip cleanup entirely since the file at that path will be overwritten on the next successful run anyway.

billion/apps/scraper/src/utils/db/video-operations.ts

Lines 119 to 134 in 1c982f6

    
           // Upload image after successful DB write, then update the row 
        
           if (jpegData) { 
        
             const storagePath = `videos/${contentType}/${contentId}.jpg`; 
        
             try { 
        
               const imageUrl = await uploadImage(storagePath, jpegData); 
        
               await db 
        
                 .update(Video) 
        
                 .set({ imageUrl }) 
        
                 .where(and(eq(Video.contentType, contentType), eq(Video.contentId, contentId))); 
        
               logger.debug(`Uploaded image to ${storagePath}`); 
        
             } catch (error) { 
        
               // Best-effort cleanup of orphaned upload 
        
               try { await deleteImage(storagePath); } catch { /* ignore */ } 
        
               logger.warn(`Image upload/update failed for ${contentType}:${contentId}, video saved without image`); 
        
             } 
        
           }

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

Separate upload and DB update into distinct try/catch blocks so that a transient DB failure doesn't delete the image at the deterministic storage path, which may already be referenced by an existing imageUrl. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 16 out of 17 changed files in this pull request and generated 6 comments.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-04T06:00:28Z


 import { db } from '@acme/db/client';
 import { Video } from '@acme/db/schema';
+import { uploadImage, deleteImage } from '@acme/db/storage';


deleteImage is imported but never used in this module. This will either fail typecheck with noUnusedLocals or add dead code; remove the import or use it where appropriate.

Suggested change

import { uploadImage, deleteImage } from '@acme/db/storage';

import { uploadImage } from '@acme/db/storage';

Copilot · 2026-04-04T06:00:28Z

+import { createClient } from "@supabase/supabase-js";
+
+const BUCKET = process.env.SUPABASE_STORAGE_BUCKET ?? "images";
+
+function getClient() {
+  const url = process.env.SUPABASE_URL;
+  const key = process.env.SUPABASE_SERVICE_ROLE_KEY;
+  if (!url || !key) {
+    throw new Error(
+      "Missing SUPABASE_URL or SUPABASE_SERVICE_ROLE_KEY for storage",
+    );
+  }
+  return createClient(url, key);


SUPABASE_STORAGE_BUCKET is read once at module import time (const BUCKET = ...). In migrate-images-to-storage.ts, dotenv is loaded after importing this module, so a bucket value set in .env won’t be picked up and the script may upload to the default bucket unintentionally. Consider reading the bucket env var inside getClient()/uploadImage() (or exporting a getter) so it reflects the runtime environment.

Copilot · 2026-04-04T06:00:28Z

-    // Hybrid image storage: Binary AI-generated images OR URL-based scraped thumbnails
-    imageData: bytea("image_data"), // Raw JPEG bytes (AI-generated)
-    imageMimeType: t.varchar("image_mime_type", { length: 50 }), // "image/jpeg"
+    // Image storage: source thumbnail URL (scraped)


The comment says "Image storage: source thumbnail URL (scraped)", but this block now contains both imageUrl (uploaded AI image) and thumbnailUrl (scraped). Updating the comment to reflect both fields will prevent future confusion during migrations/cleanup.

Suggested change

// Image storage: source thumbnail URL (scraped)

// Image storage: uploaded image URL and source thumbnail URL (scraped)

Copilot · 2026-04-04T06:00:29Z

+      try {
+        await db
+          .update(Video)
+          .set({ imageUrl })


When updating the video row after a successful upload, it only sets imageUrl. If this row previously had legacy imageData/imageMimeType/imageWidth/imageHeight, those blobs will remain stored indefinitely (and the migration script won’t clear them because it only processes rows where image_url is NULL). Consider also nulling the legacy columns in this update (or expanding the migration script to clear blobs whenever image_url is non-null).

Suggested change

.set({ imageUrl })

.set({

imageUrl,

imageData: null,

imageMimeType: null,

imageWidth: null,

imageHeight: null,

})

SUPABASE_STORAGE_BUCKET was captured at import time, before the migration script dotenv.config() could populate it — uploads silently used the default bucket. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

When a new image URL was saved, the old imageData/imageMimeType/ imageWidth/imageHeight columns were left populated — wasting storage and preventing the migration script from skipping them. Also removes unused deleteImage import. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

ThatXliner and others added 3 commits April 3, 2026 22:08

ThatXliner requested a review from Copilot April 4, 2026 05:10

ThatXliner changed the title ~~feat(db): add Supabase Storage abstraction and imageUrl column~~ Move image storage from Postgres bytea to Supabase Storage Apr 4, 2026

vercel Bot deployed to Preview April 4, 2026 05:10 View deployment

Copilot started reviewing on behalf of ThatXliner April 4, 2026 05:10 View session

Copilot AI reviewed Apr 4, 2026

View reviewed changes

address review comments

1c982f6

vercel Bot deployed to Preview April 4, 2026 05:27 View deployment

vercel Bot deployed to Preview April 4, 2026 05:35 View deployment

Format

6a49c05

vercel Bot deployed to Preview April 4, 2026 05:40 View deployment

Fix code review issues

db49802

vercel Bot deployed to Preview April 4, 2026 05:44 View deployment

ThatXliner requested a review from Copilot April 4, 2026 05:56

Copilot started reviewing on behalf of ThatXliner April 4, 2026 05:56 View session

Copilot AI reviewed Apr 4, 2026

View reviewed changes

NO DOCS

17d76c7

vercel Bot deployed to Preview April 4, 2026 06:02 View deployment

ThatXliner and others added 3 commits April 4, 2026 00:49

🐛 fix(storage): defer env read to call time

33c3dec

SUPABASE_STORAGE_BUCKET was captured at import time, before the migration script dotenv.config() could populate it — uploads silently used the default bucket. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

✏️ fix(db): correct schema comment for image fields

7966e8b

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

vercel Bot deployed to Preview April 4, 2026 07:51 View deployment

	import { uploadImage, deleteImage } from '@acme/db/storage';
	import { uploadImage } from '@acme/db/storage';

	// Image storage: source thumbnail URL (scraped)
	// Image storage: uploaded image URL and source thumbnail URL (scraped)

-          .set({ imageUrl })
+          .set({
+            imageUrl,
+            imageData: null,
+            imageMimeType: null,
+            imageWidth: null,
+            imageHeight: null,
+          })

Uh oh!

Conversation

ThatXliner commented Apr 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Deployment steps

Test plan

Uh oh!

vercel Bot commented Apr 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

ThatXliner commented Apr 4, 2026

Code review

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ThatXliner commented Apr 4, 2026 •

edited

Loading

vercel Bot commented Apr 4, 2026 •

edited

Loading