Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,25 @@ CHAT_MODEL=gemini-3.1-flash-lite
GOOGLE_OAUTH_CLIENT_ID=your_google_oauth_client_id
GOOGLE_OAUTH_SECRET=your_google_oauth_secret

# Supabase
# Neon Postgres
# DATABASE_URL is the pooled endpoint (hostname ends in -pooler) used by the running app.
# DIRECT_URL is the unpooled endpoint used by Prisma Migrate, which cannot run through a
# transaction-mode pooler. Both are shown by `neon connection-string <branch>`.
DATABASE_URL=postgresql://user:password@ep-example-pooler.region.aws.neon.tech/neondb?sslmode=require
DIRECT_URL=postgresql://user:password@ep-example.region.aws.neon.tech/neondb?sslmode=require

# Cloudflare R2 object storage (S3-compatible)
# The API token needs Object Read & Write on this bucket only. The endpoint is derived as
# https://${R2_ACCOUNT_ID}.r2.cloudflarestorage.com — do not set it separately.
R2_ACCOUNT_ID=your_cloudflare_account_id
R2_ACCESS_KEY_ID=your_r2_access_key_id
R2_SECRET_ACCESS_KEY=your_r2_secret_access_key
R2_BUCKET=your_r2_bucket_name

# Supabase (auth only — Postgres and Storage have migrated to Neon and R2)
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_ANON_KEY=your_supabase_anon_key
SUPABASE_SERVICE_ROLE_KEY=your_supabase_service_role_key
SUPABASE_DATABASE_URL=postgresql://postgres:password@db.your-project.supabase.co:5432/postgres
VITE_SUPABASE_URL=${SUPABASE_URL}
VITE_SUPABASE_ANON_KEY=${SUPABASE_ANON_KEY}

Expand Down
30 changes: 23 additions & 7 deletions RUNBOOK.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,8 @@ Copy these values from Supabase Settings:
- Project URL -> `SUPABASE_URL`
- anon public key -> `SUPABASE_ANON_KEY`
- service_role key -> `SUPABASE_SERVICE_ROLE_KEY`
- database connection string -> `SUPABASE_DATABASE_URL`

Postgres and object storage no longer come from Supabase. Take the database connection strings from Neon (`neon connection-string <branch>`, once pooled and once without `--pooled`, into `DATABASE_URL` and `DIRECT_URL`) and the four `R2_*` values from Cloudflare R2.

Create the bucket that stores Solar API GeoTIFFs. The backend expects the bucket name `geotiffs`.

Expand Down Expand Up @@ -215,10 +216,15 @@ Set the root `.env` from the table below. `VITE_*` variables are baked into the
| `CHAT_MODEL` | Gemini model name | Chat boots with the wrong or default model |
| `GOOGLE_OAUTH_CLIENT_ID` | GCP OAuth client | Google sign-in fails |
| `GOOGLE_OAUTH_SECRET` | Same OAuth client | Google sign-in fails |
| `SUPABASE_URL` | Supabase Settings -> API | Auth, storage, and backend client creation fail |
| `DATABASE_URL` | Neon pooled connection string (host ends in `-pooler`) | Runtime database access fails |
| `DIRECT_URL` | Neon unpooled connection string | Prisma Migrate cannot run |
| `R2_ACCOUNT_ID` | Cloudflare account ID; forms the S3 endpoint hostname | Storage client cannot resolve the R2 endpoint |
| `R2_ACCESS_KEY_ID` | R2 API token, Object Read & Write on the bucket | GeoTIFF upload and download fail |
| `R2_SECRET_ACCESS_KEY` | Secret half of the same R2 API token | GeoTIFF upload and download fail |
| `R2_BUCKET` | R2 bucket name holding the cached GeoTIFFs and imagery | Storage paths resolve to a nonexistent bucket |
| `SUPABASE_URL` | Supabase Settings -> API | Auth and backend client creation fail |
| `SUPABASE_ANON_KEY` | Supabase Settings -> API | Frontend Supabase client cannot sign in |
| `SUPABASE_SERVICE_ROLE_KEY` | Supabase Settings -> API | Backend cannot read/write privileged data |
| `SUPABASE_DATABASE_URL` | Supabase Settings -> Database | Prisma migrations and runtime DB access fail |
| `SUPABASE_SERVICE_ROLE_KEY` | Supabase Settings -> API | Backend cannot verify bearer tokens |
| `VITE_SUPABASE_URL` | `SUPABASE_URL` via dotenv-expand | Frontend Supabase client cannot boot |
| `VITE_SUPABASE_ANON_KEY` | `SUPABASE_ANON_KEY` via dotenv-expand | Frontend auth fails |
| `SITE_URL` | Final public origin, local or production | Supabase redirect links point to the wrong place |
Expand Down Expand Up @@ -301,10 +307,15 @@ heroku config:set \
GOOGLE_CLOUD_LOCATION="global" \
GEMINI_API_KEY="..." \
CHAT_MODEL="gemini-3.1-flash-lite-preview" \
DATABASE_URL="postgresql://...-pooler...neon.tech/neondb?sslmode=require" \
DIRECT_URL="postgresql://...neon.tech/neondb?sslmode=require" \
R2_ACCOUNT_ID="..." \
R2_ACCESS_KEY_ID="..." \
R2_SECRET_ACCESS_KEY="..." \
R2_BUCKET="solarsim" \
SUPABASE_URL="https://<supabase-ref>.supabase.co" \
SUPABASE_ANON_KEY="..." \
SUPABASE_SERVICE_ROLE_KEY="..." \
SUPABASE_DATABASE_URL="postgresql://..." \
VITE_SUPABASE_URL="https://<supabase-ref>.supabase.co" \
VITE_SUPABASE_ANON_KEY="..." \
SITE_URL="https://solarsim.tech" \
Expand Down Expand Up @@ -506,10 +517,15 @@ Supabase project deletion is still dashboard-only. After deleting the hosted pro
| `CHAT_MODEL` | Gemini model name | Chat model selector |
| `GOOGLE_OAUTH_CLIENT_ID` | GCP OAuth client | Google sign-in |
| `GOOGLE_OAUTH_SECRET` | GCP OAuth client | Google sign-in |
| `DATABASE_URL` | Neon pooled connection string | Prisma runtime queries |
| `DIRECT_URL` | Neon unpooled connection string | Prisma Migrate |
| `R2_ACCOUNT_ID` | Cloudflare account ID | R2 S3 endpoint hostname |
| `R2_ACCESS_KEY_ID` | R2 API token | GeoTIFF storage |
| `R2_SECRET_ACCESS_KEY` | R2 API token | GeoTIFF storage |
| `R2_BUCKET` | R2 bucket name | GeoTIFF storage |
| `SUPABASE_URL` | Supabase settings | Backend + auth |
| `SUPABASE_ANON_KEY` | Supabase settings | Frontend auth |
| `SUPABASE_SERVICE_ROLE_KEY` | Supabase settings | Backend privileged access |
| `SUPABASE_DATABASE_URL` | Supabase DB connection string | Prisma |
| `SUPABASE_SERVICE_ROLE_KEY` | Supabase settings | Backend token verification |
| `VITE_SUPABASE_URL` | Derived from `SUPABASE_URL` | Frontend auth |
| `VITE_SUPABASE_ANON_KEY` | Derived from `SUPABASE_ANON_KEY` | Frontend auth |
| `SITE_URL` | Final public origin | Supabase auth redirects |
Expand Down
2 changes: 2 additions & 0 deletions backend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
"test": "vitest run"
},
"dependencies": {
"@aws-sdk/client-s3": "^3.1098.0",
"@aws-sdk/s3-request-presigner": "^3.1098.0",
"@google/genai": "1.51.0",
"@prisma/client": "^6.6.0",
"@shared/types": "workspace:*",
Expand Down
7 changes: 6 additions & 1 deletion backend/src/config/env.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,15 @@ const envSchema = z
(val) => (val === '' ? undefined : val),
z.enum(['development', 'production', 'test']).default('development')
),
SUPABASE_DATABASE_URL: z.string().min(1),
DATABASE_URL: z.string().min(1),
DIRECT_URL: z.string().min(1),
GOOGLE_API_KEY: z.string().min(1),
SUPABASE_URL: z.string().url(),
SUPABASE_SERVICE_ROLE_KEY: z.string().min(1),
R2_ACCOUNT_ID: z.string().min(1),
R2_ACCESS_KEY_ID: z.string().min(1),
R2_SECRET_ACCESS_KEY: z.string().min(1),
R2_BUCKET: z.string().min(1),
FRONTEND_URL: z.string().url().optional().default('http://localhost:5173'),
PDF_TOKEN_SECRET: z.string().min(32),
GEMINI_API_KEY: z.preprocess((val) => (val === '' ? undefined : val), z.string().min(1).optional()),
Expand Down
12 changes: 12 additions & 0 deletions backend/src/config/r2.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import { S3Client } from '@aws-sdk/client-s3'
import { env } from './env.js'

/** Configured Cloudflare R2 client used by the backend for private asset storage. */
export const r2 = new S3Client({
region: 'auto',
endpoint: `https://${env.R2_ACCOUNT_ID}.r2.cloudflarestorage.com`,
credentials: {
accessKeyId: env.R2_ACCESS_KEY_ID,
secretAccessKey: env.R2_SECRET_ACCESS_KEY
}
})
97 changes: 97 additions & 0 deletions backend/src/services/__tests__/storageService.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
import { beforeEach, describe, expect, it, vi } from 'vitest'

const s3SendMock = vi.hoisted(() => vi.fn())
const createPresignedUrlMock = vi.hoisted(() => vi.fn())

vi.mock('../../config/env.js', () => ({
env: { R2_BUCKET: 'solar-assets' }
}))

vi.mock('../../config/r2.js', () => ({
r2: { send: (...args: unknown[]) => s3SendMock(...args) }
}))

vi.mock('@aws-sdk/s3-request-presigner', () => ({
getSignedUrl: (...args: unknown[]) => createPresignedUrlMock(...args)
}))

import { downloadFromStorage, getSignedUrl, uploadToStorage } from '../storageService.js'

const STORAGE_PATH = 'locations/10f3f3f6-112f-47bd-b121-8bfa7b41e1f1/monthly_flux.tif'
const R2_BUCKET = 'solar-assets'

describe('storageService', () => {
beforeEach(() => {
s3SendMock.mockReset()
createPresignedUrlMock.mockReset()
})

it('uploads the exact storage path, configured bucket, and content type', async () => {
const buffer = Buffer.from('solar-data')
s3SendMock.mockResolvedValue({})

await uploadToStorage(STORAGE_PATH, buffer, 'image/tiff')

expect(s3SendMock).toHaveBeenCalledOnce()
expect(s3SendMock.mock.calls[0]?.[0]).toMatchObject({
input: { Bucket: R2_BUCKET, Key: STORAGE_PATH, Body: buffer, ContentType: 'image/tiff' }
})
})

it('downloads object bytes as an ArrayBuffer using the exact storage path', async () => {
const bytes = new Uint8Array([1, 2, 3])
s3SendMock.mockResolvedValue({ Body: { transformToByteArray: vi.fn().mockResolvedValue(bytes) } })

const result = await downloadFromStorage(STORAGE_PATH)

expect(s3SendMock).toHaveBeenCalledOnce()
expect(s3SendMock.mock.calls[0]?.[0]).toMatchObject({ input: { Bucket: R2_BUCKET, Key: STORAGE_PATH } })
expect(result).toBeInstanceOf(ArrayBuffer)
expect([...new Uint8Array(result)]).toEqual([1, 2, 3])
})

it('creates a private signed URL with the exact storage path and requested lifetime', async () => {
createPresignedUrlMock.mockResolvedValue('https://signed.example/object')

const result = await getSignedUrl(STORAGE_PATH, 900)

expect(result).toBe('https://signed.example/object')
expect(createPresignedUrlMock).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({ input: { Bucket: R2_BUCKET, Key: STORAGE_PATH } }),
{ expiresIn: 900 }
)
})

it('uses the existing 3600 second lifetime by default', async () => {
createPresignedUrlMock.mockResolvedValue('https://signed.example/object')

await getSignedUrl(STORAGE_PATH)

expect(createPresignedUrlMock).toHaveBeenCalledWith(expect.anything(), expect.anything(), { expiresIn: 3600 })
})

it('wraps upload failures with the existing message shape', async () => {
s3SendMock.mockRejectedValue(new Error('R2 unavailable'))

await expect(uploadToStorage(STORAGE_PATH, Buffer.from('solar-data'), 'image/tiff')).rejects.toThrow(
`Storage upload failed for ${STORAGE_PATH}: R2 unavailable`
)
})

it('wraps download failures with the existing message shape', async () => {
s3SendMock.mockRejectedValue(new Error('R2 unavailable'))

await expect(downloadFromStorage(STORAGE_PATH)).rejects.toThrow(
`Storage download failed for ${STORAGE_PATH}: R2 unavailable`
)
})

it('wraps signed URL failures with the existing message shape', async () => {
createPresignedUrlMock.mockRejectedValue(new Error('R2 unavailable'))

await expect(getSignedUrl(STORAGE_PATH)).rejects.toThrow(
`Failed to create signed URL for ${STORAGE_PATH}: R2 unavailable`
)
})
})
46 changes: 32 additions & 14 deletions backend/src/services/storageService.ts
Original file line number Diff line number Diff line change
@@ -1,24 +1,31 @@
/**
* Supabase Storage wrapper for generated Solar API assets.
* Cloudflare R2 storage wrapper for generated Solar API assets.
*
* Centralises uploads, downloads, and signed URL creation for the shared
* GeoTIFF/PNG bucket used by the location pipeline.
*/

import { supabase } from '../config/supabase.js'

const BUCKET = 'geotiffs'
import { GetObjectCommand, PutObjectCommand } from '@aws-sdk/client-s3'
import { getSignedUrl as createPresignedUrl } from '@aws-sdk/s3-request-presigner'
import { env } from '../config/env.js'
import { r2 } from '../config/r2.js'

/**
* Uploads a blob to the shared storage bucket, replacing any existing object.
*
* @param storagePath - Bucket-relative object path
* @param buffer - File contents to store
* @param contentType - MIME type sent to Supabase Storage
* @param contentType - MIME type stored with the R2 object
*/
export async function uploadToStorage(storagePath: string, buffer: Buffer, contentType: string): Promise<void> {
const { error } = await supabase.storage.from(BUCKET).upload(storagePath, buffer, { contentType, upsert: true })
if (error) throw new Error(`Storage upload failed for ${storagePath}: ${error.message}`)
try {
await r2.send(
new PutObjectCommand({ Bucket: env.R2_BUCKET, Key: storagePath, Body: buffer, ContentType: contentType })
)
} catch (error) {
const message = error instanceof Error ? error.message : String(error)
throw new Error(`Storage upload failed for ${storagePath}: ${message}`)
}
}

/**
Expand All @@ -28,20 +35,31 @@ export async function uploadToStorage(storagePath: string, buffer: Buffer, conte
* @returns Raw object bytes as an ArrayBuffer
*/
export async function downloadFromStorage(storagePath: string): Promise<ArrayBuffer> {
const { data, error } = await supabase.storage.from(BUCKET).download(storagePath)
if (error) throw new Error(`Storage download failed for ${storagePath}: ${error.message}`)
return data.arrayBuffer()
try {
const response = await r2.send(new GetObjectCommand({ Bucket: env.R2_BUCKET, Key: storagePath }))
if (!response.Body) throw new Error('R2 response body is missing')

return new Uint8Array(await response.Body.transformToByteArray()).buffer
} catch (error) {
const message = error instanceof Error ? error.message : String(error)
throw new Error(`Storage download failed for ${storagePath}: ${message}`)
}
}

/**
* Creates a signed read URL for a storage object.
*
* @param storagePath - Bucket-relative object path
* @param expiresIn - URL lifetime in seconds
* @returns Temporary public URL for reading the object
* @returns Temporary signed URL for reading the object
*/
export async function getSignedUrl(storagePath: string, expiresIn = 3600): Promise<string> {
const { data, error } = await supabase.storage.from(BUCKET).createSignedUrl(storagePath, expiresIn)
if (error) throw new Error(`Failed to create signed URL for ${storagePath}: ${error.message}`)
return data.signedUrl
try {
return await createPresignedUrl(r2, new GetObjectCommand({ Bucket: env.R2_BUCKET, Key: storagePath }), {
expiresIn
})
} catch (error) {
const message = error instanceof Error ? error.message : String(error)
throw new Error(`Failed to create signed URL for ${storagePath}: ${message}`)
}
}
Loading
Loading