Skip to content

Believe self-host: Firestore→Postgres, Pinecone→pgvector, GCS→R2, Firebase Auth→JWT + fanout Company Brain - #1

Open
Jbelieve wants to merge 22 commits into
mainfrom
believe/self-host
Open

Believe self-host: Firestore→Postgres, Pinecone→pgvector, GCS→R2, Firebase Auth→JWT + fanout Company Brain#1
Jbelieve wants to merge 22 commits into
mainfrom
believe/self-host

Conversation

@Jbelieve

@Jbelieve Jbelieve commented Jun 13, 2026

Copy link
Copy Markdown
Owner

Qué hace

Self-host completo del backend Omi en infra Believe (ADR-004). Reemplaza las 4 dependencias cloud y agrega el fanout al Company Brain.

Upstream Believe
Firestore (44 módulos database/) Postgres 16 + asyncpg, 54 tablas omi_*, column-aware, encryption decorators intactos
Pinecone pgvector 768d HNSW + halfvec 3072d, embedder Ollama nomic-embed-text
GCS Cloudflare R2 single-bucket (boto3, partes multipart uniformes)
Firebase Auth JWT propio HS256 + Argon2id (t=3/m=64MiB/p=4), refresh familia + reuse-detection
— utils/believe_brain.py: fanout firmado HMAC → Company Brain (getmaas), idempotente

Vivo

https://omi.believe-global.com/health 200, cert Let's Encrypt, Contabo. 54 tablas aplicadas, auth E2E verde (register→login→request).

Tests

152 casos tests/pg/ + 5 fanout + 7 mcp-scopes verdes contra pgvector/pg16.

Decisiones

Ver specs/omi-self-host/DECISION-RECORD.md (11 decisiones congeladas) y DRIFT.md (deudas: Typesense/Hume placeholders, OPENAI hard boot dep, ENCRYPTION rotation Wave 9).

🤖 Generated with Claude Code

Jbelieve and others added 22 commits June 12, 2026 14:35
@Jbelieve @claude
chore: pin upstream SHA + DRIFT.md (Believe self-host fork)
ff7cf44 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Jbelieve @claude
feat(group-a): Postgres schema (54 tablas omi_*) + GCS→R2 swap
29ef031 
1-schema: migrations_pg/{001_init,002_auth,golden_eval}.sql — 54 tablas, RLS,
pgvector 768d + halfvec 3072d, scopes MCP enforced, idempotente, 11 índices
post-audit (FK + hot paths). Gate: aplica+re-aplica+eval verde en pgvector/pg16.

3-storage: utils/other/r2_client.py adapter boto3 (partes multipart uniformes
para R2, pool 64, mimetypes fallback) + storage.py persiste keys y firma en
lectura (logos/chat files), call sites intactos. Gate: 24 tests verdes vs MinIO.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Jbelieve @claude
feat(group-b1): repository foundations + auth JWT/Argon2id + pgvector…
4b27c2c 
… swap

- database/_repos/{runtime,pg}.py: pool asyncpg + PgRepository column-aware + run_sync
- T1 portados: conversations, memories, users, chat, notifications (encryption intacta)
- _client.py: shim RuntimeError (import-safe), Firestore eliminado del runtime T1
- 5-auth: JWT HS256 + Argon2id t=3/m=64MiB/p=4, refresh familia+reuse-detection,
  boot assertion DR#9, ADMIN_KEY audit-log, FIREBASE_GRACE lazy off-by-default
- 4-vector: vector_db.py pgvector (30 funciones API preservadas), embedder Ollama
  nomic-embed-text 768d, 004_vector.sql, BYOK embeddings removido
- requirements: +asyncpg +pgvector +argon2-cffi, -pinecone -langchain-pinecone
- Gates: 73 tests pg verdes vs pgvector/pg16; Ollama real Contabo validado (768d)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Jbelieve @claude
feat(group-b2): portar los 34 módulos database/ restantes a Postgres
900aafd 
- 24 módulos portados por agentes paralelos + 10 finales (goals, phone_calls,
  calendar_meetings, x_posts, screen_activity, advice, focus_sessions,
  phone_call_config, app_review_config, trends) — todos column-aware, API pública
  intacta, encryption decorators preservados.
- routers/agent_tools.py, fair_use_admin.py: db crudo → patrón repo.
- fix: uq_omi_action_items_idem scopeado a items activos (completed=false +
  not deleted) — el unique blanket rechazaba re-creates legítimos tras
  completar/borrar (parity con la idempotencia app-level de Firestore).
- main.py: endpoints /health (liveness) y /health/ready (pool ping) para el
  gate de deploy; boot assertion DR#9 + firebase init condicional ya presentes.
- mcp_api_key expone scopes; dev_api_key lookup por índice hashed_key.

Gate: suite tests/pg completa 138 passed, 1 skipped (Ollama real), 0 failed
contra pgvector/pg16.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Jbelieve @claude
feat(fanout): Omi conversations → Believe Company Brain (HMAC firmado)
8e67e22 
utils/believe_brain.py: push_conversation_to_brain firma X-Omi-Signature
(t=ts,v1=hmac_sha256(OMI_WEBHOOK_SHARED_SECRET, ts.body)) y empuja cada
conversación como episode source=omi a mc-company-brain-ingest (getmaas),
idempotente por x-idempotency-key omi:conversation:<id> + unique (source,
external_id) server-side. Memorias extraídas viajan en metadata. Fail-soft:
nunca tumba el pipeline. Cableado fire-and-forget en process_conversation.

Mismo contrato que Be Chat/Gmail (verificado en prod 2026-06-12). Gate:
5 tests verdes incl. verificación HMAC end-to-end + tamper detection + fail-soft.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Jbelieve @claude
build(deploy): Dockerfile.believe con base pública python:3.11-slim-b…
84ca20a 
…ookworm

La base upstream gcr.io/based-hardware-dev/python:3.11-slim-forky es privada.
Mantiene liblc3 desde fuente + jemalloc preload verbatim. test.sh registra
tests/pg (skip si DATABASE_URL no seteado, upstream CI intacto).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Jbelieve @claude
build(deploy): compose Believe para Contabo (REDIS_DB_* corregido, im…
5b73cca 
…agen local)

deploy/docker-compose.yml: omi-backend + omi-postgres (pgvector/pg16) +
omi-redis, container_name únicos, Traefik letsencrypt, network coolify externa,
OLLAMA_URL=believe-ollama, env REDIS_DB_* (no OMI_REDIS_*), BRAIN fanout.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Jbelieve @claude
fix(boot): translate client lazy + compose env_file
80dfdf3 
- utils/translation.py: TranslationServiceClient construido lazy (era eager a
  nivel módulo y exigía credenciales GCP en el import → rompía el boot sin
  SERVICE_ACCOUNT_JSON). Translation es opcional y feature-gated por user.language.
- deploy/docker-compose.yml: env_file .env para pasar TODAS las vars al container
  (Typesense/Hume placeholders de integraciones no desplegadas en Believe).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Jbelieve
docs: DRIFT.md estado deploy vivo 2026-06-13
356f04c
@Jbelieve @claude
feat(desktop): login email/password JWT propio en la app macOS
0f08e4e 
AuthService: signInWithBelieveEmail/registerWithBelieveEmail contra
/v1/auth/login|register de omi.believe-global.com, guarda nuestro JWT via
saveTokens (getIdToken lo devuelve sin tocar Firebase), refresh via /v1/auth/refresh
con flag believeAuthMode. SignInView: UI email/password. Backend default →
omi.believe-global.com. Login verificado E2E en la app corriendo (GET /v1/conversations 200).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Jbelieve @claude
fix(fanout): cliente httpx propio en push_conversation_to_brain
d4dd50c 
El fanout corre via asyncio.run en un thread de postprocess_executor (su propio
event loop). El cliente httpx compartido (get_webhook_client) está atado al loop
principal → 'TCPTransport closed / handler is closed' al reusarlo. Fix: crear un
httpx.AsyncClient fresco dentro del loop del fanout. Verificado E2E: conversación
real de Jorge llega a company_brain.episodes (título+resumen+memorias).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Jbelieve @claude
feat(windows): auth JWT propio Believe reemplaza Firebase
7107b10 
Reescribe lib/firebase.ts como modulo de auth contra el backend self-host
(/v1/auth/login|register|refresh|logout). Mantiene la superficie Firebase
(auth.currentUser, onAuthStateChanged, signOutUser, tipo User) para no tocar
los ~12 consumidores. Tokens en localStorage, refresh deduped con buffer 60s,
rotacion de refresh token, authStateReady() y photoURL para paridad de shape.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Jbelieve @claude
feat(windows): pantalla login email/password + registro
6ada029 
Reemplaza el boton 'Sign in with Google' por formulario email/password con
toggle a registro, validacion minima y manejo de error del backend.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Jbelieve @claude
refactor(windows): displayName local en vez de Firebase updateProfile
ef9109f 
setDisplayName usa updateLocalDisplayName (no hay endpoint server-side de
nombre). Quita el ultimo import de firebase/auth.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Jbelieve @claude
refactor(windows): tipo User desde lib/firebase en useAuth
c35435d 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Jbelieve @claude
refactor(windows): tipo User desde lib/firebase en Sidebar
a539342 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Jbelieve @claude
refactor(windows): tipo User desde lib/firebase en Home
24ae3f1 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Jbelieve @claude
build(windows): .env.example apunta al backend self-host Believe
1b6ebbb 
VITE_OMI_API_BASE -> https://omi.believe-global.com (sirve data + auth JWT).
Quita las vars VITE_FIREBASE_*; documenta deuda del desktop Rust backend.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Jbelieve @claude
ci(windows): workflow GitHub Actions que buildea el .exe
72b74e0 
windows-latest + node 22 + pnpm 10. Compila better-sqlite3 nativo y el OCR
helper PowerShell, corre electron-builder --win y sube el instalador como
artifact. Manual (workflow_dispatch) o tag v*-windows. Sin firmar por ahora
(SmartScreen one-time prompt); CSC_LINK/CSC_KEY_PASSWORD para firmar despues.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Jbelieve @claude
feat(macos): script firma Developer ID + notarizacion + DMG
92d6d05 
Re-firma el bundle dev con Developer ID (bottom-up: dylibs, XPC Sparkle,
Updater.app, frameworks, app), hardened runtime, entitlements sin
get-task-allow (Apple lo rechaza). Notariza con notarytool, staplea y arma
el DMG para distribucion al equipo. Autodetecta el cert del llavero.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Jbelieve @claude
docs(macos): guia distribucion notarizada (cert Developer ID + notary…
0c5d62a 
…tool)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Jbelieve @claude
feat: conectar Langfuse al get_llm() central via LangChain CallbackHa…
38c76fa 
…ndler (#2)

* feat(brain): fanout de Omi embebe episodios (nomic 768d) para búsqueda semántica

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(brain): forzar español en title/overview estructurado de conversaciones

Los resúmenes que fluyen al Company Brain de Believe (summary=overview,
decision_summary=title en believe_brain.py) deben salir SIEMPRE en español,
sin importar el idioma detectado de la conversación ni la preferencia del
usuario. Algunos salían en inglés (ej. "Radio Segment Promotes Public Notices").

Se fuerza response_language='Spanish' en get_transcript_structure y
get_reprocess_transcript_structure (conversation_processing.py) y en
get_message_structure (external_integrations.py). summarize_experience_text
no tenía instrucción de idioma; se le agrega respuesta en español.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: conectar Langfuse al get_llm() central via CallbackHandler de LangChain

Believe está instrumentando Langfuse (langfuse.believe-global.com) en todo
el stack que consume LLM. omi-believe ya centraliza todos los clientes
LangChain en get_llm()/clients.py con un usage_tracker propio, así que el
menor esfuerzo es sumar el CallbackHandler oficial de Langfuse a la misma
lista de callbacks que ya se pasa a cada cliente (OpenAI, Gemini,
OpenRouter, BYOK, llm_mini) en vez de instrumentar cada call site.

No-op si LANGFUSE_HOST/LANGFUSE_PUBLIC_KEY/LANGFUSE_SECRET_KEY no están en
el entorno, y cualquier fallo al inicializar el handler se loguea pero no
rompe el arranque.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Jbelieve