Skip to content

Repository files navigation

UGC Creator Intelligence Platform

UGC AI is a creator intelligence platform for semantic creator search, social source ingestion, evidence-backed fact extraction, campaign CRM, chat-first operations, and production-oriented task automation.

The important architectural idea is simple: the LLM is an entry point and an assistant for workflows, not the source of truth. Durable truth lives in Postgres, Qdrant, OpenSPG/KAG, Redis/Valkey, object storage, and audited service methods.

Table Of Contents

Product Scope

The platform is built for teams that source, evaluate, and operate with UGC creators across Telegram, VK, YouTube, Instagram, and similar channels.

Main product jobs:

  • Search creators semantically by natural language briefs.
  • Discover and parse creators/channels from social platforms.
  • Extract evidence-backed facts: topics, languages, brand mentions, promoted brands, countries, audience hints, and commercial signals.
  • Keep facts reviewable and graph-syncable instead of letting the model write directly into source-of-truth stores.
  • Build creator cards with evidence explaining why a result matched.
  • Manage campaigns, shortlists, statuses, notes, and CRM-style workflow state.
  • Persist chat sessions and messages so the UI behaves like a lightweight ChatGPT-style workspace.
  • Support subscriptions, membership limits, invoices, and global/RF payment providers.
  • Expose operations through a deterministic API and task engine.

Current Capabilities

Implemented backend capabilities include:

  • FastAPI application with centralized routing, CORS, metrics, and domain-error handling.
  • Cookie/session-based auth with organization membership, RBAC scopes, password reset, email verification, superuser bootstrap, and optional legacy header auth for controlled dev paths.
  • Account summary and membership management.
  • Chat sessions/messages backed by persistent identity.
  • Billing plans, usage accounting, checkout sessions, webhook handling, invoice history, Stripe and CloudPayments adapters.
  • Creator search pipeline with query planning, hard filters, semantic recall, graph recall, candidate merge, feature scoring, quality gating, reranking, and evidence-grounded responses.
  • Telegram ingestion with Telethon session support, QR/login helper endpoints, adaptive parsing settings, URL extraction, media metadata extraction, and similar-channel crawl tasks.
  • Public connector scaffolding for VK, YouTube, and Instagram discovery flows.
  • URL document enrichment that fetches exactly one source URL per URL document, extracts text/metadata/contact hints, stores artifacts, and schedules creator refresh tasks. It does not recursively crawl links inside fetched pages.
  • Media enrichment pipeline for photos/video/audio metadata, optional transcription/image-caption providers, and artifact-retention controls.
  • Ad review via local/openAI-compatible LLM provider with safe unavailable handling.
  • Fact extraction via local/openAI-compatible LLM provider. Dev/test defaults to qwen2.5:14b and fact-extractor-v3 few-shot Telegram prompts.
  • Fact validation, sensitive-fact rejection, duplicate behavior, audit metadata redaction, and OpenSPG/graph sync.
  • Redis/Valkey-compatible cache for auth/session/planner/account/billing usage paths.
  • Worker pools for crawl/parsing, graph sync, URL enrichment, media enrichment, and ad review.
  • DigitalOcean dev/test compose bundle with optional Traefik TLS, host Ollama mode, internal OpenSPG, Qdrant, Valkey, observability, and systemd autostart.

Repository Layout

High-level paths:

.
├── AGENTS.md                         # Non-negotiable engineering rules
├── TASKS_TZ.md                       # Original developer-oriented task spec
├── README.md                         # This project knowledge base
├── alembic/                          # Database migrations
├── apps/web/                         # Next.js chat/dashboard frontend
├── deployment/                       # Compose, DigitalOcean, observability
├── scripts/                          # Deploy, sync, stop, autostart helpers
├── src/ugcai/                        # Backend package
├── tests/                            # API, service, infra, worker tests
├── pyproject.toml                    # Python package/dependency metadata
└── ugc_creator_platform_summary.md   # Earlier architecture summary

Backend package layout:

src/ugcai/
├── api/              # FastAPI routes, auth dependencies, route wiring
├── connectors/       # Source adapters: Telegram, VK, YouTube, Instagram
├── core/             # Logging, metrics, shared errors/security helpers
├── domain/           # Domain models, schemas, protocols, business contracts
├── infra/            # DB repos, graph clients, Qdrant, cache, billing, LLM IO
├── runtime/          # CLI utilities, smoke tests, bootstrap/reindex/sync tools
├── services/         # Business logic and workflow orchestration
├── worker/           # Long-running worker entry point
├── bootstrap.py      # Dependency/container assembly
├── main.py           # FastAPI app factory
└── settings.py       # `UGCAI_` settings

Architecture

The codebase follows a layered architecture:

flowchart TD
  UI["Next.js Web UI"] --> API["FastAPI API Layer"]
  External["External AI/MCP Clients"] --> API
  API --> Services["Service Layer"]
  Services --> Tasks["Task Service"]
  Services --> Repos["Repositories"]
  Services --> Retrieval["Retrieval Service"]
  Services --> GraphSync["Graph Sync Service"]
  Services --> LLM["LLM Providers"]
  Workers["Worker Pools"] --> Tasks
  Tasks --> Services
  Repos --> Postgres["Postgres"]
  Retrieval --> Qdrant["Qdrant"]
  GraphSync --> OpenSPG["OpenSPG/KAG"]
  Services --> Redis["Redis/Valkey Cache"]
  Services --> S3["S3/Spaces/MinIO"]
Loading

Layer responsibilities:

  • API layer validates input, authenticates, applies org scope, maps HTTP to service calls, and returns typed schemas.
  • Service layer owns business logic, workflow orchestration, permissions, usage accounting, task creation, ranking features, and audit calls.
  • Infrastructure layer owns Postgres repositories, Qdrant clients, graph clients, billing providers, cache clients, object storage, and LLM transport.
  • Worker layer claims tasks, executes service workflows, emits task events, and stays idempotent.

Rules that matter most:

  • Do not put business logic in FastAPI route handlers.
  • Do not let LLM code write directly to Postgres, Qdrant, OpenSPG, Redis, or ClickHouse.
  • Do not create graph facts without evidence.
  • Do not use vector search as the final truth.
  • Do not perform dangerous actions without approval and audit.
  • Do not leak data across organizations.

Runtime Components

Main runtime services:

  • app: FastAPI backend.
  • web: Next.js frontend.
  • worker: crawl/parsing task pool.
  • worker-graph: fact extraction, retrieval sync, graph sync pool.
  • worker-url: URL enrichment and URL recrawl pool.
  • worker-media: media enrichment pool.
  • worker-review: ad review pool.
  • bootstrap: one-shot startup task for schema/state/bootstrap work.
  • valkey: Redis-compatible cache.
  • qdrant: vector search index.
  • neo4j: graph store used by OpenSPG bundle.
  • openspg-server: OpenSPG/KAG graph service.
  • ollama: local LLM runtime for extraction/review/vectorizer paths.
  • ollama-planner: optional dedicated planner runtime.
  • traefik: optional HTTPS reverse proxy.
  • prometheus, loki, promtail, alertmanager, grafana: observability.

Data Stores

Postgres is the operational source of truth:

  • organizations, users, memberships, roles, sessions;
  • creators, channels, posts, URL documents, media metadata;
  • campaigns, creator statuses, notes;
  • chat sessions/messages;
  • tasks and task events;
  • fact claims and validation state;
  • billing customers/subscriptions/invoices/usage;
  • audit logs.

Qdrant is a retrieval index:

  • creator/channel/post/chunk embeddings;
  • semantic recall candidates;
  • similarity and hybrid search inputs.

OpenSPG/KAG is the evidence-backed graph:

  • creator-channel ownership;
  • author/post relations;
  • brand/category/topic/language/country facts;
  • FactClaim relations with evidence references;
  • graph recall and multi-hop support.

Redis/Valkey is a short-lived cache/coordination layer:

  • session cache;
  • planner cache;
  • account summary cache;
  • billing usage cache;
  • rate-limit counters and lightweight coordination.

S3/Spaces/MinIO stores large artifacts:

  • raw social payloads;
  • fetched URL HTML/text payloads;
  • media artifacts;
  • Telegram session backup/restore artifacts when enabled.

ClickHouse is target architecture for analytics and trend workloads. The core code is prepared around the separation of transactional and analytical concerns, but ClickHouse should only be introduced when dashboard/trend volume requires it.

Core Workflows

Creator Search

Search follows a staged pipeline:

  1. Parse natural-language query into a structured search plan.
  2. Apply hard filters in Postgres.
  3. Recall semantic candidates from Qdrant.
  4. Recall fact/graph candidates from OpenSPG/KAG.
  5. Merge and deduplicate candidates.
  6. Compute ranking features: semantic relevance, graph match, evidence quality, audience fit, engagement quality, freshness, ad relevance, risk signals, and duplicate penalties.
  7. Rerank candidates.
  8. Return creator cards with evidence.
  9. Generate explanations only from retrieved evidence.

Important behavior:

  • The LLM is not the ranking engine.
  • Creators should not be returned without a clear reason/evidence path.
  • Query planning can use LLM, but a failed LLM path must fail explicitly rather than silently falling back to heuristic generation.

Telegram Ingestion

Telegram ingestion is task-based:

  1. A sourcing/channel crawl/ingest task is created.
  2. The crawl/parsing worker claims the task.
  3. The Telegram connector resolves the channel, fetches recent posts, extracts URLs/media references, and stores normalized records.
  4. Canonical catalog repositories upsert creators, channels, posts, URL refs, and media refs idempotently.
  5. Follow-up tasks are scheduled for URL enrichment, ad review, fact extraction, retrieval sync, graph sync, media enrichment, and optional deep parsing.

Current parsing policy:

  • Background parse defaults to a small recent-post window.
  • Deep parse can fetch more posts only for channels that pass configured follower thresholds.
  • Similar-channel discovery can run through channel_crawl_task and supports continuation tasks.
  • URL document enrichment fetches one URL document only; it does not recurse into links found inside the fetched page.

Fact Extraction And Graph Sync

Fact extraction steps:

  1. Load creator/channel/post bundle from Postgres.
  2. Send a strict JSON extraction request to the configured provider.
  3. Validate schema, predicate, confidence, evidence post/reference, and evidence span.
  4. Reject malformed, sensitive, unsupported, or evidence-free claims.
  5. Mark high-confidence safe claims as validated; route lower-confidence claims to review as needed.
  6. Persist claims in Postgres.
  7. Schedule or run graph sync into OpenSPG/KAG.

Fact rules:

  • Every graph fact must have evidence.
  • Extractors must not invent values outside source text.
  • Restricted/sensitive personal facts must not become normal ranking filters.
  • Duplicate facts are merged/handled without inflating confidence blindly.

URL Documents

URL document workflow:

  1. Posts contribute normalized URLs.
  2. UrlDocumentRecord deduplicates by normalized URL per organization.
  3. url_enrichment_task fetches the single source URL.
  4. Raw response and extracted text are stored in object storage.
  5. Metadata such as title, page type, language, contact points, commercial hints, and refresh schedule is written back to Postgres.
  6. Referenced creators get fact/retrieval refresh tasks.

Non-goal:

  • URL enrichment is not a website crawler. It does not follow <a href> links inside a document.

Campaign CRM

Campaign workflow supports:

  • campaign list/create/read/update;
  • adding creators to campaigns;
  • membership status and notes;
  • audit logging for write actions;
  • future expansion into outreach drafts, approvals, contracts, and ORD-related processes.

Chat

Chat is session-based:

  • users can create multiple chat sessions;
  • sessions contain multiple messages;
  • chat calls are tied to authenticated identity and organization membership;
  • billing usage is counted against chat/search/task limits;
  • search and workflow calls go through backend services, not direct frontend shortcuts.

API Surface

Main route groups:

Area Routes
Auth POST /signup, POST /login, GET /me, POST /logout, email verification, password reset
Account GET /summary, membership switching, member create/update
Billing GET /billing/history, POST /billing/checkout, provider webhooks
Chat GET /chat/sessions, POST /chat/sessions, GET /chat/sessions/{id}, POST /chat/messages
Search POST /search
Creators GET /creators/{creator_id}
Campaigns GET/POST /campaigns, campaign detail/update, campaign creator membership
Tasks POST /tasks, GET /tasks, GET /tasks/{id}, events, manual run
Sources POST /sources/{source}/channels/ingest, POST /sources/{source}/channels/crawl
Telegram Telegram-specific ingest/crawl and QR login helper routes
Reviews pending review queue, fact review, ad-post review
Admin overview, platform overview, config, data sources, data browser
Ops /health/live, /health/ready, /metrics

In Traefik /api mode, public backend paths are exposed under /api, for example /api/health/ready. Internally FastAPI still serves /health/ready.

Workers And Tasks

Long-running work is represented as tasks. Each task has:

  • task_id;
  • org_id;
  • task_type;
  • status;
  • input/output payloads;
  • plan;
  • error;
  • approval policy;
  • idempotency key;
  • timestamps;
  • task events.

Worker pools:

Pool Service Task types
crawl_parsing worker sourcing, channel expansion, channel crawl, parsing, enrichment, monitoring
graph_sync worker-graph fact extraction, retrieval sync, graph sync
url_enrichment worker-url URL enrichment, URL recrawl
media_enrichment worker-media media enrichment
review_ad worker-review ad review

Useful settings:

  • UGCAI_WORKER_POLL_INTERVAL_SECONDS
  • UGCAI_WORKER_BATCH_SIZE
  • UGCAI_WORKER_CONCURRENCY
  • UGCAI_WORKER_RESERVED_SEARCH_CONCURRENCY
  • UGCAI_WORKER_POOL_NAME
  • UGCAI_WORKER_TASK_TYPES
  • UGCAI_DEVTEST_WORKER_SCALE
  • UGCAI_DEVTEST_WORKER_GRAPH_SCALE
  • UGCAI_DEVTEST_WORKER_URL_SCALE
  • UGCAI_DEVTEST_WORKER_MEDIA_SCALE
  • UGCAI_DEVTEST_WORKER_REVIEW_SCALE

Workers claim pending tasks before execution, which prevents duplicate processing across replicas when backed by Postgres row-level locking.

LLM And Retrieval Stack

Model roles:

  • Embedding model: BGE-M3 or equivalent for retrieval/indexing.
  • Generative LLM: Qwen-class model for query planning, fact extraction, explanations, ad review, and drafting.
  • Reranker: dedicated reranker or disabled/local implementation depending on runtime settings.

Current defaults:

  • Query planner: qwen2.5:1.5b or configured local/openAI-compatible provider.
  • Fact extraction: qwen2.5:14b, prompt fact-extractor-v3.
  • Ad review: qwen2.5:1.5b, prompt ad-review-v1.
  • OpenSPG vectorizer: bge-m3:latest.

Important rule:

  • BGE-M3 is an embedding model, not a generative LLM. Do not use it for generation or reasoning.

Provider settings:

  • UGCAI_QUERY_PLANNER_BACKEND
  • UGCAI_QUERY_PLANNER_MODEL_NAME
  • UGCAI_QUERY_PLANNER_OPENAI_COMPATIBLE_BASE_URL
  • UGCAI_FACT_EXTRACTION_BACKEND
  • UGCAI_FACT_EXTRACTION_MODEL_NAME
  • UGCAI_FACT_EXTRACTION_PROMPT_VERSION
  • UGCAI_LLM_OPENAI_COMPATIBLE_BASE_URL
  • UGCAI_LLM_TIMEOUT_SECONDS
  • UGCAI_RETRIEVAL_EMBEDDING_BACKEND
  • UGCAI_RETRIEVAL_RERANKER_BACKEND

Security Model

Security posture:

  • authenticated identity is the normal access path;
  • users can belong to organizations through memberships;
  • service methods and routes enforce scopes/RBAC;
  • tenant-scoped data includes org_id;
  • write actions are audited;
  • dangerous actions require explicit approval;
  • raw secrets are not logged;
  • audit metadata is redacted, including nested dictionaries/lists;
  • checkout redirect URLs are origin-validated;
  • Stripe webhook signatures enforce timestamp tolerance;
  • proxy headers are only trusted from configured trusted proxy networks.

Important settings:

  • UGCAI_AUTH_COOKIE_NAME
  • UGCAI_AUTH_COOKIE_SECURE
  • UGCAI_AUTH_CSRF_ENFORCE_ORIGIN
  • UGCAI_AUTH_ALLOW_LEGACY_HEADER_AUTH
  • UGCAI_TRUSTED_PROXY_IPS
  • UGCAI_AUTH_BOOTSTRAP_SUPERUSER_EMAIL
  • UGCAI_AUTH_BOOTSTRAP_SUPERUSER_PASSWORD
  • UGCAI_WEB_ALLOWED_ORIGINS

Do not commit .env, real API keys, session files, cloud credentials, database passwords, Telegram secrets, payment secrets, or generated runtime data.

Billing And Accounts

Billing supports:

  • default plan limits;
  • free/pro/team limits for chat messages, searches, and tasks;
  • membership-aware usage accounting;
  • global provider path through Stripe;
  • RF provider path through CloudPayments;
  • checkout session creation;
  • webhook processing;
  • invoice history;
  • admin/superuser limit bypass where explicitly implemented in service logic.

Relevant files:

  • src/ugcai/services/billing.py
  • src/ugcai/infra/billing/stripe.py
  • src/ugcai/infra/billing/cloudpayments.py
  • src/ugcai/domain/schemas/billing.py
  • src/ugcai/api/routes/billing.py

Frontend

The frontend lives in apps/web.

It provides:

  • chat-first workspace;
  • creator/search panels;
  • task monitor;
  • review console;
  • campaign console;
  • admin console and data browser;
  • workspace settings.

Run locally:

cd apps/web
npm install
NEXT_PUBLIC_DEFAULT_API_BASE_URL=http://127.0.0.1:8000 npm run dev

The UI should talk to /chat/* for persisted chat behavior rather than calling search directly from the browser as a shortcut.

Deployment

Primary dev/test deployment docs:

  • deployment/digitalocean/devtest/README.md
  • deployment/digitalocean/devtest/RELEASE_CHECKLIST.md
  • deployment/digitalocean/README.md
  • deployment/digitalocean/terraform/prod/README.md

Dev/test topologies:

  • direct mode: public web/API ports by IP;
  • Traefik mode: 80/443, TLS, /api prefix routing;
  • internal mode: no public ports;
  • host-Ollama mode: containers point at host Ollama runtime;
  • internal graph mode: OpenSPG/Neo4j/MySQL/MinIO run in the compose bundle.

Common deploy helper:

ssh root@SERVER_IP 'cd /opt/ugcai && bash scripts/deploy_devtest.sh'

Autostart helper:

ssh root@SERVER_IP 'cd /opt/ugcai && bash scripts/install_devtest_autostart.sh'

Stop helper:

ssh root@SERVER_IP 'cd /opt/ugcai && bash scripts/stop_devtest.sh'

Health checks:

curl -sk https://YOUR_HOST/api/health/ready
curl -sk https://YOUR_HOST/api/health/live

Local Development

Install dependencies:

uv sync

Create a local env:

cp .env.example .env

Run API locally:

uv run uvicorn ugcai.main:app --reload --host 127.0.0.1 --port 8000

Run one worker iteration:

uv run python -m ugcai.worker.main --once

Run continuous worker:

uv run python -m ugcai.worker.main

Run migrations against configured Postgres:

uv run alembic upgrade head

Create a new migration after model changes:

uv run alembic revision --autogenerate -m "describe change"

Useful runtime smokes:

uv run python -m ugcai.runtime.startup --timeout 60
uv run python -m ugcai.runtime.query_planner_smoke
uv run python -m ugcai.runtime.fact_extraction_smoke
uv run python -m ugcai.runtime.ad_review_smoke
uv run python -m ugcai.runtime.telegram_live_smoke
uv run python -m ugcai.runtime.telegram_live_sourcing_smoke
uv run python -m ugcai.runtime.reindex_creator_search
uv run python -m ugcai.runtime.sync_graph

Testing

Run all backend tests:

uv run pytest -q

Run focused suites:

uv run pytest tests/test_auth_api.py tests/test_account_billing_api.py -q
uv run pytest tests/test_search_api.py tests/test_search_quality.py -q
uv run pytest tests/test_fact_extraction_service.py tests/test_fact_extraction_providers.py -q
uv run pytest tests/test_url_enrichment_service.py tests/test_url_document_repository.py -q
uv run pytest tests/test_worker_runtime.py tests/test_tasks_api.py -q

Run frontend checks:

cd apps/web
npm run lint
npm run build

Before committing backend changes, at minimum run:

uv run pytest -q
git diff --check

Operational Runbooks

Check Service Health

curl -sk https://YOUR_HOST/api/health/ready

Expected ready payload:

{
  "status": "ok",
  "database": "up",
  "cache_backend": "redis_compatible",
  "graph_backend": "openspg",
  "graph_status": "up"
}

Inspect Worker State

ssh root@SERVER_IP 'cd /opt/ugcai && docker compose --env-file deployment/digitalocean/devtest/.env ps'

Scale Workers

Preferred path: change scale in deployment/digitalocean/devtest/.env and run the deploy helper. This keeps manual deploy, CD, and systemd autostart aligned.

UGCAI_DEVTEST_WORKER_SCALE=1
UGCAI_DEVTEST_WORKER_GRAPH_SCALE=8
UGCAI_DEVTEST_WORKER_URL_SCALE=1
UGCAI_DEVTEST_WORKER_MEDIA_SCALE=1
UGCAI_DEVTEST_WORKER_REVIEW_SCALE=1
ssh root@SERVER_IP 'cd /opt/ugcai && bash scripts/deploy_devtest_fullstack.sh'

Telegram Session Reset

Use this when Telegram login/session state is broken or the user needs to log into the account again:

ssh root@SERVER_IP 'cd /opt/ugcai && docker compose --env-file deployment/digitalocean/devtest/.env stop worker && \
  docker volume ls | grep telegram && \
  docker compose --env-file deployment/digitalocean/devtest/.env up -d app'

Then open the Telegram login helper route from the web/API host and complete the QR/password flow.

Reindex Retrieval

ssh root@SERVER_IP 'cd /opt/ugcai && docker compose --env-file deployment/digitalocean/devtest/.env exec -T app \
  python -m ugcai.runtime.reindex_creator_search'

Sync Graph

ssh root@SERVER_IP 'cd /opt/ugcai && docker compose --env-file deployment/digitalocean/devtest/.env exec -T app \
  python -m ugcai.runtime.sync_graph'

Run Fact Extraction Smoke On Server

ssh root@SERVER_IP 'cd /opt/ugcai && docker compose --env-file deployment/digitalocean/devtest/.env exec -T app \
  python -m ugcai.runtime.fact_extraction_smoke'

Inspect Ollama Models

ssh root@SERVER_IP 'ollama list && ollama ps'

Troubleshooting

/openapi.json Returns 404 In Browser

In Traefik /api mode, public OpenAPI is usually available at:

https://YOUR_HOST/api/openapi.json

Internally FastAPI serves:

/openapi.json

Do not use frontend routes for API docs.

Unexpected token '<' ... is not valid JSON

The frontend received HTML instead of JSON. Typical causes:

  • frontend API base URL points at the web app instead of the backend;
  • /api prefix is missing in proxy mode;
  • OpenAPI/API route is being requested from the frontend fallback route.

Check:

curl -sk https://YOUR_HOST/api/health/ready

database is locked

This is expected only in local SQLite test/dev modes under concurrency. Remote dev/test should use Postgres. Worker runtime forces single-thread execution for SQLite to keep tests deterministic.

Monthly task limit reached

Billing usage limits are enforced by account membership plan. Superuser/admin limit bypass must be implemented explicitly in billing/account service logic, not by granting broad permissions in routes.

OpenSPG request failed ... unauthorized

Check:

  • UGCAI_OPENSPG_SERVER_URL
  • UGCAI_OPENSPG_PROJECT_ID
  • OpenSPG project bootstrap status;
  • graph service logs;
  • whether the app is pointed at the correct OpenSPG host.

Validated facts 0

Common causes:

  • extraction model returned empty JSON;
  • evidence span/reference missing;
  • confidence below validation threshold;
  • sensitive or unsupported fact rejected;
  • OpenSPG sync not run after extraction.

Current mitigation:

  • use qwen2.5:14b;
  • use fact-extractor-v3;
  • keep few-shot examples in the prompt;
  • run fact extraction smoke and inspect rejected/skipped counts.

ad_review_llm_unavailable

This means the ad-review provider could not get a usable model response. The pipeline records safe unavailable reviews instead of inventing labels. Check Ollama availability, model presence, base URL, and timeout settings.

FloodWait Or Telegram Rate Limits

Use:

  • lower crawl/parsing concurrency;
  • keep UGCAI_TELEGRAM_REQUEST_MIN_INTERVAL_SECONDS conservative;
  • keep background parse at 10 posts;
  • run deep parse only after channels pass filters;
  • avoid multiple containers writing the same Telethon session file.

Engineering Rules

Use these rules when modifying the project:

  • Keep APIs thin and services explicit.
  • Add migrations for schema changes.
  • Add tests for permission failures, org isolation, validation failures, and task behavior.
  • Prefer deterministic validation over model improvisation.
  • Keep graph facts evidence-backed.
  • Keep audit logs safe and useful.
  • Do not add broad admin permissions to bypass a real auth problem.
  • Do not silently swallow worker exceptions to keep pipelines green.
  • Avoid storing unnecessary personal data.
  • Keep generated/runtime files out of git.

Roadmap

Near-term recommended order:

  1. Harden ad-review model availability and timeout behavior.
  2. Add richer extraction golden examples for Telegram, VK, YouTube, Instagram.
  3. Add admin-visible extraction/rejection metrics by reason.
  4. Add task retry/backoff policy controls in UI.
  5. Add billing provider sandbox E2E tests.
  6. Add MCP server layer over existing services with side-effect metadata and approval policies.
  7. Add ClickHouse analytics once trend/dashboard workloads become heavy.
  8. Add production-grade backup/restore and disaster recovery docs.
  9. Add stricter creator identity resolution review workflow.
  10. Add ORD/compliance approval flows only behind explicit dangerous-action gates.

Related Documentation

  • AGENTS.md: engineering constraints and non-negotiable project rules.
  • TASKS_TZ.md: initial developer-oriented task specification.
  • ugc_creator_platform_summary.md: earlier product/architecture narrative.
  • apps/web/README.md: frontend-specific notes.
  • deployment/digitalocean/devtest/README.md: dev/test deployment bundle.
  • deployment/digitalocean/devtest/RELEASE_CHECKLIST.md: release checklist.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages