Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .ai-factory/RULES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Project Rules

- No scope creep: implement only the approved PR goal.
- One PR solves one bounded problem.
- Prefer small, reviewable changes over broad rewrites.
- Do read-only review before implementation for non-trivial, security-sensitive, or risky changes.
- Implement only after human approval when scope affects governance, security, install flows, public access, CI, changelog, or version policy.
- Do not run privileged commands unless explicitly requested.
- Do not use sudo unless explicitly requested.
- Do not run install scripts during planning or review.
- Do not start containers during planning or review unless explicitly requested.
- Do not commit secrets or generated secret files.
- Do not write generated secrets into tracked config files.
- Do not use `latest` Compose image tags.
- Do not exact-pin apt packages by default.
- Ubuntu 24.04 LTS is the primary and priority baseline.
- Ubuntu 26.04 LTS is the compatibility and validation target.
- README.md stays short; detailed guidance belongs in docs/.
- Beginner-facing docs must stay clear and avoid unnecessary jargon.
- Risky system changes must include verification and rollback notes.
- Do not broaden into Supabase scope, public Compose override, CI, changelog, version enforcement, or smoke-test evidence unless that is the approved PR goal.
68 changes: 68 additions & 0 deletions .github/workflows/quality.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
name: Quality

on:
pull_request:
push:

jobs:
static-quality:
name: Static quality
runs-on: ubuntu-24.04
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Whitespace check
run: |
if [ "${{ github.event_name }}" = "pull_request" ]; then
git fetch origin "${{ github.base_ref }}" --depth=1
git diff --check "origin/${{ github.base_ref }}...HEAD"
elif git rev-parse HEAD^ >/dev/null 2>&1; then
git diff --check HEAD^..HEAD
else
git diff --check --root HEAD
fi

- name: Bash syntax check
run: bash -n scripts/*.sh scripts/security/*.sh scripts/lib/*.sh

- name: ShellCheck
run: |
if ! command -v shellcheck >/dev/null 2>&1; then
echo "::error::ShellCheck is not available on this runner. Decide a tool installation strategy in a separate PR."
exit 1
fi
shellcheck -x --severity=warning scripts/*.sh scripts/security/*.sh scripts/lib/*.sh

- name: Compose base config
working-directory: docker-compose
run: docker compose --env-file env.example config >/tmp/install_ubuntu_ci_base.yml

- name: Compose monitoring config
working-directory: docker-compose
run: docker compose --env-file env.example -f docker-compose.yml -f docker-compose.monitoring.yml config >/tmp/install_ubuntu_ci_monitoring.yml

- name: Compose latest image guard
run: |
if grep -RInE 'image:[[:space:]]*.*:latest|image:[[:space:]]*latest' docker-compose/*.yml; then
echo "::error::Compose images must not use latest tags."
exit 1
fi

- name: Removed public override guard
run: |
if grep -RInE 'docker-compose.override.public|override.public|compose.override.public' README.md QUICKSTART.md docs docker-compose PLAN.md; then
echo "::error::Removed public Compose override is still referenced."
exit 1
fi

- name: Tracked secret-file guard
run: |
tracked_secret_files="$(git ls-files | grep -E '(^|/)\.env$|docker-compose/\.env$|\.env\.local$|\.secret$|\.secrets$|(^|/)secrets\.' || true)"
if [ -n "$tracked_secret_files" ]; then
echo "::error::Secret-like local files must not be tracked:"
printf '%s\n' "$tracked_secret_files"
exit 1
fi
28 changes: 28 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Local environment and generated secrets
.env
docker-compose/.env
*.env.local
*.secret
*.secrets
secrets.*
*.key
*.pem

# Local AI tool state
.ai-factory.json
.opencode/
.ai-factory/plans/

# Backups and logs
backups/
logs/
*.log

# Local temporary artifacts
tmp/
.tmp/
*.tmp
*.swp
*.swo
*~
.DS_Store
17 changes: 16 additions & 1 deletion PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,21 @@ This file stays in the repository root as a project roadmap and quality-gate tra
- Record quality gates that still require manual validation.
- Avoid duplicating step-by-step installation docs.

## Current PR-Sized Roadmap

| Phase | Status | Notes |
|---|---|---|
| Secrets safety PR | ✅ Completed | Committed as `89ab2bb fix: harden generated secrets handling`. |
| Project governance baseline | 🔄 Current | Add requirements, version policy, acceptance criteria, and roadmap links. |
| Ubuntu 26.04 compatibility validation | ⏳ Planned | Validate packages, scripts, Docker, Compose and profile ready checks before compatibility claims. |
| Supabase scope clarity | 🔄 Current | Narrow docs-only scope: current support is PostgreSQL with selected Supabase-related components; Full Supabase implementation is not approved and deferred. |
| Public Compose override correctness | 🔄 Current | Remove unsafe direct Compose exposure docs/file; keep SSH tunnel and Nginx/reverse proxy as public access paths. |
| Version policy enforcement | ⏳ Planned | Align package/image handling with `docs/version-policy.md`. |
| CI/quality gates | ⏳ Planned | Add automated syntax, lint and documentation checks in a separate PR. |
| Clean Ubuntu 24.04 VM smoke-test evidence | ⏳ Planned | Capture clean VM evidence for supported profile flows before release readiness claims. |
| Changelog/release readiness | ⏳ Planned | Decide changelog/release notes format and update process. |
| ai-factory rules proposal/review | ⏳ Planned | Review project rules separately; do not track `.ai-factory.json` or `.opencode/` without explicit approval. |

---

# План приведения проекта к лучшим практикам
Expand Down Expand Up @@ -129,7 +144,7 @@ This file stays in the repository root as a project roadmap and quality-gate tra
- `supabase_meta` добавлен, `supabase_studio` связан через `depends_on`
- `pgbouncer` добавлен для connection pooling
- Порты: Redis `127.0.0.1:6379`, PostgreSQL `127.0.0.1:54322`, PgBouncer `127.0.0.1:6432`
- `docker-compose.override.public.yml` — для публичного доступа (опционально)
- Публичный доступ: SSH tunnel для admin access или Nginx/reverse proxy для reviewed public access; direct Compose exposure не используется

Минимальные проверки:
```bash
Expand Down
10 changes: 5 additions & 5 deletions QUICKSTART.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@
| `proxy` | 512MB-1GB RAM, 10GB disk | база под x-ui/3x-ui/VPN/proxy panel |
| `docker-host` | 1GB RAM, 10GB disk | маленький Docker host |
| `web` | 1GB RAM, 15GB disk | web/app VPS с HTTP/HTTPS |
| `ai-stack` | 4GB RAM, 50GB disk | n8n, Supabase/PostgreSQL, Redis, pgvector, monitoring |
| `ai-stack` | 4GB RAM, 50GB disk | n8n, PostgreSQL/Supabase-related components, Redis, pgvector, monitoring |

Общее: Ubuntu 22.04 LTS или 24.04 LTS, root/sudo доступ, рабочий SSH. Для bare metal с новым железом сначала проверьте [драйверы и совместимость](docs/08-hardware-drivers.md).
Общее: Ubuntu 24.04 LTS как primary baseline, root/sudo доступ, рабочий SSH. Ubuntu 26.04 LTS — compatibility/validation target, не fully validated primary support. Для bare metal с новым железом сначала проверьте [драйверы и совместимость](docs/08-hardware-drivers.md).

## 1. Получите проект

Expand Down Expand Up @@ -108,7 +108,7 @@ sudo bash scripts/99-ready-checks.sh --profile web

## 7. AI Automation Stack

Используйте для полного Docker Compose stack: n8n, Supabase/PostgreSQL, Redis, pgvector, PgBouncer, monitoring и backups.
Используйте для Docker Compose stack: n8n, PostgreSQL with selected Supabase-related components, Redis, pgvector, PgBouncer, monitoring и backups.

```bash
sudo bash scripts/00-preflight-check.sh --profile ai-stack
Expand All @@ -121,14 +121,14 @@ docker compose ps
sudo bash ../scripts/99-ready-checks.sh --profile ai-stack
```

По умолчанию PostgreSQL, Redis, Supabase Studio и n8n привязаны к `127.0.0.1`. Для внешнего доступа используйте SSH tunnel или [Nginx](docs/07-nginx.md).
По умолчанию PostgreSQL, Redis, Supabase Studio и n8n привязаны к `127.0.0.1`. Для внешнего доступа используйте SSH tunnel или [Nginx](docs/07-nginx.md); do not expose admin services through direct Compose ports.

Критичные переменные в `docker-compose/.env`:

| Переменная | Назначение |
|------------|------------|
| `REDIS_PASSWORD` | пароль Redis |
| `SUPABASE_DB_PASSWORD` | пароль PostgreSQL/Supabase |
| `SUPABASE_DB_PASSWORD` | пароль PostgreSQL (`supabase_db`) |
| `N8N_BASIC_AUTH_PASSWORD` | пароль n8n |
| `N8N_ENCRYPTION_KEY` | ключ шифрования n8n |
| `N8N_USER_MANAGEMENT_JWT_SECRET` | JWT secret n8n |
Expand Down
13 changes: 8 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Most self-hosted projects need the same foundation before product work starts:

- hardened SSH access, firewall, fail2ban and security updates;
- Docker and Docker Compose installed correctly when containers are needed;
- PostgreSQL/Supabase, Redis and pgvector for AI workflows and RAG on larger hosts;
- PostgreSQL with selected Supabase-related components, Redis and pgvector for AI workflows and RAG on larger hosts;
- n8n main/worker setup for automation pipelines on the `ai-stack` profile;
- Nginx, SSL, monitoring, backups and readiness checks when the selected profile needs them.

Expand All @@ -24,7 +24,7 @@ This repo turns that foundation into documented steps, scripts and compose files
|------|----------|
| Server baseline | Ubuntu hardening, SSH keys, UFW, fail2ban, unattended upgrades |
| Container runtime | Docker Engine, Docker Compose, daemon configuration |
| AI automation stack | n8n, Redis, Supabase/PostgreSQL, pgvector, PgBouncer for `ai-stack` |
| AI automation stack | n8n, Redis, PostgreSQL with selected Supabase-related components, pgvector, PgBouncer for `ai-stack` |
| Production support | Nginx reverse proxy, SSL path, monitoring, backups, ready checks |
| Safety | Secret generation, closed local ports, healthchecks, version-pinned compose |
| Documentation | Step-by-step guides for VPS and local server installation |
Expand All @@ -37,7 +37,7 @@ This repo turns that foundation into documented steps, scripts and compose files
| `proxy` | Base for x-ui/3x-ui/VPN/proxy panel | minimal path + explicit service ports |
| `docker-host` | Small container host | minimal path + Docker install |
| `web` | Small web/app server | minimal path + HTTP/HTTPS reverse proxy |
| `ai-stack` | Full n8n/Supabase/Redis/pgvector stack | Docker, secrets, compose stack, service ready checks |
| `ai-stack` | n8n, Redis, PostgreSQL/Supabase-related components, pgvector stack | Docker, secrets, compose stack, service ready checks |

`4GB RAM / 50GB disk` belongs to `ai-stack`, not to every VPS. A `1 vCPU / 1GB RAM` server can still be valid for `minimal` or `proxy` with warnings.

Expand Down Expand Up @@ -71,7 +71,7 @@ For the full installation path, use [QUICKSTART.md](QUICKSTART.md). To understan

- Prepare a VPS for AI assistants and Telegram bots.
- Run n8n workflows with Redis queue mode and PostgreSQL storage.
- Build a self-hosted RAG base with Supabase/PostgreSQL and pgvector.
- Build a self-hosted RAG base with PostgreSQL, selected Supabase-related components and pgvector.
- Standardize repeatable infrastructure setup for client AI automation projects.
- Keep deployment knowledge in scripts and docs instead of one-off terminal history.

Expand All @@ -91,6 +91,9 @@ For the full installation path, use [QUICKSTART.md](QUICKSTART.md). To understan
| Guide | Description |
|-------|-------------|
| [Quick Start](QUICKSTART.md) | End-to-end installation path |
| [Project Requirements](docs/project-requirements.md) | Scope, baseline and change-control rules |
| [Version Policy](docs/version-policy.md) | Ubuntu packages, Docker and Compose image version rules |
| [Acceptance Criteria](docs/acceptance-criteria.md) | Profile-level readiness expectations |
| [System Requirements](requirements/system-requirements.md) | CPU, RAM, disk and OS requirements |
| [VPS Profiles](docs/profiles.md) | Minimal, proxy, docker-host, web and ai-stack profiles |
| [Server Security](docs/01-server-security.md) | SSH, UFW, fail2ban and hardening |
Expand All @@ -101,7 +104,7 @@ For the full installation path, use [QUICKSTART.md](QUICKSTART.md). To understan
| [Infrastructure Setup](docs/03-infrastructure-setup.md) | Stack overview and deployment order |
| [Architecture](docs/architecture.md) | Runtime components and data flow |
| [Architecture Operations](docs/architecture-operations.md) | Scaling, backups and performance notes |
| [Supabase](docs/03-supabase.md) | Self-hosted Supabase setup |
| [Supabase scope](docs/03-supabase.md) | PostgreSQL with Supabase-related Meta/Studio components |
| [n8n](docs/04-n8n.md) | n8n main/worker deployment |
| [Redis](docs/05-redis.md) | Redis setup for queues and caching |
| [pgvector](docs/06-vector-db.md) | Vector search setup for RAG |
Expand Down
18 changes: 0 additions & 18 deletions docker-compose/docker-compose.override.public.yml

This file was deleted.

2 changes: 1 addition & 1 deletion docs/01-server-security.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ systemctl status unattended-upgrades --no-pager
## Источники

- [Ubuntu Security Documentation](https://ubuntu.com/security)
- [DigitalOcean Security Best Practices](https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-22-04)
- [Ubuntu Server Documentation](https://documentation.ubuntu.com/server/)
- [UFW Documentation](https://help.ubuntu.com/community/UFW)

## See Also
Expand Down
2 changes: 1 addition & 1 deletion docs/02-docker-installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

## Предварительные требования

- Ubuntu 22.04 LTS или Ubuntu 24.04 LTS
- Ubuntu 24.04 LTS primary baseline. Ubuntu 26.04 LTS is a compatibility/validation target only.
- Права root или sudo
- Стабильное интернет-соединение

Expand Down
34 changes: 20 additions & 14 deletions docs/03-infrastructure-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

# Инфраструктура для мультиагентных ассистентов

Это руководство описывает установку и настройку всех компонентов инфраструктуры для мультиагентных ассистентов на базе n8n с Supabase, Redis и pgvector.
Это руководство описывает установку и настройку компонентов инфраструктуры для мультиагентных ассистентов на базе n8n, PostgreSQL with selected Supabase-related components, Redis и pgvector.

## Предварительные требования

Expand All @@ -16,7 +16,7 @@

## Обзор компонентов

- **Supabase** - Self-hosted база данных PostgreSQL с аутентификацией и API
- **PostgreSQL/Supabase-related components** - PostgreSQL using Supabase Postgres image, Supabase Meta and Studio where implemented
- **pgvector** - Расширение PostgreSQL для векторного поиска
- **PgBouncer** - пул соединений PostgreSQL (для n8n)
- **Redis** - Кэш и очередь задач
Expand All @@ -27,7 +27,7 @@

Каждый компонент имеет отдельное руководство:

1. **[Установка Supabase](03-supabase.md)** - Self-hosted PostgreSQL с pgvector
1. **[Supabase-related PostgreSQL](03-supabase.md)** - PostgreSQL с Supabase Meta/Studio и pgvector
2. **[Установка Redis](05-redis.md)** - Кэш и очередь задач
3. **[Настройка векторной БД](06-vector-db.md)** - pgvector таблицы и функции
4. **[Установка n8n](04-n8n.md)** - Платформа автоматизации с воркерами
Expand All @@ -42,25 +42,31 @@

## Рекомендуемый порядок установки

1. **Supabase** - базовая инфраструктура БД
0. **Secrets** - обязательный `.env` для `ai-stack`
```bash
sudo bash scripts/12-generate-secrets.sh --profile ai-stack
```
См. [13-secrets.md](13-secrets.md) для подробностей.

1. **PostgreSQL with selected Supabase-related components** - базовая инфраструктура БД
```bash
sudo bash scripts/04-setup-supabase.sh
```
См. [03-supabase.md](03-supabase.md) для подробностей

2. **Redis** - очередь задач (можно установить параллельно с Supabase)
2. **Redis** - очередь задач (можно установить параллельно с PostgreSQL)
```bash
sudo bash scripts/06-setup-redis.sh
```
См. [05-redis.md](05-redis.md) для подробностей

3. **Векторная БД** - настройка pgvector в Supabase
3. **Векторная БД** - настройка pgvector в PostgreSQL
```bash
sudo bash scripts/07-setup-vector-db.sh
```
См. [06-vector-db.md](06-vector-db.md) для подробностей

4. **n8n** - требует Supabase и Redis
4. **n8n** - требует PostgreSQL и Redis
```bash
sudo bash scripts/05-setup-n8n.sh
```
Expand All @@ -76,17 +82,17 @@ cd docker-compose
docker compose --env-file .env up -d
```

Для публичного доступа (опционально):
```bash
docker compose -f docker-compose.yml -f docker-compose.override.public.yml up -d
```
Не запускайте `scripts/12-generate-secrets.sh` для `minimal`, `proxy`, `docker-host` или `web`: эти профили не используют compose secrets.

Доступ по умолчанию остаётся только через localhost. Для админского доступа используйте SSH tunnel, например `ssh -L 5678:127.0.0.1:5678 user@server`, а для reviewed public access используйте [Nginx/reverse proxy](07-nginx.md).

Примечания:
- Порты БД и Redis привязаны к `127.0.0.1` для безопасности.
- Для внешнего доступа используйте SSH‑туннель или reverse proxy.
- Supabase Studio, n8n, Prometheus и Grafana не должны публиковаться наружу напрямую через direct Compose ports.
- Для внешнего доступа используйте SSH‑туннель или Nginx/reverse proxy.
- Supabase Studio требует сервис `supabase_meta` (он включён в compose).
- n8n подключается к БД через PgBouncer (`localhost:6432` для хоста).
- n8n по умолчанию доступен только на `127.0.0.1`, для внешнего доступа используйте override.
- n8n по умолчанию доступен только на `127.0.0.1`; не открывайте его direct Compose port публично.
- Для production используйте защищённый `.env` или Docker secrets.
- Рекомендуется регулярная ротация паролей (минимум раз в 90 дней).

Expand Down Expand Up @@ -145,5 +151,5 @@ docker exec redis redis-cli -a YOUR_PASSWORD --rdb /data/dump.rdb
## See Also

- [Architecture](architecture.md) — компоненты и потоки данных.
- [Supabase](03-supabase.md) — база данных и pgvector foundation.
- [Supabase scope](03-supabase.md) — PostgreSQL and implemented Supabase-related components.
- [n8n](04-n8n.md) — automation runtime поверх инфраструктуры.
Loading
Loading