AI Knowledge Infrastructure System (AKIS)

A production-grade, private RAG (Retrieval-Augmented Generation) system for organizations. AKIS enables natural language querying over internal knowledge sources with high accuracy, source traceability, and permission-aware access control.

🎯 Key Features

Multi-Source Ingestion: Notion, Google Drive, GitHub, PDFs, local files
Hybrid Search: Combines semantic (vector) and keyword-based retrieval
LLM Agnostic: Supports OpenAI, Anthropic, and self-hosted models
Permission-Aware: Role-based access control (RBAC)
Source Attribution: Every answer includes verifiable citations
Uncertainty Handling: Explicit "insufficient data" responses when appropriate
Client Infrastructure: Deploy on your cloud or on-premise servers
No External Training: Your data stays private

📋 Requirements

Docker & Docker Compose (recommended)
Python 3.11+ (for local development)
8GB+ RAM
LLM API access (OpenAI, Anthropic) or local model server

🚀 Quick Start

1. Clone and Configure

git clone <repository-url>
cd akis
cp .env.example .env

2. Configure Environment

Edit .env with your settings:

# LLM Configuration (choose one)
LLM_PROVIDER=openai
LLM_MODEL=gpt-4-turbo-preview
OPENAI_API_KEY=your_openai_key_here

# Or use Anthropic
# LLM_PROVIDER=anthropic
# LLM_MODEL=claude-3-opus-20240229
# ANTHROPIC_API_KEY=your_anthropic_key_here

# Data Source Credentials
NOTION_API_KEY=your_notion_key
GOOGLE_CREDENTIALS_PATH=./credentials/google_creds.json
GITHUB_TOKEN=your_github_token

# Security
SECRET_KEY=generate_a_secure_random_string_here
POSTGRES_PASSWORD=secure_database_password

3. Start Services

docker-compose up -d

This will start:

PostgreSQL (metadata & user management)
Redis (task queue)
Elasticsearch (keyword search)
ChromaDB (vector storage)
AKIS API
Celery workers (background processing)

4. Initialize Database

docker-compose exec api python scripts/setup_db.py
docker-compose exec api python scripts/init_indices.py

5. Access the System

API: http://localhost:8000
API Docs: http://localhost:8000/api/docs (development only)
Health Check: http://localhost:8000/health

📖 Usage

Query API

curl -X POST http://localhost:8000/api/v1/query/ask \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -d '{
    "query": "What is our deployment process for production?"
  }'

Response:

{
  "query": "What is our deployment process for production?",
  "answer": "Based on the engineering docs, the deployment process involves...",
  "citations": [
    {
      "source_title": "Production Deployment Guide",
      "source_system": "notion",
      "excerpt": "..."
    }
  ],
  "confidence": 0.85,
  "has_sufficient_context": true
}

Admin: Add Data Source

curl -X POST http://localhost:8000/api/v1/admin/sources \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ADMIN_TOKEN" \
  -d '{
    "source_type": "notion",
    "config": {
      "database_id": "your_notion_database_id"
    },
    "sync_schedule": "0 */6 * * *"
  }'

🏗️ Architecture

┌─────────────┐
│   Client    │
└──────┬──────┘
       │
       ▼
┌─────────────────────────────────────────┐
│           FastAPI REST API               │
│  ┌────────────┐      ┌────────────┐     │
│  │   Query    │      │   Admin    │     │
│  │   Routes   │      │   Routes   │     │
│  └─────┬──────┘      └─────┬──────┘     │
└────────┼─────────────────┼──────────────┘
         │                 │
         ▼                 ▼
┌─────────────────────────────────────────┐
│         Core Services                    │
│  ┌───────────┐  ┌──────────┐  ┌───────┐ │
│  │ Retriever │  │   LLM    │  │ RBAC  │ │
│  │  (Hybrid) │  │ Gateway  │  │       │ │
│  └─────┬─────┘  └────┬─────┘  └───────┘ │
└────────┼─────────────┼──────────────────┘
         │             │
         ▼             ▼
┌─────────────────────────────────────────┐
│          Data Layer                      │
│  ┌──────────┐  ┌───────────┐  ┌───────┐ │
│  │ Vector   │  │ Keyword   │  │  SQL  │ │
│  │   DB     │  │  Search   │  │  DB   │ │
│  │(ChromaDB)│  │(Elastic)  │  │(Pgsql)│ │
│  └──────────┘  └───────────┘  └───────┘ │
└─────────────────────────────────────────┘
         ▲
         │
┌────────┴────────┐
│  Ingestion &    │
│  Processing     │
│  (Celery)       │
└─────────────────┘

🔧 Configuration

LLM Providers

Switch models without code changes:

# In .env or config
LLM_PROVIDER=openai
LLM_MODEL=gpt-4-turbo-preview

# Or switch to Anthropic
LLM_PROVIDER=anthropic
LLM_MODEL=claude-3-opus-20240229

# Or use local model via Ollama
LLM_PROVIDER=local
LOCAL_LLM_URL=http://ollama:11434
LLM_MODEL=llama2

Chunking Strategy

Customize per document type:

# Text documents
CHUNK_SIZE=1000
CHUNK_OVERLAP=200

# Code files
CODE_CHUNK_SIZE=1500

Hybrid Search Weighting

Balance semantic vs keyword search:

# 0.0 = pure keyword, 1.0 = pure semantic
HYBRID_ALPHA=0.5

🔐 Security

Authentication

AKIS uses JWT tokens for authentication:

# Generate token (admin only)
POST /api/v1/admin/users/token
{
  "username": "user@company.com",
  "password": "secure_password"
}

Role-Based Access Control

Three roles supported:

Admin: Full system access
User: Query access only
Auditor: Read-only log access

Data Isolation

Each deployment is fully isolated:

Dedicated database schemas
Separate vector collections
Encrypted storage and transit

📊 Monitoring

Health Checks

/health: Basic health check
/ready: Readiness probe (checks dependencies)

Metrics (if enabled)

ENABLE_METRICS=true
METRICS_PORT=9090

Access Prometheus metrics at http://localhost:9090/metrics

Audit Logging

All queries and access events are logged:

ENABLE_AUDIT_LOGGING=true

View logs:

docker-compose logs -f api

🧪 Testing

# Run all tests
pytest

# With coverage
pytest --cov=src --cov-report=html

# Specific test suite
pytest tests/test_ingestion/

📦 Deployment

Production Checklist

Security:
- Change SECRET_KEY to secure random value
- Set strong POSTGRES_PASSWORD
- Use HTTPS/TLS for API
- Enable firewall rules
- Rotate API keys regularly
Performance:
- Adjust API_WORKERS based on CPU
- Configure resource limits in docker-compose
- Set up database connection pooling
- Enable caching where appropriate
Reliability:
- Configure backup strategy
- Set up monitoring/alerting
- Test disaster recovery
- Document runbooks
Scaling:
- Use external managed services (RDS, ElasticSearch)
- Deploy behind load balancer
- Implement rate limiting
- Set up horizontal pod autoscaling (K8s)

Kubernetes Deployment

Helm charts available in /deploy/kubernetes/:

helm install akis ./deploy/kubernetes/akis-chart \
  --set api.replicas=3 \
  --set postgresql.enabled=false \
  --set postgresql.host=your-rds-endpoint

🛠️ Development

Local Setup

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Install development dependencies
pip install -r requirements-dev.txt

# Run locally
uvicorn src.api.main:app --reload

Code Quality

# Format code
black src/ tests/

# Lint
ruff src/ tests/

# Type checking
mypy src/

📝 API Documentation

Full API documentation available at /api/docs (development mode).

Core Endpoints

Query

POST /api/v1/query/ask - Submit natural language query
GET /api/v1/query/history - Get query history

Admin

POST /api/v1/admin/sources - Add data source
GET /api/v1/admin/sources - List data sources
POST /api/v1/admin/sources/{id}/sync - Trigger sync
GET /api/v1/admin/users - Manage users

🤝 Contributing

See CONTRIBUTING.md for guidelines.

📄 License

Proprietary - Contact for licensing information.

🆘 Support

Documentation: /docs
Issues: GitHub Issues
Email: support@example.com

🗺️ Roadmap

Current (v1.0)

✅ Multi-source ingestion
✅ Hybrid search
✅ LLM agnostic architecture
✅ Basic RBAC

Future (v1.1+)

Built with ❤️ for organizations that value their knowledge

Name		Name	Last commit message	Last commit date
alembic		alembic
config		config
scripts		scripts
src		src
tests		tests
.env.example		.env.example
Dockerfile		Dockerfile
README.md		README.md
alembic.ini		alembic.ini
docker-compose.yml		docker-compose.yml
requirements.txt		requirements.txt

Latest commit

History

Folders and files

Repository files navigation