AI Studio is a Next.js 16 enterprise application built with modern cloud-native architecture principles. It provides AI-powered tools with role-based access control, featuring multiple LLM providers, document processing, and knowledge management capabilities.
📊 Visual Diagrams: See
/docs/diagrams/for comprehensive architectural diagrams including:
- Frontend: Next.js 16 with App Router, React 19
- UI Components: Shadcn UI + Tailwind CSS
- TypeScript: Strict type safety across the application
- AI SDK: Vercel AI SDK v6 for LLM integration
- Providers:
- OpenAI (GPT-5, GPT-4, GPT-3.5)
- Google AI (Gemini models)
- Amazon Bedrock (Claude, Llama)
- Azure OpenAI
- Streaming: Server-Sent Events (SSE) for real-time responses
- Embeddings: Vector search for knowledge retrieval
- Auth Provider: AWS Cognito with Google OAuth federation
- Session Management: NextAuth v5 with JWT strategy
- RBAC: Role-based access control with tool-specific permissions
- Security Headers: CSRF protection, CSP, secure cookies
- Database: AWS Aurora Serverless v2 (PostgreSQL)
- ORM: Drizzle ORM with postgres.js driver (type-safe queries)
- Caching: 5-minute TTL for settings
- IaC: AWS CDK (TypeScript)
- Hosting: AWS ECS Fargate with Application Load Balancer
- Container: Next.js 16 SSR on Fargate with auto-scaling
- Storage: S3 with lifecycle policies
- Monitoring: CloudWatch with structured logging + ADOT
- Network: VPC with public/private/isolated subnets
┌─────────────────┐ ┌──────────────┐ ┌─────────────────┐
│ │ │ │ │ │
│ Client (React) │────▶│ Next.js │────▶│ AWS Cognito │
│ │ │ App Router │ │ + Google │
└─────────────────┘ └──────────────┘ └─────────────────┘
│
▼
┌──────────────────┐
│ │
│ Server Actions │
│ & API Routes │
│ │
└──────────────────┘
│
┌──────────────┴──────────────┐
▼ ▼
┌──────────────┐ ┌──────────────┐
│ │ │ │
│ AI Providers│ │ Drizzle ORM │
│ (Factory) │ │ (postgres) │
│ │ │ │
└──────────────┘ └──────────────┘
│ │
▼ ▼
┌───────────────────────┐ ┌──────────────┐
│ OpenAI/Google/Bedrock │ │Aurora Server-│
│ Azure APIs │ │ less v2 │
└───────────────────────┘ └──────────────┘
- React Server Components (default)
- Client components with
"use client"directive - Shadcn UI components with Tailwind CSS
- Form handling with react-hook-form
- Server actions return
ActionState<T>pattern - Business logic isolation
- Request ID tracking for tracing
- Comprehensive logging and error handling
- Database adapter (
/lib/db) - Authentication utilities (
/lib/auth) - AI provider factory (
/app/api/chat/lib) - AWS service clients (S3, CloudWatch)
- Settings management with caching
All server actions return a consistent response structure:
interface ActionState<T> {
isSuccess: boolean
data?: T
error?: {
code: string
message: string
details?: unknown
}
message?: string
}Unified interface for multiple AI providers:
createProviderModel(provider: string, modelId: string): Promise<LanguageModel>Every operation gets a unique request ID:
const requestId = generateRequestId()
const log = createLogger({ requestId, action: "actionName" })Database-first configuration with environment fallback:
// Check database → Fall back to env → Cache result
await getSetting('OPENAI_API_KEY')users- User accounts linked to Cognitoroles- Available roles (Admin, Staff)user_roles- User-role associationscapabilities- Role-gated UI feature registry (synced fromlib/capabilities/manifest.ts)role_capabilities- Role-capability associations
models- AI model configurationsconversations- Chat sessionsmessages- Chat messages with usage trackingtoken_usage- Token consumption tracking
repositories- GitHub repository metadatarepository_files- Indexed file contentdocuments- Uploaded documentsembeddings- Vector embeddings for search
assistant_architects- AI assistant configurationsassistant_tools- Tool assignmentsassistant_executions- Execution history
- User initiates sign-in via
/auth/signin - Redirected to Cognito hosted UI
- Google OAuth authentication
- Cognito returns authorization code
- NextAuth exchanges for JWT tokens
- Session stored in HTTP-only cookies
- Role Hierarchy: Admin → Staff
- Tool-Based Permissions: Granular feature access
- Session Validation: Server-side JWT verification
- CSRF Protection: State parameter validation
- SQL Injection: Parameterized queries only
- XSS Prevention: Input sanitization, CSP headers
- Secrets Management: AWS Secrets Manager
- PII Handling: Automatic log redaction
- Settings: 5-minute TTL cache
- Model Configs: In-memory caching
- S3 Client: Connection pooling
- Database: postgres.js connection pooling
- Chat Responses: SSE for real-time streaming
- File Processing: Chunked uploads for large files
- Assistant Execution: Progressive updates
- Route-based: Automatic with App Router
- Component-level: Dynamic imports for heavy components
- Library-level: Lazy loading for document processors
- JSON format in production
- Request ID correlation
- Performance metrics
- User context injection
{
"timestamp": "2025-08-20T10:00:00Z",
"level": "info",
"requestId": "abc123",
"userId": "user-456",
"action": "chat.completion",
"duration": 1234,
"tokens": 500
}- Typed error codes (60+ categories)
- Appropriate severity levels
- Stack traces in development
- User-friendly messages in production
All resources defined in AWS CDK:
/infra/
├── lib/
│ ├── stacks/
│ │ ├── auth-stack.ts # Cognito configuration
│ │ ├── database-stack.ts # Aurora Serverless v2
│ │ ├── frontend-stack.ts # ECS Fargate + ALB
│ │ ├── guardrails-stack.ts # Bedrock Guardrails + DynamoDB + SNS
│ │ └── storage-stack.ts # S3 buckets
│ └── constructs/
│ ├── security/ # IAM, service roles
│ ├── network/ # VPC patterns
│ ├── compute/ # Lambda, ECS
│ └── monitoring/ # CloudWatch, ADOT
└── database/
└── schema/ # SQL migrations
- Development: Feature branches, rapid iteration
- Staging: Integration testing, QA
- Production: Blue-green deployments
- Files 001-005: Initial schema (immutable)
- Files 010+: Incremental migrations
- Lambda-based automatic execution
- Transaction-wrapped for consistency
Assistant Architect supports external tool integration, enabling AI assistants to perform actions beyond text generation. Tools are executed within isolated environments and provide capabilities like web search and code execution.
- Provider: SerpAPI integration
- Models: GPT-5, Gemini Pro
- Capabilities: Real-time web search, current information retrieval
- Execution: Asynchronous with 15-second timeout
- Caching: Query-based caching with 5-minute TTL
- Runtime: Python 3.9+ in isolated sandbox
- Models: GPT-5, GPT-4o, Gemini Pro
- Libraries: NumPy, Pandas, Matplotlib, SciPy, Scikit-learn
- Execution: Stateless containers with 30-second timeout
- Security: No file system access, no network access
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ │ │ │ │ │
│ Assistant │────▶│ Tool Registry │────▶│ Model Capability│
│ Architect UI │ │ & Validation │ │ Matrix │
│ │ │ │ │ │
└─────────────────┘ └──────────────────┘ └─────────────────┘
│ │
▼ ▼
┌─────────────────┐ ┌──────────────────┐
│ │ │ │
│ Prompt Chain │────▶│ Tool Execution │
│ Configuration │ │ Lambda │
│ │ │ │
└─────────────────┘ └──────────────────┘
│
┌──────────┴──────────┐
▼ ▼
┌──────────────┐ ┌──────────────┐
│ │ │ │
│ Web Search │ │Code Interpreter│
│ Service │ │ Runtime │
│ │ │ │
└──────────────┘ └──────────────┘
-
Model Compatibility Check
interface ModelToolMatrix { [modelId: string]: { supportedTools: ToolType[] limitations: string[] performance: PerformanceMetrics } }
-
Tool Registry Lookup
async function getAvailableToolsForModel(modelId: string): Promise<Tool[]> { // Check model capabilities in ai_models table // Return intersection of model support and enabled tools // Apply user permission filtering }
-
Validation Pipeline
- Model compatibility verification
- User permission checks
- Tool configuration validation
- Security constraint enforcement
interface ToolExecution {
id: string
assistantArchitectId: number
promptId: string
enabledTools: string[]
status: 'pending' | 'running' | 'completed' | 'failed'
results: ToolResult[]
}- Queue: AWS SQS for reliable task distribution
- Workers: Lambda functions with model-specific configurations
- Coordination: Step Functions for complex workflows
- Monitoring: CloudWatch metrics and distributed tracing
interface ToolResult {
toolType: 'web_search' | 'code_interpreter'
status: 'success' | 'error' | 'timeout'
output: string
metadata: {
executionTime: number
resourceUsage: ResourceMetrics
cacheHit?: boolean
}
error?: {
code: string
message: string
details: unknown
}
}-- SECURITY NOTE: The following SQL examples are for documentation purposes only.
-- In production code, ALWAYS use Drizzle ORM type-safe queries
-- through executeQuery/executeTransaction to prevent SQL injection attacks.
-- Enhanced chain_prompts table
ALTER TABLE chain_prompts ADD COLUMN enabled_tools JSONB DEFAULT '[]';
ALTER TABLE chain_prompts ADD COLUMN tool_settings JSONB DEFAULT '{}';
-- Tool execution tracking
CREATE TABLE tool_executions (
id SERIAL PRIMARY KEY,
assistant_architect_id INTEGER REFERENCES assistant_architects(id),
user_id INTEGER REFERENCES users(id),
status VARCHAR(20) NOT NULL,
input_data JSONB NOT NULL,
started_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
completed_at TIMESTAMP WITH TIME ZONE,
error_message TEXT
);
-- Tool result storage
CREATE TABLE tool_results (
id SERIAL PRIMARY KEY,
execution_id INTEGER REFERENCES tool_executions(id),
tool_type VARCHAR(50) NOT NULL,
output_data TEXT,
metadata JSONB DEFAULT '{}',
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);-- SECURITY NOTE: The following SQL examples are for documentation purposes only.
-- In production code, ALWAYS use Drizzle ORM type-safe queries
-- through executeQuery/executeTransaction to prevent SQL injection attacks.
-- Enhanced ai_models table
ALTER TABLE ai_models ADD COLUMN capabilities JSONB DEFAULT '{}';
-- Example capabilities structure (use parameterized queries in actual implementation)
UPDATE ai_models SET capabilities = '{
"tools": ["web_search", "code_interpreter"],
"maxToolCalls": 5,
"parallelExecution": true,
"timeoutSeconds": 30
}' WHERE model_id = 'gpt-5';- Runtime: AWS Lambda with isolated execution contexts
- Container Features:
- Read-only root filesystem
- Non-root user execution (UID 1000)
- No privileged access or capabilities
- Isolated process namespace (PID isolation)
- Restricted file system access (no /tmp persistence)
- Security Context:
securityContext: runAsNonRoot: true runAsUser: 1000 readOnlyRootFilesystem: true allowPrivilegeEscalation: false capabilities: drop: ["ALL"]
- Code Interpreter: Complete network isolation - no outbound internet access
- Web Search Tool: Restricted to approved domains only via allowlist
- DNS Resolution: Limited to AWS internal DNS for security
- Firewall Rules:
- Block all outbound traffic except HTTPS to approved endpoints
- No inbound network access permitted
- VPC security groups with explicit deny-all default
- Memory Limits:
- Code Interpreter: 512MB maximum (configurable per model)
- Web Search: 256MB maximum
- Hard limits enforced at container level
- CPU Limits:
- Code Interpreter: 0.5 vCPU maximum with burst capability
- Web Search: 0.25 vCPU maximum
- Timeout enforcement: 30 seconds hard limit
- Disk I/O:
- Ephemeral storage only (no persistent volumes)
- 512MB maximum temporary space
- Automatic cleanup after execution
-
Dangerous Input Patterns Blocked:
// Command injection patterns /[;&|`$(){}[\]\\]/g // Shell metacharacters /\b(eval|exec|system)\b/gi // Dangerous functions /import\s+os|subprocess/gi // System module imports /__import__|getattr/gi // Dynamic imports // Path traversal patterns /\.\.[\/\\]/g // Directory traversal /\/etc\/|\/proc\/|\/dev\//gi // System directories // Network access patterns /socket|urllib|requests/gi // Network libraries /http[s]?:\/\//gi // URL patterns
-
Code Execution Restrictions:
- No file system write access outside
/tmp - Blocked system calls:
socket,fork,exec - Import restrictions:
os,subprocess,socket,urllib - Memory allocation limits to prevent resource exhaustion
- No file system write access outside
-
Input Sanitization:
- Maximum input size: 100KB for code, 10KB for search queries
- UTF-8 encoding validation
- SQL injection pattern detection
- XSS pattern filtering for all outputs
-
Real-time Monitoring:
- Resource usage tracking (CPU, memory, disk)
- Network connection attempts (blocked and logged)
- Suspicious pattern detection in code execution
- Failed execution attempt analysis
-
Security Alerting:
- Immediate alerts for blocked dangerous patterns
- Resource limit violations
- Repeated security violations by user
- Anomalous execution patterns
-
Audit Logging:
- All tool execution inputs and outputs
- Security violation attempts with user context
- Resource usage metrics for capacity planning
- Performance metrics for optimization
- Temporary Storage: Tool results stored with automatic cleanup
- Encryption: All tool data encrypted in transit and at rest
- Audit Logging: Comprehensive logging of tool usage and access
- Data Residency: Configurable data processing regions
interface ToolCache {
webSearch: {
keyPattern: string // hash of query + parameters
ttl: number // 5 minutes for search results
maxSize: number // 1000 entries per instance
}
codeExecution: {
keyPattern: string // hash of code + inputs
ttl: number // 1 hour for deterministic code
maxSize: number // 100 entries per instance
}
}- Concurrent Execution: Max 10 tools per user simultaneously
- Memory Allocation: Dynamic scaling based on tool complexity
- CPU Throttling: Intelligent resource allocation
- Timeout Handling: Graceful degradation with partial results
interface ToolMetrics {
executionTime: {
p50: number // Target: <15s
p95: number // Target: <30s
p99: number // Target: <45s
}
successRate: number // Target: >95%
errorRate: {
timeout: number // Target: <2%
validation: number // Target: <1%
system: number // Target: <1%
}
resourceUtilization: {
memory: number
cpu: number
network: number
}
}- Custom metrics for tool execution performance
- Automated alerting for failure rate thresholds
- Dashboard with real-time tool usage statistics
- Log aggregation for debugging and optimization
- Tool Timeout: Graceful degradation with partial results
- API Rate Limiting: Exponential backoff with queue management
- Resource Exhaustion: Load balancing and capacity scaling
- Validation Failures: Clear error messages and retry options
interface RecoveryPolicy {
retryAttempts: number // Max 3 attempts
backoffStrategy: 'exponential' | 'linear'
fallbackOptions: string[] // Alternative tools or cached results
userNotification: boolean // Inform user of degraded service
}The AI Studio streaming infrastructure has evolved through two major architectural migrations to optimize performance, cost, and user experience.
Timeline: January 2025 Problem: AWS Amplify does not support HTTP streaming, causing buffered responses and poor UX Solution: Migrated to ECS Fargate with Application Load Balancer
Key Changes:
- Enabled HTTP/2 streaming via ECS Fargate containers
- Eliminated 30-second Amplify timeout limitation
- Achieved real-time AI response streaming
- Reduced time-to-first-token from 2-5 seconds to <1 second
Impact:
- ✅ Real-time streaming responses (like ChatGPT/Claude)
- ✅ No timeout limits for long-running AI models
- ✅ Professional user experience
- ✅ Full control over containerized deployment
See: ADR-002: Streaming Architecture Migration
Timeline: October 2025 (PR #340) Problem: Redundant Lambda + SQS polling architecture adding latency and cost Solution: Direct ECS execution for all AI streaming
Key Changes:
- Removed streaming Lambda workers and SQS queues
- Direct ECS execution via HTTP/2 streaming
- Simplified architecture (fewer moving parts)
- Eliminated 1-5 second SQS polling delay
Cost Savings:
- Lambda streaming workers: -$20-30/month
- SQS requests: -$5-10/month
- Total savings: ~$35-40/month (~40% reduction)
Performance Improvements:
- 1-5 second latency reduction (no SQS polling)
- Eliminated Lambda cold starts for streaming
- Faster job execution start time
- More consistent response times
Architecture Simplification:
- Single execution path (ECS only)
- Fewer infrastructure components to maintain
- Unified monitoring and logging
- Easier debugging and troubleshooting
Components Retained:
- SQS + Lambda for background processing:
- Document processing (
file-processor) - Textract processing (
textract-processor) - URL processing (
url-processor) - Embedding generation (
embedding-generator)
- Document processing (
- These remain appropriate for asynchronous, non-streaming tasks
See: ADR-003: ECS Streaming Migration
┌──────────────┐
│ Client │
└──────┬───────┘
│ HTTP/2 Streaming
▼
┌──────────────┐
│ CloudFront │ (Optional CDN)
└──────┬───────┘
│
▼
┌──────────────┐
│ ALB │ ← Chunked Transfer Encoding
└──────┬───────┘ ← Server-Sent Events (SSE)
│
▼
┌──────────────┐
│ ECS Fargate │ ← Real-time AI streaming
│ (Next.js) │ ← No timeout limits
└──────┬───────┘
│
├─────────────┐
▼ ▼
┌──────────┐ ┌─────────────┐
│ Aurora │ │ Lambda │
│ServerlessV2│ │ Workers │
│ │ │ (Background)│
└──────────┘ └─────────────┘
Features:
- Nexus Chat: Real-time streaming conversations
- Model Compare: Side-by-side model comparison with streaming
- Assistant Architect: Multi-prompt tool execution
- All Features: Unlimited response times, progressive rendering
Benefits Summary:
| Metric | Before (Amplify + Lambda) | After (ECS Direct) | Improvement |
|---|---|---|---|
| Time-to-first-token | 2-5 seconds | <1 second | 75-90% faster |
| Streaming latency | 1-5 seconds (polling) | Real-time (<100ms) | Instant |
| Monthly cost | ~$100-140 | ~$60-100 | ~$40 savings |
| Infrastructure components | 8 services | 4 services | 50% reduction |
| Timeout limit | 30 seconds (Amplify) | None (ECS) | Unlimited |
Migration Impact:
- Zero user-facing changes (seamless migration)
- Better performance and lower costs
- Simpler architecture for maintenance
- All features continue to work as expected
REST API for external integrations, providing programmatic access to assistants, decisions, and chat. Authenticated via API key (sk- prefix), OAuth JWT, or session cookie. Rate-limited at 60 requests/minute by default.
- OpenAPI spec:
docs/API/v1/openapi.yaml - Base path:
/api/v1/ - Auth: Bearer token (API key or OAuth access token)
JWT-based authorization for external applications using Authorization Code Flow with PKCE.
- Endpoints:
/api/oauth/authorize,/api/oauth/token,/api/oauth/userinfo - Token lifetimes: Access (15min), Refresh (24hr), ID token (15min)
- Scopes:
openid,profile,email,mcp:read,mcp:write,api:read,api:write - Admin UI:
/admin/oauth-clientsfor client registration
Model Context Protocol server exposing AI Studio capabilities as tools for external AI agents.
- Tools: search_decisions, capture_decision, list_assistants, execute_assistant, get_context
- Transport: HTTP with SSE streaming
- Auth: API key or OAuth access token
Structured decision capture and retrieval system for organizational knowledge management.
- Capture: Decisions with context, alternatives, criteria, and outcomes
- Graph: Relationship-based navigation between related decisions
- Search: Full-text and semantic search across decision history
Amazon Bedrock Guardrails integration for content filtering, with narrowly scoped Comprehend-backed detect-only PII checks for Nexus memory and published content.
- Stack:
GuardrailsStack(Bedrock + SNS) - Features: Content filtering, topic blocking, detect-only PII gates, real-time alerts
- Inference contract: Allowed request and response text is never rewritten
- Multi-modal support (images, audio)
- WebSocket support for real-time collaboration
- Edge runtime optimization
- Distributed caching with Redis
- Horizontal scaling with container orchestration
/app → Pages and API routes
/actions → Server-side business logic
/components → Reusable UI components
/lib → Shared utilities and adapters
/types → TypeScript definitions
/infra → AWS CDK infrastructure
- Files: kebab-case (
user-actions.ts) - Components: PascalCase matching filename
- Server Actions: camelCase with
Actionsuffix - Database: snake_case for tables/columns
- Zero TypeScript errors (
npm run typecheck) - Zero ESLint violations (
npm run lint) - Comprehensive logging (no console methods)
- E2E tests for new features
- 80% code coverage minimum
- Next.js Documentation
- AWS CDK Guide
- Vercel AI SDK
- Internal CLAUDE.md - AI assistant guidelines