Python Agentic AI Chatbot

A starter template for building agentic AI applications with multi-step reasoning, tool calling, and RAG capabilities.

---

📋 Table of Contents

1. Overview
2. Architecture
3. Key Features
4. Project Structure
5. Technology Stack
6. Getting Started
7. Configuration
8. Architecture Decisions
9. Extending the Project
10. License

---

Overview

This project is a modular, extensible framework for building agentic AI applications in Python. It follows industry best practices inspired by frameworks like Microsoft Semantic Kernel, LangChain, and LlamaIndex.

Core Capabilities

• Agentic Reasoning: Multi-step task planning and execution
• Tool Calling: Automatic tool invocation based on agent decisions
• RAG (Retrieval-Augmented Generation): Semantic search over documents
• Flexible LLM Integration: Support for OpenAI, GitHub Models, Gemini
• Streaming API: Real-time responses via FastAPI
• Chat UI: React frontend with WebSocket support

---

Architecture

System Architecture Diagram

User (React Frontend)
  ↓
API Gateway (FastAPI)
  ↓
Orchestrator (Task Router)
  ↓
Agent (Reasoning Engine)
  ↓
Tool Manager (Search, Math, FileReader, MCP)
  ↓
Memory Layer (Vector DB, RAG)
  ↓
LLM Provider (OpenAI / GitHub / Gemini)
  ↓
Response Stream → React UI

Component Overview

Component	Purpose	Technologies
Frontend	User interaction & chat UI	React, TypeScript
API	HTTP/WebSocket server	FastAPI, Uvicorn
Agent	Decision making & planning	Python async, reasoning logic
Tools	Extensible capabilities	Search, Math, File I/O, MCP
Memory	Context & semantic search	ChromaDB, Vector embeddings
LLM	Language model calls	OpenAI, GitHub Models, Gemini

---

Key Features

✅ Ask anything - Natural language queries with full context
✅ Multi-step reasoning - Agent plans and executes complex tasks
✅ Tool calling - Automatic tool selection and invocation
✅ Search capabilities - Web search, document search, code search
✅ RAG integration - Semantic search over your documents
✅ Streaming responses - Real-time output via WebSocket
✅ Modular design - Easy to add new tools, providers, memory backends
✅ Production-ready - Docker support, structured logging, error handling

---

Project Structure

ntg-python-agent/
│
├── src/
│   ├── agent/
│   │   ├── __init__.py
│   │   ├── runner.py          # Agent loop
│   │   ├── planner.py         # Reasoning / planning
│   │   └── context.py         # Agent state
│   │
│   ├── llm/
│   │   ├── __init__.py
│   │   ├── provider.py        # OpenAI / GitHub / Gemini wrapper
│   │   └── models.py
│   │
│   ├── tools/
│   │   ├── __init__.py
│   │   ├── search.py
│   │   ├── math.py
│   │   └── tool_registry.py
│   │
│   ├── memory/
│   │   ├── __init__.py
│   │   ├── vector_store.py
│   │   └── embedder.py
|   |
│   └── server/
│      ├── main.py            # FastAPI application
│      └── routers/           # API endpoints (future)
│
├── tests/                      # Unit & integration tests
├── pyproject.toml             # Python project configuration
├── .env                       # Environment variables
├── .gitignore
├── LICENSE
└── README.md

---

Technology Stack

Layer	Technology	Version
Runtime	Python	3.12+
API Framework	FastAPI	Latest
Async	asyncio	Built-in
LLM Integration	OpenAI SDK	Latest
Environment	python-dotenv	Latest
Frontend	React	18+
Containerization	Docker	Latest

---

Getting Started

Prerequisites

• Python 3.12 or higher (required by project configuration)
• pip, uv package manager
• API keys for LLM providers (OpenAI, etc.)
• Optional: Docker for containerized deployment

Installation

Option 1: Standard pip installation

1. Clone the repository

   git clone https://github.com/nashtech-garage/ntg-python-agent.git
   cd ntg-python-agent

2. Create virtual environment

   python -m venv .venv
   source .venv/bin/activate  # on Linux/macOS
   .venv\Scripts\activate # On Windows

3. Install pre-commit in venv

   python -m pip install pre-commit
   pre-commit --version # Check version

4. Install base dependencies

   pip install -e .

   This installs:

   • openai>=1.60.0 - OpenAI API client
   • fastapi>=0.115.0 - Web framework
   • uvicorn>=0.30.0 - ASGI server
   • httpx>=0.27.0 - HTTP client
   • python-dotenv>=1.0.1 - Environment variables
   • pydantic>=2.7.0 - Data validation

5. Install optional dependencies (as needed)

   For RAG/Memory features:

   pip install -e ".[rag]"

   Includes: faiss-cpu, sentence-transformers, chromadb

   For development:

   pip install -e ".[dev]"

   Includes: pre-commit, ruff, pytest

6. Configure environment variables: Copy and paste all from .env.example to .env file

7. Run the server

   uvicorn src.server.main:app --reload

   Server runs at: http://localhost:8000

Option 2: Using uv (faster, recommended)

# Install uv (one-time)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Clone and setup
git clone https://github.com/nashtech-garage/ntg-python-agent.git
cd ntg-python-agent

# Sync dependencies
uv sync --all-groups

# Run
uv run uvicorn src.server.main:app --reload

---

Configuration

Environment Variables

Create a .env file in the project root with the following variables:

# Required: your OpenAI API key
OPENAI_API_KEY=sk-proj-xxx

How to obtain an OpenAI API key

Follow these steps to create and obtain an OpenAI API key (secret):

1. Create an account or sign in at the OpenAI platform:
   • https://platform.openai.com/signup (or https://platform.openai.com/login)
2. Verify your email address and finish any required account setup.
3. Open the API keys page:
   • https://platform.openai.com/account/api-keys
4. Click Create new secret key (or similar). Copy the generated key immediately — you won't be able to see it again in full.
5. Paste the copied key into your project's .env file as the value of OPENAI_API_KEY.

Dependency Groups

The project defines dependencies in pyproject.toml:

Core Dependencies (always installed):

• openai>=1.60.0 - OpenAI API client for LLM calls
• fastapi>=0.115.0 - Modern web framework with async support
• uvicorn>=0.30.0 - ASGI web server
• httpx>=0.27.0 - Async HTTP client
• python-dotenv>=1.0.1 - Load environment variables from .env
• pydantic>=2.7.0 - Data validation and settings management

Optional: RAG Group - For retrieval-augmented generation:

pip install -e ".[rag]"

• faiss-cpu>=1.8.0 - Vector search library
• sentence-transformers>=3.0.0 - Embedding models
• chromadb>=0.5.0 - Vector database

Optional: Dev Group - For development and testing:

pip install -e ".[dev]"

• pre-commit>=4.5.0 - Git hooks framework
• ruff>=0.14.8 - Python linter and formatter
• pytest>=8.3.0 - Testing framework

Project Metadata

• Name: ntg-python-agent
• Version: 0.1.0
• Python Version: >=3.12 (required)
• License: MIT
• Repository: github.com/nashtech-garage/ntg-python-agent

---

Architecture Decisions

This section documents key architectural decisions made in this project.

ADR-001: Modular LLM Provider System

Status: Accepted

Context: Supporting multiple LLM providers (OpenAI, GitHub Models, Gemini) requires flexibility.

Decision: Implement provider abstraction layer with pluggable implementations.

Rationale:

• Reduces vendor lock-in
• Allows easy switching between providers
• Supports cost optimization (choose cheapest provider per task)
• Enables A/B testing different models

Consequences:

• ✅ Easy to add new providers
• ✅ Consistent interface across providers
• ⚠️ Need to handle provider-specific features gracefully

Implementation: /src/agent/llm.py - Base class for all LLM providers

---

ADR-002: Async-First Architecture

Status: Accepted

Context: Agent operations are I/O bound (API calls, DB queries, tool invocations).

Decision: Use Python asyncio throughout the codebase.

Rationale:

• Better resource utilization (non-blocking I/O)
• Natural fit for FastAPI
• Handles concurrent requests efficiently
• Enables real-time streaming responses

Consequences:

• ✅ High concurrency support
• ✅ Responsive APIs
• ⚠️ Requires careful handling of blocking operations

Implementation: All agent methods are async def, FastAPI handles async endpoints

---

ADR-003: Vector DB for Memory & RAG

Status: Planned

Context: Need efficient semantic search over documents for context retrieval.

Decision: Use ChromaDB as primary vector database with embedding service.

Rationale:

• Lightweight, in-process option for development
• Production options available (Pinecone, Weaviate, Milvus)
• Fast approximate nearest neighbor search
• Supports metadata filtering

Consequences:

• ✅ Fast semantic search
• ✅ Flexible storage options
• ⚠️ Requires embedding model selection

---

📄 License

MIT License