Penguin - FastAPI ML Project Scaffolder

A production-ready Rust CLI tool that scaffolds modern, fully-functional FastAPI-based machine learning APIs with built-in templates, automatic dependency management, and smart project initialization.

Features

3 Built-in Templates

Minimal — Single model, simple request/response (ideal for single detectors)
Multi-model — Model registry pattern for managing multiple models
Async-heavy — Queue-based inference with background job processing (Redis-ready)

Framework Support

PyTorch (--framework torch)
ONNX Runtime (--framework onnx)
TensorFlow Lite (--framework tflite)
Automatic dependency injection based on framework selection

Smart Project Setup

Auto-creates Python virtual environment
Auto-installs all dependencies
Optional --no-setup flag to scaffold files only

Production-Ready Boilerplate

Pydantic models for type-safe request/response handling
FastAPI lifespan events for model loading/cleanup
Docker support (Dockerfile included)
Environment configuration (.env.example)
Comprehensive README in generated projects

Model Management

Add new models to existing projects with penguin add model
Each model gets its own inference stub and utilities
Template-based for consistency

Project Initialization

Initialize existing FastAPI projects with penguin init
Automatic venv + dependency setup
Validates project structure

Get Started Instantly

cargo install penguin-ml
penguin new my-detector --template minimal --framework torch
cd my-detector && source venv/bin/activate && python src/main.py

Visit http://localhost:8000/docs to see your API!

Installation

Option 1: From Crates.io (Recommended)

cargo install penguin-ml

Then verify installation:

penguin --help

Option 2: From Source

git clone https://github.com/yourusername/penguin
cd penguin
cargo install --path .

Option 3: Download Pre-built Binary

Download the latest release from GitHub Releases:

chmod +x penguin
mv penguin ~/.local/bin/  # or /usr/local/bin/

Quick Start

1. Create a New Project

# Interactive mode (prompts for template and framework)
penguin new my-detector

# With explicit options
penguin new my-detector --template minimal --framework torch

# Skip venv setup (manual setup later)
penguin new my-detector --template multi-model --framework onnx --no-setup

2. Start the API Server

cd my-detector
source venv/bin/activate
python src/main.py

Visit http://localhost:8000/docs to see interactive API documentation (Swagger UI).

3. Add Models to Your Project

penguin add model my_custom_detector

This creates:

src/models/my_custom_detector/
├── inference.py    # Model loading and inference logic
├── utils.py        # Preprocessing/postprocessing utilities
└── __init__.py

4. Initialize Existing Projects

If you have an existing FastAPI project and want to add venv + dependencies:

cd /path/to/existing/project
penguin init

Commands

`penguin new <project-name>`

Scaffold a new FastAPI ML project.

Options:

-t, --template <TEMPLATE> — Choose template: minimal, multi-model, async-heavy
-f, --framework <FRAMEWORK> — Choose framework: torch, onnx, tflite
--no-setup — Skip venv creation and dependency installation

Examples:

penguin new detector-api --template minimal --framework torch
penguin new multi-model-api --template multi-model --framework onnx --no-setup
penguin new queue-api --template async-heavy --framework tflite

`penguin add model <model-name>`

Add a new model to an existing penguin project.

Usage:

cd your-project
penguin add model yolo_v5
penguin add model faster_rcnn

Output:

src/models/<model-name>/inference.py — Model loading and prediction logic
src/models/<model-name>/utils.py — Helper functions
src/models/<model-name>/__init__.py — Package initialization

`penguin init`

Initialize venv and dependencies in an existing FastAPI project.

Usage:

cd existing-fastapi-project
penguin init

Requirements:

Project must contain src/app.py or src/main.py
requirements.txt must exist

Templates

Minimal Template

Best for single-model inference tasks.

API Endpoints:

GET / — Root endpoint
GET /api/health — Health check
POST /api/predict — Run inference

Structure:

src/
├── main.py              # Entry point
├── app.py               # FastAPI app factory
├── api/
│   ├── routes.py        # API endpoints
│   └── schemas.py       # Pydantic models
├── core/
│   └── config.py        # Settings & configuration
└── models/
    └── model_1/
        ├── inference.py # Model inference
        └── utils.py     # Utility functions

Multi-Model Template

Best for projects with multiple detectors/models.

API Endpoints:

GET /api/health — Health check
GET /api/models — List available models
POST /api/predict/{model_name} — Run inference with specific model

Key Components:

ModelRegistry — Central model management
/api/models endpoint shows registered models
Route-based model selection

Structure:

src/
├── main.py
├── app.py
├── api/
│   ├── routes.py
│   └── schemas.py
├── core/
│   ├── config.py
│   └── model_registry.py   # Model management
└── models/
    ├── detector_1/
    └── detector_2/

Async-Heavy Template

Best for long-running inference tasks (image processing, batch jobs, etc.).

API Endpoints:

GET /api/health — Health check
POST /api/submit-job — Submit async inference job
GET /api/job-status/{job_id} — Check job progress
GET /api/results/{job_id} — Retrieve job results (when ready)

For Production: Replace in-memory queue with Redis or RabbitMQ for persistence and multi-worker support.

Structure:

src/
├── main.py
├── app.py
├── api/
│   ├── routes.py
│   └── schemas.py
├── core/
│   ├── config.py
│   └── job_queue.py        # Queue management
└── workers/
    └── inference.py        # Background job processing

Framework Support

PyTorch (`--framework torch`)

Includes:

torch==2.11.0
torchvision==0.26.0
numpy==2.4.4

Example:

penguin new torch-detector --template minimal --framework torch

ONNX Runtime (`--framework onnx`)

Includes:

onnx==1.21.0
onnxruntime==1.24.4
numpy==2.4.4

Example:

penguin new onnx-api --template multi-model --framework onnx

TensorFlow Lite (`--framework tflite`)

Includes:

tensorflow==2.21.0
numpy==2.4.4

Example:

penguin new tflite-server --template async-heavy --framework tflite

Generated Project Structure

Each generated project follows this structure:

my-project/
├── src/
│   ├── main.py                 # Entry point
│   ├── app.py                  # FastAPI app factory
│   ├── api/
│   │   ├── __init__.py
│   │   ├── routes.py           # API endpoints
│   │   └── schemas.py          # Pydantic models
│   ├── core/
│   │   ├── __init__.py
│   │   └── config.py           # Settings (pydantic-settings)
│   └── models/
│       └── model_1/            # Your model(s)
│           ├── __init__.py
│           ├── inference.py    # Model class & predict()
│           └── utils.py        # Preprocessing/postprocessing
├── requirements.txt            # Python dependencies (framework-aware)
├── .env.example                # Environment template
├── Dockerfile                  # Production containerization
├── README.md                   # Project-specific docs
└── venv/                       # Virtual environment (created by penguin new)

Usage Examples

Example 1: Create a Single-Model Detection API

# Create minimal project with PyTorch
penguin new face-detector --template minimal --framework torch

cd face-detector
source venv/bin/activate

# Edit src/models/model_1/inference.py to load your face detection model
# Edit src/api/routes.py to handle face detection input/output
# Start server
python src/main.py

Visit http://localhost:8000/docs and test the /api/predict endpoint.

Example 2: Multi-Model Detection System

# Create multi-model project
penguin new detection-system --template multi-model --framework onnx

cd detection-system
source venv/bin/activate

# Add multiple detectors
penguin add model person_detector
penguin add model vehicle_detector
penguin add model pose_estimator

# Update src/core/config.py to register models
# Update each model's inference.py with actual ONNX logic
# Start server
python src/main.py

Now you can:

curl http://localhost:8000/api/models
# {"models": ["person_detector", "vehicle_detector", "pose_estimator"], "count": 3}

curl -X POST http://localhost:8000/api/predict/person_detector \
  -H "Content-Type: application/json" \
  -d '{"data": "base64-encoded-image"}'

Example 3: Async Batch Processing

# Create async-heavy project for long-running jobs
penguin new batch-processor --template async-heavy --framework tflite --no-setup

cd batch-processor
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Edit src/workers/inference.py for actual inference logic
python src/main.py

Usage:

# Submit a job
JOB=$(curl -X POST http://localhost:8000/api/submit-job \
  -H "Content-Type: application/json" \
  -d '{"data": "input"}' | jq -r .job_id)

# Poll for status
curl http://localhost:8000/api/job-status/$JOB

# Get results when complete
curl http://localhost:8000/api/results/$JOB

Configuration

Environment Variables

Each project includes a .env.example file:

cp .env.example .env
# Edit .env with your configuration

Common Variables:

HOST — Server host (default: 0.0.0.0)
PORT — Server port (default: 8000)
DEBUG — Enable debug mode (default: false)
RELOAD — Auto-reload on code changes (default: true)

See .env.example for template-specific variables.

Model Configuration

Edit src/core/config.py to customize model paths and settings:

class Settings(BaseSettings):
    model_name: str = "model_1"
    model_path: str = "models/model_1"  # Change this
    # ... other settings

Docker Deployment

Each generated project includes a Dockerfile:

# Build image
docker build -t my-detector .

# Run container
docker run -p 8000:8000 my-detector

# With environment variables
docker run -p 8000:8000 \
  -e PORT=8000 \
  -e RELOAD=false \
  my-detector

Troubleshooting

Issue: "Python 3 not found"

Solution:

# Install Python 3.8+
# macOS
brew install python@3.11

# Ubuntu/Debian
sudo apt-get install python3.11 python3.11-venv

# Then try again
penguin new my-project

Issue: `pip install` fails during project creation

Solution: Use --no-setup and install manually:

penguin new my-project --framework torch --no-setup
cd my-project
python -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

Issue: Model not loading in inference.py

Solution: Edit src/models/<model>/inference.py and implement the _load_model() method:

def _load_model(self):
    """Load your actual model here."""
    import torch
    self.model = torch.load("path/to/model.pth")

Issue: Virtual environment already exists

Solution: Remove and recreate:

rm -rf venv
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Development

Build from Source

git clone https://github.com/yourusername/penguin
cd penguin
cargo build --release

Binary will be at: target/release/penguin

Run Tests

cargo test

Publishing to Crates.io

The project is already published on crates.io:

# Install from crates.io
cargo install penguin-ml

# To publish updates:
# 1. Update version in Cargo.toml
# 2. cargo publish

Current version: v0.1.0 (penguin-ml on crates.io)

Contributing

Contributions welcome! Areas for improvement:

Remote template registry support
Additional framework templates (JAX, Hugging Face Transformers)
Advanced monitoring/metrics
WebSocket support for streaming inference
Multi-GPU deployment helpers

Architecture

Modules

scaffolder.rs — Template rendering via Tera, file generation
dependencies.rs — Framework-specific dependency mapping
python_env.rs — Virtual environment and pip integration
progress.rs — CLI progress spinners and feedback

All templates are compiled into the binary using include_str!() — zero external dependencies.

Performance & Binary Size

Binary size: ~10 MB (optimized)
Startup: < 100ms
Project scaffold: < 1 second
Dependency install: 1-5 minutes (network dependent)

License

MIT

Support

For issues, questions, or feature requests:

Open a GitHub issue
Check existing issues
Review generated project READMEs for template-specific docs

Roadmap

Remote template registry for custom templates
penguin doctor — Environment diagnostics
Hot-reload file watcher
Multiple Python version support
Pre-commit hooks scaffolding
CI/CD pipeline templates (GitHub Actions, GitLab CI)
Kubernetes deployment helpers
Monitoring integration (Prometheus, DataDog)

Happy building with Penguin!

Name		Name	Last commit message	Last commit date
Latest commit History 6 Commits
src		src
templates		templates
.gitignore		.gitignore
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
LICENSE		LICENSE
README.md		README.md

Folders and files

Latest commit

History

Repository files navigation

Penguin - FastAPI ML Project Scaffolder

Features

Get Started Instantly

Installation

Option 1: From Crates.io (Recommended)

Option 2: From Source

Option 3: Download Pre-built Binary

Quick Start

1. Create a New Project

2. Start the API Server

3. Add Models to Your Project

4. Initialize Existing Projects

Commands

penguin new <project-name>

penguin add model <model-name>

penguin init

Templates

Minimal Template

Multi-Model Template

Async-Heavy Template

Framework Support

PyTorch (--framework torch)

ONNX Runtime (--framework onnx)

TensorFlow Lite (--framework tflite)

Generated Project Structure

Usage Examples

Example 1: Create a Single-Model Detection API

Example 2: Multi-Model Detection System

Example 3: Async Batch Processing

Configuration

Environment Variables

Model Configuration

Docker Deployment

Troubleshooting

Issue: "Python 3 not found"

Issue: pip install fails during project creation

Issue: Model not loading in inference.py

Issue: Virtual environment already exists

Development

Build from Source

Run Tests

Publishing to Crates.io

Contributing

Architecture

Modules

Performance & Binary Size

License

Support

Roadmap

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 1

Contributors

Uh oh!

Languages

`penguin new <project-name>`

`penguin add model <model-name>`

`penguin init`

PyTorch (`--framework torch`)

ONNX Runtime (`--framework onnx`)

TensorFlow Lite (`--framework tflite`)

Issue: `pip install` fails during project creation