A serverless, multi-tenant REST API that turns any CSV of postal codes into a queryable, secure, private database - deployed entirely on AWS with zero servers to manage.
Status: This project was built as a personal project, deployed live on AWS via RapidAPI, and has since been taken down. The infrastructure-as-code, application logic, and test suite in this repository are the complete, production-ready source.
Companies often maintain their own postal/pincode master data in spreadsheets - custom delivery zones, regional risk scores, serviceability flags, SLA tiers. When a backend team needs this data available as an API, the options are:
- Build and host your own lookup service (DynamoDB + Lambda + API Gateway plumbing, user isolation, quota management, etc.)
- Use a public postal code API that knows nothing about your business-specific columns.
PincodeDB is option 3: upload your CSV, get a private, queryable API back in seconds. Your data stays isolated. Nobody else can see or query it. You can also fall back to a global public Indian pincode dataset (~155K records) when your private data doesn't have a match.
The entire backend is 100% serverless on AWS, defined in a single template.yaml (AWS SAM / CloudFormation).
flowchart TB
Client["Client / RapidAPI Marketplace"]
subgraph API["API Gateway (REST + OpenAPI 3.0)"]
direction LR
Auth["Lambda Authorizer\n(JIT User Provisioning)"]
end
subgraph Lambdas["Lambda Functions (Python 3.13, ARM64)"]
direction LR
Health["Health"]
Lookup["Lookup"]
Batch["Batch Lookup"]
Strategy["Search Strategy"]
Upload["Upload"]
ListDS["List Datasets"]
Status["Get Status"]
Delete["Delete Dataset"]
end
subgraph Storage["Storage Layer"]
direction LR
DDB_Pincode[("DynamoDB\nPincodeTable\n(GSI: Pincode, Dataset)")]
DDB_Users[("DynamoDB\nPincodeUsers\n(GSI: Marketplace, Tier)")]
DDB_Datasets[("DynamoDB\nPincodeDatasets\n(GSI: Owner)")]
S3["S3 Upload Bucket\n(AES-256, Versioned)"]
end
subgraph Ingestion["Async Ingestion Pipeline"]
S3Proc["S3 Processor Lambda\n(CSV Parse + Validate + Ingest)"]
end
subgraph Ops["Observability & Operations"]
direction LR
CW["CloudWatch\nAlarms + Metrics"]
DLQ["SQS Dead-Letter\nQueues"]
SNS["SNS Ops Topic"]
SSM["SSM Parameter Store\n(Kill Switch + Secrets)"]
XRay["X-Ray Tracing"]
end
subgraph Scheduled["Scheduled Jobs"]
Reset["Reset Usage Lambda\n(Monthly Cron)"]
end
Client -->|"HTTPS"| API
Auth -->|"Allow/Deny + Context"| Lambdas
Auth -.->|"Read/Write"| DDB_Users
Auth -.->|"Read"| SSM
Lookup & Batch & Strategy -->|"Query"| DDB_Pincode
Upload -->|"Presigned POST"| S3
Upload -.->|"Read"| DDB_Users
ListDS & Status -->|"Query"| DDB_Datasets
Delete -->|"Delete"| DDB_Pincode
Delete -->|"Delete"| DDB_Datasets
Delete -.->|"Update"| DDB_Users
S3 -->|"S3 Event Trigger\n(ObjectCreated: *.csv)"| S3Proc
S3Proc -->|"Batch Write"| DDB_Pincode
S3Proc -->|"Update Status"| DDB_Datasets
S3Proc -.->|"Update Quotas"| DDB_Users
Reset -->|"Reset Counters"| DDB_Users
Lambdas -.->|"Metrics"| CW
S3Proc -.->|"Metrics"| CW
Lambdas -.->|"Failed Events"| DLQ
CW -.->|"Alarm"| SNS
Lambdas -.->|"Traces"| XRay
style Client fill:#f9f,stroke:#333,stroke-width:2px
style API fill:#ff9900,stroke:#333,stroke-width:2px,color:#000
style Storage fill:#3b48cc,stroke:#333,stroke-width:2px,color:#fff
style Ingestion fill:#2d8f2d,stroke:#333,stroke-width:2px,color:#fff
style Ops fill:#666,stroke:#333,stroke-width:2px,color:#fff
style Scheduled fill:#8B4513,stroke:#333,stroke-width:2px,color:#fff
sequenceDiagram
participant C as Client
participant AG as API Gateway
participant UL as Upload Lambda
participant S3 as S3 Bucket
participant SP as S3 Processor Lambda
participant DB as DynamoDB
C->>AG: POST /postal-code/upload
AG->>UL: Invoke (with auth context)
UL->>DB: Check user tier & quotas
UL-->>C: Presigned S3 POST URL + datasetId
C->>S3: Direct upload (multipart/form-data)
S3-->>C: HTTP 204 No Content
S3--)SP: S3 Event Trigger (ObjectCreated)
SP->>S3: Get CSV object
SP->>SP: Parse CSV, validate schema,<br/>check quotas
SP->>DB: Batch write pincode records
SP->>DB: Update dataset status = SUCCEEDED
SP->>DB: Update user quotas
loop Poll until complete
C->>AG: GET /dataset/{id}/status
AG->>DB: Read dataset record
DB-->>C: {status: PROCESSING | SUCCEEDED | FAILED}
end
| Service | Purpose |
|---|---|
| API Gateway | REST API with OpenAPI 3.0 spec, Lambda authorizer for auth |
| Lambda (Python 3.13, ARM64) | 10 functions - auth, lookup, batch lookup, search strategy, upload, S3 processor, delete, status, list datasets, usage reset |
| DynamoDB | 3 tables with 7 GSIs total for multi-access-pattern queries |
| S3 | Encrypted upload bucket with presigned POST URLs, lifecycle policies |
| SSM Parameter Store | Kill switch and proxy secret storage |
| SQS | Dead-letter queues for failed invocations (ingest DLQ + general DLQ) |
| CloudWatch | Alarms on Lambda errors and DLQ depth, embedded custom metrics |
| SNS | Ops alert topic wired to alarms |
| X-Ray | Distributed tracing enabled on all functions |
| EventBridge | Monthly cron to reset usage counters for paid tiers |
All postal code records share a single DynamoDB table (PincodeTable), partitioned by internalUserId. A user's queries can only ever hit their own partition. The table schema:
- Partition Key:
internalUserId(tenant identifier) - Sort Key:
pincode_officename(composite:{pincode}#{officeName}) - GSI PincodeIndex:
internalUserId+pincode- for "find this pincode across all my datasets" - GSI DatasetIndex:
datasetId- for "find all records in this specific dataset"
This means a single table serves all tenants with zero cross-tenant data leakage, while supporting three different query patterns efficiently.
There is no separate signup or onboarding flow. The Lambda authorizer handles everything:
- Validates the RapidAPI proxy secret (via SSM Parameter Store)
- Looks up the user by their RapidAPI marketplace ID
- If the user doesn't exist, creates them on the spot with the correct tier
- If the user exists but their subscription tier has changed (upgrade/downgrade), updates them immediately
- Returns an IAM policy + context (userId, tier, clientId) that downstream Lambdas consume
This eliminates an entire class of "user sync" problems between the marketplace and the backend.
Users never send CSV data through API Gateway (which has a 10MB payload limit and would waste Lambda execution time). Instead:
POST /postal-code/uploadreturns a presigned S3 POST with size limits enforced by the presigned policy (tied to the user's tier)- The client uploads directly to S3 (server-side encryption enforced)
- An S3 event trigger fires the S3 Processor Lambda, which parses the CSV, validates it, and writes to DynamoDB
- The user polls
GET /dataset/{id}/statusuntil processing completes
This keeps the upload path off the API Gateway entirely and lets S3 handle the heavy lifting.
Lower tiers (FREE, BUSINESS) require a structured CSV with known columns (pincode, country, area/officeName, district, state, etc.). The S3 processor normalizes headers (e.g., zip_code -> pincode, province -> stateName) and maps them to a canonical schema.
Higher tiers (GROWTH, ENTERPRISE) unlock dynamic schema: upload any CSV with a pincode column and arbitrary extra columns. The extra columns are stored as a customData map in DynamoDB and returned as-is in query responses. This means a logistics company can upload pincode,delivery_sla,risk_score,is_serviceable and query all of it back without any schema definition.
Defined in shared/tier.py and enforced across upload, ingestion, and query:
| Tier | RapidAPI Plan | Max File Size | Max Total Rows | Max Datasets | Dynamic Schema |
|---|---|---|---|---|---|
| FREE | BASIC | 50 KB | 250 | 1 | No |
| BUSINESS | PRO | 10 MB | 1,000,000 | 50 | No |
| GROWTH | ULTRA | 20 MB | 5,000,000 | 200 | Yes |
| ENTERPRISE | MEGA | 50 MB | 25,000,000 | 1,000 | Yes |
FREE tier users get exactly one upload (enforced via one_time_upload_used flag). Paid tier usage counters reset monthly via a scheduled Lambda.
- Proxy secret validation - Every request is verified against a secret shared with RapidAPI, stored in SSM Parameter Store (SecureString). Prevents direct API Gateway access bypassing the marketplace.
- Tenant isolation - All DynamoDB queries are scoped to
internalUserId. Ownership checks on dataset delete and status endpoints. - S3 hardening - Public access fully blocked, HTTPS-only (deny insecure transport), AES-256 server-side encryption enforced on every upload, versioning enabled,
DeleteObjectdenied on raw uploads via IAM policy. - Kill switch - An SSM parameter (
/pincode-api/enabled) that, when set tofalse, makes the authorizer deny all requests. Instant global shutoff without a deployment.
| Method | Path | Auth | Description |
|---|---|---|---|
GET |
/health |
Public | Health check |
GET |
/postal-code/search |
Required | Lookup a single pincode (all datasets or a specific one, with optional public fallback) |
POST |
/postal-code/batch-search |
Required | Batch lookup up to 100 pincodes in one call |
POST |
/postal-code/search-strategy |
Required (GROWTH+) | Chained search with custom priority order (specific dataset -> all private -> public) |
POST |
/postal-code/upload |
Required | Get a presigned URL to upload a CSV |
GET |
/datasets |
Required | List all your uploaded datasets |
GET |
/dataset/{datasetId}/status |
Required | Check processing status of a dataset |
DELETE |
/dataset/{datasetId} |
Required (BUSINESS+) | Delete a private dataset and its records |
The full OpenAPI 3.0 specification is embedded in template.yaml.
Every user-facing Lambda emits custom CloudWatch metrics via aws-embedded-metrics:
LatencyMs- end-to-end handler latencyRequestsOk/RequestsError- success/failure countersRowsIngested- rows written per ingestionRequestedPins- batch size per batch lookupSkippedTriggers- duplicate S3 events that were safely ignored
CloudWatch Alarms are configured for:
- Lookup Lambda errors (fires on any error within a 60s window)
- Ingest DLQ depth (fires if any message lands in the dead-letter queue)
Both route to an SNS topic for alerting.
pincode-db-api/
src/
auth/app.py # Lambda authorizer (JIT user provisioning + tier sync)
lookup/app.py # Single pincode lookup with fallback
batch_lookup/app.py # Batch lookup (up to 100 pincodes)
search_strategy/app.py # Custom priority chain search (premium)
upload/app.py # Presigned S3 POST URL generation
s3handler/app.py # S3 event processor (CSV parse, validate, ingest)
delete_dataset/app.py # Dataset deletion with quota adjustment
get_status/app.py # Dataset processing status check
list_datasets/app.py # List user's datasets
reset_usage/app.py # Monthly usage counter reset (scheduled)
health/app.py # Health check
shared/
tier.py # Tier definitions and limits
tests/
conftest.py # Shared fixtures (moto mock AWS, DynamoDB table setup)
test_*.py # Test suite (~40 tests covering all Lambdas)
template.yaml # SAM/CloudFormation template (entire infrastructure)
ingest_public_data.py # One-time script to load public Indian pincode data
Tests use moto to mock AWS services (DynamoDB, S3, SSM) in-process. No real AWS calls are made during testing.
# Setup
python -m venv .venv
source .venv/bin/activate # or .\.venv\Scripts\Activate.ps1 on Windows
pip install -r requirements.txt
pip install -r tests/requirements-dev.txt
# Run tests with coverage
pytestCoverage spans all Lambda handlers including happy paths, authorization failures, quota enforcement, encoding errors, race condition handling, and pagination.
CI runs on every push via GitHub Actions: lint (ruff) -> format check (black) -> tests + coverage -> SAM validate.
Note: This project was previously deployed and has since been taken down. These instructions are preserved for reference.
# Build
sam build
# Deploy (interactive first time)
sam deploy --guidedThe SAM template creates all resources (API Gateway, 10 Lambdas, 3 DynamoDB tables, S3 bucket, SQS queues, CloudWatch alarms, SNS topic, SSM parameters) in a single stack. DynamoDB tables and the S3 bucket have DeletionPolicy: Retain to prevent accidental data loss on stack deletion.