JobFlow API

Backend API for automated job discovery, tracking, email notifications, and data export.

Overview

JobFlow API is a FastAPI-based backend application that aggregates job listings from multiple sources, stores them in PostgreSQL, and provides authenticated users with tools to search, save, organize, and export jobs.

The application includes a complete authentication system with email verification, background workers for automated scraping and notifications, database migrations using Alembic, and an admin module for maintenance tasks.

The project focuses on building a modular backend architecture by separating routing, business logic, database operations, background workers, and external services into dedicated components.

---

Features

Authentication

• User registration and login
• JWT authentication
• Email verification
• Password reset via email
• Password change
• Secure password hashing

Job Discovery

• Multi-source scraping — RemoteOK, Adzuna, Arbeitnow, Python.org Jobs
• Search by title, company, location, source, or keyword
• Sorting by any field (asc/desc)
• Pagination with metadata (page, total, has_next, has_previous)
• Automatic job expiry using valid_until and last_seen tracking

User Features

• Save jobs and track application status (saved → applied → interview → rejected)
• Personal dashboard — total jobs, saved, applied, interviews, rejected, added today
• Manage notification keywords
• Export saved jobs to PDF (ReportLab)
• Export saved jobs to CSV

Email Services (via Resend)

• Welcome email
• Email verification OTP
• Password reset OTP
• Job notification digest

Background Workers

• Scraper — runs every 10 minutes, fetches from all 4 sources, deduplicates by link, refreshes last_seen on existing jobs
• Notification worker — runs every 2 hours, matches new jobs against user keywords, sends digest email, marks jobs as processed
• Cleanup — runs daily, deletes expired unsaved jobs, marks expired saved jobs inactive, removes unverified accounts

Admin

• Manual scrape trigger
• Manual cleanup trigger
• System stats (total/active/inactive jobs, users, saved jobs, keywords, sources)

---

Tech Stack

Layer	Technology
Language	Python 3.13
Framework	FastAPI
Database	PostgreSQL
ORM	SQLAlchemy 2.0
Validation	Pydantic v2
Migrations	Alembic
Authentication	JWT (PyJWT) + Argon2
Email	Resend
Scraping	Requests + BeautifulSoup
PDF Reports	ReportLab
Server	Uvicorn

---

Project Structure

.
├── main.py               # App entry point + lifespan (background workers)
├── database.py           # PostgreSQL connection + session
├── models.py             # SQLAlchemy models (Jobs, User, SavedJobs, Keywords)
├── schemas.py            # Pydantic request/response schemas
├── enums.py              # Status, SortField, SortOrder enums
├── config.json           # Scraper keywords, countries, intervals
├── alembic/              # Database migrations
├── assets/               # Screenshots for README
├── routes/
│   ├── auth.py           # Register, login, verify, password reset
│   ├── jobs.py           # Browse and search jobs
│   ├── user.py           # Save jobs, dashboard, keywords, exports
│   └── admin.py          # Scrape, cleanup, stats
├── scraper/
│   ├── manager.py        # Orchestrates all scrapers + cleanup trigger
│   ├── adzuna.py
│   ├── remoteOK.py
│   ├── arbeitnow.py
│   ├── python_jobs.py
│   └── utils.py
├── security/
│   ├── jwt_handler.py    # Token creation + current user dependencies
│   └── password.py       # Hashing + OTP generation
└── services/
    ├── email_service.py         # All transactional emails via Resend
    ├── notification_service.py  # Keyword matching + notification logic
    ├── notification_worker.py   # Async notification loop
    ├── job_insert_update.py     # Upsert logic for scraped jobs
    ├── cleanup_service.py       # Expired job cleanup
    └── export_service.py        # PDF and CSV generation

---

API Endpoints

Authentication — /auth

Method	Endpoint	Description
POST	/auth/register	Register a new user
POST	/auth/login	Login and get JWT token
PATCH	/auth/password_change	Change password
POST	/auth/verify_email	Send email verification OTP
POST	/auth/verify_email/confirm	Confirm OTP and verify account
POST	/auth/forgot_password	Send password reset OTP
POST	/auth/reset_password	Reset password with OTP

Jobs — /jobs

Method	Endpoint	Description
GET	/jobs/	Browse jobs with filters, sorting, and pagination
GET	/jobs/{job_id}	Get job by ID

Query parameters for GET /jobs/:

• title, company, location, source — field-level filters
• keyword — searches across title, company, location, source
• sort_by — any job field
• sort_order — asc or desc
• page, limit — pagination (default: page 1, limit 20, max 100)

Users — /user

Method	Endpoint	Description
GET	/user/me	Get current user profile
GET	/user/dashboard	Personal dashboard stats
PATCH	/user/me/preferences	Update notification preferences
POST	/user/save/{job_id}	Save a job
GET	/user/	Get all saved jobs
GET	/user/{job_id}	Get saved job by ID
PATCH	/user/edit/{id}	Update saved job status or notes
DELETE	/user/remove/{id}	Remove a saved job
POST	/user/keywords	Add a notification keyword
GET	/user/keywords	Get all keywords
DELETE	/user/delete{id}	Delete a keyword
GET	/user/export_pdf	Export saved jobs as PDF
GET	/user/export_csv	Export saved jobs as CSV

Admin — /admin

Method	Endpoint	Description
POST	/admin/scrape	Manually trigger a scrape
POST	/admin/cleanup	Manually trigger cleanup
GET	/admin/stats	System-wide stats

---

Screenshots

Swagger UI

Authentication

Jobs API

User Endpoints

Admin Endpoints

Email Notification

PDF Export

CSV Export

---

Installation

1. Clone the repository

git clone https://github.com/qw3rty-dev/jobflow-api.git
cd jobflow-api

2. Install dependencies

pip install -r requirements.txt

3. Create your .env file

cp .env.example .env

Fill in your values:

APPLICATION_ID=your_adzuna_application_id
APPLICATION_KEY=your_adzuna_application_key
DATABASE_URL=postgresql://username:password@localhost:5432/jobflow_db
SECRET_KEY=your_super_secret_key
RESEND_API_KEY=re_your_resend_api_key
FROM_EMAIL=noreply@yourdomain.com

4. Run database migrations

alembic upgrade head

5. Start the server

uvicorn main:app --reload

Background workers (scraper, notification worker, cleanup) start automatically with the server via FastAPI's lifespan.

Open http://127.0.0.1:8000/docs for interactive API docs.

---

Configuration

config.json controls scraper behavior without touching code:

{
    "adzuna_keywords": ["backend", "frontend", "intern", "junior", ...],
    "countries": ["gb", "us", "in", "ca", "au", "de"],
    "results_per_page": 20,
    "notification_interval": 7200,
    "scraper_interval": 600
}

• scraper_interval — how often jobs are fetched (seconds, default 600 = 10 min)
• notification_interval — how often notification emails are sent (seconds, default 7200 = 2 hrs)

---

Future Improvements

• Docker support for simplified deployment
• Redis caching for frequently queried data
• Celery or RQ for robust background task queuing
• OAuth login with Google and GitHub
• Additional job sources (Wellfound, Greenhouse, Lever)
• AI-powered job recommendations based on saved jobs and user preferences
• Job analytics dashboard with application insights

---