Search Console Agent

An automated SEO reporting system that generates daily insights from Google Search Console data using AI-powered analysis.

Deployed on AWS infrastructure with automated daily reports delivered via email.

🚀 Live Demo

• Application: https://main.d3vozze6u0rukp.amplifyapp.com/
• Repository: https://github.com/muhmdusman/seo-agent

☁️ AWS Architecture

This application is fully deployed on AWS using:

• AWS Amplify - Frontend hosting with CI/CD
• AWS Elastic Beanstalk - Backend API (FastAPI)
• AWS RDS PostgreSQL - Database
• AWS Lambda - Automated report generation
• AWS EventBridge - Daily scheduling (8 AM UTC)
• AWS CloudWatch - Logging and monitoring

Tech Stack

🏗️ How It Works

The system automates SEO monitoring by:

1. Authentication - Users sign in with Google OAuth to access Search Console data
2. Site Selection - Choose from verified Search Console properties
3. Data Collection - Fetch 30-day performance metrics automatically
4. AI Analysis - Generate insights using Mistral AI
5. Report Delivery - Email reports daily at 8 AM UTC via AWS Lambda
6. Continuous Monitoring - AWS EventBridge ensures reports run every day

Architecture Diagram

User Browser
    ↓
AWS Amplify (Frontend)
    ↓
AWS Elastic Beanstalk (Backend API)
    ↓
AWS RDS PostgreSQL (Database)

AWS EventBridge (Daily 8 AM UTC)
    ↓
AWS Lambda (Report Generation)
    ↓
Google Search Console API → Mistral AI → SMTP Email

✨ Features

• 🔐 Google OAuth 2.0 authentication with webmasters.readonly scope
• 📊 Google Search Console integration for performance data
• 🤖 AI-powered insights using Mistral API
• 📧 Automated email delivery via SMTP
• ⏰ Daily automated reports via AWS Lambda + EventBridge
• 🌐 Multi-site support - manage multiple Search Console properties
• 📱 Responsive web interface built with Next.js
• ☁️ Fully cloud-deployed on AWS infrastructure

Prerequisites

Requirement	Version	Notes
Python	3.11+	Pinned in backend/.python-version
uv	latest	Dependency and venv management
Node.js	20+	Next.js 16
Docker + Compose	latest	Local PostgreSQL
Google Cloud project	—	OAuth client + Search Console API
Mistral API key	—	Generates the recommendations

Google Cloud setup

Do this first. Most setup failures happen here, not in the code.

1. Create or select a project in the Google Cloud Console.
2. Enable Google Search Console API under APIs & Services → Library.
3. Under Google Auth Platform → Clients, create an OAuth client ID of type Web application and add this authorized redirect URI:

   http://localhost:8000/api/v1/auth/google/callback
   

4. Under Google Auth Platform → Audience, set User type to External and register the scopes openid, email, and .../auth/webmasters.readonly.
5. Add your own Google account under Test users.

Important

webmasters.readonly is a sensitive scope. While the app sits in Testing, only accounts on the test-user list can complete the flow, and everyone else gets Error 403: access_denied before the consent screen appears. Expect an "unverified app" warning on first sign-in; continue via Advanced.

Configuration

Create a .env in the repository root:

APP_NAME="Search Console Agent"
DEBUG=true
APP_URL="http://localhost:8000"
FRONTEND_URL="http://localhost:3000"

DATABASE_URL="postgresql+psycopg://postgres:postgres@localhost:5433/app_db"

GOOGLE_CLIENT_ID="your-client-id.apps.googleusercontent.com"
GOOGLE_CLIENT_SECRET="your-client-secret"
GOOGLE_REDIRECT_URI="http://localhost:8000/api/v1/auth/google/callback"

JWT_SECRET="generate-a-long-random-string"
JWT_ALGORITHM="HS256"
ACCESS_TOKEN_EXPIRE_MINUTES=60
REFRESH_TOKEN_EXPIRY_DAYS=7

MISTRAL_API_KEY="your-mistral-api-key"

# SMTP Configuration (for email reports)
SMTP_HOST="smtp.gmail.com"
SMTP_PORT=587
SMTP_USER="your-email@gmail.com"
SMTP_PASSWORD="your-app-specific-password"
SMTP_FROM_EMAIL="your-email@gmail.com"
SMTP_FROM_NAME="Search Console Agent"

Variable	Purpose
APP_NAME	FastAPI application title shown in the OpenAPI docs
DEBUG	FastAPI debug mode and SQLAlchemy statement echo
APP_URL	Declared in settings; currently unused by request handling
FRONTEND_URL	Post-OAuth redirect target and the only allowed CORS origin
DATABASE_URL	Async SQLAlchemy connection string; also used by Alembic
GOOGLE_CLIENT_ID / GOOGLE_CLIENT_SECRET	Identify and authenticate the application to Google. One pair serves every user; per-user tokens live in the database
GOOGLE_REDIRECT_URI	Must match the Google Cloud registration byte for byte
JWT_SECRET / JWT_ALGORITHM	Sign and verify this app's own access and refresh tokens
ACCESS_TOKEN_EXPIRE_MINUTES / REFRESH_TOKEN_EXPIRY_DAYS	Token lifetimes
MISTRAL_API_KEY	Authenticates ChatMistralAI for the analysis call
SMTP_HOST / SMTP_PORT	SMTP server connection details for sending email reports
SMTP_USER / SMTP_PASSWORD	SMTP authentication credentials (use app-specific password for Gmail)
SMTP_FROM_EMAIL / SMTP_FROM_NAME	Email sender information for automated reports

The frontend reads one browser-visible value from frontend/.env.local:

NEXT_PUBLIC_API_BASE_URL=http://localhost:8000/api/v1

Setup and run

Quick Start (Recommended)

Start everything with a single command:

./dev.sh

This starts database + backend + frontend. Press Ctrl+C to stop all services.

Individual Services

Backend only:

cd backend
./start.sh

The start.sh script will:

• Check for .env file and create a symlink to it
• Start the PostgreSQL database using Docker Compose
• Wait for the database to become healthy
• Run database migrations if needed
• Start the FastAPI backend server with hot-reload

Press Ctrl+C to stop the backend server, then run:

./stop.sh

The stop.sh script gracefully stops the PostgreSQL database container.

Frontend only:

cd frontend
npm run dev

Quick Reference

Command	What it does
./dev.sh	Start everything (database + backend + frontend)
./backend/start.sh	Start PostgreSQL database + run migrations + start FastAPI backend
./backend/stop.sh	Stop PostgreSQL database container
npm run dev	Start frontend development server (from frontend folder)

Service URLs

• Frontend: http://localhost:3000
• Backend API: http://127.0.0.1:8000
• API Docs: http://127.0.0.1:8000/docs
• Adminer (DB): http://localhost:8081

Manual Setup (Advanced)

If you prefer to start services individually:

Backend:

cd backend && ln -sfn ../.env .env
uv sync
docker compose -f db/docker-compose.yaml up -d
uv run alembic upgrade head
uv run uvicorn main:app --host 127.0.0.1 --port 8000 --reload

Frontend:

cd frontend
npm install
npm run dev

Verify

curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:8000/openapi.json    # 200
curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:8000/api/v1/auth/me  # 401 without cookies

API reference

All routes are mounted under /api/v1.

Method	Path	Auth	Description
GET	/auth/google	—	302 redirect to Google's consent screen
GET	/auth/google/callback	—	Exchanges code, persists user/account/credential/session, sets cookies, redirects to the frontend
GET	/auth/me	Cookie	Returns the authenticated local user ID
POST	/auth/logout	—	Clears both auth cookies
GET	/search-console/sites	Cookie	Lists Search Console properties for the linked Google account
GET	/agent/weekly	—	SSE stream of analysis progress and the final recommendation

The weekly stream emits data: {"message": "..."} frames. Five literal strings are treated as progress (Getting Google credentials..., Fetching Search Console..., Scraping website..., Thinking..., Completed.); anything else is the final Markdown result.

Project structure

.
├── backend/
│   ├── main.py                  # FastAPI app, CORS, lifespan
│   ├── api/routes/              # auth, search_console, agents (SSE)
│   ├── agents/weekly_agent.py   # Analysis pipeline + prompt
│   ├── tools/                   # LangChain tool wrappers
│   ├── services/                # OAuth, JWT, Search Console, scraper
│   ├── models/                  # User, OAuthAccount, OAuthCredential, Session
│   ├── dependencies/auth.py     # Cookie/JWT authentication
│   ├── db/                      # Async engine + docker-compose
│   └── alembic/                 # Migration history
└── frontend/
    ├── src/app/                 # App Router: landing, callback, dashboard
    ├── src/components/          # Site selector, analysis display, UI primitives
    └── src/lib/                 # API client, auth helpers, config, types

Data model

users 1 ── * oauth_accounts 1 ── 1 oauth_credentials
  └── * sessions

oauth_credentials holds the Google access token, refresh token, and expiry for each linked account. sessions holds this application's own session state: a hashed refresh token, expiry, and revocation flag. The two are deliberately separate.

Known limitations

This is a working project, not a hardened production deployment. Contributions welcome on any of these:

• GET /agent/weekly takes user_id as a query parameter and does not apply the cookie authentication dependency, so it neither verifies the caller nor checks that the requested site belongs to them.
• Auth cookies are set with secure=False, which is fine over local HTTP but unsuitable for an HTTPS deployment.
• Google access and refresh tokens are stored as plain columns rather than encrypted at rest.
• The SSE protocol is untyped: progress versus result is decided by literal string matching, so changing backend wording changes UI classification.
• The Alembic revision graph drops and recreates sessions across revisions; reconcile against a deployed schema before upgrading an existing database.

Contributing

Issues and pull requests are welcome.

1. Fork the repository and create a branch from main.
2. Keep the layering intact: HTTP in api/, business and provider logic in services/, agent-callable adapters in tools/, workflow orchestration in agents/.
3. Run uv run alembic revision --autogenerate -m "..." for any model change.
4. Verify with npx tsc --noEmit and npm run lint in frontend/.
5. Open a PR describing the change and how you tested it.

Authors

Built by Muhammad Usman and Muhaddis.

License

Released under the MIT License.

Topics

ai-seo-agent seo seo-tools google-search-console ai-agent llm langchain mistral-ai fastapi nextjs react typescript python postgresql sqlalchemy oauth2 server-sent-events tailwindcss seo-automation search-console-api