GitHub Repo Tracker

A Spring Boot service that tracks GitHub repositories, stores metadata and commit history in a persistent database, and polls GitHub for updates on a schedule.

Features

• Tracks a repository by owner/name and stores repository metadata
• Pulls commit history from the GitHub public API
• Stores commit data with a SHA-based primary key to prevent duplicates
• Polls repositories on a configurable schedule to keep commit data current
• Supports resetting the collection window to a custom start date
• Exposes query-friendly APIs for commit lookups and top authors

Tech stack

• Java 17
• Spring Boot 4.1.1
• Spring Data JPA
• PostgreSQL (runtime)
• H2 (tests)

Local setup

1. Start PostgreSQL locally (for example with Docker):

   docker run --name github_tracker_db -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=github_tracker -p 5432:5432 -d postgres:16

2. Configure the database environment if needed:

   export DB_USER=postgres
   export DB_PASSWORD=postgres

3. Optional: provide a GitHub token for a higher rate limit:

   export GITHUB_TOKEN=ghp_your_token_here

4. Run the application:

   ./mvnw spring-boot:run

Configuration

src/main/resources/application.properties includes the key runtime values:

spring.application.name=GitHubRepoTracker
spring.datasource.url=jdbc:postgresql://localhost:5432/github_tracker
spring.datasource.username=${DB_USER:postgres}
spring.datasource.password=${DB_PASSWORD:postgres}
spring.jpa.hibernate.ddl-auto=update
spring.jpa.open-in-view=false

github.base-url=https://api.github.com
github.token=${GITHUB_TOKEN:}
github.poll-interval-ms=3600000
github.default-since=2024-01-01T00:00:00Z
github.page-size=100
github.max-pages-per-sync=50
github.request-timeout-ms=30000
github.seed-repositories=chromium/chromium

API summary

• POST /api/repositories — track a repository
• GET /api/repositories — list tracked repositories
• GET /api/repositories/{owner}/{name} — get repository metadata
• POST /api/repositories/{owner}/{name}/sync — trigger a manual sync
• POST /api/repositories/{owner}/{name}/reset — reset commit collection and set since-date
• GET /api/repositories/{owner}/{name}/commits — fetch commits by owner and repo name
• GET /api/repositories/by-name/{name}/commits — fetch commits by repository name
• GET /api/repositories/{owner}/{name}/top-authors?limit=10 — get top authors by commit count

Example:

curl -X POST http://localhost:8080/api/repositories \
  -H 'Content-Type: application/json' \
  -d '{"owner":"chromium","name":"chromium"}'

curl 'http://localhost:8080/api/repositories/chromium/chromium/top-authors?limit=5'
curl 'http://localhost:8080/api/repositories/by-name/chromium/commits?page=0&size=10'

Efficient database queries

1) Top N commit authors by commit count

SQL:

SELECT author_name, COUNT(*) AS commit_count
FROM commit
WHERE repository_id = :repoId
GROUP BY author_name
ORDER BY commit_count DESC
LIMIT :n;

JPA repository query:

@Query("""
    select c.authorName as authorName, count(c) as commitCount
    from Commit c
    where c.repository.id = :repoId
    group by c.authorName
    order by count(c) desc
    """)
List<AuthorCommitCount> findTopAuthorsByRepositoryId(@Param("repoId") Long repoId, Pageable pageable);

2) Retrieve commits for a repository by repository name

SQL:

SELECT c.*
FROM commit c
JOIN tracked_repository r ON r.id = c.repository_id
WHERE r.name = :name
ORDER BY c.author_date DESC;

JPA repository query:

Page<Commit> findByRepository_NameOrderByAuthorDateDesc(String name, Pageable pageable);

Data model

tracked_repository

Stores repository metadata and sync settings.

Columns include:

• id
• owner
• name
• description
• url
• language
• forks_count
• stars_count
• open_issues_count
• watchers_count
• github_created_at
• github_updated_at
• since_date
• last_synced_at

commit

Stores one row per GitHub commit, keyed by SHA to avoid duplicates.

Columns include:

• sha
• repository_id
• message
• author_name
• author_email
• author_date
• url

Resetting collection to a point in time

To reset a repository and re-fetch commits from a chosen timestamp:

curl -X POST 'http://localhost:8080/api/repositories/chromium/chromium/reset' \
  -H 'Content-Type: application/json' \
  -d '{"sinceDate":"2026-01-01T00:00:00Z"}'

The code deletes existing commit rows for the repository, updates since_date, and syncs from that point forward.

Testing

Run the project test suite:

./mvnw test

This includes repository query tests and a sync-service unit test covering deduplication.

Notes

• The scheduler polls tracked repositories on a fixed delay and updates the last sync timestamp.
• Duplicate commit SHAs are ignored, which ensures the database mirrors GitHub without repeated rows.
• Seed repositories can be configured in github.seed-repositories.