overleaf-to-git
===============

Archive an Overleaf project's *full edit history* into a local Git repository,
one commit per Overleaf revision, preserving authors, co-authors, and the
original timestamps. Overleaf offers no supported way to export this history,
so the tool reconstructs it from the same JSON endpoints the "History" tab uses
in a browser.

It works against overleaf.com and against a self-hosted Overleaf/ShareLaTeX
instance (set --base-url), which makes it handy when migrating projects to your
own server.


Install
-------

The project requires Python 3.10 or newer and is managed with `uv`
(https://docs.astral.sh/uv/):

    git clone https://github.com/zambonin/overleaf-to-git.git
    cd overleaf-to-git
    uv sync                      # core install
    uv sync --extra browser      # also enable --from-browser cookie import

Run it as a module or via the installed script:

    uv run overleaf-to-git --help
    uv run python -m overleaf_to_git --help


Authentication
--------------

No password is sent and no browser cookie database is read by default. Copy your
session cookie once from your browser's developer tools (Application -> Cookies),
the value named `overleaf_session2` on overleaf.com, and pass it in:

    export OVERLEAF_SESSION='s%3A...'        # the cookie value
    uv run overleaf-to-git --list

Alternatives:

  * `--from-browser firefox` imports cookies automatically (needs the optional
    `browser_cookie3` dependency; install with `uv sync --extra browser`).
  * Self-hosted instances usually name the cookie `sharelatex.sid`; pass
    `--base-url https://overleaf.example.org --cookie-name sharelatex.sid`.


Usage
-----

    # list projects (newest first), their owners and their dashboard tags
    uv run overleaf-to-git --list

    # archive projects 1 to 4 and 7, non-interactively
    uv run overleaf-to-git --projects "1-4 7"

    # archive exact projects reproducibly (repeat for several projects)
    uv run overleaf-to-git --project-id 59d3b117fea3c15129b60c74

    # archive every project carrying a dashboard tag (repeatable)
    uv run overleaf-to-git --tag '~published'

    # ...and file the repositories under a directory per tag
    uv run overleaf-to-git --tag '~published' --group-by-tag

    # omit --projects and --tag to pick interactively from the printed list

Each chosen project becomes a Git repository under --output-dir, which defaults
to `~/.local/share/overleaf-to-git/` ($XDG_DATA_HOME). The repositories are the
product, so they live in the data directory and never in the cache, which
anything is entitled to delete.


Re-running
----------

Repositories are named `<projectId>-<slug>` and found again by their id, so a
re-run updates the repository it made last time instead of building a second
one, and follows a project renamed on Overleaf. Only the projects that actually
changed are reported when a run finishes.

Imported commits are deterministic, so rebuilding an unchanged history
reproduces its commit ids exactly and adds nothing. Signing in UTC keeps the
importing machine's timezone out of those ids.

Commits of your own on top of the imported history are never discarded: they are
kept on a `local/<timestamp>` branch. Such a branch is deleted once a later
import turns out to contain its commits anyway.
An archive repository with uncommitted changes is rejected rather than reset;
these repositories are otherwise machine-owned.


Responsible use & caching
-------------------------

Requests are rate limited (diffs at 1/second, version ZIPs at 1/2 minutes) and
the tool backs off when the server signals a limit (HTTP 429/500, `Retry-After`)
or when the network fails, retrying with exponential backoff rather than losing
a long import to one timeout. A wait longer than a few seconds is announced with
a countdown, and Ctrl-C stops it cleanly.

The budgets are tunable, since the useful value depends on the instance and the
network: `--timeout` and `--retries` as flags, and the ZIP quota as
`OVERLEAF_ZIP_PERIOD` (seconds per ZIP; it is read before the flags exist).

Every downloaded diff and ZIP is cached under `--cache-dir` (default
`~/.cache/overleaf-to-git/<project-id>/`), so an interrupted archive resumes
without touching Overleaf's servers again for anything it already fetched.
Cache entries are validated and installed atomically; an interrupted write is
never mistaken for a completed download.
Projects are fetched and archived one at a time, so an interrupted run keeps the
projects it had already finished. Keep the cache around until you are done.

Binary files are reconstructed from version ZIPs, downloading the minimal set of
ZIPs by normalizing add/rename/remove sequences first.


Notes
-----

Files opened on Overleaf can be marked "modified" with no visible change (a user
rapidly adding then removing content); such revisions are still committed (empty
if needed) and noted in the commit message. Use `git-filter-repo` afterwards if
you want to prune or rewrite the imported history.

DISCLAIMER: this may not align with Overleaf's Acceptable Use Policy regarding
"scraping". The author's position is that Overleaf provides no supported
interface to archive your own project history; this tool merely automates what a
user could do by clicking through every revision in the History tab.