Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 49 additions & 26 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,42 +6,27 @@ on:
pull_request:
branches: [main]

permissions:
contents: read

jobs:
test:
name: Test
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- uses: docker/setup-buildx-action@v3

- name: Build soffice image
uses: docker/build-push-action@v6
with:
context: benchmarks
file: benchmarks/Dockerfile.soffice-debian
tags: soffice:latest
load: true
cache-from: type=gha
cache-to: type=gha,mode=max
- uses: actions/checkout@v6

- name: Start soffice
run: |
docker run --detach --name soffice \
--publish 2002:2002 \
--volume /tmp:/tmp \
soffice
timeout 30 bash -c 'until nc -z localhost 2002; do sleep 1; done'
- uses: cachix/install-nix-action@v31

- uses: erlef/setup-beam@v1
id: beam
with:
elixir-version: 1.19.5
otp-version: 28.3
elixir-version: 1.20.2
otp-version: 29.0.3

- name: Restore dependencies cache
uses: actions/cache@v4
uses: actions/cache@v5
with:
path: |
deps
Expand All @@ -50,7 +35,7 @@ jobs:
restore-keys: ${{ runner.os }}-mix-

- name: Restore PLT cache
uses: actions/cache@v4
uses: actions/cache@v5
with:
path: priv/plts
key: ${{ runner.os }}-plt-otp${{ steps.beam.outputs.otp-version }}-${{ hashFiles('**/mix.lock') }}-${{ github.sha }}
Expand All @@ -67,8 +52,46 @@ jobs:
- name: Check formatting
run: mix format --check-formatted

- name: Run tests
run: mix test --include lo26
- name: Check unused dependencies
run: mix deps.unlock --check-unused

- name: Check dependency cycles
run: mix xref graph --format cycles --fail-above 0

- name: Run unit tests without soffice
run: mix test

- name: Build documentation
run: mix docs --warnings-as-errors

- name: Audit Hex dependencies
run: mix hex.audit

- name: Check Nix flake
run: nix flake check

- name: Run dialyzer
run: mix dialyzer

- uses: docker/setup-buildx-action@v4

- name: Build soffice image
uses: docker/build-push-action@v7
with:
context: benchmarks
file: benchmarks/Dockerfile.soffice-debian
tags: soffice:latest
load: true
cache-from: type=gha
cache-to: type=gha,mode=max

- name: Start soffice
run: |
docker run --detach --name soffice \
--publish 2002:2002 \
--volume /tmp:/tmp \
soffice
timeout 30 bash -c 'until nc -z localhost 2002; do sleep 1; done'

- name: Run integration tests
run: mix test --include lo26
8 changes: 4 additions & 4 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,15 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6

- uses: erlef/setup-beam@v1
with:
elixir-version: "1.19.5"
otp-version: "28.3"
elixir-version: "1.20.2"
otp-version: "29.0.3"

- name: Restore dependencies cache
uses: actions/cache@v4
uses: actions/cache@v5
with:
path: |
deps
Expand Down
70 changes: 52 additions & 18 deletions PERFORMANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@ docker compose --file benchmarks/docker-compose.yml up --detach --wait
mix run benchmarks/bench.exs
```

Both URP and Gotenberg run LibreOffice 26.2.0 on Debian (glibc).
The results below were recorded on July 14, 2026, on Apple M3 Max with
Elixir 1.20.2 and Erlang/OTP 29.0.3. URP's Debian container ran
LibreOffice 26.2.4.2; Gotenberg 8.32.0 bundled LibreOffice 26.2.2.2.
The fixture uses Liberation fonts only — regenerate with
`uv run --with python-docx --with Pillow --with numpy benchmarks/generate_fixture.py`
(pass `--size 15` for the large variant).
Expand All @@ -22,22 +24,39 @@ The fixture uses Liberation fonts only — regenerate with

```
Name ips average deviation median 99th %
URP 1.05 0.95 s ±7.60% 0.94 s 1.20 s
Gotenberg 0.81 1.23 s ±7.47% 1.19 s 1.42 s
URP 1.37 0.73 s ±4.74% 0.72 s 0.82 s
Gotenberg 0.84 1.19 s ±5.46% 1.16 s 1.37 s
```

**15.5 MB input → 62 MB PDF:**

```
Name ips average deviation median 99th %
URP 0.145 6.87 s ±7.44% 6.73 s 7.44 s
Gotenberg 0.087 11.45 s ±1.21% 11.45 s 11.54 s
URP 0.196 5.10 s ±2.12% 5.05 s 5.22 s
Gotenberg 0.135 7.39 s ±0.88% 7.36 s 7.46 s
```

**27% faster** for small documents, **67% faster** for large ones.
The gap grows because Gotenberg's Go/HTTP overhead (multipart parsing,
queue management, response framing) scales with document size, while
URP talks to soffice directly over a TCP socket.
URP had **39% lower average latency** for the small document and **31%
lower average latency** for the large document. The absolute advantage
grew from 0.46 s to 2.29 s. These measurements compare the complete
stacks, including their slightly different LibreOffice patch versions;
they do not isolate dependency or runtime upgrades individually.

### Process overhead sanity check

`benchmarks/convert.exs` compares the persistent URP connection with a
cold `soffice --convert-to` process and Gotenberg using the 33 KB
`sample3.docx` fixture. Across five timed iterations, the averages were:

| Method | Average | Range |
|--------|---------|-------|
| URP | 46 ms | 43–49 ms |
| Gotenberg | 153 ms | 146–160 ms |
| LibreOffice CLI | 280 ms | 267–302 ms |

This is a process-overhead check, not an apples-to-apples transport
benchmark: URP reuses a live office process, while the CLI measurement
starts a new process for every conversion.

## I/O strategies

Expand All @@ -53,10 +72,25 @@ and transfers them over URP in ~6 round-trips. **Stream I/O** (`:stream`)
pipes bytes over the URP socket via XInputStream/XOutputStream — no temp
disk, but more round-trips.

Stream input is the bottleneck (~40-50% slower) because ZIP-based formats
(docx, xlsx, pptx) require thousands of XInputStream/XSeekable random-access
round-trips. Stream output adds negligible overhead — soffice writes in
fixed [32 767-byte chunks](https://github.com/LibreOffice/core/blob/libreoffice-26-2-0/sfx2/source/doc/docfile.cxx#L2573),
Current results on Elixir 1.20.2 / OTP 29.0.3:

| Strategy | 2.7 MB | 16.2 MB | 35.0 MB |
|----------|--------|---------|---------|
| File → file | 1.13 s | 7.66 s | 39.17 s |
| File → stream | 1.19 s | 8.06 s | 37.19 s |
| Stream → file | 1.72 s | 11.59 s | 41.53 s |
| Stream → stream | 1.64 s | 9.06 s | 46.02 s |

Stream input remains the bottleneck because ZIP-based formats (docx,
xlsx, pptx) require thousands of XInputStream/XSeekable random-access
round-trips, but its measured penalty now ranges from roughly 12% to 52%
depending on document size and output mode. Stream output remains within
about 5% of file output. The 35 MB fixture completes only one iteration
per scenario with the default 10-second Benchee window, so treat those
figures as directional rather than statistically stable.

Soffice writes stream output in fixed
[32 767-byte chunks](https://github.com/LibreOffice/core/blob/libreoffice-26-2-0/sfx2/source/doc/docfile.cxx#L2573),
so the round-trip count is predictable.

| Strategy | Input | Output | Best for |
Expand Down Expand Up @@ -109,22 +143,22 @@ and can be mitigated with
It bundles OpenJDK 11, 130+ Noto font packages, and 450 packages
total.

As of March 2026, Alpine ships LO 25.8.x (Still) while Debian
As of July 2026, Alpine ships LO 25.8.x (Still) while Debian
trixie-backports has 26.2.x (Fresh). Carlito is missing from the
stock Alpine image (`apk add font-carlito` to fix).

| Setup | 2.6 MB | 15.5 MB | LO version | Image size |
|-------|--------|---------|------------|------------|
| URP → Debian glibc | 0.94 s | 6.73 s | 26.2.0 | ~564 MB |
| URP → Alpine musl | 1.20 s | 11.11 s | 25.8.1 | ~1.78 GB |
| Gotenberg (Debian glibc) | 1.19 s | 11.45 s | 26.2.0 | ~1.86 GB |
| URP → Debian glibc | 0.72 s | 5.05 s | 26.2.4.2 | ~564 MB |
| URP → Alpine musl | 1.18 s | 7.94 s | 25.8.1.1 | ~1.78 GB |
| Gotenberg (Debian glibc) | 1.16 s | 7.36 s | 26.2.2.2 | ~1.68 GB |

<details>
<summary>Reproducing the strace analysis</summary>

```sh
docker compose --file benchmarks/docker-compose.yml up --detach --wait
SOFFICE=benchmarks-soffice-1
SOFFICE=benchmarks-soffice-alpine-1
docker exec $SOFFICE apk add --no-cache strace
docker exec $SOFFICE pgrep -f soffice.bin # note the PID

Expand Down
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ no Python, no wrappers, no sidecars.
## Installation

```elixir
{:urp, "~> 0.8"}
{:urp, "~> 0.10"}
```

## Prerequisites
Expand Down Expand Up @@ -69,6 +69,14 @@ URP.Test.stub(fn _input, _opts -> {:ok, "/tmp/fake.pdf"} end)
assert {:ok, _} = MyApp.generate_invoice(order)
```

When soffice is unavailable, tests tagged `:integration` are excluded automatically.
Run the complete suite, including the LibreOffice 26.2+ coverage, with:

```sh
docker compose --file benchmarks/docker-compose.yml up --detach --wait soffice
mix test --include lo26
```

### Telemetry

Every operation emits `[:urp, :call, :stop]` with queue, service, and
Expand Down
5 changes: 2 additions & 3 deletions benchmarks/Dockerfile.soffice-debian
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Minimal Debian soffice with LO from trixie-backports (same version as Gotenberg).
# Minimal Debian soffice with LibreOffice 26.2+ from trixie-backports.
# Only free fonts — Liberation (Arial/Times metric-compatible) and Carlito (Calibri).
FROM debian:trixie-slim

Expand All @@ -16,5 +16,4 @@ ENV PORT=2002

EXPOSE ${PORT}

CMD soffice --headless --norestore \
--accept="socket,host=${HOST},port=${PORT},tcpNoDelay=1;urp;"
CMD ["sh", "-c", "exec soffice --headless --norestore --accept=\"socket,host=${HOST},port=${PORT},tcpNoDelay=1;urp;\""]
14 changes: 7 additions & 7 deletions benchmarks/bench.exs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ gotenberg_url = "http://localhost:3002/forms/libreoffice/convert"
IO.puts("Fixture: #{fixture} (#{div(byte_size(docx_bytes), 1024)} KB)\n")

# Gotenberg-equivalent FilterData — matches DefaultOptions() from
# https://github.com/gotenberg/gotenberg/blob/v8.27.0/pkg/modules/libreoffice/api/api.go
# https://github.com/gotenberg/gotenberg/blob/v8.32.0/pkg/modules/libreoffice/api/api.go
gotenberg_filter_data = [
ExportFormFields: true,
AllowDuplicateFieldNames: false,
Expand Down Expand Up @@ -52,26 +52,26 @@ Application.ensure_all_started(:telemetry)
{:ok, _} = Finch.start_link(name: Req.Finch)
form_data = [files: {docx_bytes, filename: fixture, content_type: "application/octet-stream"}]

# ── Debian pool (port 2003) ──
# ── Alpine pool (port 2003) ──

{:ok, _} = URP.Pool.start_link(name: :debian, host: "localhost", port: 2003, pool_size: 1)
{:ok, _} = URP.Pool.start_link(name: :alpine, host: "localhost", port: 2003, pool_size: 1)

# ── Warmup all services ──

{:ok, _} = URP.convert({:binary, docx_bytes}, urp_opts)
{:ok, _} = URP.Pool.convert(:debian, {:binary, docx_bytes}, urp_opts)
{:ok, _} = URP.Pool.convert(:alpine, {:binary, docx_bytes}, urp_opts)
Req.post!(gotenberg_url, form_multipart: form_data)

# ── Benchmark ──

Benchee.run(
%{
"URP → Alpine musl" => fn ->
"URP → Debian glibc" => fn ->
{:ok, pdf} = URP.convert({:binary, docx_bytes}, urp_opts)
pdf
end,
"URP → Debian glibc" => fn ->
{:ok, pdf} = URP.Pool.convert(:debian, {:binary, docx_bytes}, urp_opts)
"URP → Alpine musl" => fn ->
{:ok, pdf} = URP.Pool.convert(:alpine, {:binary, docx_bytes}, urp_opts)
pdf
end,
"Gotenberg (HTTP)" => fn ->
Expand Down
6 changes: 3 additions & 3 deletions benchmarks/convert.exs
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
# Simple benchmark: URP vs CLI shell-out vs Gotenberg
#
# All three use LibreOffice 25.8.1 for apples-to-apples comparison.
# Record each service's LibreOffice version when comparing benchmark results.
#
# Prerequisites:
# 1. soffice container on port 2002 (URP)
# 2. soffice-cli container with /fixtures mount (CLI — separate instance to avoid lock)
# 3. gotenberg:8.23.1 on GOTENBERG_PORT (HTTP API)
# 3. Gotenberg service from docker-compose.yml on GOTENBERG_PORT (HTTP API)
#
# cd benchmarks && docker compose up -d
# docker compose --file benchmarks/docker-compose.yml up --detach --wait
#
# Run:
# nix develop --command mix run benchmarks/convert.exs
Expand Down
30 changes: 20 additions & 10 deletions benchmarks/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,29 +5,39 @@
# nix develop --command mix run benchmarks/bench.exs
#
# Services:
# soffice — Alpine (musl), URP on port 2002
# soffice-debian — Debian (glibc), URP on port 2003, LO from trixie-backports
# soffice — Debian (glibc), URP on port 2002, LO 26.2+ from trixie-backports
# soffice-cli — idle Debian container for cold CLI shell-out comparisons
# soffice-alpine — Alpine (musl), URP on port 2003, comparison service
# gotenberg — Gotenberg full stack (Debian glibc), HTTP on port 3002

services:
soffice:
image: libreofficedocker/alpine:3.23
command:
- soffice
- --headless
- --norestore
- --accept=socket,host=0.0.0.0,port=2002,tcpNoDelay=1;urp;
build:
context: .
dockerfile: Dockerfile.soffice-debian
ports:
- "2002:2002"

soffice-debian:
soffice-cli:
build:
context: .
dockerfile: Dockerfile.soffice-debian
container_name: soffice-cli
command: ["sleep", "infinity"]
volumes:
- ./fixtures:/fixtures:ro

soffice-alpine:
image: libreofficedocker/alpine:3.23
command:
- soffice
- --headless
- --norestore
- --accept=socket,host=0.0.0.0,port=2002,tcpNoDelay=1;urp;
ports:
- "2003:2002"

gotenberg:
image: gotenberg/gotenberg:8.27.0
image: gotenberg/gotenberg:8.32.0
ports:
- "3002:3000"
Loading
Loading