Synapse hangs during initial database schema migration on ARM (Raspberry Pi)

[js-blog]

Description

Summary

Synapse hangs during initial database schema migration when running on ARM (Raspberry Pi 4).
The same PostgreSQL instance works correctly when accessed by Synapse on amd64.

The process appears to stall during schema delta application and does not progress further.

---

Environment

• Synapse version: 1.151.0

• Python version: 3.12

• Architecture: ARM64 (Raspberry Pi 4)

• OS: (Ubuntu 24.04)

• Deployment: Synapse running in container (Incus)

• PostgreSQL: 17.x/18.x

• Version: (please fill)

• Location: remote (LAN)

• Network latency: low (same LAN)

---

Expected behaviour

Synapse should complete database initialisation and continue startup.

---

Actual behaviour

Synapse hangs during database schema migration and does not proceed.

Logs stop at:

Applying schema deltas for v73
Applying schema 73/01event_failed_pull_attempts.sql
Applying schema 73/02add_pusher_enabled.sql

No further progress is observed.

---

Reproduction steps

1. Start Synapse with a fresh PostgreSQL database
2. Observe logs during initial startup

Command used:

./env/bin/python -m synapse.app.homeserver --config-path homeserver.yaml

---

Additional observations

• The issue only occurs when Synapse runs on ARM (Raspberry Pi)
• Running Synapse on amd64 against the same PostgreSQL instance works correctly
• Changing connection parameters (connection pool, keepalive, sslmode) does not affect the issue

---

Database state (PostgreSQL)

Observed PostgreSQL session state during hang:

pid  |        state        | wait_event_type | wait_event | query
------+---------------------+-----------------+------------+------------------------------------------------------------
6012 | idle in transaction | Client          | ClientRead | INSERT INTO applied_schema_deltas (version, file)
VALUES (73,'73/01event_failed_pull_attempts.sql')

This suggests that:

• The SQL statement has already been executed
• PostgreSQL is waiting for the client to continue (ClientRead)
• The process appears to be stuck on the client side (Synapse), not inside PostgreSQL

---

Additional context

This appears to be related to database initialisation or migration behaviour on ARM rather than connectivity or configuration issues.

---

Steps to reproduce

1. Start Synapse with a fresh PostgreSQL database
2. Observe logs during initial startup

Command used:

./env/bin/python -m synapse.app.homeserver --config-path homeserver.yaml

Homeserver

another

Synapse Version

matrix-synapse 1.151.0

Installation Method

pip (from PyPI)

Database

PostgreSQL 17.x/18.x

Workers

Single process

Platform

rpi4

Configuration

No response

Relevant log output

--------------------------------------------------------------------------------
2026-04-27 04:12:19,800 - synapse.config.logger - 377 - WARNING - main - ***** STARTING SERVER *****
2026-04-27 04:12:19,801 - synapse.config.logger - 378 - WARNING - main - Server /home/synapse/synapse/env/lib/python3.12/site-packages/synapse/app/homeserver.py version 1.151.0
2026-04-27 04:12:19,801 - synapse.config.logger - 383 - WARNING - main - Copyright (c) 2023 New Vector, Inc
2026-04-27 04:12:19,801 - synapse.config.logger - 384 - WARNING - main - Licensed under the AGPL 3.0 license. Website: https://github.com/element-hq/synapse
2026-04-27 04:12:19,801 - synapse.config.logger - 387 - INFO - main - Server hostname: chat.onmypc.org
2026-04-27 04:12:19,802 - synapse.config.logger - 388 - INFO - main - Public Base URL: https://chat.onmypc.org/
2026-04-27 04:12:19,802 - synapse.config.logger - 389 - INFO - main - Instance name: master
2026-04-27 04:12:19,802 - synapse.config.logger - 390 - INFO - main - Twisted reactor: EPollReactor
2026-04-27 04:12:19,802 - synapse.app.homeserver - 417 - INFO - main - Setting up server
2026-04-27 04:12:19,802 - synapse.server - 635 - INFO - main - Setting up.
2026-04-27 04:12:19,853 - synapse.storage.databases - 92 - INFO - main - [database config 'master']: Checking database server
2026-04-27 04:12:19,858 - synapse.storage.databases - 95 - INFO - main - [database config 'master']: Preparing for databases ['main', 'state']
2026-04-27 04:12:19,858 - synapse.storage.prepare_database - 132 - INFO - main - ['main', 'state']: Checking existing schema version
2026-04-27 04:12:19,886 - synapse.storage.prepare_database - 162 - INFO - main - ['main', 'state']: Initialising new database
2026-04-27 04:12:20,730 - synapse.storage.prepare_database - 430 - INFO - main - Applying schema deltas for v73
2026-04-27 04:12:20,733 - synapse.storage.prepare_database - 538 - INFO - main - Applying schema 73/01event_failed_pull_attempts.sql
2026-04-27 04:12:20,746 - synapse.storage.prepare_database - 538 - INFO - main - Applying schema 73/02add_pusher_enabled.sql
2026-04-27 04:27:59,693 - synapse.app._base - 251 - ERROR - main - Exception during startup
Traceback (most recent call last):
  File "/home/synapse/synapse/env/lib/python3.12/site-packages/synapse/storage/prepare_database.py", line 169, in prepare_database
    _setup_new_database(cur, database_engine, databases=databases)
  File "/home/synapse/synapse/env/lib/python3.12/site-packages/synapse/storage/prepare_database.py", line 295, in _setup_new_database
    _upgrade_existing_database(
  File "/home/synapse/synapse/env/lib/python3.12/site-packages/synapse/storage/prepare_database.py", line 539, in _upgrade_existing_database
    database_engine.execute_script_file(cur, absolute_path)
  File "/home/synapse/synapse/env/lib/python3.12/site-packages/synapse/storage/engines/_base.py", line 154, in execute_script_file
    cls.executescript(cursor, f.read())
  File "/home/synapse/synapse/env/lib/python3.12/site-packages/synapse/storage/engines/postgres.py", line 263, in executescript
    cursor.execute(f"COMMIT; BEGIN TRANSACTION; {script}")
  File "/home/synapse/synapse/env/lib/python3.12/site-packages/synapse/storage/database.py", line 458, in execute
    self._do_execute(self.txn.execute, sql, parameters)
  File "/home/synapse/synapse/env/lib/python3.12/site-packages/synapse/storage/database.py", line 520, in _do_execute
    return func(sql, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.OperationalError: could not receive data from server: Connection timed out

Anything else that would be useful to know?

No response

[reivilibre]

It's hard to say for sure, but I'm afraid this looks like a network problem, maybe instability with the network interface on the Pi; given both the client and the server appear to have become disconnected from one another; this isn't 'hanging' AFAICT as you get a clear exception written to the logs.

Does it make progress and get through more migrations if you restart Synapse?

How is your Pi connected to the Postgres server? Via the onboard Ethernet or something else? Are you using a stable power supply?

In your position, I would try to conduct some network testing on your Pi, including when it is under CPU load.

There might also be forum threads around with more diagnosis suggestions specifically for the Pi 4.

[saisharan0103]

I was able to reproduce the issue by starting Synapse with a fresh PostgreSQL database on a Raspberry Pi 4, and I observed the logs hanging during the schema delta application. My hypothesis is that the problem may lie in the synapse/storage/schema.py:apply_schema_deltas function, where it processes the SQL scripts.

To address this, I suggest the following approaches:

• Investigate if there are any ARM-specific SQL syntax or execution path issues in the migration scripts.
• Add logging around the execution of each SQL file to identify which specific command is causing the hang.

I'm happy to open a focused PR if that's the direction — or if there's a reason it's been left like this, good to know. Have there been any known issues with ARM compatibility in previous migrations?