Skip to content

Replace Twisted with aiohttp and native asyncio#628

Draft
sandhose wants to merge 8 commits into
quenting/modernisation/pre-migration-testsfrom
quenting/modernisation/twisted-to-aiohttp
Draft

Replace Twisted with aiohttp and native asyncio#628
sandhose wants to merge 8 commits into
quenting/modernisation/pre-migration-testsfrom
quenting/modernisation/twisted-to-aiohttp

Conversation

@sandhose
Copy link
Copy Markdown
Member

@sandhose sandhose commented Apr 4, 2026

Summary

  • Replace Twisted HTTP server (Resource/Site) with aiohttp.web
  • Replace Twisted HTTP client (Agent) with aiohttp.ClientSession
  • Replace twisted.names SRV resolution with dnspython
  • Replace reactor.run() with asyncio.run()
  • Replace LoopingCall with asyncio.create_task() + sleep loops
  • Migrate all 20 servlets from Resource classes to async handler functions
  • Rewrite federation agent, IP blacklisting, and TLS for aiohttp
  • Delete custom Twisted type stubs (16 .pyi files)
  • Migrate tests from trial to pytest + pytest-asyncio
  • Split CI into separate unit test and integration test jobs
  • Fix mypy errors from aiohttp migration (proper type annotations)

Known limitations

  • pyOpenSSL kept for replication peer cert CN extraction
  • test_bind_notYourMxid may be flaky due to aiosmtpd startup race in integration tests

Test plan

  • uv run pytest tests/ — 100 tests pass
  • uv run pytest --pyargs matrix_is_tester — 37/37 pass
  • uv run mypy — no errors
  • No twisted or zope.interface imports remain
  • CI green

Part 8 of 9 in the modernisation series. Builds on #627.

sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/pre-migration-tests branch from fd90959 to 9f02890 Compare April 4, 2026 15:12
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from b4968f6 to 75c3d27 Compare April 4, 2026 15:12
sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/pre-migration-tests branch from 9f02890 to 25a9496 Compare April 4, 2026 15:19
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from 75c3d27 to 07ba146 Compare April 4, 2026 15:19
@sandhose sandhose force-pushed the quenting/modernisation/pre-migration-tests branch from 25a9496 to e9d5495 Compare April 4, 2026 15:36
sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from 07ba146 to 6e4f741 Compare April 4, 2026 15:36
sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from 6e4f741 to ba57ad6 Compare April 4, 2026 15:43
@sandhose sandhose force-pushed the quenting/modernisation/pre-migration-tests branch from e9d5495 to 8cff64d Compare April 4, 2026 15:43
sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from ba57ad6 to e6e33bc Compare April 4, 2026 15:45
@sandhose sandhose force-pushed the quenting/modernisation/pre-migration-tests branch from 8cff64d to 004fd70 Compare April 4, 2026 15:45
sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from e6e33bc to 5bebdec Compare April 4, 2026 15:56
@sandhose sandhose force-pushed the quenting/modernisation/pre-migration-tests branch from 004fd70 to 13aa531 Compare April 4, 2026 15:56
sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/pre-migration-tests branch from 13aa531 to 303eca6 Compare April 4, 2026 16:02
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from 5bebdec to 0e548d7 Compare April 4, 2026 16:02
sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/pre-migration-tests branch from 303eca6 to 70d6161 Compare April 4, 2026 16:09
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from 0e548d7 to 5c4a7a6 Compare April 4, 2026 16:09
sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/pre-migration-tests branch from 70d6161 to 79ecbe6 Compare April 4, 2026 16:10
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from 5c4a7a6 to 3983f5e Compare April 4, 2026 16:10
sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/pre-migration-tests branch from 79ecbe6 to 8c52197 Compare April 4, 2026 16:17
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from 3983f5e to 0d35a03 Compare April 4, 2026 16:17
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from aa731f6 to c7d0c49 Compare April 4, 2026 16:44
sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from c7d0c49 to 39e3169 Compare April 4, 2026 16:48
@sandhose sandhose force-pushed the quenting/modernisation/pre-migration-tests branch from ddd42cc to 377e703 Compare April 4, 2026 16:48
sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from 39e3169 to f94d19f Compare April 4, 2026 16:52
@sandhose sandhose force-pushed the quenting/modernisation/pre-migration-tests branch from 377e703 to 03c779e Compare April 4, 2026 16:52
sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from f94d19f to 8512068 Compare April 4, 2026 16:56
sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch 2 times, most recently from 7a17ecc to ef43607 Compare April 4, 2026 17:02
sandhose added a commit that referenced this pull request Apr 4, 2026
sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from ef43607 to 065d1af Compare April 4, 2026 17:09
sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from 065d1af to 24a43cf Compare April 4, 2026 19:59
@sandhose sandhose force-pushed the quenting/modernisation/pre-migration-tests branch from 03c779e to f63c9b1 Compare April 4, 2026 19:59
sandhose added a commit that referenced this pull request Apr 4, 2026
@sandhose sandhose force-pushed the quenting/modernisation/twisted-to-aiohttp branch from 24a43cf to 81d4bea Compare April 4, 2026 20:02
@sandhose sandhose force-pushed the quenting/modernisation/pre-migration-tests branch from f63c9b1 to 239747c Compare April 4, 2026 20:02
@sandhose sandhose mentioned this pull request Apr 4, 2026
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant