Skip to content

fix: stabilize identify integration tests under parallel xdist load#1378

Open
acul71 wants to merge 4 commits into
mainfrom
fix/parallel-identify-test-flakes
Open

fix: stabilize identify integration tests under parallel xdist load#1378
acul71 wants to merge 4 commits into
mainfrom
fix/parallel-identify-test-flakes

Conversation

@acul71

@acul71 acul71 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Harden identify handler to wait for listen addresses and fall back to transport addrs when advertising (libp2p/identity/identify/identify.py)
  • Add test synchronization helpers and explicit _identify_peer waits for load-sensitive integration tests
  • Split make test into parallel (-m "not serial_only") and serial phases for six flaky-under-load tests
  • Register @pytest.mark.serial_only in pyproject.toml

Closes #1377

Root cause

Six identify/host/quic tests fail only under full-suite pytest -n auto (~3060 tests), not in isolation. Investigation notes were kept local under downloads/ (gitignored, not in the repo).

Not caused by #1372/#1374 logic changes; #1372 increased parallel suite load and exposed latent timing races.

Changed files

Area Files
Production libp2p/identity/identify/identify.py, libp2p/tools/utils.py
Tests tests/utils/identify_test_helpers.py, identify/host/quic test files
Infra Makefile, pyproject.toml, newsfragments/1377.bugfix.rst

Test plan

  • make lint
  • make typecheck
  • make test (parallel + serial phases)
  • make linux-docs
  • pytest -n auto --timeout=1200 tests/core (CI tox parity — 2807 passed locally)
  • Confirm 6 previously failing tests pass in serial phase

Notes for reviewers

  • wait_for_peerstore_addrs added to libp2p/tools/utils.py but not wired into global connect() (caused suite-wide timeouts during investigation)
  • Makefile split affects local make pr only; CI uses tox pytest -n auto tests/core, which passes with the production/test hardening alone

Made with Cursor

acul71 and others added 3 commits July 9, 2026 21:39
Co-authored-by: Cursor <cursoragent@cursor.com>
Split make test into parallel and serial_only phases, harden identify
handler address publication, and add test synchronization for the six
load-sensitive identify/host/quic tests that failed under pytest -n auto.

Co-authored-by: Cursor <cursoragent@cursor.com>
Closes #1377

Co-authored-by: Cursor <cursoragent@cursor.com>
The downloads/ tree is gitignored; keep INVESTIGATION.md local only.

Co-authored-by: Cursor <cursoragent@cursor.com>
@sumanjeet0012

Copy link
Copy Markdown
Contributor

@acul71 @seetadev

I created a fresh virtual environment, installed all dependencies, and ran the make pr command on macOS.

All tests passed successfully; however, I encountered the following runtime warnings:

  1. During test execution:

[gw4] [ 98%] PASSED tests/interop/zig_libp2p/test_zig_basic.py::test_zig_libp2p_placeholder
/Users/sumanjeet/code/py-libp2p/.venv/lib/python3.10/site-packages/_pytest/unraisableexception.py:33:
RuntimeWarning: coroutine 'AsyncMockMixin._execute_mock_call' was never awaited
gc.collect()
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

  1. Warnings summary:

tests/core/transport/webrtc/test_connection.py::TestOpenStream::test_open_stream_increments_ids_by_2
tests/core/transport/webrtc/test_connection.py::TestOpenStream::test_open_stream_on_closed_connection_raises
tests/core/transport/webrtc/test_connection.py::TestClose::test_close_is_idempotent
/Users/sumanjeet/.local/share/uv/python/cpython-3.10.19-macos-aarch64-none/lib/python3.10/unittest/mock.py:2135:
RuntimeWarning: coroutine 'AsyncMockMixin._execute_mock_call' was never awaited
setattr(_type, entry, MagicProxy(entry, self))
Enable tracemalloc to get the object allocation traceback.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

Aside from these warnings, the test suite completed successfully.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Six identify/host/quic tests fail under full-suite pytest-xdist load

2 participants