Description
Given a folder contains multiple test modules each of which defining a test function that references the page fixture and one parametrized variable, when calling pytest --collect-only, then the list of test cases is not in module order but shuffled at seemingly random.
The issue occurs since pytest version 8.4.0. Tested with python 3.13.7, playwright 1.56.0 and pytest-playwright 0.7.1.
How to reproduce
pip install playwright pytest pytest-playwright
test_a.py
import pytest
@pytest.mark.parametrize("param", ["apple", "orange", "pear"])
def test_1(page, param):
assert Truetest_b.py
import pytest
@pytest.mark.parametrize("param", ["dog", "cat", "mouse"])
def test_2(page, param):
assert TrueCalling pytest --collect-only results in
<Package tests>
<Module test_a.py>
<Function test_1[chromium-apple]>
<Module test_b.py>
<Function test_2[chromium-dog]>
<Module test_a.py>
<Function test_1[chromium-orange]>
<Module test_b.py>
<Function test_2[chromium-cat]>
<Module test_a.py>
<Function test_1[chromium-pear]>
<Module test_b.py>
<Function test_2[chromium-mouse]>
Note: After removing the page fixture from one of the tests, the collection order is as expected:
<Package tests>
<Module test_a.py>
<Function test_1[apple]>
<Function test_1[orange]>
<Function test_1[pear]>
<Module test_b.py>
<Function test_2[chromium-dog]>
<Function test_2[chromium-cat]>
<Function test_2[chromium-mouse]>
Description
Given a folder contains multiple test modules each of which defining a test function that references the page fixture and one parametrized variable, when calling
pytest --collect-only, then the list of test cases is not in module order but shuffled at seemingly random.The issue occurs since pytest version 8.4.0. Tested with python 3.13.7, playwright 1.56.0 and pytest-playwright 0.7.1.
How to reproduce
pip install playwright pytest pytest-playwrighttest_a.py
test_b.py
Calling
pytest --collect-onlyresults inNote: After removing the page fixture from one of the tests, the collection order is as expected: