Plugin slows down distributed execution using xdist

[GergelyKalmar]

It seems that pytest-pylint is not playing very well with pytest-xdist. On a relatively small project:

> pytest -n 0 -k pylint
=== test session starts ===
platform linux -- Python 3.8.10, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
...
plugins: xdist-2.3.0, pylint-0.18.0
...
collected 203 items / 136 deselected / 67 selected
--------------------------------------------------------------------------------
Linting files
...
--------------------------------------------------------------------------------
...
=== 3 failed, 64 skipped, 136 deselected in 5.90s ===

> pytest -n logical -k pylint
=== test session starts ===
platform linux -- Python 3.8.10, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
...
plugins: xdist-2.3.0, pylint-0.18.0
gw0 [67] / gw1 [67] / gw2 [67] / gw3 [67] / gw4 [67] / gw5 [67] / gw6 [67] / gw7 [67]
ssssssssssssssFsssssssssssssssssssssssssssssssssssssssFsFssssssssss
...
=== 3 failed, 64 skipped in 21.43s ===

It appears that all pylint tests are executed on all workers in parallel, moreover, they are executed before the regular tests are executed, which makes the run overall 4 times slower in a distributed test suite.

I am not sure why, but I have also noticed that when running tests in a distributed fashion then tests don't get skipped as they usually are (on the second run), moreover duplicate-code errors don't appear reliably either. I understand that originally the idea was to run pylint tests independently of pytest's collection in one run, but it seems that this is not working as expected either.