How do I assure pytest runs sets of tests in specific order?

[ssbarnea]

I am looking for a generic solution for running inexpensive unit tests before heavier functional (or integration) tests without having to execute pytest twice, and in a way that does not break xdist or coverage.

Think about scale case, where a project already have hundreds of tests, but it would be reasonable to assume that tests are already grouped (folders) by their type. A setup like test/unit/ and test/functional being quite common, an natural.

I have seen some approaches so far but none that I would count as sustainable as adding decorators to hundreds of tests is not really handy.

I mention that I do not care about order of tests inside a "group", still it would also be useful to have a special decorator that assures some tests are run in serial.

Does anyone have an example where this issue was addressed?

[symonk]

Hi @ssbarnea I assume given your final points that some tests need to run serially / isolated as part of a more parallel run, are you using xdist? If so you could try to write your own custom scheduler that completes the initial ones before handing out the more 'functional' ones to the worker nodes. We did something pretty much exactly like this except all of our python tests are 'functional' (unit in another language - perl!) However we implemented our own @pytest.mark.isolated marker and built a scheduler that finished the parallel ones before starting the isolated ones within one subprocess (well, w/e execnet uses). That said it wasen't entirely happy sailing, we ended up binning it after a few months and just went to the simple old multiple pytest invocations.