Preliminaries

I prepared some sample tests for playing with. These are some simple page title checks. We have one module test_google.py with two unit tests that check the titles of www.google.com and mail.google.com:

# test_google.pyimport unittestfrom selenium import webdriverclass GoogleTests(unittest.TestCase):    def setUp(self):        self.driver = webdriver.Chrome()    def tearDown(self):        self.driver.close()    def test_google_page_title(self):        self.driver.get('https://www.google.com')        assert self.driver.title == 'Google'    def test_gmail_page_title(self):        self.driver.get('https://mail.google.com')        assert self.driver.title == 'Gmail'

The second module is test_stackoverflow.py that contains one test that checks the title of stackoverflow.com:

# test_stackoverflow.pyimport unittestfrom selenium import webdriverclass StackoverflowTests(unittest.TestCase):    def setUp(self):        self.driver = webdriver.Chrome()    def tearDown(self):        self.driver.close()    def test_so_page_title(self):        self.driver.get('https://stackoverflow.com')        assert 'Stack Overflow' in self.driver.title

Running the tests with the bare unittest runner yields:

$ python setup.py testrunning testrunning egg_info...running build_exttest_gmail_page_title (test_google.GoogleTests) ... oktest_google_page_title (test_google.GoogleTests) ... oktest_so_page_title (test_stackoverflow.StackoverflowTests) ... ok----------------------------------------------------------------------Ran 3 tests in 11.657sOK

Migration to pytest

Install pytest via pip:

$ pip install pytest

pytest supports unit tests out of the box, so we don't need to touch the tests, we can run them immediately. Trying out the pytest runner:

$ pytest -v================ test session starts ================platform darwin -- Python 3.6.3, pytest-3.2.5, py-1.5.2, pluggy-0.4.0 -- /Users/hoefling/.virtualenvs/stackoverflow/bin/python3.6cachedir: .cacherootdir: /Users/hoefling/projects/private/stackoverflow/so-47439103, inifile:collected 3 items                                                                                                                                                                                   test_google.py::GoogleTests::test_gmail_page_title PASSEDtest_google.py::GoogleTests::test_google_page_title PASSEDtest_stackoverflow.py::StackoverflowTests::test_so_page_title PASSED================ 3 passed in 13.81 seconds ================

Running tests in parallel

This requires pytest-xdist plugin for pytest. Install it via pip:

$ pip install pytest-xdist

The plugin is installed now, but won't be active by default, so if you run pytest again, you won't notice any difference. Use the numprocesses key to parallelize the test execution. This denotes the number of processes that are reserved to run the tests, here I use the auto value to spawn as many processes as many CPUs my machine has:

$ pytest -v --numprocesses=auto================ test session starts ================platform darwin -- Python 3.6.3, pytest-3.2.5, py-1.5.2, pluggy-0.4.0 -- /Users/hoefling/.virtualenvs/stackoverflow/bin/python3.6cachedir: .cacherootdir: /Users/hoefling/projects/private/stackoverflow/so-47439103, inifile:plugins: xdist-1.20.1, forked-0.2[gw0] darwin Python 3.6.3 cwd: /Users/hoefling/projects/private/stackoverflow/so-47439103[gw1] darwin Python 3.6.3 cwd: /Users/hoefling/projects/private/stackoverflow/so-47439103[gw2] darwin Python 3.6.3 cwd: /Users/hoefling/projects/private/stackoverflow/so-47439103[gw3] darwin Python 3.6.3 cwd: /Users/hoefling/projects/private/stackoverflow/so-47439103[gw0] Python 3.6.3 (v3.6.3:2c5fed86e0, Oct  3 2017, 00:32:08)  -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)][gw1] Python 3.6.3 (v3.6.3:2c5fed86e0, Oct  3 2017, 00:32:08)  -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)][gw2] Python 3.6.3 (v3.6.3:2c5fed86e0, Oct  3 2017, 00:32:08)  -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)][gw3] Python 3.6.3 (v3.6.3:2c5fed86e0, Oct  3 2017, 00:32:08)  -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]gw0 [3] / gw1 [3] / gw2 [3] / gw3 [3]scheduling tests via LoadSchedulingtest_google.py::GoogleTests::test_google_page_title test_stackoverflow.py::StackoverflowTests::test_so_page_title test_google.py::GoogleTests::test_gmail_page_title [gw0] PASSED test_google.py::GoogleTests::test_gmail_page_title [gw1] PASSED test_google.py::GoogleTests::test_google_page_title [gw2] PASSED test_stackoverflow.py::StackoverflowTests::test_so_page_title ================ 3 passed in 7.81 seconds ================

You will see that all the three tests run in parallel by three chrome instances opened simultaneously. Each test runs in own process, so they don't interfere with each other. Also, notice that both test methods from the GoogleTests class also run in parallel, so this is not restricted to tests in different modules or classes.

Integration with setup.py

When I first started the migration to pytest, one of the conditions I had was that the command python setup.py test should still work so we don't need to memorize an extra pytest command to run the tests and so we also don't have to adapt all our utility scripts or build jobs on our integration server, so here are the steps to update your setup.py script:

1. Add the following packages to test requirements:

   from setuptools import setupsetup(    ...    tests_require=[        'pytest',        'pytest-runner',  # this one is needed to install distutils command for pytest        'pytest-xdist'    ],)

2. Add an alias to setup.cfg:

   [aliases]test=pytest

3. Add a configuration section for pytest in setup.cfg:

   [tool:pytest]addopts=--verbose --numprocesses=auto

Now, if you run python setup.py test, the correct runner will be invoked and the xdist plugin will be active by default.

Additional notes

I personally really like pytest as it offers much more than a plain test execution - you can write tests as pure functions (no wrapping into TestCase classes required), collect tests without executing them, easily rerun only tests that failed recently, hook the code coverage measurement with multiple reports in different formats and many more. Refer to the official docs for more details, it is really worth the reading time!