If I have multiple tests in one class and a preceding test fails, how do I have it skip or exit the class instead of testing the remaining tests?
I think i found what will solve this problem. It's called incremental testing
in py.test framework: https://pytest.org/latest/example/simple.html#incremental-testing-test-steps
I tested locally - worked like a charm!
Used same conftest.py
as in the py.test docs.Slightly different test file:
# content of test_incremental.pyimport pytest@pytest.mark.incrementalclass TestUserHandling: def test_login(self): assert 1 def test_modification(self): assert 0 def test_deletion(self): assert 1 def test_foo(self): assert 1class TestAnotherStuff: def test_normal(self): assert 1
mac:[incremental_testing]: py.test -v test_incremental.py ============================================================================== test session starts ===============================================================================platform darwin -- Python 2.7.11, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Pythoncachedir: .cacherootdir: /Users/dmitry.tokarev/Repos/playground/incremental_testing, inifile: plugins: hidecaptured-0.1.2, instafail-0.3.0collected 5 items test_incremental.py::TestUserHandling::test_login PASSEDtest_incremental.py::TestUserHandling::test_modification FAILEDtest_incremental.py::TestUserHandling::test_deletion xfailtest_incremental.py::TestUserHandling::test_foo xfailtest_incremental.py::TestAnotherStuff::test_normal PASSED==================================================================================== FAILURES ====================================================================================_______________________________________________________________________ TestUserHandling.test_modification _______________________________________________________________________self = <test_incremental.TestUserHandling instance at 0x103b7a5a8> def test_modification(self):> assert 0E assert 0test_incremental.py:10: AssertionError================================================================= 1 failed, 2 passed, 2 xfailed in 0.02 seconds ==================================================================
Why don't you create dependencies in you tests for example:
def test_mainTest(): ...@dependsOn(test_mainTest)def test_dependantTest(): ...def test_non_dependant_test(): ...
Use flags -x
or --exitfirst
to exit instantly on first error or failed test.
Or --maxfail=num
to exit after first num failures or error.
http://pytest.org/latest/usage.html#stopping-after-the-first-or-n-failures