How do you generate dynamic (parameterized) unit tests in Python?
This is called "parametrization".
There are several tools that support this approach. E.g.:
The resulting code looks like this:
from parameterized import parameterizedclass TestSequence(unittest.TestCase): @parameterized.expand([ ["foo", "a", "a",], ["bar", "a", "b"], ["lee", "b", "b"], ]) def test_sequence(self, name, a, b): self.assertEqual(a,b)
Which will generate the tests:
test_sequence_0_foo (__main__.TestSequence) ... oktest_sequence_1_bar (__main__.TestSequence) ... FAILtest_sequence_2_lee (__main__.TestSequence) ... ok======================================================================FAIL: test_sequence_1_bar (__main__.TestSequence)----------------------------------------------------------------------Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/parameterized/parameterized.py", line 233, in <lambda> standalone_func = lambda *a: func(*(a + p.args), **p.kwargs) File "x.py", line 12, in test_sequence self.assertEqual(a,b)AssertionError: 'a' != 'b'
For historical reasons I'll leave the original answer circa 2008):
I use something like this:
import unittestl = [["foo", "a", "a",], ["bar", "a", "b"], ["lee", "b", "b"]]class TestSequense(unittest.TestCase): passdef test_generator(a, b): def test(self): self.assertEqual(a,b) return testif __name__ == '__main__': for t in l: test_name = 'test_%s' % t[0] test = test_generator(t[1], t[2]) setattr(TestSequense, test_name, test) unittest.main()
Using unittest (since 3.4)
Since Python 3.4, the standard library unittest
package has the subTest
context manager.
See the documentation:
Example:
from unittest import TestCaseparam_list = [('a', 'a'), ('a', 'b'), ('b', 'b')]class TestDemonstrateSubtest(TestCase): def test_works_as_expected(self): for p1, p2 in param_list: with self.subTest(): self.assertEqual(p1, p2)
You can also specify a custom message and parameter values to subTest()
:
with self.subTest(msg="Checking if p1 equals p2", p1=p1, p2=p2):
Using nose
The nose testing framework supports this.
Example (the code below is the entire contents of the file containing the test):
param_list = [('a', 'a'), ('a', 'b'), ('b', 'b')]def test_generator(): for params in param_list: yield check_em, params[0], params[1]def check_em(a, b): assert a == b
The output of the nosetests command:
> nosetests -vtestgen.test_generator('a', 'a') ... oktestgen.test_generator('a', 'b') ... FAILtestgen.test_generator('b', 'b') ... ok======================================================================FAIL: testgen.test_generator('a', 'b')----------------------------------------------------------------------Traceback (most recent call last): File "/usr/lib/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "testgen.py", line 7, in check_em assert a == bAssertionError----------------------------------------------------------------------Ran 3 tests in 0.006sFAILED (failures=1)
This can be solved elegantly using Metaclasses:
import unittestl = [["foo", "a", "a",], ["bar", "a", "b"], ["lee", "b", "b"]]class TestSequenceMeta(type): def __new__(mcs, name, bases, dict): def gen_test(a, b): def test(self): self.assertEqual(a, b) return test for tname, a, b in l: test_name = "test_%s" % tname dict[test_name] = gen_test(a,b) return type.__new__(mcs, name, bases, dict)class TestSequence(unittest.TestCase): __metaclass__ = TestSequenceMetaif __name__ == '__main__': unittest.main()