How do you generate dynamic (parameterized) unit tests in Python? How do you generate dynamic (parameterized) unit tests in Python? python python

How do you generate dynamic (parameterized) unit tests in Python?

This is called "parametrization".

There are several tools that support this approach. E.g.:

The resulting code looks like this:

from parameterized import parameterizedclass TestSequence(unittest.TestCase):    @parameterized.expand([        ["foo", "a", "a",],        ["bar", "a", "b"],        ["lee", "b", "b"],    ])    def test_sequence(self, name, a, b):        self.assertEqual(a,b)

Which will generate the tests:

test_sequence_0_foo (__main__.TestSequence) ... oktest_sequence_1_bar (__main__.TestSequence) ... FAILtest_sequence_2_lee (__main__.TestSequence) ... ok======================================================================FAIL: test_sequence_1_bar (__main__.TestSequence)----------------------------------------------------------------------Traceback (most recent call last):  File "/usr/local/lib/python2.7/site-packages/parameterized/", line 233, in <lambda>    standalone_func = lambda *a: func(*(a + p.args), **p.kwargs)  File "", line 12, in test_sequence    self.assertEqual(a,b)AssertionError: 'a' != 'b'

For historical reasons I'll leave the original answer circa 2008):

I use something like this:

import unittestl = [["foo", "a", "a",], ["bar", "a", "b"], ["lee", "b", "b"]]class TestSequense(unittest.TestCase):    passdef test_generator(a, b):    def test(self):        self.assertEqual(a,b)    return testif __name__ == '__main__':    for t in l:        test_name = 'test_%s' % t[0]        test = test_generator(t[1], t[2])        setattr(TestSequense, test_name, test)    unittest.main()

Using unittest (since 3.4)

Since Python 3.4, the standard library unittest package has the subTest context manager.

See the documentation:


from unittest import TestCaseparam_list = [('a', 'a'), ('a', 'b'), ('b', 'b')]class TestDemonstrateSubtest(TestCase):    def test_works_as_expected(self):        for p1, p2 in param_list:            with self.subTest():                self.assertEqual(p1, p2)

You can also specify a custom message and parameter values to subTest():

with self.subTest(msg="Checking if p1 equals p2", p1=p1, p2=p2):

Using nose

The nose testing framework supports this.

Example (the code below is the entire contents of the file containing the test):

param_list = [('a', 'a'), ('a', 'b'), ('b', 'b')]def test_generator():    for params in param_list:        yield check_em, params[0], params[1]def check_em(a, b):    assert a == b

The output of the nosetests command:

> nosetests -vtestgen.test_generator('a', 'a') ... oktestgen.test_generator('a', 'b') ... FAILtestgen.test_generator('b', 'b') ... ok======================================================================FAIL: testgen.test_generator('a', 'b')----------------------------------------------------------------------Traceback (most recent call last):  File "/usr/lib/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/", line 203, in runTest    self.test(*self.arg)  File "", line 7, in check_em    assert a == bAssertionError----------------------------------------------------------------------Ran 3 tests in 0.006sFAILED (failures=1)

This can be solved elegantly using Metaclasses:

import unittestl = [["foo", "a", "a",], ["bar", "a", "b"], ["lee", "b", "b"]]class TestSequenceMeta(type):    def __new__(mcs, name, bases, dict):        def gen_test(a, b):            def test(self):                self.assertEqual(a, b)            return test        for tname, a, b in l:            test_name = "test_%s" % tname            dict[test_name] = gen_test(a,b)        return type.__new__(mcs, name, bases, dict)class TestSequence(unittest.TestCase):    __metaclass__ = TestSequenceMetaif __name__ == '__main__':    unittest.main()