Python unittests in Jenkins? Python unittests in Jenkins? jenkins jenkins

Python unittests in Jenkins?


sample tests:

tests.py:

# tests.pyimport randomtry:    import unittest2 as unittestexcept ImportError:    import unittestclass SimpleTest(unittest.TestCase):    @unittest.skip("demonstrating skipping")    def test_skipped(self):        self.fail("shouldn't happen")    def test_pass(self):        self.assertEqual(10, 7 + 3)    def test_fail(self):        self.assertEqual(11, 7 + 3)

JUnit with pytest

run the tests with:

py.test --junitxml results.xml tests.py

results.xml:

<?xml version="1.0" encoding="utf-8"?><testsuite errors="0" failures="1" name="pytest" skips="1" tests="2" time="0.097">    <testcase classname="tests.SimpleTest" name="test_fail" time="0.000301837921143">        <failure message="test failure">self = <tests.SimpleTest testMethod=test_fail>    def test_fail(self):>       self.assertEqual(11, 7 + 3)E       AssertionError: 11 != 10tests.py:16: AssertionError</failure>    </testcase>    <testcase classname="tests.SimpleTest" name="test_pass" time="0.000109910964966"/>    <testcase classname="tests.SimpleTest" name="test_skipped" time="0.000164031982422">        <skipped message="demonstrating skipping" type="pytest.skip">/home/damien/test-env/lib/python2.6/site-packages/_pytest/unittest.py:119: Skipped: demonstrating skipping</skipped>    </testcase></testsuite>

JUnit with nose

run the tests with:

nosetests --with-xunit

nosetests.xml:

<?xml version="1.0" encoding="UTF-8"?><testsuite name="nosetests" tests="3" errors="0" failures="1" skip="1">    <testcase classname="tests.SimpleTest" name="test_fail" time="0.000">        <failure type="exceptions.AssertionError" message="11 != 10">            <![CDATA[Traceback (most recent call last):File "/opt/python-2.6.1/lib/python2.6/site-packages/unittest2-0.5.1-py2.6.egg/unittest2/case.py", line 340, in runtestMethod()File "/home/damien/tests.py", line 16, in test_failself.assertEqual(11, 7 + 3)File "/opt/python-2.6.1/lib/python2.6/site-packages/unittest2-0.5.1-py2.6.egg/unittest2/case.py", line 521, in assertEqualassertion_func(first, second, msg=msg)File "/opt/python-2.6.1/lib/python2.6/site-packages/unittest2-0.5.1-py2.6.egg/unittest2/case.py", line 514, in _baseAssertEqualraise self.failureException(msg)AssertionError: 11 != 10]]>        </failure>    </testcase>    <testcase classname="tests.SimpleTest" name="test_pass" time="0.000"></testcase>    <testcase classname="tests.SimpleTest" name="test_skipped" time="0.000">        <skipped type="nose.plugins.skip.SkipTest" message="demonstrating skipping">            <![CDATA[SkipTest: demonstrating skipping]]>        </skipped>    </testcase></testsuite>

JUnit with nose2

You would need to use the nose2.plugins.junitxml plugin. You can configure nose2 with a config file like you would normally do, or with the --plugin command-line option.

run the tests with:

nose2 --plugin nose2.plugins.junitxml --junit-xml tests

nose2-junit.xml:

<testsuite errors="0" failures="1" name="nose2-junit" skips="1" tests="3" time="0.001">  <testcase classname="tests.SimpleTest" name="test_fail" time="0.000126">    <failure message="test failure">Traceback (most recent call last):  File "/Users/damien/Work/test2/tests.py", line 18, in test_fail    self.assertEqual(11, 7 + 3)AssertionError: 11 != 10</failure>  </testcase>  <testcase classname="tests.SimpleTest" name="test_pass" time="0.000095" />  <testcase classname="tests.SimpleTest" name="test_skipped" time="0.000058">    <skipped />  </testcase></testsuite>

JUnit with unittest-xml-reporting

Append the following to tests.py

if __name__ == '__main__':    import xmlrunner    unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test-reports'))

run the tests with:

python tests.py

test-reports/TEST-SimpleTest-20131001140629.xml:

<?xml version="1.0" ?><testsuite errors="1" failures="0" name="SimpleTest-20131001140629" tests="3" time="0.000">    <testcase classname="SimpleTest" name="test_pass" time="0.000"/>    <testcase classname="SimpleTest" name="test_fail" time="0.000">        <error message="11 != 10" type="AssertionError"><![CDATA[Traceback (most recent call last):  File "tests.py", line 16, in test_fail    self.assertEqual(11, 7 + 3)AssertionError: 11 != 10]]>     </error>    </testcase>    <testcase classname="SimpleTest" name="test_skipped" time="0.000">        <skipped message="demonstrating skipping" type="skip"/>    </testcase>    <system-out><![CDATA[]]>    </system-out>    <system-err><![CDATA[]]>    </system-err></testsuite>


I would second using nose. Basic XML reporting is now built in. Just use the --with-xunit command line option and it will produce a nosetests.xml file. For example:

nosetests --with-xunit

Then add a "Publish JUnit test result report" post build action, and fill in the "Test report XMLs" field with nosetests.xml (assuming that you ran nosetests in $WORKSPACE).


You can install the unittest-xml-reporting package to add a test runner that generates XML to the built-in unittest.

We use pytest, which has XML output built in (it's a command line option).

Either way, executing the unit tests can be done by running a shell command.