What is the JUnit XML format specification that Hudson supports?
I did a similar thing a few months ago, and it turned out this simple format was enough for Hudson to accept it as a test protocol:
<testsuite tests="3"> <testcase classname="foo1" name="ASuccessfulTest"/> <testcase classname="foo2" name="AnotherSuccessfulTest"/> <testcase classname="foo3" name="AFailingTest"> <failure type="NotEnoughFoo"> details about failure </failure> </testcase></testsuite>
This question has answers with more details: Spec. for JUnit XML Output
I just grabbed the junit-4.xsd that others have linked to and used a tool named XMLSpear to convert the schema to a blank XML file with the options shown below. This is the (slightly cleaned up) result:
<?xml version="1.0" encoding="UTF-8"?><testsuites disabled="" errors="" failures="" name="" tests="" time=""> <testsuite disabled="" errors="" failures="" hostname="" id="" name="" package="" skipped="" tests="" time="" timestamp=""> <properties> <property name="" value=""/> </properties> <testcase assertions="" classname="" name="" status="" time=""> <skipped/> <error message="" type=""/> <failure message="" type=""/> <system-out/> <system-err/> </testcase> <system-out/> <system-err/> </testsuite></testsuites>
Some of these items can occur multiple times:
- There can only be one
testsuites
element, since that’s how XML works, but there can be multipletestsuite
elements within thetestsuites
element. - Each
properties
element can have multipleproperty
children. - Each
testsuite
element can have multipletestcase
children. - Each
testcase
element can have multipleerror
,failure
,system-out
, orsystem-err
children.
The top answer of the question Anders Lindahl refers to an xsd file.
Personally I found this xsd file also very useful (I don't remember how I found that one). It looks a bit less intimidating, and as far as I used it, all the elements and attributes seem to be recognized by Jenkins (v1.451)
One thing though: when adding multiple <failure ...
elements, only one was retained in Jenkins. When creating the xml file, I now concatenate all the failures in one.
Update 2016-11 The link is broken now. A better alternative is this page from cubic.org: JUnit XML reporting file format, where a nice effort has been taken to provide a sensible documented example. Example and xsd are copied below, but their page looks waay nicer.
sample JUnit XML file
<?xml version="1.0" encoding="UTF-8"?><!-- a description of the JUnit XML format and how Jenkins parses it. See also junit.xsd --><!-- if only a single testsuite element is present, the testsuites element can be omitted. All attributes are optional. --><testsuites disabled="" <!-- total number of disabled tests from all testsuites. --> errors="" <!-- total number of tests with error result from all testsuites. --> failures="" <!-- total number of failed tests from all testsuites. --> name="" tests="" <!-- total number of successful tests from all testsuites. --> time="" <!-- time in seconds to execute all test suites. --> > <!-- testsuite can appear multiple times, if contained in a testsuites element. It can also be the root element. --> <testsuite name="" <!-- Full (class) name of the test for non-aggregated testsuite documents. Class name without the package for aggregated testsuites documents. Required --> tests="" <!-- The total number of tests in the suite, required. --> disabled="" <!-- the total number of disabled tests in the suite. optional --> errors="" <!-- The total number of tests in the suite that errored. An errored test is one that had an unanticipated problem, for example an unchecked throwable; or a problem with the implementation of the test. optional --> failures="" <!-- The total number of tests in the suite that failed. A failure is a test which the code has explicitly failed by using the mechanisms for that purpose. e.g., via an assertEquals. optional --> hostname="" <!-- Host on which the tests were executed. 'localhost' should be used if the hostname cannot be determined. optional --> id="" <!-- Starts at 0 for the first testsuite and is incremented by 1 for each following testsuite --> package="" <!-- Derived from testsuite/@name in the non-aggregated documents. optional --> skipped="" <!-- The total number of skipped tests. optional --> time="" <!-- Time taken (in seconds) to execute the tests in the suite. optional --> timestamp="" <!-- when the test was executed in ISO 8601 format (2014-01-21T16:17:18). Timezone may not be specified. optional --> > <!-- Properties (e.g., environment settings) set during test execution. The properties element can appear 0 or once. --> <properties> <!-- property can appear multiple times. The name and value attributres are required. --> <property name="" value=""/> </properties> <!-- testcase can appear multiple times, see /testsuites/testsuite@tests --> <testcase name="" <!-- Name of the test method, required. --> assertions="" <!-- number of assertions in the test case. optional --> classname="" <!-- Full class name for the class the test method is in. required --> status="" time="" <!-- Time taken (in seconds) to execute the test. optional --> > <!-- If the test was not executed or failed, you can specify one the skipped, error or failure elements. --> <!-- skipped can appear 0 or once. optional --> <skipped/> <!-- Indicates that the test errored. An errored test is one that had an unanticipated problem. For example an unchecked throwable or a problem with the implementation of the test. Contains as a text node relevant data for the error, for example a stack trace. optional --> <error message="" <!-- The error message. e.g., if a java exception is thrown, the return value of getMessage() --> type="" <!-- The type of error that occured. e.g., if a java execption is thrown the full class name of the exception. --> ></error> <!-- Indicates that the test failed. A failure is a test which the code has explicitly failed by using the mechanisms for that purpose. For example via an assertEquals. Contains as a text node relevant data for the failure, e.g., a stack trace. optional --> <failure message="" <!-- The message specified in the assert. --> type="" <!-- The type of the assert. --> ></failure> <!-- Data that was written to standard out while the test was executed. optional --> <system-out></system-out> <!-- Data that was written to standard error while the test was executed. optional --> <system-err></system-err> </testcase> <!-- Data that was written to standard out while the test suite was executed. optional --> <system-out></system-out> <!-- Data that was written to standard error while the test suite was executed. optional --> <system-err></system-err> </testsuite></testsuites>
JUnit XSD file
<?xml version="1.0" encoding="UTF-8" ?><!-- from https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-model/src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/junit-4.xsd --><xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"> <xs:element name="failure"> <xs:complexType mixed="true"> <xs:attribute name="type" type="xs:string" use="optional"/> <xs:attribute name="message" type="xs:string" use="optional"/> </xs:complexType> </xs:element> <xs:element name="error"> <xs:complexType mixed="true"> <xs:attribute name="type" type="xs:string" use="optional"/> <xs:attribute name="message" type="xs:string" use="optional"/> </xs:complexType> </xs:element> <xs:element name="properties"> <xs:complexType> <xs:sequence> <xs:element ref="property" maxOccurs="unbounded"/> </xs:sequence> </xs:complexType> </xs:element> <xs:element name="property"> <xs:complexType> <xs:attribute name="name" type="xs:string" use="required"/> <xs:attribute name="value" type="xs:string" use="required"/> </xs:complexType> </xs:element> <xs:element name="skipped" type="xs:string"/> <xs:element name="system-err" type="xs:string"/> <xs:element name="system-out" type="xs:string"/> <xs:element name="testcase"> <xs:complexType> <xs:sequence> <xs:element ref="skipped" minOccurs="0" maxOccurs="1"/> <xs:element ref="error" minOccurs="0" maxOccurs="unbounded"/> <xs:element ref="failure" minOccurs="0" maxOccurs="unbounded"/> <xs:element ref="system-out" minOccurs="0" maxOccurs="unbounded"/> <xs:element ref="system-err" minOccurs="0" maxOccurs="unbounded"/> </xs:sequence> <xs:attribute name="name" type="xs:string" use="required"/> <xs:attribute name="assertions" type="xs:string" use="optional"/> <xs:attribute name="time" type="xs:string" use="optional"/> <xs:attribute name="classname" type="xs:string" use="optional"/> <xs:attribute name="status" type="xs:string" use="optional"/> </xs:complexType> </xs:element> <xs:element name="testsuite"> <xs:complexType> <xs:sequence> <xs:element ref="properties" minOccurs="0" maxOccurs="1"/> <xs:element ref="testcase" minOccurs="0" maxOccurs="unbounded"/> <xs:element ref="system-out" minOccurs="0" maxOccurs="1"/> <xs:element ref="system-err" minOccurs="0" maxOccurs="1"/> </xs:sequence> <xs:attribute name="name" type="xs:string" use="required"/> <xs:attribute name="tests" type="xs:string" use="required"/> <xs:attribute name="failures" type="xs:string" use="optional"/> <xs:attribute name="errors" type="xs:string" use="optional"/> <xs:attribute name="time" type="xs:string" use="optional"/> <xs:attribute name="disabled" type="xs:string" use="optional"/> <xs:attribute name="skipped" type="xs:string" use="optional"/> <xs:attribute name="timestamp" type="xs:string" use="optional"/> <xs:attribute name="hostname" type="xs:string" use="optional"/> <xs:attribute name="id" type="xs:string" use="optional"/> <xs:attribute name="package" type="xs:string" use="optional"/> </xs:complexType> </xs:element> <xs:element name="testsuites"> <xs:complexType> <xs:sequence> <xs:element ref="testsuite" minOccurs="0" maxOccurs="unbounded"/> </xs:sequence> <xs:attribute name="name" type="xs:string" use="optional"/> <xs:attribute name="time" type="xs:string" use="optional"/> <xs:attribute name="tests" type="xs:string" use="optional"/> <xs:attribute name="failures" type="xs:string" use="optional"/> <xs:attribute name="disabled" type="xs:string" use="optional"/> <xs:attribute name="errors" type="xs:string" use="optional"/> </xs:complexType> </xs:element></xs:schema>