Continuous integration: ensure new commits are covered with tests

I know you can configure Jenkins to verify that there is at least one test file as part of the commit. That would not assure good test coverage, but at least you would know there was some kind of test related changes.

Some coverage tools (like cobertura) support excluding packages. This way, you can exclude all the old code (assuming it can be pattern matched) and have cobertura check only new code (which covers new commits).

I hope this helps.

For option 2, you can use the Jenkins JaCoCo plugin to track the code coverage for each build and set the build result to passed or failed depending on the coverage metrics.

I like option 1 better, too, but I don't know of a built-in way for Jenkins to do this. It should be fairly easy (at least at the class level) to post-process the coverage data and combine it with the SVN revision info, something like:

1. Parse the JaCoCo output files and find classes that have 0% coverage
2. Get the file(s) that changed for this build from the SVN revision details (Jenkins makes the revision numbers available in environment variables, SVN_REVISION if there is only one for this build or SVN_REVISION_1, SVN_REVISION_2, ... for multiple)
3. Print an error message if any of the changed classes have 0% coverage
4. Use the Jenkins Text Finder plugin to fail the build if the error message is printed.

This isn't a full solution, it gets trickier for new methods or lines that aren't covered by tests. Gives me an idea for a new Jenkins plugin ;-)