How to categorize build failures with jenkins?

The Build Failure Analyzer plugin lets you set regular expressions for categorizing current and future builds. Once the regular expressions have been set, detection is automatic for current and future builds. This may be exactly what you're after.

I have also implemented a lightweight, manual failure categorization plug-in for Jenkins, which provides a simple drop-down and additional text field for categorizing failures by hand. This was done for a research project studying build failures. Categorizations are currently saved as separate XMLs in each build's directory, to have no impact on Jenkins' own files. Categories are displayed in build listings and in a separate categorization page for each build.

I regret to say that the plugin is not currently shared anywhere, as it contains some hard-codings for a specific project. If there is demand for such a plug-in, I can re-visit the code and publish it open source. However, for your purposes it seems like the automatic solution of Build Failure Analyzer would be more suitable.

The plug-in is very simple and similar things are not hard to implement. If you wish to make something similar yourself, or perhaps extend the automated Build Failure Analyzer plug-in to fit your needs, the Jenkins plugin tutorial and Extend Jenkins pages are a good starting point for Jenkins plug-in development. For persistence, getting familiar with XStream would be beneficial, as that is what Jenkins and most plug-ins internally use to serialize data to XML.

For reporting purposes you can use build description (to edit it manually in order to see how it works go to http://[jenkins_server]/job/[job_name]/[build_number]/editDescription, or click on 'add description' on the build status page).

Programmatically you can set via Jenkins CLI (set-build-description command); or from Groovy via Jenkins Java API - Run::setDescription() method.

Groovy scripts can be invoked either via Groovy Plugin or Groovy PostBuild Plugin, or both.

This tool I worked on, might help you: https://github.com/nacuteodor/SearchInJenkinsLogs . This can be integrated with a Jenkins job. You can find a searched text specifying a regular expression in builds artifacts, or tests failures and it will print the build nodes and failed tests reports links sorted by build and tests. Also, it has a feature to group the common similar failures using a maximum difference threshold, sorted by group failures count.

You could use this tool for finding crashes in artifacts logs, that you would like to be reported and fixed a.s.a.p, and you can make the job sent an email notification or fail the tool job build. I will might add some other features in the future.