Jenkins validates the JMeter build as successful when it’s actually failed
Set "Use error thresholds on single build" from 0 to 1
.
This should mean that the jenkins build fails if at least 1 error is in your JMeter test case, where a 0 ignores the error count (If I remember this correctly).
Sounds like a Performance Plugin issue, I believe you should report it via Jenkins Bug Tracker
In the meantime to work this around I would suggest returning non-zero exit code for the JMeter build step, the fastest and the easiest way is running your JMeter test using Taurus tool as a wrapper. Taurus has powerful Pass/Fail Criteria subsystem where you can define the conditions of your build failure. If the condition will be met Taurus execution step will be finished with code 3
. Jenkins is smart enough to automatically fail command-line tasks with non-zero exit code.
Minimal working Taurus config file:
---execution: scenario: my-testscenarios: my-test: script: C:\Users\maria\Desktop\Automation\WIP\Test-page.jmxservices:- module: passfail criteria: - succ<100%, stop as failed
More information, if needed: Meetup Recap on Using Taurus to Automate JMeter and Jenkins Tests
You can also change JMeter exit code directly form the JMeter test via i.e. JSR223 Scripting Elements like:
if (some condition) { System.exit(1);}
but in this case you won't have much flexibility in terms of setting failure criteria
I set my Select evaluation mode to standard mode, Set the radio button on Error Threshold and set Unstable and Failed to 1