Jenkins - Running install tests on remote machine and reporting results back to Jenkins
I didn't actually need any plugins for this. I simply set up the job to run on the slave, had a build step that ran some tests and generated an xml file (similar to the jUnit xml results) and then added a post build step to look at the jUnit results (even though the tests weren't jUnit tests).
This worked a charm and I have builds being marked as unstable if they fail tests that I specify, like did they install an rpm and did such and such happen.
I was able to get the latest stable builds as the latest stable builds are coppied to a file server anyway, any failed builds don't go there so that was simple.