Running tests as usual against docker containers or dockerize tests?

I prefer your option (3) i.e. to include test code in the production deployable artifact (the docker image)

Will quote Alister Scott from GTAC 2015 which I attended:

Don’t be afraid to add testability specific features to your app that don’t serve a functional purpose. I recently had to get new tyres on my car and realized that a lot of tyres have testability features called tread indicators. These don’t serve a functional purpose

For integration and e2e tests, i.e. tests that require more than 1 docker image to be used, I prefer CI tool that, through docker-compose, and a separate git repo for these tests, orchestrates the creation of all containers that are needed for the larger test. Again the docker images used should be the exact same as for production except what varies is the configuration (e.g. environment variables) that make the tests point to test data and/or staging services.

I don't think you'd dockerize the tests themselves as the process that runs the tests.

For example if you need to run unit tests in php with phpunit, you would dockerize phpunit and use that to run tests against your code, and similarly for any other test framework you use (i.e. testng, junit... etc.)

Here's an example of the Dockerfile I use to encapsulate Java/Maven/TestNG for a Java test project which includes some selenium tests:

FROM maven:3-jdk-8# Selectively add stuff we needCOPY pom.xml /usr/src/testng/# Get a clean build immediately after and 'go-offline' to improve subsequent buildsRUN cd /usr/src/testng && mvn dependency:go-offlineCOPY src /usr/src/testng/srcWORKDIR /usr/src/testng/# Additional support files that's needed but not for the buildCOPY supportfiles /usr/src/testng/supportfilesCMD [ "mvn test" ]

Look closely at the screenshot I provide on this page: https://github.com/djangofan/karate-test-prime-example This is an example project I made that shows how to do a test container in context with a service and returning an exit code at the end of the tests.

Then when your code is released by a pipeline, before it goes live, your Docker container will include an exit code that allows your pipeline to decide whether to roll it back or not.