How can I see normal print output created during pytest run?
pytest captures the stdout from individual tests and displays them only on certain conditions, along with the summary of the tests it prints by default.
Extra summary info can be shown using the '-r' option:
shows the captured output of passed tests.
shows the captured output of failed tests (default behaviour).
The formatting of the output is prettier with -r than with -s.
Is there any way to print to the console AND capture the output so that it shows in the junit report?
In UNIX, this is commonly referred to as teeing. Ideally, teeing rather than capturing would be the py.test default. Non-ideally, neither py.test nor any existing third-party py.test plugin (...that I know of, anyway) supports teeing – despite Python trivially supporting teeing out-of-the-box.
Monkey-patching py.test to do anything unsupported is non-trivial. Why? Because:
- Most py.test functionality is locked behind a private
_pytestpackage not intended to be externally imported. Attempting to do so without knowing what you're doing typically results in the public
pytestpackage raising obscure exceptions at runtime. Thanks alot, py.test. Really robust architecture you got there.
- Even when you do figure out how to monkey-patch the private
_pytestAPI in a safe manner, you have to do so before running the public
pytestpackage run by the external
py.testcommand. You cannot do this in a plugin (e.g., a top-level
conftestmodule in your test suite). By the time py.test lazily gets around to dynamically importing your plugin, any py.test class you wanted to monkey-patch has long since been instantiated – and you do not have access to that instance. This implies that, if you want your monkey-patch to be meaningfully applied, you can no longer safely run the external
py.testcommand. Instead, you have to wrap the running of that command with a custom setuptools
testcommand that (in order):
- Monkey-patches the private
- Calls the public
pytest.main()function to run the
- Monkey-patches the private
This answer monkey-patches py.test's
--capture=no options to capture stderr but not stdout. By default, these options capture neither stderr nor stdout. This isn't quite teeing, of course. But every great journey begins with a tedious prequel everyone forgets in five years.
Why do this? I shall now tell you. My py.test-driven test suite contains slow functional tests. Displaying the stdout of these tests is helpful and reassuring, preventing leycec from reaching for
killall -9 py.test when yet another long-running functional test fails to do anything for weeks on end. Displaying the stderr of these tests, however, prevents py.test from reporting exception tracebacks on test failures. Which is completely unhelpful. Hence, we coerce py.test to capture stderr but not stdout.
Before we get to it, this answer assumes you already have a custom setuptools
test command invoking py.test. If you don't, see the Manual Integration subsection of py.test's well-written Good Practices page.
Do not install pytest-runner, a third-party setuptools plugin providing a custom setuptools
test command also invoking py.test. If pytest-runner is already installed, you'll probably need to uninstall that pip3 package and then adopt the manual approach linked to above.
Assuming you followed the instructions in Manual Integration highlighted above, your codebase should now contain a
PyTest.run_tests() method. Modify this method to resemble:
class PyTest(TestCommand): . . . def run_tests(self): # Import the public "pytest" package *BEFORE* the private "_pytest" # package. While importation order is typically ignorable, imports can # technically have side effects. Tragicomically, that is the case here. # Importing the public "pytest" package establishes runtime # configuration required by submodules of the private "_pytest" package. # The former *MUST* always be imported before the latter. Failing to do # so raises obtuse exceptions at runtime... which is bad. import pytest from _pytest.capture import CaptureManager, FDCapture, MultiCapture # If the private method to be monkey-patched no longer exists, py.test # is either broken or unsupported. In either case, raise an exception. if not hasattr(CaptureManager, '_getcapture'): from distutils.errors import DistutilsClassError raise DistutilsClassError( 'Class "pytest.capture.CaptureManager" method _getcapture() ' 'not found. The current version of py.test is either ' 'broken (unlikely) or unsupported (likely).' ) # Old method to be monkey-patched. _getcapture_old = CaptureManager._getcapture # New method applying this monkey-patch. Note the use of: # # * "out=False", *NOT* capturing stdout. # * "err=True", capturing stderr. def _getcapture_new(self, method): if method == "no": return MultiCapture( out=False, err=True, in_=False, Capture=FDCapture) else: return _getcapture_old(self, method) # Replace the old with the new method. CaptureManager._getcapture = _getcapture_new # Run py.test with all passed arguments. errno = pytest.main(self.pytest_args) sys.exit(errno)
To enable this monkey-patch, run py.test as follows:
python setup.py test -a "-s"
Stderr but not stdout will now be captured. Nifty!
Extending the above monkey-patch to tee stdout and stderr is left as an exercise to the reader with a barrel-full of free time.