FAQ | This is a LIVE service | Changelog

Recommendation: Disable coverage, black & flake8 when running tox

When running tox, all of coverage, black & flake8 run.

This doesn't seem beneficial, since:

  • Pre-commit hook enforces that commits can't be made locally, let alone pushed, when any of the outputs of those 3 fails.
  • None of these 3 change the code logic, so it doesn't affect the test results.

In practice, I find that including them leads to:

  • extra time spent waiting for tests to run
  • extra output which has to be visually filtered before & after test results output (= added brain strain)

There is already a dedicated command to run black and flake8: pre-commit run --all-files

This is a more beneficial method of applying these tools currently, since:

  • When run via pre-commit, black automatically updates files with issues detected.
  • When run via tox, black doesn't automatically update files with issues detected.
  • flake8 commonly fails when black fails.

So I recommend disabling their inclusion when running tox; their inclusion seems counter-productive.

The only caveat is that coverage doesn't seem to be run by pre-commit.

So it may make sense to only include coverage when running tox. However, please see related concern (2) below which causes issues with that.

Two related concerns (these may need to be separate issues, I don't yet understand well enough the boilerplate structure to know right now):

  1. When tox is run and these 3 tools are included, the output consistently references threading yet in reality those lines relate to coverage. For example:
threading: OK ✔ in 12.51 seconds
  1. When running tox with an individual TestCase or test method, output indicates that "threading" has failed. In fact what's happened is that coverage has failed, and the reason is that it's assessed there aren't enough unit tests. It succeeds when all tests are run. So coverage isn't currently configured to be compatible with tox when specific tests/test cases are passed to tox.

e.g. poetry poe tox -- project/test/test_logging.py::LoggingTestCase

Edited by Nick Brown