Recommendation: Disable coverage, black & flake8 when running tox
When running tox
, all of coverage
, black
& flake8
run.
This doesn't seem beneficial, since:
- Pre-commit hook enforces that commits can't be made locally, let alone pushed, when any of the outputs of those 3 fails.
- None of these 3 change the code logic, so it doesn't affect the test results.
In practice, I find that including them leads to:
- extra time spent waiting for tests to run
- extra output which has to be visually filtered before & after test results output (= added brain strain)
There is already a dedicated command to run black
and flake8
: pre-commit run --all-files
This is a more beneficial method of applying these tools currently, since:
- When run via
pre-commit
,black
automatically updates files with issues detected. - When run via
tox
,black
doesn't automatically update files with issues detected. -
flake8
commonly fails whenblack
fails.
So I recommend disabling their inclusion when running tox
; their inclusion seems counter-productive.
The only caveat is that coverage
doesn't seem to be run by pre-commit
.
So it may make sense to only include coverage
when running tox
. However, please see related concern (2) below which causes issues with that.
Two related concerns (these may need to be separate issues, I don't yet understand well enough the boilerplate structure to know right now):
- When
tox
is run and these 3 tools are included, the output consistently referencesthreading
yet in reality those lines relate tocoverage
. For example:
threading: OK ✔ in 12.51 seconds
- When running
tox
with an individual TestCase or test method, output indicates that "threading" has failed. In fact what's happened is thatcoverage
has failed, and the reason is that it's assessed there aren't enough unit tests. It succeeds when all tests are run. Socoverage
isn't currently configured to be compatible withtox
when specific tests/test cases are passed totox
.
e.g. poetry poe tox -- project/test/test_logging.py::LoggingTestCase