add slowest 100 tests to pytest output #84
Conversation
| @@ -100,7 +100,7 @@ if [ "x${DTEST_SPLIT_CHUNK}" != "x" ] ; then | |||
| SPLIT_TESTS=$(split -n r/${DTEST_SPLIT_CHUNK} ${WORKSPACE}/test_list.txt) | |||
| fi | |||
|
|
|||
| PYTEST_OPTS="-vv --log-cli-level=DEBUG --junit-xml=nosetests.xml --junit-prefix=${DTEST_TARGET} -s" | |||
| PYTEST_OPTS="-vv --durations=100 --log-cli-level=DEBUG --junit-xml=nosetests.xml --junit-prefix=${DTEST_TARGET} -s" | |||
There was a problem hiding this comment.
awesome idea!
note, this will be the slowest 100 in each split.
and… you can sort all tests by duration here: https://ci-cassandra.apache.org/job/Cassandra-4.1/lastSuccessfulBuild/testReport/
|
hah! So what you're saying is this is redundant and would require more TLC to centralize. I'm good w/that so long as there's a place to work on this; I can also call it out on the project status updates. Maybe getting more focus on cleaning up CI will help push back against us accumulating more and more bloat in the suites. Some of those runtimes on that linked page are... really really really long.
|
they'e accumulated, so you gotta drill down, which is kinda annoying as it hides what the longest test methods might actually be… |
On each run, we can trivially add the slowest 100 runs to the test output to give us something to target with potential optimization, comparison to in-jvm dtest, or deprecation.
100 is... very arbitrary. 10 might be fine. Or 50. Or 0 so it prints all! No real opinion there.
In a similar vein to the JIRA on unit test runtimes being lopsided and hot-spotty: https://issues.apache.org/jira/browse/CASSANDRA-17371