bitcoin: Overview of slow unit tests
At last IRC meeting we’ve discussed trying to speed up the unit tests, as they’ve become much too slow.
For this purpose I’ve run test_bitcoin --log_level=test_suite
and picked out the slowest from the list.
The top 20 is (updated 2018-04-14 by @lucash-dev):
Test | Time (μs) |
---|---|
test_big_witness_transaction |
39133014 |
test_CheckQueue_Correct_Random |
14808019 |
knapsack_solver_test |
9304171 |
checkinputs_test |
6370348 |
CreateNewBlock_validity |
6105558 |
test_CheckQueue_Memory |
4230330 |
versionbits_test |
3835090 |
coins_cache_simulation_test |
3205869 |
updatecoins_simulation_test |
2499840 |
merkle_test |
2391688 |
rescan |
1900315 |
SelectCoins_test |
1731901 |
passphrase |
1730167 |
ListCoins |
1689793 |
test_CheckQueue_Catches_Failure |
1563869 |
test_CheckQueue_UniqueCheck |
1521864 |
PrevectorTestInt |
1409636 |
test_CheckQueue_Correct_Max |
1401687 |
bnb_search_test |
1147974 |
cuckoocache_generations |
978637 |
I think (open for discussion, of course) it is unreasonable for a unit test case to take more than ~1s (mind you, we have 242 of them, so each test is supposed to be quick) so we should take a look at each of these up to merkle_test
.
It could be as simple in some cases as getting rid of (reasonably heavy) BOOST_*
tests in inner loops, moving the checks up to a higher level.
Edit: I added the script to create this table to maintainer-tools: https://github.com/bitcoin-core/bitcoin-maintainer-tools/blob/master/unittest-statistics.py
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 27 (26 by maintainers)
Commits related to this issue
- Merge #10099: Slightly Improve Unit Tests for Checkqueue 8c2f4b888 Expose more parallelism with relaxed atomics (suggested in #9938). Fix a test to check the exclusive or of two properties rather tha... — committed to bitcoin/bitcoin by sipa 7 years ago
- Merge #13309: Directly operate with CMutableTransaction in SignSignature 6b8b63af1461dc11ffd813401e2c36fa44656715 Generic TransactionSignatureCreator works with both CTransaction and CMutableTransact... — committed to bitcoin/bitcoin by laanwj 6 years ago
- Merge #13404: [tests] speed up of tx_validationcache_tests by reusing of CTransaction. ebebedce20 speed up of tx_validationcache_tests by reusing of CTransaction. (lucash.dev@gmail.com) Pull request... — committed to bitcoin/bitcoin by deleted user 6 years ago
- Merge #13419: [tests] Speed up knapsack_solver_test by not recreating wallet 100 times. a679109be40491222c458fdbef58f68509dae0bd Speed up knapsack_solver_test by not recreating wallet 100 times. (luc... — committed to bitcoin/bitcoin by laanwj 6 years ago
- Merge #10099: Slightly Improve Unit Tests for Checkqueue 8c2f4b888 Expose more parallelism with relaxed atomics (suggested in #9938). Fix a test to check the exclusive or of two properties rather tha... — committed to PastaPastaPasta/dash by sipa 7 years ago
- Merge #10099: Slightly Improve Unit Tests for Checkqueue 8c2f4b888 Expose more parallelism with relaxed atomics (suggested in #9938). Fix a test to check the exclusive or of two properties rather tha... — committed to PastaPastaPasta/dash by sipa 7 years ago
- Merge #10099: Slightly Improve Unit Tests for Checkqueue 8c2f4b888 Expose more parallelism with relaxed atomics (suggested in #9938). Fix a test to check the exclusive or of two properties rather tha... — committed to PastaPastaPasta/dash by sipa 7 years ago
- Merge #10099: Slightly Improve Unit Tests for Checkqueue 8c2f4b888 Expose more parallelism with relaxed atomics (suggested in #9938). Fix a test to check the exclusive or of two properties rather tha... — committed to PastaPastaPasta/dash by sipa 7 years ago
- Merge #10099: Slightly Improve Unit Tests for Checkqueue 8c2f4b888 Expose more parallelism with relaxed atomics (suggested in #9938). Fix a test to check the exclusive or of two properties rather tha... — committed to PastaPastaPasta/dash by sipa 7 years ago
- Merge #10099: Slightly Improve Unit Tests for Checkqueue 8c2f4b888 Expose more parallelism with relaxed atomics (suggested in #9938). Fix a test to check the exclusive or of two properties rather tha... — committed to PastaPastaPasta/dash by sipa 7 years ago
- Merge #10099: Slightly Improve Unit Tests for Checkqueue 8c2f4b888 Expose more parallelism with relaxed atomics (suggested in #9938). Fix a test to check the exclusive or of two properties rather tha... — committed to PastaPastaPasta/dash by sipa 7 years ago
- Merge #10099: Slightly Improve Unit Tests for Checkqueue 8c2f4b888 Expose more parallelism with relaxed atomics (suggested in #9938). Fix a test to check the exclusive or of two properties rather tha... — committed to PastaPastaPasta/dash by sipa 7 years ago
- Merge #10099: Slightly Improve Unit Tests for Checkqueue 8c2f4b888 Expose more parallelism with relaxed atomics (suggested in #9938). Fix a test to check the exclusive or of two properties rather tha... — committed to PastaPastaPasta/dash by sipa 7 years ago
- Merge #10099: Slightly Improve Unit Tests for Checkqueue 8c2f4b888 Expose more parallelism with relaxed atomics (suggested in #9938). Fix a test to check the exclusive or of two properties rather tha... — committed to PastaPastaPasta/dash by sipa 7 years ago
- Merge #13309: Directly operate with CMutableTransaction in SignSignature 6b8b63af1461dc11ffd813401e2c36fa44656715 Generic TransactionSignatureCreator works with both CTransaction and CMutableTransact... — committed to UdjinM6/dash by laanwj 6 years ago
- Merge #13404: [tests] speed up of tx_validationcache_tests by reusing of CTransaction. ebebedce20 speed up of tx_validationcache_tests by reusing of CTransaction. (lucash.dev@gmail.com) Pull request... — committed to UdjinM6/dash by deleted user 6 years ago
Latest run on my machine (2019 Macbook pro) still found a bunch of tests over a second:
test_big_witness_transaction
CreateNewBlock_validity
knapsack_solver_test
test_CheckQueue_Correct_Random
updatecoins_simulation_test
checkinputs_test
SelectCoins_test
coins_cache_simulation_test
versionbits_test
Though in the middle of the list above, my old laptop (2013 MacBook Pro) runs into a bunch of trouble with
test_CheckQueue_Correct_Random
. It consistently comes above 30 seconds. Here are the others that are over a second:test_CheckQueue_Correct_Random
test_big_witness_transaction
CreateNewBlock_validity
knapsack_solver_test
test_CheckQueue_Memory
updatecoins_simulation_test
checkinputs_test
coins_cache_simulation_test
versionbits_test
Should we add a “good first issue” label to this?