jqwik: Performance degradation with 1.4.0

Using 1.4.0 (as opposed to 1.3.10) we experience a severe performance degradation compared to 1.3.10 for some tests.

The time required to execute (at least some tests) seems to increase super-linear with the number of tries. According to a first quick glance edgeCases are heavily involved. I’ll try to investigate this further and/or come up with a reproducer in the following days.

One of the affected test classes (containing 17 tests) shows the following behavior:

tries	1.3.10	1.4.0
100	2.5s	4.0s
1000	8s	80s
2000	14s	228s

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 32 (16 by maintainers)

Most upvoted comments

@jlink I think there is no need to rash this decision to drop backward compatibility so fast:

I don’t know if what we are doing with jqwik is representative or an outlier.
Everyone who is dissatisfied with 1.4.0 can still use 1.3.10 (as we do for now).
Also I have no idea how many installations rely on the unique behavior (for us the adaption was a very minor thing).
Although I guess if your adaptions fix the problem with my example, they will probably also fix our other tests, I can’t confirm this for sure until I’ve tried it (tomorrow).

tmohme on Feb 14, 2021

@osi Turns out, even a “well enough” implementation requires more effort than I’m willing to invest.

I’ll go ahead with a 1.5.0 soonish.

jlink on Feb 14, 2021