jqwik: ContainerGenerator#next might yield different results for the same seed due to state in noDuplicatesHadToBeSwitchedOff

Testing Problem

I think Generator#next should yield the same results when given the same Random inputs. Unfortunately, Generator#next might flip the generation algorithm and reuse it for the subsequent executions, so the generated value depends on both input random (which is expected), and past history of executions (which is not expected)

I do not know what are the consequences, however, I think non-reproducibility makes shrinking worse.

Suggested Solution

Avoid stateful flip of the generation algorithm.

For instance: a) Temporary remove boolean noDuplicatesHadToBeSwitchedOff and the relevant code that flips sizeGenerator b) Try querying size estimations from the downstream generator

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 16 (7 by maintainers)

Commits related to this issue

Most upvoted comments

I more or less went with your suggestion; additionally, I added a check for the maximum unique elements to make the number of cases where a repeated fail will slow down generation less probable.

Has been deployed to “1.7.0-SNAPSHOT”

Optimizing memoization remains an open issue.