jqwik: ContainerGenerator#next might yield different results for the same seed due to state in noDuplicatesHadToBeSwitchedOff
Testing Problem
I think Generator#next
should yield the same results when given the same Random
inputs.
Unfortunately, Generator#next
might flip the generation algorithm and reuse it for the subsequent executions, so the generated value depends on both input random
(which is expected), and past history of executions
(which is not expected)
I do not know what are the consequences, however, I think non-reproducibility makes shrinking worse.
Suggested Solution
Avoid stateful flip of the generation algorithm.
For instance:
a) Temporary remove boolean noDuplicatesHadToBeSwitchedOff
and the relevant code that flips sizeGenerator
b) Try querying size estimations from the downstream generator
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 16 (7 by maintainers)
Commits related to this issue
- ContainerGenerator#next might yield different results for the same seed due to state in noDuplicatesHadToBeSwitchedOff Generator#next should yield the same results when given the same Random inputs. ... — committed to vlsi/jqwik by vlsi 2 years ago
- ContainerGenerator#next might yield different results for the same seed due to state in noDuplicatesHadToBeSwitchedOff Generator#next should yield the same results when given the same Random inputs. ... — committed to vlsi/jqwik by vlsi 2 years ago
- ContainerGenerator#next might yield different results for the same seed due to state in noDuplicatesHadToBeSwitchedOff Generator#next should yield the same results when given the same Random inputs. ... — committed to vlsi/jqwik by vlsi 2 years ago
- ContainerGenerator#next might yield different results for the same seed due to state in noDuplicatesHadToBeSwitchedOff Generator#next should yield the same results when given the same Random inputs. ... — committed to vlsi/jqwik by vlsi 2 years ago
- ContainerGenerator#next might yield different results for the same seed due to state in noDuplicatesHadToBeSwitchedOff Generator#next should yield the same results when given the same Random inputs. ... — committed to vlsi/jqwik by vlsi 2 years ago
- ContainerGenerator#next might yield different results for the same seed due to state in noDuplicatesHadToBeSwitchedOff Generator#next should yield the same results when given the same Random inputs. ... — committed to vlsi/jqwik by vlsi 2 years ago
I more or less went with your suggestion; additionally, I added a check for the maximum unique elements to make the number of cases where a repeated fail will slow down generation less probable.
Has been deployed to “1.7.0-SNAPSHOT”
Optimizing memoization remains an open issue.