quickcheck: Infinite Repetition/Never Ending Test with `f32` and `f64`.

Issue

I’ve got a test that uses the following structure:

quickcheck! {
    fn f32_quickcheck(f: f32) -> bool {
        if f.is_special() {
            true
        } else {
            let string = f.to_string();
            f == string.parse::<f32>()
        }
    }

The minimum failing input is:

f32::from_bits(0b11001111000000000000000000000000);   // -2147483600.0

Adding print debugging with --nocapture shows that this value, -2147483600.0, is called repeatedly, without ever stopping, if the checks pass. The actual implementation uses parsing for an arbitrary radix (using big-integer arithmetic), and a float formatter, but pressing enter in the terminal has printed values rapidly being produced afterwards, showing that the quickcheck is completing each iteration faster than the newline can show (and therefore that the test is stalling out indefinitely due to quickcheck, not the actual test code).

A similar result also occurs for f64 with the following structure:

quickcheck! {
    fn f64_quickcheck(f: f64) -> bool {
        if f.is_special() {
            true
        } else {
            let string = f.to_string();
            f == string.parse::<f64>()
        }
    }

The minimal failing input is:

f64::from_bits(0b1100001111100000000000000000000000000000000000000000000000000000) // f=-9223372036854776000.0

It’s worth noting that these have a practically identical bit pattern, which may help debugging this issue.

The currently rustc/QuickCheck version is:

$ rustc --version
rustc 1.55.0-nightly (b41936b92 2021-07-20)

[[package]]
name = "quickcheck"
version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "588f6378e4dd99458b60ec275b4477add41ce4fa9f64dcba6f15adccb19b50d6"
dependencies = [
 "env_logger",
 "log",
 "rand 0.8.4",
]

Debugging/Solution

Is there any way to print the seed so I can more reliably debug these issues? Although the same infinite loop is reproducing on each subsequent run, this is somewhat difficult to debug, given that it uses a random seed and never exits, making it very hard to submit a PR.

About this issue

Original URL
State: open
Created 3 years ago
Comments: 17

Commits related to this issue

Added a formatter for generic radix floats. - Added more tests for binary floats. - Added support for mandatory signs in integer writers. - Added support for generic radix floats. - Added extensive u... — committed to Alexhuszagh/rust-lexical-experimental by Alexhuszagh 3 years ago

Most upvoted comments

Also reproduces with i32::MIN but not with e.g. i32::MIN + 1.

Reproducer:

use quickcheck::Arbitrary;
i32::MIN.shrink().for_each(|v| println!("{}", v));

Looking at the SignedShrinker, it’s quite obvious that the shrinker is quaranteed to yield a result if it was initialized from i32::MIN: https://github.com/BurntSushi/quickcheck/blob/defde6fb0ce20b0c8c4e672aa9ae821f7d1f5b38/src/arbitrary.rs#L827 Which results in the endless iteration.

I’m honestly surprised this didn’t surface any earlier.

neithernut on Aug 9, 2021

@neithernut Seems like it has, both #285 and this bug seem to be fixed by the changes. I can’t reproduce it exactly with the cases, since running 50 or so repeats never had had those specific bit patterns occur, however, all the failing cases in the shrinker before had a bit pattern of 0b1100111XXXXXXXXXXXXXXXXXXXXXXXXX, so I created the following test:

#[cfg(test)]
mod tests {
    use quickcheck::quickcheck;

    quickcheck! {
        fn case_failure(a: f32) -> bool {
            let mask = 0b11001110000000000000000000000000;
            a.to_bits() & mask != mask
        }
    }
}

The minimum failing case, as expected was 0b11001110000000000000000000000000, or -536870900.0, which is fixed in your branch, but overflows the stack in the master branch. This now seems to be a further extension of #285.

0b11001000000000000000000000000000 does not cause a stack overflow on either, while 0b11001100000000000000000000000000 does produce a stack overflow for the master branch, so I believe this is a useful metric that the issue is fixed.

Alexhuszagh on Aug 30, 2021

Can you verify that #296 solves your problem?

neithernut on Aug 9, 2021