image: Undefined behavior detected by miri

I just discovered an UB whilst testing another project with miri. ~~Primitive::from_slice and~~ Primitive::from_slice_mut perform an invalid unsafe cast, which is detected by miri.

EDIT: #1358 showed that it is just a matter of a wrong as_ptr fn, therefore Primitive::from_slice is likely to be unaffected.

Expected

cargo +nightly miri test should not raise any problem.

Actual behaviour

test buffer_::test::mut_iter triggers UB.

Reproduction steps

Just run miri on tests ☺️

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 16 (8 by maintainers)

Most upvoted comments

No test failed with HEAD at e0261ce3b8b612650b2a24b8da22fc5410d23fa1 (but going back to 32bae1f463ed2197bc7c53be610cc179e15c387e reproduces the UB failure from the OP).

Sanitised log (the tests I skipped because they OOMed are marked accordingly):

Nov 13 17:16:08 tarta systemd[24223]: Started /home/nabijaczleweli/.cargo/bin/cargo +nightly-2020-11-09 miri test.
Nov 13 17:16:09 tarta cargo[19983]:    Compiling image v0.23.11 (/home/nabijaczleweli/uwu/image)
Nov 13 17:16:09 tarta cargo[19983]:     Finished test [unoptimized + debuginfo] target(s) in 0.13s
Nov 13 17:16:09 tarta cargo[19983]:      Running target/x86_64-unknown-linux-gnu/debug/deps/image-3c9cbb519f35a53e
Nov 13 17:16:21 tarta cargo[19983]: running 137 tests
Nov 13 17:16:22 tarta cargo[19983]: test animation::tests::duration_approx ... ok
Nov 13 17:16:22 tarta cargo[19983]: test animation::tests::duration_outlier ... ok
Nov 13 17:16:22 tarta cargo[19983]: test animation::tests::fps_30 ... ok
Nov 13 17:16:28 tarta cargo[19983]: test animation::tests::precise ... ok
Nov 13 17:16:28 tarta cargo[19983]: test animation::tests::simple ... ok
Nov 13 17:19:58 tarta cargo[19983]: test animation::tests::small ... ok
Nov 13 17:19:58 tarta cargo[19983]: test buffer_::test::get_pixel ... ok
Nov 13 17:19:58 tarta cargo[19983]: test buffer_::test::mut_iter ... ok
Nov 13 17:19:58 tarta cargo[19983]: test buffer_::test::nonzero_width_zero_height ... ok
Nov 13 17:19:58 tarta cargo[19983]: test buffer_::test::pixels_on_large_buffer ... ok
Nov 13 17:19:58 tarta cargo[19983]: test buffer_::test::slice_buffer ... ok
Nov 13 17:19:59 tarta cargo[19983]: test buffer_::test::zero_width_nonzero_height ... ok
Nov 13 17:19:59 tarta cargo[19983]: test buffer_::test::zero_width_zero_height ... ok
Nov 13 17:19:59 tarta cargo[19983]: test codecs::bmp::decoder::test::test_bitfield_len ... ok
Nov 13 17:20:06 tarta cargo[19983]: test codecs::bmp::encoder::tests::huge_files_return_error ... ok
Nov 13 17:20:06 tarta cargo[19983]: test codecs::bmp::encoder::tests::round_trip_3px_rgb ... ok
Nov 13 17:20:26 tarta cargo[19983]: test codecs::bmp::encoder::tests::round_trip_gray ... ok
Nov 13 17:20:45 tarta cargo[19983]: test codecs::bmp::encoder::tests::round_trip_graya ... ok
Nov 13 17:20:46 tarta cargo[19983]: test codecs::bmp::encoder::tests::round_trip_single_pixel_rgb ... ok
Nov 13 17:20:46 tarta cargo[19983]: test codecs::bmp::encoder::tests::round_trip_single_pixel_rgba ... ok
Nov 13 17:20:46 tarta cargo[19983]: test codecs::farbfeld::tests::read_rect_1x2 ... ok
Nov 13 17:20:46 tarta cargo[19983]: test codecs::farbfeld::tests::read_rect_2x1 ... ok
Nov 13 17:20:47 tarta cargo[19983]: test codecs::farbfeld::tests::read_rect_2x2 ... ok
Nov 13 17:20:47 tarta cargo[19983]: test codecs::farbfeld::tests::read_rect_2x3 ... ok
Nov 13 17:20:47 tarta cargo[19983]: test codecs::farbfeld::tests::read_rect_in_stream ... ok
Nov 13 17:20:47 tarta cargo[19983]: test codecs::hdr::decoder::read_line_u8_test ... ok
Nov 13 17:20:48 tarta cargo[19983]: test codecs::hdr::decoder::split_at_first_test ... ok
Nov 13 17:20:51 tarta cargo[19983]: test codecs::hdr::encoder::noruncombine_test ... ok
Nov 13 17:20:52 tarta cargo[19983]: test codecs::hdr::encoder::runiterator_test ... ok
Nov 13 17:23:30 tarta cargo[19983]: test codecs::hdr::encoder::to_rgbe8_test ... ok
Nov 13 18:01:22 tarta cargo[19983]: test codecs::jpeg::decoder::tests::cmyk_to_rgb_correct ... ok
Nov 13 18:01:31 tarta cargo[19983]: test codecs::jpeg::decoder::tests::test_assorted_colors ... ok
Nov 13 18:01:38 tarta cargo[19983]: test codecs::jpeg::encoder::tests::grayscale_roundtrip_sanity_check ... warning: thread support is experimental. For example, Miri does not detect data races yet.
Nov 13 18:01:40 tarta cargo[19983]: ok
Nov 13 18:01:41 tarta cargo[19983]: test codecs::jpeg::encoder::tests::jfif_header_density_check ... ok
Nov 13 18:01:54 tarta cargo[19983]: test codecs::jpeg::encoder::tests::roundtrip_sanity_check ... ok
Nov 13 18:02:06 tarta cargo[19983]: test codecs::jpeg::encoder::tests::test_bgra16 ... ok
Nov 13 18:02:06 tarta cargo[19983]: test codecs::jpeg::encoder::tests::test_build_frame_header ... ok
Nov 13 18:02:06 tarta cargo[19983]: test codecs::jpeg::encoder::tests::test_build_huffman_segment ... ok
Nov 13 18:02:06 tarta cargo[19983]: test codecs::jpeg::encoder::tests::test_build_jfif_header ... ok
Nov 13 18:02:06 tarta cargo[19983]: test codecs::jpeg::encoder::tests::test_build_quantization_segment ... ok
Nov 13 18:02:07 tarta cargo[19983]: test codecs::jpeg::encoder::tests::test_build_scan_header ... ok
Nov 13 18:02:11 tarta cargo[19983]: test codecs::jpeg::encoder::tests::test_image_too_large ... ok
                                    test codecs::png::tests::ensure_no_decoder_off_by_one ... skipped
Nov 14 15:59:13 tarta cargo[31496]: test codecs::png::tests::underlying_error ... ok
Nov 14 15:59:13 tarta cargo[31496]: test codecs::pnm::autobreak::tests::test_aligned_writes ... ok
Nov 14 15:59:13 tarta cargo[31496]: test codecs::pnm::autobreak::tests::test_greater_writes ... ok
Nov 14 15:59:14 tarta cargo[31496]: test codecs::pnm::decoder::tests::pam_blackandwhite ... ok
Nov 14 15:59:15 tarta cargo[31496]: test codecs::pnm::decoder::tests::pam_grayscale ... ok
Nov 14 15:59:15 tarta cargo[31496]: test codecs::pnm::decoder::tests::pam_rgb ... ok
Nov 14 15:59:16 tarta cargo[31496]: test codecs::pnm::decoder::tests::pbm_ascii ... ok
Nov 14 15:59:16 tarta cargo[31496]: test codecs::pnm::decoder::tests::pbm_ascii_nospace ... ok
Nov 14 15:59:16 tarta cargo[31496]: test codecs::pnm::decoder::tests::pbm_binary ... ok
Nov 14 15:59:16 tarta cargo[31496]: test codecs::pnm::decoder::tests::pbm_binary_ascii_termination ... ok
Nov 14 15:59:17 tarta cargo[31496]: test codecs::pnm::decoder::tests::pgm_ascii ... ok
Nov 14 15:59:17 tarta cargo[31496]: test codecs::pnm::decoder::tests::pgm_binary ... ok
Nov 14 15:59:20 tarta cargo[31496]: test codecs::pnm::tests::roundtrip_gray ... ok
Nov 14 15:59:22 tarta cargo[31496]: test codecs::pnm::tests::roundtrip_rgb ... ok
Nov 14 15:59:23 tarta cargo[31496]: test codecs::pnm::tests::roundtrip_u16 ... ok
Nov 14 15:59:23 tarta cargo[31496]: test codecs::tga::encoder::tests::round_trip_3px_rgb ... ok
Nov 14 15:59:24 tarta cargo[31496]: test codecs::tga::encoder::tests::round_trip_gray ... ok
Nov 14 15:59:24 tarta cargo[31496]: test codecs::tga::encoder::tests::round_trip_graya ... ok
Nov 14 15:59:24 tarta cargo[31496]: test codecs::tga::encoder::tests::round_trip_single_pixel_bgr ... ok
Nov 14 15:59:24 tarta cargo[31496]: test codecs::tga::encoder::tests::round_trip_single_pixel_bgra ... ok
Nov 14 15:59:24 tarta cargo[31496]: test codecs::tga::encoder::tests::round_trip_single_pixel_rgb ... ok
Nov 14 15:59:25 tarta cargo[31496]: test codecs::tga::encoder::tests::round_trip_single_pixel_rgba ... ok
Nov 14 15:59:25 tarta cargo[31496]: test codecs::tga::encoder::tests::test_image_height_too_large ... ok
Nov 14 15:59:25 tarta cargo[31496]: test codecs::tga::encoder::tests::test_image_width_too_large ... ok
Nov 14 15:59:25 tarta cargo[31496]: test codecs::webp::vp8::test::test_add_residue ... ok
Nov 14 16:00:07 tarta cargo[31496]: test codecs::webp::vp8::test::test_avg2 ... ok
Nov 14 16:00:07 tarta cargo[31496]: test codecs::webp::vp8::test::test_avg2_specific ... ok
Nov 14 19:07:02 tarta cargo[31496]: test codecs::webp::vp8::test::test_avg3 ... ok
Nov 14 19:07:02 tarta cargo[31496]: test codecs::webp::vp8::test::test_edge_pixels ... ok
Nov 14 19:07:02 tarta cargo[31496]: test codecs::webp::vp8::test::test_predict_bhepred ... ok
Nov 14 19:07:03 tarta cargo[31496]: test codecs::webp::vp8::test::test_predict_bldpred ... ok
Nov 14 19:07:03 tarta cargo[31496]: test codecs::webp::vp8::test::test_predict_brdpred ... ok
Nov 14 19:07:03 tarta cargo[31496]: test codecs::webp::vp8::test::test_predict_bvepred ... ok
Nov 14 19:07:03 tarta cargo[31496]: test codecs::webp::vp8::test::test_top_pixels ... ok
Nov 14 19:07:03 tarta cargo[31496]: test color::tests::test_apply_with_alpha_bgr ... ok
Nov 14 19:07:03 tarta cargo[31496]: test color::tests::test_apply_with_alpha_bgra ... ok
Nov 14 19:07:04 tarta cargo[31496]: test color::tests::test_apply_with_alpha_rgb ... ok
Nov 14 19:07:04 tarta cargo[31496]: test color::tests::test_apply_with_alpha_rgba ... ok
Nov 14 19:07:04 tarta cargo[31496]: test color::tests::test_apply_without_alpha_bgr ... ok
Nov 14 19:07:04 tarta cargo[31496]: test color::tests::test_apply_without_alpha_bgra ... ok
Nov 14 19:07:04 tarta cargo[31496]: test color::tests::test_apply_without_alpha_rgb ... ok
Nov 14 19:07:04 tarta cargo[31496]: test color::tests::test_apply_without_alpha_rgba ... ok
Nov 14 19:07:05 tarta cargo[31496]: test color::tests::test_blend_luma_alpha ... ok
Nov 14 19:07:05 tarta cargo[31496]: test color::tests::test_blend_rgba ... ok
Nov 14 19:07:05 tarta cargo[31496]: test color::tests::test_lossless_conversions ... ok
Nov 14 19:07:05 tarta cargo[31496]: test color::tests::test_map_with_alpha_bgr ... ok
Nov 14 19:07:05 tarta cargo[31496]: test color::tests::test_map_with_alpha_bgra ... ok
Nov 14 19:07:06 tarta cargo[31496]: test color::tests::test_map_with_alpha_rgb ... ok
Nov 14 19:07:06 tarta cargo[31496]: test color::tests::test_map_with_alpha_rgba ... ok
Nov 14 19:07:06 tarta cargo[31496]: test color::tests::test_map_without_alpha_bgr ... ok
Nov 14 19:07:06 tarta cargo[31496]: test color::tests::test_map_without_alpha_bgra ... ok
Nov 14 19:07:06 tarta cargo[31496]: test color::tests::test_map_without_alpha_rgb ... ok
Nov 14 19:07:06 tarta cargo[31496]: test color::tests::test_map_without_alpha_rgba ... ok
Nov 14 19:07:08 tarta cargo[31496]: test dynimage::test::image_dimensions ... ok
Nov 14 19:08:16 tarta cargo[31496]: test dynimage::test::open_16bpc_png ... ok
Nov 14 19:08:16 tarta cargo[31496]: test dynimage::test::test_empty_file ... ok
Nov 14 19:08:16 tarta cargo[31496]: test error::tests::test_send_sync_stability ... ok
Nov 14 19:10:27 tarta cargo[31496]: test flat::tests::aliasing_view ... ok
Nov 14 19:10:27 tarta cargo[31496]: test flat::tests::image_buffer_conversion ... ok
Nov 14 19:10:27 tarta cargo[31496]: test flat::tests::mutable_view ... ok
Nov 14 19:10:27 tarta cargo[31496]: test flat::tests::normal_forms ... ok
Nov 14 19:10:29 tarta cargo[31496]: test image::tests::image_formats_are_recognized ... ok
Nov 14 19:10:29 tarta cargo[31496]: test image::tests::test_can_nest_views ... ok
Nov 14 19:10:29 tarta cargo[31496]: test image::tests::test_can_subimage_clone_nonmut ... ok
Nov 14 19:10:29 tarta cargo[31496]: test image::tests::test_generic_image_copy_within_bl ... ok
Nov 14 19:10:30 tarta cargo[31496]: test image::tests::test_generic_image_copy_within_br ... ok
Nov 14 19:10:30 tarta cargo[31496]: test image::tests::test_generic_image_copy_within_oob ... ok
Nov 14 19:10:30 tarta cargo[31496]: test image::tests::test_generic_image_copy_within_tl ... ok
Nov 14 19:10:30 tarta cargo[31496]: test image::tests::test_generic_image_copy_within_tr ... ok
Nov 14 19:10:30 tarta cargo[31496]: test image::tests::test_image_alpha_blending ... ok
Nov 14 19:10:31 tarta cargo[31496]: test image::tests::test_image_format_from_path ... ok
Nov 14 19:10:31 tarta cargo[31496]: test image::tests::test_in_bounds ... ok
Nov 14 19:10:35 tarta cargo[31496]: test image::tests::test_load_rect ... ok
Nov 14 19:10:35 tarta cargo[31496]: test imageops::affine::test::test_flip_horizontal ... ok
Nov 14 19:10:35 tarta cargo[31496]: test imageops::affine::test::test_flip_horizontal_in_place ... ok
Nov 14 19:10:36 tarta cargo[31496]: test imageops::affine::test::test_flip_vertical ... ok
Nov 14 19:10:36 tarta cargo[31496]: test imageops::affine::test::test_flip_vertical_in_place ... ok
Nov 14 19:10:36 tarta cargo[31496]: test imageops::affine::test::test_rotate180 ... ok
Nov 14 19:10:36 tarta cargo[31496]: test imageops::affine::test::test_rotate180_in_place ... ok
Nov 14 19:10:37 tarta cargo[31496]: test imageops::affine::test::test_rotate270 ... ok
Nov 14 19:10:37 tarta cargo[31496]: test imageops::affine::test::test_rotate90 ... ok
Nov 14 19:10:37 tarta cargo[31496]: test imageops::colorops::test::test_dither ... ok
                                    test imageops::sample::tests::test_issue_186 ... skipped
Nov 14 20:18:45 tarta cargo[9848]: test imageops::tests::test_image_coordinate_overflow ... ok
Nov 14 20:18:47 tarta cargo[9848]: test imageops::tests::test_image_horizontal_gradient_limits ... ok
Nov 14 20:18:48 tarta cargo[9848]: test imageops::tests::test_image_in_image ... ok
Nov 14 20:18:57 tarta cargo[9848]: test imageops::tests::test_image_in_image_outside_of_bounds ... ok
Nov 14 20:18:59 tarta cargo[9848]: test imageops::tests::test_image_outside_image_no_wrap_around ... ok
Nov 14 20:19:00 tarta cargo[9848]: test imageops::tests::test_image_vertical_gradient_limits ... ok
Nov 14 20:19:07 tarta cargo[9848]: test math::utils::test::resize_bounds_correctly_height ... ok
Nov 14 20:19:14 tarta cargo[9848]: test math::utils::test::resize_bounds_correctly_width ... ok
Nov 14 20:19:14 tarta cargo[9848]: test math::utils::test::resize_handles_fill ... ok
Nov 14 20:19:14 tarta cargo[9848]: test math::utils::test::resize_handles_overflow ... ok
Nov 14 20:19:15 tarta cargo[9848]: test utils::test::gray_to_luma8_skip ... ok
Nov 14 20:19:15 tarta cargo[9848]: test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 126 filtered out
Nov 14 20:19:23 tarta cargo[9848]:      Running target/x86_64-unknown-linux-gnu/debug/deps/reference_images-d8ceb47829515b8c
Nov 14 20:19:23 tarta cargo[9848]: warning: use of deprecated associated function `image::DynamicImage::to_rgba`: replaced by `to_rgba8`
Nov 14 20:19:23 tarta cargo[9848]:   --> tests/reference_images.rs:49:28
Nov 14 20:19:23 tarta cargo[9848]:    |
Nov 14 20:19:23 tarta cargo[9848]: 49 |             Ok(img) => img.to_rgba(),
Nov 14 20:19:23 tarta cargo[9848]:    |                            ^^^^^^^
Nov 14 20:19:23 tarta cargo[9848]:    |
Nov 14 20:19:23 tarta cargo[9848]:    = note: `#[warn(deprecated)]` on by default
Nov 14 20:19:24 tarta cargo[9848]: warning: use of deprecated associated function `image::DynamicImage::to_rgba`: replaced by `to_rgba8`
Nov 14 20:19:24 tarta cargo[9848]:    --> tests/reference_images.rs:159:28
Nov 14 20:19:24 tarta cargo[9848]:     |
Nov 14 20:19:24 tarta cargo[9848]: 159 |             Ok(img) => img.to_rgba(),
Nov 14 20:19:24 tarta cargo[9848]:     |                            ^^^^^^^
Nov 14 20:19:24 tarta cargo[9848]: warning: use of deprecated associated function `image::DynamicImage::to_rgba`: replaced by `to_rgba8`
Nov 14 20:19:24 tarta cargo[9848]:    --> tests/reference_images.rs:260:52
Nov 14 20:19:24 tarta cargo[9848]:     |
Nov 14 20:19:24 tarta cargo[9848]: 260 |                     Ok(img) => test_img = Some(img.to_rgba()),
Nov 14 20:19:24 tarta cargo[9848]:     |                                                    ^^^^^^^
Nov 14 20:20:02 tarta cargo[9848]: running 4 tests
Nov 14 20:20:13 tarta cargo[9848]: test bad_bmps ... ok
Nov 14 20:20:24 tarta cargo[9848]: test check_hdr_references ... warning: thread support is experimental. For example, Miri does not detect data races yet.
Nov 14 21:54:30 tarta cargo[9848]: ok
                                   test check_references ... skipped
                                   test render_images ... skipped
                                   test jqeg_qualitys ... skipped
Nov 15 17:46:39 tarta cargo[11254]: test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 4 filtered out
Nov 15 17:46:40 tarta cargo[11254]:      Running target/x86_64-unknown-linux-gnu/debug/deps/save_jpeg-63ceb4a52bcb3705
Nov 15 17:47:01 tarta cargo[11254]: running 0 tests
Nov 15 17:47:01 tarta cargo[11254]: test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out
Nov 15 17:47:01 tarta cargo[11254]:      Running target/x86_64-unknown-linux-gnu/debug/deps/truncate_images-85dea5657e967dd4
Nov 15 17:48:32 tarta cargo[11254]: running 9 tests
Nov 15 17:48:32 tarta cargo[11254]: test truncate_bmp ... ignored
Nov 15 17:48:32 tarta cargo[11254]: test truncate_farbfeld ... ignored
Nov 15 17:48:32 tarta cargo[11254]: test truncate_gif ... ignored
Nov 15 17:48:32 tarta cargo[11254]: test truncate_hdr ... ignored
Nov 15 17:48:32 tarta cargo[11254]: test truncate_ico ... ignored
Nov 15 17:48:32 tarta cargo[11254]: test truncate_jpg ... ignored
Nov 15 17:48:33 tarta cargo[11254]: test truncate_png ... ignored
Nov 15 17:48:33 tarta cargo[11254]: test truncate_tga ... ignored
Nov 15 17:48:33 tarta cargo[11254]: test truncate_tiff ... ignored
Nov 15 17:48:33 tarta cargo[11254]: test result: ok. 0 passed; 0 failed; 9 ignored; 0 measured; 0 filtered out
Nov 15 17:48:35 tarta cargo[11254]:    Doc-tests image
Nov 15 17:48:35 tarta cargo[11254]: Running doctests is not currently supported by Miri.
Nov 15 17:48:35 tarta systemd[24223]: miri.service: Succeeded.

Raw log here, just in case.

The truncation tests took over 12 hours to compile and OOMed immediately, so they remain unchecked.

If there’s anything else I can run miri on now or in future, I’m of course more than happy to.

nabijaczleweli on Nov 16, 2020

I have a 96G amd64 machine, I’ll spin the tests there as well.

nabijaczleweli on Nov 13, 2020

Good points on bytemuck being MIRI tested without issue. Then we can close this for now. That said, any additional small and quickly executed tests that cover some of the unsafe code would be appreciated, in any dependency as well 😃

@lu-zero Thanks for the dedication. Due to your efforts, we say that image had a literal CPU-month dedicated to running them (960:43.10). Feel free to stop at any point if you need your computer. If you can, comment with the log here so any one reading this later knows which tests were not covered yet.

HeroicKatora on Nov 14, 2020

I can tell you that codecs::png::tests::ensure_no_decoder_off_by_one is probably showing a pathological case in miri.

1632769 root      20   0  118.3g 107.5g 142272 S 100.0  42.4 960:43.10 miri

lu-zero on Nov 14, 2020

After 4h20 of CPU time (plus a single restart to add -Zmiri-disable-isolation 40 minutes in :v), I’m at 43/137 tests with miri. I’d rather not restart this for reasons I assume obvious, and it’s currently at 52.5G resident set, so I can hardly run another set of tests in parallel (which miri is not doing for some reason, so those four hours are on one out of the twenty four CPUs, I’m pretty sure, which kinda sucks since they’re relatively slow).

If there’s an easy way to find out which tests resolve to unsafe blocks, as you say, then I’ll be happy to restrict to those tests in future, but I don’t know of one nor particularly feel like going through over a hundred tests when I can just let’er rip overnight instead tbh.

nabijaczleweli on Nov 13, 2020

It also shouldn’t be strictly necessary to run all tests. Only those which eventually might call into unsafe code might yield interesting results in miri, apart from bugs in the compiler or standard library. I’m sure many tests are very verbose in this regard, in particular the reference image rendering test suite test a miniscule amount of unsafe code as many decoders are entirely safe.

HeroicKatora on Nov 13, 2020

I did not try, but I am trying now to run all the tests with miri. 😉

dodomorandi on Nov 12, 2020