vvenc: Tests fail with VS 2022 17.8.0: 37% tests passed, 24 tests failed out of 38

Description

VVenC stopped working for me, and I suspected that the upgrade of VS 2022 was the problem. I tried on another workstation that I did not use for some time. It had version 17.5.1 and my tests that include VVenC were running successfully. After that I upgraded VS 2022 to the latest version 17.8.0 and then vvencFFapp crashed like it did on my first system.

To create a meaningful bug report that does not bring in the specifics of my project, I tried to run the VVenC tests and (fortunately) they fail. I assume that they should all pass. At this point I am running out of systems that have an older VS 2022 version so I cannot try.

How to reproduce

1. Open x64 Native Tools Command prompt for VS 2022
2. cd C:\Data\dev
3. mkdir fraunhoferhhi
4. cd fraunhoferhhi
5. git clone https://github.com/fraunhoferhhi/vvenc.git
6. cd vvenv
7. mkdir build
8. cd build
9. cmake .. -DCMAKE_BUILD_TYPE=RelWithDebInfo -G Ninja
10. ninja
11. ninja test

Software versions

• VS 2022 17.8.0 (MSVC 19.38.33130.0)
• CMake 3.24.2
• VVenC revision 2217e10

Output of cmake .. -DCMAKE_BUILD_TYPE=RelWithDebInfo -G Ninja

-- The C compiler identification is MSVC 19.38.33130.0
-- The CXX compiler identification is MSVC 19.38.33130.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Professional/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Professional/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CMAKE_MODULE_PATH: updating module path to: C:/Data/dev/fraunhoferhhi/vvenc/cmake/modules
-- Performing Test SUPPORTED_Werror_unused_command_line_argument
-- Performing Test SUPPORTED_Werror_unused_command_line_argument - Failed
-- Performing Test SUPPORTED/arch:AVX2
-- Performing Test SUPPORTED/arch:AVX2 - Success
-- Performing Test HAVE_INTRIN_mm_storeu_si16
-- Performing Test HAVE_INTRIN_mm_storeu_si16 - Success
-- Performing Test HAVE_INTRIN_mm_storeu_si32
-- Performing Test HAVE_INTRIN_mm_storeu_si32 - Success
-- Performing Test HAVE_INTRIN_mm_storeu_si64
-- Performing Test HAVE_INTRIN_mm_storeu_si64 - Success
-- Performing Test HAVE_INTRIN_mm_loadu_si32
-- Performing Test HAVE_INTRIN_mm_loadu_si32 - Success
-- Performing Test HAVE_INTRIN_mm_loadu_si64
-- Performing Test HAVE_INTRIN_mm_loadu_si64 - Success
-- Performing Test HAVE_INTRIN_mm_cvtsi128_si64
-- Performing Test HAVE_INTRIN_mm_cvtsi128_si64 - Success
-- Performing Test HAVE_INTRIN_mm_extract_epi64
-- Performing Test HAVE_INTRIN_mm_extract_epi64 - Success
-- Performing Test HAVE_INTRIN_mm256_zeroupper
-- Performing Test HAVE_INTRIN_mm256_zeroupper - Success
-- Performing Test HAVE_INTRIN_mm256_loadu2_m128i
-- Performing Test HAVE_INTRIN_mm256_loadu2_m128i - Success
-- Performing Test HAVE_INTRIN_mm256_set_m128i
-- Performing Test HAVE_INTRIN_mm256_set_m128i - Success
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Found Threads: TRUE
CMake Warning at CMakeLists.txt:227 (message):
  ITT_PATH c:/Program Files (x86)/IntelSWTools/VTune Amplifier/ not found,
  ignoring option VVCEncoderLib_ENABLE_ITT


-- Configuring done
-- Generating done
-- Build files have been written to: C:/Data/dev/fraunhoferhhi/vvenc/build

Output of ninja

[128/128] Linking CXX executable C:\Data\dev\fraunhoferhhi\vvenc\bin\relwithdebinfo-static\vvencFFapp.exe

Output of ninja test

C:\Data\dev\fraunhoferhhi\vvenc\build>ninja test
[0/1] Running tests...Test project C:/Data/dev/fraunhoferhhi/vvenc/build
      Start  1: Test_vvencinterfacetest
 1/38 Test  #1: Test_vvencinterfacetest ..........................***Exception: Illegal  1.06 sec
      Start  2: Test_vvenclibtest-parameter_range
 2/38 Test  #2: Test_vvenclibtest-parameter_range ................   Passed    0.68 sec
      Start  3: Test_vvenclibtest-calling_order
 3/38 Test  #3: Test_vvenclibtest-calling_order ..................   Passed    0.11 sec
      Start  4: Test_vvenclibtest-input_params
 4/38 Test  #4: Test_vvenclibtest-input_params ...................   Passed    0.08 sec
      Start  5: Test_vvenclibtest-sdk_default
 5/38 Test  #5: Test_vvenclibtest-sdk_default ....................   Passed    0.23 sec
      Start  6: Test_vvenclibtest-sdk_stringapi_interface
 6/38 Test  #6: Test_vvenclibtest-sdk_stringapi_interface ........   Passed    0.12 sec
      Start  7: Test_vvenclibtest-timestamps
 7/38 Test  #7: Test_vvenclibtest-timestamps .....................   Passed    7.70 sec
      Start  8: Test_vvencapp-tooltest
 8/38 Test  #8: Test_vvencapp-tooltest ...........................***Exception: Illegal  1.98 sec
      Start  9: Test_vvencFFapp-tooltest
 9/38 Test  #9: Test_vvencFFapp-tooltest .........................***Exception: Illegal  1.98 sec
      Start 10: Test_compare_output-tooltest
Failed test dependencies: Test_vvencFFapp-tooltest Test_vvencapp-tooltest
10/38 Test #10: Test_compare_output-tooltest .....................***Not Run   0.00 sec
      Start 11: Test_vvencFFapp-tooltest-Scalar
11/38 Test #11: Test_vvencFFapp-tooltest-Scalar ..................   Passed    2.08 sec
      Start 12: Test_compare_output-tooltest-Scalar
Failed test dependencies: Test_vvencapp-tooltest
12/38 Test #12: Test_compare_output-tooltest-Scalar ..............***Not Run   0.00 sec
      Start 13: Test_vvencapp-faster
13/38 Test #13: Test_vvencapp-faster .............................***Exception: Illegal  0.16 sec
      Start 14: Test_vvencFFapp-faster
14/38 Test #14: Test_vvencFFapp-faster ...........................***Exception: Illegal  0.19 sec
      Start 15: Test_compare_output-faster
Failed test dependencies: Test_vvencFFapp-faster Test_vvencapp-faster
15/38 Test #15: Test_compare_output-faster .......................***Not Run   0.00 sec
      Start 16: Test_vvencapp-fast
16/38 Test #16: Test_vvencapp-fast ...............................***Exception: Illegal  0.23 sec
      Start 17: Test_vvencFFapp-fast
17/38 Test #17: Test_vvencFFapp-fast .............................***Exception: Illegal  0.22 sec
      Start 18: Test_compare_output-fast
Failed test dependencies: Test_vvencFFapp-fast Test_vvencapp-fast
18/38 Test #18: Test_compare_output-fast .........................***Not Run   0.00 sec
      Start 19: Test_vvencapp-medium
19/38 Test #19: Test_vvencapp-medium .............................***Exception: Illegal  0.35 sec
      Start 20: Test_vvencFFapp-medium
20/38 Test #20: Test_vvencFFapp-medium ...........................***Exception: Illegal  0.35 sec
      Start 21: Test_compare_output-medium
Failed test dependencies: Test_vvencFFapp-medium Test_vvencapp-medium
21/38 Test #21: Test_compare_output-medium .......................***Not Run   0.00 sec
      Start 22: Test_vvencapp-slow
22/38 Test #22: Test_vvencapp-slow ...............................   Passed    0.56 sec
      Start 23: Test_vvencFFapp-slow
23/38 Test #23: Test_vvencFFapp-slow .............................   Passed    0.59 sec
      Start 24: Test_compare_output-slow
24/38 Test #24: Test_compare_output-slow .........................   Passed    0.12 sec
      Start 25: Test_vvencapp-medium_noqpa_0thr
25/38 Test #25: Test_vvencapp-medium_noqpa_0thr ..................***Exception: Illegal  0.23 sec
      Start 26: Test_vvencFFapp-medium_noqpa_0thr
26/38 Test #26: Test_vvencFFapp-medium_noqpa_0thr ................***Exception: Illegal  0.24 sec
      Start 27: Test_compare_output-medium_noqpa_0thr
Failed test dependencies: Test_vvencFFapp-medium_noqpa_0thr Test_vvencapp-medium_noqpa_0thr
27/38 Test #27: Test_compare_output-medium_noqpa_0thr ............***Not Run   0.00 sec
      Start 28: Test_vvencFFapp-lowdelay_medium
28/38 Test #28: Test_vvencFFapp-lowdelay_medium ..................   Passed    0.51 sec
      Start 29: Test_vvencapp-medium_rc2p
29/38 Test #29: Test_vvencapp-medium_rc2p ........................***Exception: Illegal  0.42 sec
      Start 30: Test_vvencFFapp-medium_rc2p
30/38 Test #30: Test_vvencFFapp-medium_rc2p ......................***Exception: Illegal  0.41 sec
      Start 31: Test_compare_output-medium_rc2p
Failed test dependencies: Test_vvencFFapp-medium_rc2p Test_vvencapp-medium_rc2p
31/38 Test #31: Test_compare_output-medium_rc2p ..................***Not Run   0.00 sec
      Start 32: Test_vvencFFapp-medium_rc2p_statsFile1_exp
32/38 Test #32: Test_vvencFFapp-medium_rc2p_statsFile1_exp .......   Passed    0.15 sec
      Start 33: Test_vvencFFapp-medium_rc2p_statsFile2_exp
33/38 Test #33: Test_vvencFFapp-medium_rc2p_statsFile2_exp .......***Exception: Illegal  0.39 sec
      Start 34: Test_compare_output-medium_rc2p_statsFile_exp
Failed test dependencies: Test_vvencFFapp-medium_rc2p_statsFile2_exp Test_vvencapp-medium_rc2p
34/38 Test #34: Test_compare_output-medium_rc2p_statsFile_exp ....***Not Run   0.00 sec
      Start 35: Test_vvencapp-medium_rc2p_statsFile1_easy
35/38 Test #35: Test_vvencapp-medium_rc2p_statsFile1_easy ........   Passed    0.15 sec
      Start 36: Test_vvencapp-medium_rc2p_statsFile2_easy
36/38 Test #36: Test_vvencapp-medium_rc2p_statsFile2_easy ........***Exception: Illegal  0.39 sec
      Start 37: Test_compare_output-medium_rc2p_statsFile_easy
Failed test dependencies: Test_vvencFFapp-medium_rc2p_statsFile2_exp Test_vvencapp-medium_rc2p_statsFile2_easy
37/38 Test #37: Test_compare_output-medium_rc2p_statsFile_easy ...***Not Run   0.00 sec
      Start 38: Cleanup_remove_temp_files
38/38 Test #38: Cleanup_remove_temp_files ........................   Passed    0.12 sec

37% tests passed, 24 tests failed out of 38

Total Test time (real) =  21.95 sec

The following tests FAILED:
          1 - Test_vvencinterfacetest (ILLEGAL)
          8 - Test_vvencapp-tooltest (ILLEGAL)
          9 - Test_vvencFFapp-tooltest (ILLEGAL)
         10 - Test_compare_output-tooltest (Not Run)
         12 - Test_compare_output-tooltest-Scalar (Not Run)
         13 - Test_vvencapp-faster (ILLEGAL)
         14 - Test_vvencFFapp-faster (ILLEGAL)
         15 - Test_compare_output-faster (Not Run)
         16 - Test_vvencapp-fast (ILLEGAL)
         17 - Test_vvencFFapp-fast (ILLEGAL)
         18 - Test_compare_output-fast (Not Run)
         19 - Test_vvencapp-medium (ILLEGAL)
         20 - Test_vvencFFapp-medium (ILLEGAL)
         21 - Test_compare_output-medium (Not Run)
         25 - Test_vvencapp-medium_noqpa_0thr (ILLEGAL)
         26 - Test_vvencFFapp-medium_noqpa_0thr (ILLEGAL)
         27 - Test_compare_output-medium_noqpa_0thr (Not Run)
         29 - Test_vvencapp-medium_rc2p (ILLEGAL)
         30 - Test_vvencFFapp-medium_rc2p (ILLEGAL)
         31 - Test_compare_output-medium_rc2p (Not Run)
         33 - Test_vvencFFapp-medium_rc2p_statsFile2_exp (ILLEGAL)
         34 - Test_compare_output-medium_rc2p_statsFile_exp (Not Run)
         36 - Test_vvencapp-medium_rc2p_statsFile2_easy (ILLEGAL)
         37 - Test_compare_output-medium_rc2p_statsFile_easy (Not Run)
Errors while running CTest
Output from these tests are in: C:/Data/dev/fraunhoferhhi/vvenc/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.

FAILED: CMakeFiles/test.util
cmd.exe /C "cd /D C:\Data\dev\fraunhoferhhi\vvenc\build && "C:\Program Files\CMake\bin\ctest.exe" --force-new-ctest-process "
ninja: build stopped: subcommand failed.

Please let me know if there is something else that I can test.