MINGW-packages: `Cabal` is broken after updating `mingw-w64-x86_64-crt-git` package

Hi, I can’t build sample haskell application on Windows with MSYS2 by Cabal tool. It happened after update mingw-w64-x86_64-crt-git package.

We are maintainers of virtual-environments repository and this issue blocked the rollout of the new Windows images for GitHub Actions.

Sample repro: hello world application: https://github.com/vsafonkin/sample-msys-crt-git-package-bug Successful build with mingw-w64-x86_64-crt-git package version 9.0.0.6373.5be8fcd83-1: https://github.com/vsafonkin/sample-msys-crt-git-package-bug/actions/runs/1876488973 Failed build with mingw-w64-x86_64-crt-git package version 9.0.0.6373.5be8fcd83-2: https://github.com/vsafonkin/sample-msys-crt-git-package-bug/actions/runs/1876499498

Error:

Failed to build exe:haskell-say-exe from haskell-say-1.0.0.0.
Build log (
C:\cabal\logs\ghc-9.2.1\haskell-say-1.0.0.0-7116a2e9f87[5](https://github.com/vsafonkin/sample-msys-crt-git-package-bug/runs/5275068585?check_suite_focus=true#step:4:5)0afc93c33140953997584183[6](https://github.com/vsafonkin/sample-msys-crt-git-package-bug/runs/5275068585?check_suite_focus=true#step:4:6)ecf.log
):
Preprocessing executable 'haskell-say-exe' for haskell-say-1.0.0.0..
Building executable 'haskell-say-exe' for haskell-say-1.0.0.0..
[1 of 2] Compiling Main             ( app\Main.hs, dist\build\haskell-say-exe\haskell-say-exe-tmp\Main.o )
[2 of 2] Compiling Paths_haskell_say ( dist\build\haskell-say-exe\autogen\Paths_haskell_say.hs, dist\build\haskell-say-exe\haskell-say-exe-tmp\Paths_haskell_say.o )
cabal.exe: Failed to build exe:haskell-say-exe from haskell-say-1.0.0.0. See
Linking dist\\build\\haskell-say-exe\\haskell-say-exe.exe ...
the build log above for details.
C://tools//ghc-9.2.1//mingw//bin/ld.exe: C:/tools/ghc-9.2.1/mingw/bin/../lib/gcc/x[8](https://github.com/vsafonkin/sample-msys-crt-git-package-bug/runs/5275068585?check_suite_focus=true#step:4:8)6_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/lib/../lib/crt2.o:crtexe.c:(.rdata$.refptr.mingw_app_type[.refptr.mingw_app_type]+0x0): undefined reference to `mingw_app_type'

C://tools//ghc-[9](https://github.com/vsafonkin/sample-msys-crt-git-package-bug/runs/5275068585?check_suite_focus=true#step:4:9).2.1//mingw//bin/ld.exe: C:/tools/ghc-9.2.1/mingw/bin/../lib/gcc/x86_64-w64-mingw32/[10](https://github.com/vsafonkin/sample-msys-crt-git-package-bug/runs/5275068585?check_suite_focus=true#step:4:10).2.0/../../../../x86_64-w64-mingw[32](https://github.com/vsafonkin/sample-msys-crt-git-package-bug/runs/5275068585?check_suite_focus=true#step:4:32)/lib/../lib/crt2.o:crtexe.c:(.rdata$.refptr.mingw_initcharmax[.refptr.mingw_initcharmax]+0x0): undefined reference to `mingw_initcharmax'
C://tools//ghc-9.2.1//mingw//bin/ld.exe: C:/tools/ghc-9.2.1/mingw/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/lib/../lib/crt2.o:crtexe.c:(.rdata$.refptr.mingw_initltssuo_force[.refptr.mingw_initltssuo_force]+0x0): undefined reference to `mingw_initltssuo_force'
C://tools//ghc-9.2.1//mingw//bin/ld.exe: C:/tools/ghc-9.2.1/mingw/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/lib/../lib/crt2.o:crtexe.c:(.rdata$.refptr.mingw_initltsdyn_force[.refptr.mingw_initltsdyn_force]+0x0): undefined reference to `mingw_initltsdyn_force'
C://tools//ghc-9.2.1//mingw//bin/ld.exe: C:/tools/ghc-9.2.1/mingw/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/lib/../lib/crt2.o:crtexe.c:(.rdata$.refptr.mingw_initltsdrot_force[.refptr.mingw_initltsdrot_force]+0x0): undefined reference to `mingw_initltsdrot_force'
collect2.exe: error: ld returned 1 exit status
`gcc.exe' failed in phase `Linker'. (Exit code: 1)

About this issue

Original URL
State: open
Created 2 years ago
Comments: 18 (6 by maintainers)

Most upvoted comments

@hasufell thanks. This ABI incompatibility isn’t easy to fix post mortem, but can be easily worked around.

The error is coming from

 .bss           0x0000000000532370       0x10 C:\ghcup\msys64\mingw64\lib/libmingw32.a(lib64_libmingw32_a-tlssup.o)
                0x0000000000532370                __mingw_initltssuo_force
                0x0000000000532378                __mingw_initltsdrot_force
                0x000000000053237c                _tls_index
                0x0000000000532374                __mingw_initltsdyn_force

Though the symbol is internal it seems it’s use is spread across more than one package. The problem is that the folder libmingw32.a is in isn’t explicitly overridden by the GHC driver. This means that paths cabal pass it can override it as it’s an implicit path.

We should add the libraries path to the driver, can you file a ticket on the GHC bug tracker.

There’s only one workaround I can think of that’ll work for both ghc and ghci which is to add this folder as well to the extra lib-dirs for cabal. You must add this before anything else

This folder is essentially $ghc_top_install_dir/mingw/x86_64-w64-mingw32/lib/ that should fix it and prevent future ABI breaks like these.

Mistuke on Feb 21, 2022

There’s only one workaround I can think of that’ll work for both ghc and ghci which is to add this folder as well to the extra lib-dirs for cabal. You must add this before anything else

I see a problem with this… because it depends on the GHC version in use. global extra-lib-dirs isn’t smart enough for this.

hasufell on Feb 21, 2022

From the above log posted it looks like a external mingw version (I think included in ghc) is used.

Yes, GHC ships a very small toolchain with it, not a full msys2 distribution. And it’s hardcoded to use that one afaik.

hasufell on Feb 21, 2022

I can reproduce this. The way you can go around this issue is to disable msys2 integration in cabal:

remove extra-include-dirs: ... from the global cabal config (I think on virtual-machines it’s in C:\cabal\config? Otherwise it’s somewhere in AppData)
remove extra-lib-dirs: ... as well

For user this means that cabal won’t be able to see msys2 installed packages and can’t link to them.

I was under the impression that these settings are not enabled anyway: https://github.com/actions/virtual-environments/blob/60dc1b5f7a1a046eceaea0e5667beefdead26d65/images/win/scripts/Installers/Install-Haskell.ps1#L41

Invoke-Command -ScriptBlock ([ScriptBlock]::Create($bootstrapHaskell)) -ArgumentList $false, $true, $true, $false, $true, $false, $false, $ghcupPrefix, "", $msysPath, $cabalDir

The 5th argument NoAdjustCabalConfig should instruct the installer to not perform this config change.

It’s possible that that is a bug in the installer: https://gitlab.haskell.org/haskell/ghcup-hs/-/blob/ab36d4418e508fdd9d5955c7ee7c5996a7c4d759/scripts/bootstrap/bootstrap-haskell#L528 (this should probably return 0).

However, this will still leave users CI broken that need msys2 libdir/include dirs.

hasufell on Feb 21, 2022