swift: SwiftCompilerSources doesn't work on Windows ARM64

Description

The compiler binary from the Windows ARM64 toolchain build crashes (segfaults) right after it’s launched and it cannot even compile a hello world program.

https://download.swift.org/development/windows10-arm64/swift-DEVELOPMENT-SNAPSHOT-2024-06-03-a/swift-DEVELOPMENT-SNAPSHOT-2024-06-03-a-windows10-arm64.exe https://ci-external.swift.org/job/swift-main-windows-toolchain-arm64/

Reproduction

Compile the hello world swift program

Stack dump

swiftc hello.swift -o hello
error: compile command failed due to exception 5 (use -v to see invocation)
Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the crash backtrace.
Stack dump:
0.      Program arguments: C:/Users/hiroshi/AppData/Local/Programs/Swift/Toolchains/0.0.0+Asserts/usr/bin/swift-frontend.exe -frontend -c -primary-file C:\\Users\\hiroshi\\hello.swift -target aarch64-unknown-windows-msvc -Xllvm -aarch64-use-tbi -disable-objc-interop -sdk C:\\Users\\hiroshi\\AppData\\Local\\Programs\\Swift\\Platforms\\0.0.0\\Windows.platform\\Developer\\SDKs\\Windows.sdk -color-diagnostics -empty-abi-descriptor -Xcc -working-directory -Xcc C:\\Users\\hiroshi -resource-dir C:\\Users\\hiroshi\\AppData\\Local\\Programs\\Swift\\Toolchains\\0.0.0+Asserts\\usr\\lib\\swift -module-name hello -plugin-path C:\\Users\\hiroshi\\AppData\\Local\\Programs\\Swift\\Toolchains\\0.0.0+Asserts\\usr\\bin -plugin-path C:\\Users\\hiroshi\\AppData\\Local\\Programs\\Swift\\Toolchains\\0.0.0+Asserts\\usr\\local\\bin -o C:\\Users\\hiroshi\\AppData\\Local\\Temp\\TemporaryDirectory.M79sPs\\hello-1.o
1.      Swift version 6.0-dev (LLVM 6a4cb0f4db7bec7, Swift 76d5dd44894ccb0)
2.      Compiling with effective version 5.10
3.      While evaluating request ExecuteSILPipelineRequest(Run pipelines { SILGen Passes } on SIL for hello)
4.      While running pass #1 SILFunctionTransform "LifetimeDependenceInsertion" on SILFunction "@main".
Exception Code: 0xC0000005
 #0 0x00007ff734b3c154 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0x13fc154)
 #1 0x00007ff7338bac08 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0x17ac08)
 #2 0x00007ff73377b018 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0x3b018)
 #3 0x00007ff733757b78 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0x17b78)
 #4 0x00007ff73432a5e8 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0xbea5e8)
 #5 0x00007ff734347e0c (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0xc07e0c)
 #6 0x00007ff734346d20 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0xc06d20)
 #7 0x00007ff734340310 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0xc00310)
 #8 0x00007ff734340670 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0xc00670)
 #9 0x00007ff73433ffec (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0xbfffec)
#10 0x00007ff7343937a8 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0xc537a8)
#11 0x00007ff734337b14 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0xbf7b14)
#12 0x00007ff7343406f4 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0xc006f4)
#13 0x00007ff73432a664 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0xbea664)
#14 0x00007ff733d87370 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0x647370)
#15 0x00007ff733b1d9d4 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0x3dd9d4)
#16 0x00007ff733b1e5a0 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0x3de5a0)
#17 0x00007ff733b1394c (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0x3d394c)
#18 0x00007ff733b24df8 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0x3e4df8)
#19 0x00007ff733b1d414 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0x3dd414)
#20 0x00007ff733b1d690 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0x3dd690)
#21 0x00007ff733b1f540 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0x3df540)
#22 0x00007ff733aaf364 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0x36f364)
#23 0x00007ff733aaeef8 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0x36eef8)
#24 0x00007ff7399aac08 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0x626ac08)
#25 0x00007ff7399aaca4 (C:\Users\hiroshi\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swift-frontend.exe+0x626aca4)
#26 0x00007ff84ce12310 (C:\Windows\System32\KERNEL32.DLL+0x12310)
#27 0x00007ff84f395b2c (C:\Windows\SYSTEM32\ntdll.dll+0x75b2c)

Expected behavior

No crash

Environment

Windows ARM64

Additional information

No response

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 64 (64 by maintainers)

Commits related to this issue

Most upvoted comments

We discussed this last week and it looks like the easiest way to go forward is to directly upgrade the hosttools to 6.0. First we need to merge the fixes to 6.0 (next week) then we can upgrade to 6.0 for Windows. This has the advantage that we don’t need to merge the fixes to 5.10. cc @shahmishal

I also locally tested bootstrapping the Windows/ARM64 toolchains by using a release/6.0 toolchain

Thanks!

CI is now upgraded to new version of Visual Studio / MSVC version. @compnerd verified the issue was solved with the version change.

@hjyamauchi Thanks for the ping! It will take a few weeks until we can make the upgrade. We’ll let you know when we start the process.

That’s great!

My local main branch build is looking good though there is currently a different build issue in the win arm64 CI.

I have up https://github.com/swiftlang/swift/pull/78032 as a draft to update the pinned toolchain for the main branch to the above 6.0 build and enable SwiftCompilerSources for win/arm64 in the main build.

CC @eeckstein

I locally tested the new 6.0 build: https://download.swift.org/swift-6.0-branch/windows10-arm64/swift-6.0-DEVELOPMENT-SNAPSHOT-2024-12-03-a/swift-6.0-DEVELOPMENT-SNAPSHOT-2024-12-03-a-windows10-arm64.exe a little bit and it seems in a working condition.

I’m testing a main branch build using this new 6.0 build as the pinned toolchain (hosttools) now. Will update later

yes, that should be it

Right, I enabled it for Windows after 6.0 (and then disabled it again for Windows arm64)

I managed to figure out a set of additional PRs to get the failing tests fixed in https://github.com/swiftlang/swift/pull/77277. It’s ready for review. @rjmccall @eeckstein @compnerd

The PRs for release/6.0 have been approved and ready to merge.

Created cherrypick PRs for release/5.10:

However,

An update:

Both the inline bridging mode and cherrypicking the fixes to 5.10 and using it to bootstrap main at head locally worked fine.

So it seems like if we can get the PR in one way or another, I think that the rest should go smoothly.

CC @eeckstein @egorzhdan @compnerd @rjmccall @hyp @DougGregor

A plan to get the Windows ARM64 compiler working again:

With https://github.com/swiftlang/swift/pull/76324 included, I can locally confirm that the Windows ARM64 compiler is a working state as far as I can tell.

There were the two categories of the fixes so far, (a) adding missing inreg to sret in IRGen and (b) adding constructors to bridged C++ types to match the struct return calling convention between Swift/C++ in the SwiftCompilerSources code.

Noting (a) is in the codegen and we need that fix on the bootstrap/pinned toolchain side, as opposed to the currently-being-built toolchain side, we need to have the fix in and update the bootstrap/pinned toolchain. That is, simply a new build with all the fixes isn’t sufficient to fix the issue, but rather we need a two-step fix: have all the fixes in, built a new toolchain and then use that to bootstrap another toolchain build.

I did a one-cycle of self-hosting cross-compiling build on main (do a win/x64 toolchain build after the fixes, use the just-built win/x64 toolchain as the bootstrap/pinned toolchain for a win/arm64-on-win/x64 cross-compile build) and the final win/arm64 appears to be good to the extent I locally tested. It can work as a bootstrap/pinned toolchain for a native win/arm64 build and can build the arm64 version of our internal app in arm64 to the same extent as the X64 version (currently hits a known build failure).

@compnerd suggested backporting the fixes to the 5.10 release and use it as a new bootstrap toolchain for main. So I will try and see if this works.

The rough steps:

  • Merge https://github.com/swiftlang/swift/pull/76324 in main
  • Backport the fixes to release/5.10, create a new 5.10 build
  • Enable SwiftCompilerSources in main
  • Check if that can bootstrap a new main build (if not, debug further)
  • Update the bootstrap toolchain used for the main CI with the above new 5.10 build
  • Do a new build in the main CI
  • Done

(and doing similar steps for release/6)

The review for https://github.com/swiftlang/swift/pull/76324 seems to be not fast-progressing, I’d appreciate anyone’s help there.

CC @eeckstein @egorzhdan @compnerd @rjmccall @hyp @DougGregor

@eeckstein This PR needs to be merged before enabling SwiftCompilerSources on Windows arm64: https://github.com/swiftlang/swift/pull/76324

If I add constructors to the other bridged C++ types so their value types are indirectly returned, the SwiftCompilerSource-enabled ARM64 compiler seems to be able to successfully build the toolchain in my local setup. This needs more testing and validation as I’d need to test at head and figure out how to revive the self-hosting build, but this is a good sign!

Here’s a draft PR: https://github.com/swiftlang/swift/pull/76589 which is a follow-up to the previous https://github.com/swiftlang/swift/pull/76433

CC @eeckstein @kavon @egorzhdan @rjmccall @hyp @compnerd @DougGregor

CC @eeckstein @kavon @egorzhdan @rjmccall @hyp @compnerd

I think I figured out the next (OnoneSimplification) crash referred to above:

The crash happens because of how a struct value type is returned mismatches:

(extension in Optimizer):SIL.BuiltinInst.simplify(Optimizer.SimplifyContext) -> () (Swift)

    swift_frontend!$s3SIL11BuiltinInstC9OptimizerE8simplifyyyAD15SimplifyContextVF:
00007ff6`4a4ab034 ff0307d1 sub     sp, sp, #0x1C0
...
00007ff6`4a4ab16c e9450494 bl      swift_frontend!$s3SIL15SubstitutionMapV16replacementTypesAA17OptionalTypeArrayVvg (7ff64a5bc910) --------> calling SIL.SubstitutionMap.replacementTypes.getter : SIL.OptionalTypeArray (below)
00007ff6`4a4ab170 e20300aa mov     x2, x0 --------------------------------------------------------------------------------------------------> Expecting OptionalTypeArray directly returned in x0 and x1
00007ff6`4a4ab174 e30301aa mov     x3, x1
00007ff6`4a4ab178 e0031faa mov     x0, xzr
00007ff6`4a4ab17c e10302aa mov     x1, x2
00007ff6`4a4ab180 e20303aa mov     x2, x3
00007ff6`4a4ab184 6c480494 bl      swift_frontend!$s3SIL17OptionalTypeArrayVyAA0C0VSgSicig (7ff64a5bd334)
...

SIL.SubstitutionMap.replacementTypes.getter : SIL.OptionalTypeArray (Swift)

    swift_frontend!$s3SIL15SubstitutionMapV16replacementTypesAA17OptionalTypeArrayVvg: 
00007ff6`4a5bc910 041a6114 b       swift_frontend!BridgedTypeArray::fromReplacementTypes (7ff64be03120)  -----> calls fromReplacementTypes below

fromReplacementTypes (C++)

    swift_frontend!BridgedTypeArray::fromReplacementTypes:
00007ff6`4be03120 ff4300d1 sub     sp, sp, #0x10
00007ff6`4be03124 f37b00a9 stp     x19, lr, [sp]
00007ff6`4be03128 ff8300d1 sub     sp, sp, #0x20
00007ff6`4be0312c f30300aa mov     x19, x0
00007ff6`4be03130 e10300f9 str     x1, [sp]
00007ff6`4be03134 e1230091 add     x1, sp, #8
00007ff6`4be03138 e0030091 mov     x0, sp
00007ff6`4be0313c d3443794 bl      swift_frontend!swift::SubstitutionMap::getReplacementTypes (7ff64cbd4488)
00007ff6`4be03140 1000c03d ldr     q16, [x0]
00007ff6`4be03144 e00313aa mov     x0, x19
00007ff6`4be03148 083e184e umov    x8, v16.d[1]
00007ff6`4be0314c 700200fd str     d16, [x19] -------------> returning OptionalTypeArray indirectly via x0, a mismatch!
00007ff6`4be03150 680600f9 str     x8, [x19, #8] ----------> returning OptionalTypeArray indirectly via x0, a mismatch!
00007ff6`4be03154 ff830091 add     sp, sp, #0x20
00007ff6`4be03158 f37b40a9 ldp     x19, lr, [sp]
00007ff6`4be0315c ff430091 add     sp, sp, #0x10
00007ff6`4be03160 c0035fd6 ret

Apparently, on Windows ARM64, how a struct value type is returned is (more) sensitive to the conditions including whether a user-defined constructor exists, etc. See this doc.

That caused a calling convention mismatch between the non-USED_IN_CPP_SOURCE (Swift) side and the USE_IN_CPP_SOURCE (C++) side and resulted in a crash.

It seems like the original intent is to control the visibility of some C++ types but it also had an unintended consequence on how the struct type is returned on Windows ARM64!

The draft fix PR is here, pending the CI results. The included test demonstrates the issue and the fix.

After this fix (along with https://github.com/swiftlang/swift/pull/76324) the SwiftCompilerSources-enabled windows/arm64 compiler can now build a hello-world program with -O and the toolchain up to a further point.

Yet, this fix didn’t seem to have fully fixed the SwiftCompilerSources-enabled windows/arm64 compiler yet.

But I will check whether this issue exists in the other bridging struct types and if so, whether fix those will fully fix it or not.

@eeckstein I’m looking into it. I think I found another indirect result calling convention mismatch but haven’t fully figured it out.

Awesome! Thanks a lot!