swift-package-manager: [SR-14008] "Library not loaded: /usr/lib/swift/libswift_Differentiation.dylib" issue for `swift test`

Previous ID SR-14008
Radar None
Original Reporter @dan-zheng
Type Bug

Attachment: Download

Environment

https://swift.org/builds/development/xcode/swift-DEVELOPMENT-SNAPSHOT-2020-12-23-a/swift-DEVELOPMENT-SNAPSHOT-2020-12-23-a-osx.pkg

Apple Swift version 5.3-dev (LLVM bcee1b98f3b26c5, Swift dbcf7fe1bf76226)
Target: x86_64-apple-darwin19.6.0

Additional Detail from JIRA
Votes 2
Component/s Package Manager
Labels Bug, AutoDiff
Assignee None
Priority Medium

md5: 4ecdf44eee77ebefd73035c76ce61be5

Issue Description:

With development snapshot toolchains, swift build/swift run for executable/library targets importing _Differentiation has no problem.

However, running swift test for test targets importing _Differentiation fails with a dynamic linker issue:

$ swift test
[7/7] Linking SwiftDifferentiationPackageTests

* Build Completed!
2020-12-30 14:28:23.465 xctest[20739:26994456] The bundle “SwiftDifferentiationPackageTests.xctest” couldn’t be loaded because it is damaged or missing necessary resources. Try reinstalling the bundle.
2020-12-30 14:28:23.465 xctest[20739:26994456] (dlopen_preflight(/Users/danielzheng/Differentiation/.build/x86_64-apple-macosx/debug/SwiftDifferentiationPackageTests.xctest/Contents/MacOS/SwiftDifferentiationPackageTests): Library not loaded: /usr/lib/swift/libswift_Differentiation.dylib
  Referenced from: /Users/danielzheng/Differentiation/.build/x86_64-apple-macosx/debug/SwiftDifferentiationPackageTests.xctest/Contents/MacOS/SwiftDifferentiationPackageTests
  Reason: image not found)

Setting DYLD_LIBRARY_PATH to the toolchain library directory containing libswift_Differentiation.dylib does not help:

$ DYLD_LIBRARY_PATH=/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2020-12-23-a.xctoolchain/usr/lib/swift/macosx swift test
2020-12-30 14:29:55.586 xctest[20925:26997481] The bundle “SwiftDifferentiationPackageTests.xctest” couldn’t be loaded because it is damaged or missing necessary resources. Try reinstalling the bundle.
2020-12-30 14:29:55.586 xctest[20925:26997481] (dlopen_preflight(/Users/danielzheng/Differentiation/.build/x86_64-apple-macosx/debug/SwiftDifferentiationPackageTests.xctest/Contents/MacOS/SwiftDifferentiationPackageTests): Library not loaded: /usr/lib/swift/libswift_Differentiation.dylib
  Referenced from: /Users/danielzheng/Differentiation/.build/x86_64-apple-macosx/debug/SwiftDifferentiationPackageTests.xctest/Contents/MacOS/SwiftDifferentiationPackageTests
  Reason: image not found)

However, manually invoking the xctest command from swift test -v with DYLD_LIBRARY_PATH does work:

$ DYLD_LIBRARY_PATH=/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2020-12-23-a.xctoolchain/usr/lib/swift/macosx /Applications/Xcode-beta12_3.app/Contents/Developer/usr/bin/xctest /Users/danielzheng/Differentiation/.build/x86_64-apple-macosx/debug/SwiftDifferentiationPackageTests.xctest
Test Suite 'All tests' started at 2020-12-30 14:35:43.472
Test Suite 'SwiftDifferentiationPackageTests.xctest' started at 2020-12-30 14:35:43.473
Test Suite 'DifferentiationTests' started at 2020-12-30 14:35:43.473
Test Case '-[DifferentiationTests.DifferentiationTests testExample]' started.
Test Case '-[DifferentiationTests.DifferentiationTests testExample]' passed (0.112 seconds).
Test Suite 'DifferentiationTests' passed at 2020-12-30 14:35:43.585.
     Executed 1 test, with 0 failures (0 unexpected) in 0.112 (0.112) seconds
Test Suite 'SwiftDifferentiationPackageTests.xctest' passed at 2020-12-30 14:35:43.585.
     Executed 1 test, with 0 failures (0 unexpected) in 0.112 (0.112) seconds
Test Suite 'All tests' passed at 2020-12-30 14:35:43.585.
     Executed 1 test, with 0 failures (0 unexpected) in 0.112 (0.113) seconds

Maybe this is because swift test does not pass the DYLD_LIBRARY_PATH environment variable to the xctest invocation?

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Comments: 34 (13 by maintainers)

Most upvoted comments

These messages indicate that the library libswift_Concurrency.dylib has been loaded twice. This is a significant problem that must be fixed.

That error is present on trunk snapshots, but not on 5.7 branch snapshots. For example, the 2022-05-04 and 2022-06-08 trunk snapshots produce the concurrency error. The 2022-06-04 v5.7 and 2022-06-13 v5.7 snapshots do not. I was building the package using the June 13, 2022 v5.7 Development Snapshot in https://github.com/apple/swift-package-manager/issues/4454#issuecomment-1160881425. Same behavior on the June 8 trunk snapshot though.

Terminal output (before, then after adding platforms to Package.swift)
(base) philipturner@m1-max-mbp MyPackage % rm -rf .build
(base) philipturner@m1-max-mbp MyPackage % swift build
objc[76736]: Class _TtCs25CheckedContinuationCanary is implemented in both /usr/lib/swift/libswift_Concurrency.dylib (0x235b06670) and /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-06-08-a.xctoolchain/usr/lib/swift/macosx/libswift_Concurrency.dylib (0x1028e88c8). One of the two will be used. Which one is undefined.
objc[76736]: Class _TtCs17DispatchQueueShim is implemented in both /usr/lib/swift/libswift_Concurrency.dylib (0x235b06700) and /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-06-08-a.xctoolchain/usr/lib/swift/macosx/libswift_Concurrency.dylib (0x1028e8958). One of the two will be used. Which one is undefined.
objc[76736]: Class _TtCs9MainActor is implemented in both /usr/lib/swift/libswift_Concurrency.dylib (0x235b06790) and /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-06-08-a.xctoolchain/usr/lib/swift/macosx/libswift_Concurrency.dylib (0x1028e89f0). One of the two will be used. Which one is undefined.
warning: Usage of /Users/philipturner/Library/org.swift.swiftpm/collections.json has been deprecated. Please delete it and use the new /Users/philipturner/Library/org.swift.swiftpm/configuration/collections.json instead.
Building for debugging...
[2/2] Emitting module MyPackage
Build complete! (1.31s)
(base) philipturner@m1-max-mbp MyPackage % swift test
objc[76816]: Class _TtCs25CheckedContinuationCanary is implemented in both /usr/lib/swift/libswift_Concurrency.dylib (0x235b06670) and /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-06-08-a.xctoolchain/usr/lib/swift/macosx/libswift_Concurrency.dylib (0x1044808c8). One of the two will be used. Which one is undefined.
objc[76816]: Class _TtCs17DispatchQueueShim is implemented in both /usr/lib/swift/libswift_Concurrency.dylib (0x235b06700) and /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-06-08-a.xctoolchain/usr/lib/swift/macosx/libswift_Concurrency.dylib (0x104480958). One of the two will be used. Which one is undefined.
objc[76816]: Class _TtCs9MainActor is implemented in both /usr/lib/swift/libswift_Concurrency.dylib (0x235b06790) and /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-06-08-a.xctoolchain/usr/lib/swift/macosx/libswift_Concurrency.dylib (0x1044809f0). One of the two will be used. Which one is undefined.
warning: Usage of /Users/philipturner/Library/org.swift.swiftpm/collections.json has been deprecated. Please delete it and use the new /Users/philipturner/Library/org.swift.swiftpm/configuration/collections.json instead.
Building for debugging...
[3/3] Linking MyPackagePackageTests
Build complete! (14.48s)
2022-06-20 18:39:56.054 xctest[76878:8415670] The bundle “MyPackagePackageTests.xctest” couldn’t be loaded. Try reinstalling the bundle.
2022-06-20 18:39:56.054 xctest[76878:8415670] (dlopen(/Users/philipturner/Desktop/MyPackage/.build/arm64-apple-macosx/debug/MyPackagePackageTests.xctest/Contents/MacOS/MyPackagePackageTests, 0x0109): Symbol not found: _$sSa16_DifferentiationAA14DifferentiableRzlE0B4ViewVMn
  Referenced from: /Users/philipturner/Desktop/MyPackage/.build/arm64-apple-macosx/debug/MyPackagePackageTests.xctest/Contents/MacOS/MyPackagePackageTests
  Expected in: /usr/lib/swift/libswift_Differentiation.dylib)
(base) philipturner@m1-max-mbp MyPackage % otool -L .build/arm64-apple-macosx/debug/MyPackagePackageTests.xctest/Contents/MacOS/MyPackagePackageTests
.build/arm64-apple-macosx/debug/MyPackagePackageTests.xctest/Contents/MacOS/MyPackagePackageTests:
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.100.3)
	/usr/lib/swift/libswiftAppKit.dylib (compatibility version 1.0.0, current version 109.0.0, weak)
	/usr/lib/swift/libswiftCore.dylib (compatibility version 1.0.0, current version 0.0.0)
	/usr/lib/swift/libswiftCoreData.dylib (compatibility version 1.0.0, current version 19.0.0, weak)
	/usr/lib/swift/libswiftCoreFoundation.dylib (compatibility version 1.0.0, current version 14.0.0, weak)
	/usr/lib/swift/libswiftCoreGraphics.dylib (compatibility version 1.0.0, current version 2.0.0, weak)
	/usr/lib/swift/libswiftCoreImage.dylib (compatibility version 1.0.0, current version 2.0.0, weak)
	/usr/lib/swift/libswiftDarwin.dylib (compatibility version 1.0.0, current version 0.0.0, weak)
	/usr/lib/swift/libswiftDispatch.dylib (compatibility version 1.0.0, current version 11.0.0, weak)
	/usr/lib/swift/libswiftFoundation.dylib (compatibility version 1.0.0, current version 72.105.0)
	/usr/lib/swift/libswiftIOKit.dylib (compatibility version 1.0.0, current version 1.0.0, weak)
	/usr/lib/swift/libswiftMetal.dylib (compatibility version 1.0.0, current version 261.13.0, weak)
	/usr/lib/swift/libswiftObjectiveC.dylib (compatibility version 1.0.0, current version 3.0.0, weak)
	/usr/lib/swift/libswiftQuartzCore.dylib (compatibility version 1.0.0, current version 3.0.0, weak)
	/usr/lib/swift/libswiftXPC.dylib (compatibility version 1.0.0, current version 1.1.0, weak)
	/usr/lib/swift/libswift_Differentiation.dylib (compatibility version 1.0.0, current version 0.0.0)
	@rpath/XCTest.framework/Versions/A/XCTest (compatibility version 1.0.0, current version 20501.0.0)
	@rpath/libXCTestSwiftSupport.dylib (compatibility version 1.0.0, current version 1.0.0)
(base) philipturner@m1-max-mbp MyPackage % rm -rf .build                                                                                             
(base) philipturner@m1-max-mbp MyPackage % swift build
objc[76886]: Class _TtCs25CheckedContinuationCanary is implemented in both /usr/lib/swift/libswift_Concurrency.dylib (0x235b06670) and /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-06-08-a.xctoolchain/usr/lib/swift/macosx/libswift_Concurrency.dylib (0x1044148c8). One of the two will be used. Which one is undefined.
objc[76886]: Class _TtCs17DispatchQueueShim is implemented in both /usr/lib/swift/libswift_Concurrency.dylib (0x235b06700) and /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-06-08-a.xctoolchain/usr/lib/swift/macosx/libswift_Concurrency.dylib (0x104414958). One of the two will be used. Which one is undefined.
objc[76886]: Class _TtCs9MainActor is implemented in both /usr/lib/swift/libswift_Concurrency.dylib (0x235b06790) and /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-06-08-a.xctoolchain/usr/lib/swift/macosx/libswift_Concurrency.dylib (0x1044149f0). One of the two will be used. Which one is undefined.
warning: Usage of /Users/philipturner/Library/org.swift.swiftpm/collections.json has been deprecated. Please delete it and use the new /Users/philipturner/Library/org.swift.swiftpm/configuration/collections.json instead.
Building for debugging...
[2/2] Compiling MyPackage MyPackage.swift
Build complete! (1.00s)
(base) philipturner@m1-max-mbp MyPackage % swift test
objc[76929]: Class _TtCs25CheckedContinuationCanary is implemented in both /usr/lib/swift/libswift_Concurrency.dylib (0x235b06670) and /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-06-08-a.xctoolchain/usr/lib/swift/macosx/libswift_Concurrency.dylib (0x1045908c8). One of the two will be used. Which one is undefined.
objc[76929]: Class _TtCs17DispatchQueueShim is implemented in both /usr/lib/swift/libswift_Concurrency.dylib (0x235b06700) and /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-06-08-a.xctoolchain/usr/lib/swift/macosx/libswift_Concurrency.dylib (0x104590958). One of the two will be used. Which one is undefined.
objc[76929]: Class _TtCs9MainActor is implemented in both /usr/lib/swift/libswift_Concurrency.dylib (0x235b06790) and /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-06-08-a.xctoolchain/usr/lib/swift/macosx/libswift_Concurrency.dylib (0x1045909f0). One of the two will be used. Which one is undefined.
warning: Usage of /Users/philipturner/Library/org.swift.swiftpm/collections.json has been deprecated. Please delete it and use the new /Users/philipturner/Library/org.swift.swiftpm/configuration/collections.json instead.
Building for debugging...
[3/3] Linking MyPackagePackageTests
Build complete! (14.11s)
2022-06-20 18:40:37.120 xctest[76994:8416593] The bundle “MyPackagePackageTests.xctest” couldn’t be loaded. Try reinstalling the bundle.
2022-06-20 18:40:37.120 xctest[76994:8416593] (dlopen(/Users/philipturner/Desktop/MyPackage/.build/arm64-apple-macosx/debug/MyPackagePackageTests.xctest/Contents/MacOS/MyPackagePackageTests, 0x0109): Symbol not found: _$sSa16_DifferentiationAA14DifferentiableRzlE0B4ViewVMn
  Referenced from: /Users/philipturner/Desktop/MyPackage/.build/arm64-apple-macosx/debug/MyPackagePackageTests.xctest/Contents/MacOS/MyPackagePackageTests
  Expected in: /usr/lib/swift/libswift_Differentiation.dylib)
(base) philipturner@m1-max-mbp MyPackage % otool -L .build/arm64-apple-macosx/debug/MyPackagePackageTests.xctest/Contents/MacOS/MyPackagePackageTests
.build/arm64-apple-macosx/debug/MyPackagePackageTests.xctest/Contents/MacOS/MyPackagePackageTests:
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.100.3)
	/usr/lib/swift/libswiftAppKit.dylib (compatibility version 1.0.0, current version 109.0.0, weak)
	/usr/lib/swift/libswiftCore.dylib (compatibility version 1.0.0, current version 0.0.0)
	/usr/lib/swift/libswiftCoreData.dylib (compatibility version 1.0.0, current version 19.0.0, weak)
	/usr/lib/swift/libswiftCoreFoundation.dylib (compatibility version 1.0.0, current version 14.0.0, weak)
	/usr/lib/swift/libswiftCoreGraphics.dylib (compatibility version 1.0.0, current version 2.0.0, weak)
	/usr/lib/swift/libswiftCoreImage.dylib (compatibility version 1.0.0, current version 2.0.0, weak)
	/usr/lib/swift/libswiftDarwin.dylib (compatibility version 1.0.0, current version 0.0.0, weak)
	/usr/lib/swift/libswiftDispatch.dylib (compatibility version 1.0.0, current version 11.0.0, weak)
	/usr/lib/swift/libswiftFoundation.dylib (compatibility version 1.0.0, current version 72.105.0)
	/usr/lib/swift/libswiftIOKit.dylib (compatibility version 1.0.0, current version 1.0.0, weak)
	/usr/lib/swift/libswiftMetal.dylib (compatibility version 1.0.0, current version 261.13.0, weak)
	/usr/lib/swift/libswiftObjectiveC.dylib (compatibility version 1.0.0, current version 3.0.0, weak)
	/usr/lib/swift/libswiftQuartzCore.dylib (compatibility version 1.0.0, current version 3.0.0, weak)
	/usr/lib/swift/libswiftXPC.dylib (compatibility version 1.0.0, current version 1.1.0, weak)
	/usr/lib/swift/libswift_Differentiation.dylib (compatibility version 1.0.0, current version 0.0.0)
	@rpath/XCTest.framework/Versions/A/XCTest (compatibility version 1.0.0, current version 20501.0.0)
	@rpath/libXCTestSwiftSupport.dylib (compatibility version 1.0.0, current version 1.0.0)

Also, here’s my /usr/lib and /usr/lib/swift:

Most dylibs whose install names are under these directories do not actually exist there as separate files; instead, the dynamic loader picks them up from the system-wide dyld shared cache.