fastlane: [scan] Failures with error "Exit status: 65"

New Issue Checklist

Issue Description

We use scan in our applications to test PR builds. We have recently been having lots of failed builds with the generic error: Exit status: 65, which occurs after seeing this:

Signing /Users/jenkins/Library/Developer/Xcode/DerivedData/WhiteLabelClient-ehxpakjqhonkevfqqupggshezzeo/Build/Products/Debug-iphonesimulator/WhiteLabelClientUITests-Runner.app/PlugIns/WhiteLabelClientUITests.xctest
2018-08-22 13:16:08.609 xcodebuild[925:7134107] Error Domain=IDETestOperationsObserverErrorDomain Code=6 "Early unexpected exit, operation never finished bootstrapping - no restart will be attempted" UserInfo={NSLocalizedDescription=Early unexpected exit, operation never finished bootstrapping - no restart will be attempted}

From what I have gathered, when this occurs, I have noticed issues with the simulator on those Jenkins boxes. It will either not be open, or it will be unresponsive, so I believe its something to do with the simulator not being opened correctly, or it time. I from time to time will see a “Springboard quit unexpectedly” error on our VMs, but I am unsure if that is related.

The odd thing is that it is not happening all of the time, it is intermittent between builds. What seems to help clear it up is resetting the simulator on our Jenkins boxes (using snapshot reset_simulators), but, eventually they failures come back. Another thing that seems to help is restarting the boxes completely. Sometimes simply re-running the build will clear it up as well. What is also odd, is we tell scan to ignore our UI tests, which you see in the output above (WhiteLabelClientUITests).

Things I have tried:

  1. Killing all simulators before running scan
  2. Not killing all simulators before running scan
  3. Manually starting a sim before running a build sh "xcrun instruments -t 'Blank' -l 1 -w \"iPhone X (${simOSVersion})\" || sleep 30" (got that idea from https://circleci.com/blog/xcodebuild-exit-code-65-what-it-is-and-how-to-solve-for-ios-and-macos-builds/)
  4. Setting ENV['FASTLANE_EXPLICIT_OPEN_SIMULATOR'] = '2' in my fastfile
  5. Setting ENV['FASTLANE_EXPLICIT_OPEN_SIMULATOR'] = '1' in my fastfile
  6. Setting ENV['FASTLANE_EXPLICIT_OPEN_SIMULATOR'] = '0' in my fastfile
  7. Several combinations of the things above

This is probably happening to 70% of our builds, which has caused some havoc when getting to the end of releases, and every PR build that fails can set us back. I am not sure if this is a bug, or I am doing something wrong in my setup.

Any help would be greatly appreciated! At one point a few weeks back, we did not see these issues at all.

Here is our fastfile for reference:

lane :urbn_scan do |options|
	ENV['FASTLANE_XCODE_LIST_TIMEOUT'] = '300'
	ENV['FASTLANE_XCODEBUILD_SETTINGS_TIMEOUT'] = '300'
	ENV['FASTLANE_XCODEBUILD_SETTINGS_RETRIES'] = '20'
	ENV['FASTLANE_EXPLICIT_OPEN_SIMULATOR'] = '2'
	
	opts = {
		workspace: options[:workspaceName],
		scheme: options[:scheme],
		device: options[:simName],
		clean: true,
		buildlog_path: "./build",
		output_directory: "./build"
    	}
		
	testsToSkip = options[:testsToSkip]
		
	if testsToSkip then
		opts[:skip_testing] = testsToSkip
	end
		
	scan(opts)
end
Complete output when running fastlane, including the stack trace and command used
+ fastlane urbn_scan workspaceName:AppRepo/Client-WhiteLabel/WhiteLabelClient.xcworkspace scheme:Debug 'simName:iPhone X' testsToSkip:WhiteLabelClientUITests
[13:04:45]: Get started using a Gemfile for fastlane https://docs.fastlane.tools/getting-started/ios/setup/#use-a-gemfile
[13:04:46]: ------------------------------
[13:04:46]: --- Step: default_platform ---
[13:04:46]: ------------------------------
[13:04:46]: Driving the lane 'urbn_scan' 🚀
[13:04:46]: ------------------
[13:04:46]: --- Step: scan ---
[13:04:46]: ------------------
[13:04:46]: $ xcodebuild -showBuildSettings -workspace AppRepo/Client-WhiteLabel/WhiteLabelClient.xcworkspace -scheme Debug
Resuming build at Wed Aug 22 13:08:16 EDT 2018 after Jenkins restart
Waiting to resume part of iOS-WhiteLabel » dev #4034: ???
Cannot contact worker4: java.io.IOException: remote file operation failed: /Users/jenkins/Documents/jenkins-workspace/workspace/iOS-WhiteLabel_dev-V2EDLBBHOMCZUYAQRXD53KGJEN6LMKWS45T5O5HJKBJGV2MMEUAA/MobileBuildTools/Pipelines at hudson.remoting.Channel@7d774507:worker4: hudson.remoting.ChannelClosedException: Channel "unknown": Remote call on worker4 failed. The channel is closing down or has closed down
Ready to run at Wed Aug 22 13:08:19 EDT 2018
Timeout set to expire in 56 min
[13:05:02]: $ xcodebuild -showBuildSettings -workspace AppRepo/Client-WhiteLabel/WhiteLabelClient.xcworkspace -scheme Debug

±---------------------------±-------------------------------------------------------------------------------------------------+
|                                                   Summary for scan 2.102.0                                                    |
±---------------------------±-------------------------------------------------------------------------------------------------+
| workspace                  | AppRepo/Client-WhiteLabel/WhiteLabelClient.xcworkspace                                           |
| scheme                     | Debug                                                                                            |
| device                     | iPhone X                                                                                         |
| clean                      | true                                                                                             |
| buildlog_path              | ./build                                                                                          |
| output_directory           | ./build                                                                                          |
| skip_testing               | [“WhiteLabelClientUITests”]                                                                      |
| derived_data_path          | /Users/jenkins/Library/Developer/Xcode/DerivedData/WhiteLabelClient-ehxpakjqhonkevfqqupggshezzeo |
| skip_build                 | false                                                                                            |
| output_types               | html,junit                                                                                       |
| include_simulator_logs     | false                                                                                            |
| disable_concurrent_testing | false                                                                                            |
| should_zip_build_products  | false                                                                                            |
| result_bundle              | false                                                                                            |
| open_report                | false                                                                                            |
| skip_slack                 | false                                                                                            |
| slack_only_on_failure      | false                                                                                            |
| use_clang_report_name      | false                                                                                            |
| fail_build                 | true                                                                                             |
| xcode_path                 | /Applications/Xcode.app                                                                          |
±---------------------------±-------------------------------------------------------------------------------------------------+


[13:05:05]: Killing all running simulators
…
…
▸ Running script ‘[CP] Embed Pods Frameworks’
▸ Running script ‘[CP] Copy Pods Resources’
▸ Touching WhiteLabelClientUITests.xctest
▸ Signing /Users/jenkins/Library/Developer/Xcode/DerivedData/WhiteLabelClient-ehxpakjqhonkevfqqupggshezzeo/Build/Products/Debug-iphonesimulator/WhiteLabelClientUITests-Runner.app/PlugIns/WhiteLabelClientUITests.xctest
2018-08-22 13:16:08.609 xcodebuild[925:7134107] Error Domain=IDETestOperationsObserverErrorDomain Code=6 “Early unexpected exit, operation never finished bootstrapping - no restart will be attempted” UserInfo={NSLocalizedDescription=Early unexpected exit, operation never finished bootstrapping - no restart will be attempted}


Testing failed:
Test target WhiteLabelClientTests encountered an error (Early unexpected exit, operation never finished bootstrapping - no restart will be attempted)
** TEST FAILED **
[13:16:08]: Exit status: 65

Environment

✅ fastlane environment ✅

Stack






Key
Value






OS
10.13.6




Ruby
2.5.1




Bundler?
false




Git
git version 2.15.2 (Apple Git-101.1)




Installation Source
~/.gem/bin/fastlane




Host
Mac OS X 10.13.6 (17G65)




Ruby Lib Dir
/usr/local/Cellar/ruby/2.5.1/lib




OpenSSL Version
OpenSSL 1.0.2o  27 Mar 2018




Is contained
false




Is homebrew
false




Is installed via Fabric.app
false




Xcode Path
/Applications/Xcode.app/Contents/Developer/




Xcode Version
9.4.1






System Locale






Variable
Value







LANG
en_US.UTF-8
✅




LC_ALL






LANGUAGE








fastlane files:


No Fastfile found


No Appfile found


fastlane gems






Gem
Version
Update-Status






fastlane
2.102.0
✅ Up-To-Date






Loaded fastlane plugins:


No plugins Loaded


Loaded gems





Gem
Version






did_you_mean
1.2.0




slack-notifier
2.3.2




atomos
0.1.2




CFPropertyList
2.3.5




claide
1.0.2




colored2
3.1.2




nanaimo
0.2.4




xcodeproj
1.5.7




rouge
2.0.7




xcpretty
0.2.8




terminal-notifier
1.7.1




unicode-display_width
1.1.3




terminal-table
1.7.3




plist
3.2.0




public_suffix
2.0.5




addressable
2.5.1




multipart-post
2.0.0




word_wrap
1.0.0




tty-screen
0.6.3




tty-cursor
0.5.0




tty-spinner
0.8.0




babosa
1.0.2




colored
1.2




highline
1.7.8




commander-fastlane
4.4.6




excon
0.54.0




faraday
0.11.0




unf_ext
0.0.7.2




unf
0.1.4




domain_name
0.5.20161129




http-cookie
1.0.3




faraday-cookie_jar
0.0.6




fastimage
2.1.0




gh_inspector
1.1.2




mini_magick
4.5.1




multi_json
1.12.1




multi_xml
0.6.0




rubyzip
1.2.1




security
0.1.3




xcpretty-travis-formatter
0.0.4




dotenv
2.2.0




bundler
1.16.1




faraday_middleware
0.11.0.1




naturally
2.1.0




simctl
1.6.3




uber
0.0.15




declarative
0.0.9




declarative-option
0.1.0




representable
3.0.4




retriable
2.1.0




mime-types-data
3.2016.0521




mime-types
3.1




little-plugger
1.1.4




logging
2.1.0




jwt
1.5.6




memoist
0.15.0




os
0.9.6




signet
0.7.3




googleauth
0.5.1




httpclient
2.8.3




google-api-client
0.21.2




emoji_regex
0.1.1




json
2.1.0




openssl
2.1.0




date
1.0.0




fileutils
1.0.2




ipaddr
1.2.0








generated on: 2018-08-22

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 6
  • Comments: 19

Most upvoted comments

To be fair, this isn’t entirely a fastlane issue. The same error can occur when running the tests with xcodebuild (which is what fastlane uses). Fastlane is still in the wrong here because Xcode still reports partial test results when a test target fails to start, but fastlane just gives up.

The only workaround I’ve found is to run tests with xcodebuild directly and restart the test if you find that specific error message in Xcode’s test report. I’ve detailed this here: https://silverhammermba.github.io/blog/2019/03/12/slowlane

+2
silverhammermba on Mar 18, 2019

@lukasburns I followed a solution I found here: https://github.com/bitrise-io/bitrise.io/issues/5

That solution is not necessarily using fastlane, but I translated that to my scan calls.

I am now in my fastfile doing 2 scan calls:

  • The first with build_for_testing: true
  • The second with test_without_building: true. Do NOT clean: true in the second call

Doing this did help some, but we are still seeing the occasional Exit status: 65 failures, but not nearly as often so I wouldn’t say I “solved” it.

Heres some more info as well: https://stackoverflow.com/questions/37922146/xctests-failing-on-physical-device-canceling-tests-due-to-timeout/40790171#40790171

+2
ryang1428 on Sep 24, 2018
Read more comments on GitHub
← fastlane: Scan Does Not Output Tests Results
fastlane: Scan hangs on touching .xctests →