appium: Test execution is very slow on iOS 13

@cosminstirbu We are seeing something extremely similar to your case after just upgrading from Xcode 10.3 and iOS 12.2 to Xcode 11.3.1 and iOS 13.3

Often the app will wait 60-70 seconds for tapping

We are on Native iOS / Android apps

In our production tests suite (184 tests), even after we’ve implemented the workaround, the duration increased from 2h 55m with Xcode 10.3 and iOS 12.4 to 5h 35m with Xcode 11.3.1 and iOS 13.3

I created a sample app in ios 13.

I found that finding elements was fast. it was clicking them that was slow. until I used:

# idk which of these actually makes it faster.
"includeNonModalElements": True
"useFirstMatch": True
"snapshotTimeout": 1

after that clicking was very fast.

however my original app that my job requires to me automate did not see any such improvement. 😦 Wish I could figure out why.

EDIT: scratch that, I commented out all 3 of those lines, and it was fast. idk why it was slow the first time.

Is the target element(s) inside a web view?

Hi @mykola-mokhnach, I have the same issue on an element in webview, input text to element in webview is very slow, inputing 17 characters takes 1 minute and 26 seconds. `2019-11-05 17:19:25:219 - [debug] [WD Proxy] Proxying [POST /element/DA020000-0000-0000-C769-010000000000/value] to [POST http://localhost:8100/session/4E09AFE6-5E02-4067-8436-6C1376452DAE/element/DA020000-0000-0000-C769-010000000000/value] with body: {“value”:[“t”,“e”,“s”,“t”," “,“n”,“o”,“t”,“e”,” ",“i”,“t”,“a”,“l”,“i”,“c”,“s”],“text”:“test note italics”}

2019-11-05 17:20:51:013 - [debug] [WD Proxy] Got response with status 200: {“value”:null,“sessionId”:“4E09AFE6-5E02-4067-8436-6C1376452DAE”}`

You can try out appium@rc which will be 1.17.1 It has commits until 2.14.2 in https://github.com/appium/WebDriverAgent/commits/master Appium 1.17.0 has the version 2.7.4.

It is appreciated if you can detect the cause is XCTest framework side or Appium still facing

@mrk-han If you re-install the beta of appium (npm uninstall -g appium && npm install -g appium@beta) you ought to get the latest dependencies, including the change in appium-webdriveragent.

Ok I’ve found that if you disable animations, ios 13 is faster.

func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
    UIView.setAnimationsEnabled(false)
}

I created a simple app like below and measured the result.

test code

• Appium (Ruby client, with useFirstMatch)

puts Benchmark.measure {
  @driver.find_element(:accessibility_id, 'Button'  ).click
}

• XCTest (Swift) (Picked the first measurement since it could cache)

    func testExample() {
        measureMetrics([XCTPerformanceMetric.wallClockTime], automaticallyStartMeasuring: true, for: {
            XCUIApplication().buttons["Button"].tap()
        })
    }

Result

• Xcode 11.2, iOS 13.2.2, iPhone 8 simulator
  • Appium (1.16-beta)
    • 0.62s
  • XCTest
    • 0.61s
• Xcode 11.2, iOS 12.4, iPhone 8 simulator
  • Appium (1.16-beta)
    • 0.41s
  • XCTest
    • 0.60s

XCTest were almost the same. Appium became almost same time as vanilla XCTest in iOS13. Appium was faster than XCTest for iOS 12.4…?

many thanks @KazuCocoa and @owlsnakes (for raising and struggling with this issue). My automation works again on iOS 13.4.1 after a very long time RIP

I have installed the latest appium 1.17.1 and run the WebdriverAgentRunner app (with the edited product bundle Identifier to replace the default one) instead of the previous running Integration App

You can try out appium@rc which will be 1.17.1 It has commits until 2.14.2 in https://github.com/appium/WebDriverAgent/commits/master Appium 1.17.0 has the version 2.7.4.

It is appreciated if you can detect the cause is XCTest framework side or Appium still facing

Moving from appium 1.17 to 1.17.1-rc.1 made my test duration decreased from 146022ms to 21096ms.

Thank you

I’m still having this issue.

ios 13.3 appium 1.17

RIP I guess?

FWIW, I was seeing very long click times doing some testing on an ios simulator yesterday. OSX 10.15.1, xcode 11.3.1, ios 13.3 iphone 8 simulator. Click times were taking ~ 120s.

Based on this thread I installed a ios 12.4 simulator and re-ran the tests. Clicks were fast, but somehow the initial driver start, took a long time. In other words the simulator would respond to clicks quickly, but only after a long delay starting up. The app was visible but the initial POST to /session took a long time to respond.

Next, I switched back to ios 13.3 simulator. Now things appear to be working – fast startup and fast clicks.

FWIW.

Hi @anhmainvg - we’re building native apps, we don’t interact with any embedded web views so I can’t tell if those were impacted by the iOS 13 migration.

Our device tests still run on iOS 12 for now, however I don’t except significant differences if we include the Click() workaround. Once we migrate to iOS 13 for our device tests as well, I’ll come back with our results.

hi @cosminstirbu , what is the stack of your team development, please? I meant is your team building native apps or hybrid apps? our team used the Ionic and I use the webdriverio appium to automate the apps, but after I upgraded real devices to iOS 13, the tests running very slow. I yet to implement the workaround solution. Thanks

After further investigations my team found something even more interesting (and silly).

In our production test suite, we were using iPhone 7 simulator. Unfortunately, as of Xcode 11, there isn’t a default iPhone 7 simulator, so Appium created one (and teared it down) every test …, adding more than one minute per test.

After switching to iPhone 8 simulator, everything is back to normal, we have 3h test execution time on iOS 13 with iPhone 8 simulator and Click() workaround.

What about real devices?

After further investigations my team found something even more interesting (and silly).

In our production test suite, we were using iPhone 7 simulator. Unfortunately, as of Xcode 11, there isn’t a default iPhone 7 simulator, so Appium created one (and teared it down) every test …, adding more than one minute per test.

After switching to iPhone 8 simulator, everything is back to normal, we have 3h test execution time on iOS 13 with iPhone 8 simulator and Click() workaround.

or… it was resolved by XCode 11.3. @ganjarpanji do you still see this on XCode 11.3?

Awesome. @KazuCocoa what about without useFirstMatch? - afaik that flag will change/break existing tests which is not ideal for us.

Thank you.