XCTest Performance Test Failed on Device with Error: Lost connection to test manager service.

Hi all,


I'm using UI Testing and performance testing in XCode 7 to test against the application. I used following XCTest API to run performance test.

measureMetrics(self.dynamicType.defaultPerformanceMetrics(), automaticallyStartMeasuring: false, forBlock: {

//someFunc

})


someFunc() {

//some preparation before star measuring

self.startMeasuring()

let lastView = app.otherElements["uitest.performancelogendingview"]

expectationForPredicate(exists, evaluatedWithObject: lastView, handler: nil)

waitForExpectationsWithTimeout(60, handler: nil)

self.stopMeasuring()

// some tear down work

}

It works well on simulator. But if it runs on device, there's a problem. The test would launch and process correctly. However every time just after repeat 10 runs, the test fails, before it tears down. Is this caused by a crash? Fail can only be observed on device. It's reproducible stably.


15:38:21.813 Xcode[28802:2800126] Test operation failure: Lost connection to test manager service.

15:38:21.813 Xcode[28802:2800126] _finishWithError:Error Domain=IDETestOperationsObserverErrorDomain Code=4 "Lost connection to test manager service." UserInfo={NSLocalizedDescription=Lost connection to test manager service.} didCancel: 1

Could anyone help to give any suggestions? Thank you!

I found the cause by myself: it should be an issue when trying to read baseline file after finish running test on device. Once I set peformance baseline, the lost connection error would reproduce. If I delete baseline plist files, no error. I believe this is an Xcode bug for performance testing on device.

>I believe this is an Xcode bug for performance testing on device.


Feel free to use the Report Bugs link, below right - add your report # to this thread for reference, thanks.

XCTest Performance Test Failed on Device with Error: Lost connection to test manager service.
 
 
Q