Create and run unit tests, performance tests, and UI tests for your Xcode project using XCTest.

Posts under XCTest tag

200 Posts

Post

Replies

Boosts

Views

Activity

Struggling to add a test target to an existing project in Xcode
Sorry if this question is too vague, however I've tried this multiple times and see the same result. I'm pretty sure I'm doing something wrong, but don't know what. I have a Multiplatform (iOS and macOS) project that builds, and runs I add a new target of type Unit Test Bundle I click the diamond in the margin beside the XCTestCase declaration at the top of the new Test file The target project builds, then Xcode says 'Testing...' and it stays like this forever. I've also tried creating a new scheme that targets the target created in the above steps. attempting to run my tests behaves the same. The top status bar will get stuck saying 'Testing...' and never get anywhere. I'm pretty sure this is something basic. thanks, in advance for any guidance. Mike
2
0
1.7k
Nov ’24
Do I need to down load iOS 18 for testing on iPhone 11 having iOS version 17.6.1 ?
I am using two iPhone11 devices, having iOS v17.6.1 for automation testing using Seleniumv4.X-Appium v2.x. Today I updated the mac to Sequoia15 version which updated the XCode to v16. Just after this update, I could not able to run the previously running automation script. The console message is "org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Response code 500. Message: Unable to launch WebDriverAgent. Original error: xcodebuild failed with code 70. This usually indicates an issue with the local Xcode setup or WebDriverAgent project configuration or the driver-to-platform version mismatch."
3
0
1.3k
Nov ’24
Running local package's tests in main app's test plan
Hi, I work on a iOS application which includes several local swift packages containing modularised code. Each of those local swift packages has a test target defined in their respective Package.swift, and when opening a local package folder standalone in Xcode, then the tests run without issues. However I would like in the main app's test plan to add the local package test targets and have them all run together when the main app's default test plan is run. Xcode allows me to select local package test targets within the main app's test plan (also within Test portion of the main app's scheme), however attempting to build for testing throws a "Module '…' was not compiled for testing". Any ideas on how to achieve this goal (run local package tests in conjunction with main app's tests) and avoid that error? Thanks Peter
2
5
1.1k
Nov ’24
How to run Catalyst xctest bundle with xcrun
I'm trying to run an xctest bundle, built for catalyst, with xcrun. i.e. xcrun xctest path_to_my_test.xctest It fails, complaining that: The bundle "MyBundle" couldn't be loaded because it is damaged or missing necessary resources. Try reinstalling the bundle.....but incompatible platform (have 'MacCatalyst', need 'macOS') So it seems like because I'm not executing it in a sim it wants the bundle to be a macOS bundle. But I would have thought it would be possible to run a Mac Catalyst target directly on a macOS host the same as a native macOS test target. Is this not possible?
0
0
456
Nov ’24
Run test specific setup logic on different target
Our use case is that we need to run a CLI script (on Mac) to set up backend configuration before each test case (UI test on real iOS device). The setup logic is test specific, so using pre-build scripts is not an option. Is this support in any way or what are best practices around such a setup? Ideally, we would still be able to execute the tests from within Xcode.
0
0
393
Nov ’24
Unable to inspect SwiftUI view embedded inside UITableView
We have been trying to migrate screens that were developed using UITool Kit to SwiftUI. In the process we have some screens that have SwiftUI embedded inside the UITool kit view. Our developers have defined accessibility ids for all elements in these views and these are inspectable using the native iOS xcode inspector. However when i try inspecting it with the appium inspector i get an empty list with no elements in the hierarchy tree. Attaching a screenshot of the element when inspecting through the native xcode accessibility inspector, Attaching a screenshot of the same screen when inspected through the appium inspector, Also tried printing the XCTest UI dump using appium method, `driver().executeScript("mobile:source", Map.ofEntries(Map.entry("format","description"))) The UI tree i get is the same that i get when inspecting through the appium inspector. Requesting support from the Apple team based on this ticket, [https://github.com/appium/appium/issues/20759)
0
0
395
Nov ’24
Not seeing signposts when profiling a unit test (Xcode 16.1)
While I was recently profiling some code from a Swift library, I noticed that XCTest added in signposts for the measurement tests, which I found really helpful to "home in" on the code I wanted to profile digging around in the stack trace. I tried to add my own signposts to provide just a bit of my own markers in there, but while it compiles and profiles equivalently, the signposts just aren't showing up. This is with Xcode 16.1, macOS Sequoia (15.1) and a swift library, using XCTest and profiling within one of the unit tests. Is there something in this sequence that doesn't allow the library to set up signposts and have instruments collect them? The flow I'm using: import os let subsystem = "MyLibrary" class MyClass { let logger: Logger = .init(subsystem: subsystem, category: "fastloop") let signposter: OSSignposter init() { signposter = OSSignposter(logger: logger) } func goFast() { let signpostId = signposter.makeSignpostID() let state = signposter.beginInterval("tick", id: signpostId) // ... do a bunch of work here - all synchronous signposter.endInterval("tick", state) } } Is there something I'm doing incorrectly in using this API, or not enabling to allow those signposts to be collected by the profiler? I do see the signposts that XCTests injects into the system, just not any of the ones I'm creating.
1
0
881
Nov ’24
XCUITest Screen Recording
In WWDC23: Fix failures faster with Xcode test reports, the presenter showed off an Xcode feature called Automation Explorer, that allows playback of a screen recording corresponding to an XCUITest. For the life of me, I have not been able to find out how to enable/use this feature in Xcode 16. I am using Test Plans and have set the UI Testing configuration settings for Automatic Screen Capture to On, and keep all and Preferred Capture Format to Video, but am not seeing any screen recordings being produced after running the tests. Am I doing something wrong?
1
0
951
Nov ’24
Run identical UI/Unit tests on different build targets
I have different versions of my iOS App (written in SwiftUI). The app on the store, the App Clip, and one or two next version apps not yet released (e.g. A/B comparison). All good. But now I've started creating UI and Unit tests and I'm confused about how to get this working. Each build target has its own scheme. And in that scheme I have a Test plan for that target. E.g. The App Clip scheme has an App Clip test plan. Since all the app variants are very similar, I only have one set of unit tests and one set of UI tests so each test plan includes the same unit test target and the UI test target. Problem: When I selected a scheme (e.g. for the App Clip) and ran the tests, it turned out that all the tests ran for another build target, not the target of the scheme. I think this might be because within the definition of the test target there's a field specifying the host application. I.e. the build target. Question: How can I set up my project so that the test plan uses the relevant target build? Or do I have to duplicate all the test targets (one for each target)? Or do I have to manually change each test target before running it for a particular build target?
1
0
667
Nov ’24
Enable local network access during iOS UI test in iOS14
We are building an iOS app that connects to a device using Bluetooth. To test unhappy flow scenarios for this app, we'd like to power cycle the device we are connecting to by using an IoT power switch that connects to the local network using WiFi (a Shelly Plug-S). In my test code on iOS13, I was able to do a local HTTP call to the IP address of the power switch and trigger a power cycle using its REST interface. In iOS 14 this is no longer possible, probably due to new restrictions regarding local network usage without permissions (see: https://developer.apple.com/videos/play/wwdc2020/10110 ). When running the test and trying a local network call to the power switch in iOS14, I get the following error: Task <D206B326-1820-43CA-A54C-5B470B4F1A79>.<2> finished with error [-1009] Error Domain=NSURLErrorDomain Code=-1009 "The internet connection appears to be offline." UserInfo={_kCFStreamErrorCodeKey=50, NSUnderlyingError=0x2833f34b0 {Error Domain=kCFErrorDomainCFNetwork Code=-1009 "(null)" UserInfo={_kCFStreamErrorCodeKey=50, _kCFStreamErrorDomainKey=1}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <D206B326-1820-43CA-A54C-5B470B4F1A79>.<2>, _NSURLErrorRelatedURLSessionTaskErrorKey=("LocalDataTask <D206B326-1820-43CA-A54C-5B470B4F1A79>.<2>"), NSLocalizedDescription=The internet connection appears to be offline., NSErrorFailingURLStringKey=http://192.168.22.57/relay/0?turn=on, NSErrorFailingURLKey=http://192.168.22.57/relay/0?turn=on, _kCFStreamErrorDomainKey=1} An external network call (to google.com) works just fine in the test. I have tried fixing this by adding the following entries to the Info.plist of my UI test target: <key>NSLocalNetworkUsageDescription</key> <string>Local network access is needed for tests</string> <key>NSBonjourServices</key> <array> <string>_http._tcp</string> </array> <key>NSAppTransportSecurity</key> <dict> <key>NSAllowsArbitraryLoads</key> <true/> </dict> However, this has no effect. I have also tried adding these entries to the Info.plist of my app target to see if that makes a difference, but it doesn't. I'd also rather not add these entries to my app's Info.plist, because the app does not need local network access. Only the test does. Does anyone know how to enable local network access during an iOS UI test in iOS14?
13
1
20k
Nov ’24
Crash calling UIHostingController from a Swift Package
Hello team, We recently found a EXC_BAD_ACCESS crash when using UIHostingControllers on a SPM local Package in our application. This is happening from time to time when we run the app on simulators using Debug configurations. Also, this issue is consistent when we run application tests, there is something weird that we found after making a research... If we disable app test target and only keep packages tests, the crash is not happening, but as soon as we re-enable the app test target from the test suite the crash returns. This is the full error message: Thread 1: EXC_BAD_ACCESS (code=1, address=0xbad4017) A bad access to memory terminated the process. Is this s known issue? We've been performing explorations for some days and couldn't find any real solution for this. A basic call on UIHostingController inside of a SPM local Package is enough to make the app crash, nothing custom being done: UIHostingController(rootView: MyView())
2
0
606
Nov ’24
macOS Sequoia doesn't respect Full Disk Access for UITests-Runner
We develop and test App for macOS. We start to see system alert - "UlTests-Runner" would like to access data from other apps on each UITest run. Our test suite does cleanup of files generated by App so we need access outside of UITests-Runner sandbox. We enabled Full Disk Access for UITests-Runner at Settings -> Privacy & Security -> Full Disk Access but unfortunately still see this alert. Is there any way to permanently remove/hide this alert or remove sandbox for 'UITests-Runner' since we want to run tests on CI and having this alert is not an option? Note: everything works fine on previous versions of macOS. Environment: macOS - 15.1 (24B83) Xcode - Version 16.1 (16B40)
0
1
501
Nov ’24
XCUITest target invalid config sets both USES_XCTRUNNER and either TEST_HOST or RUNTIME_TEST_HOST
I am trying to add an XCUITest target to an existing application. When I try to run the tests, I get the error "Invalid configuration: MyProjectUITests sets both USES_XCTRUNNER and either TEST_HOST or RUNTIME_TEST_HOST". I have seen information stating that it might be because I have multiple apps built from different build configurations in the same scheme, but I could not resolve the issue with any of the suggestions I saw. I've attached a screenshot of the error message and a link to a project stripped down to just the existing configuration to help with diagnosing this issue. I would really love to be able to get some UI tests running. My Project
1
0
679
Nov ’24
Xcode not generating code coverage.
No coverage is generated for private xxxx.framework, but tests are running under this framework. This was perfectly working before and started giving this error after updating to Xcode 16. Failed to generate coverage for target xxxx.framework at paths ( "/Users/xxxxxx/Desktop/xcode/Debug-iphonesimulator/Common/xxxxxx.framework/xxxx" ): No object file for requested architecture The operation couldn’t be completed. (XCTHCoverageReportGenerationErrorDomain error 2.). (Underlying Error: No object file for requested architecture)
2
1
1k
Nov ’24
iOS simulator freezes on macOS Sequoia during test execution
Hello! 👋 We are seeing a bug on macOS Sequoia related to the test running. When attempting to run tests, the iOS simulator becomes stuck indefinitely. This can occur whether we run tests for a specific module, all unit tests, or even a single test. We narrowed down the issue and is due to the OS failing to copy some named pipes (FIFO). For example: db.realm.note db.realm.management/access_control.new_commit.cv db.realm.management/access_control.pick_writer.cv We saw in the CoreSimulator log file the following error: NSCocoaErrorDomain Code=512 ""access_control.new_commit.cv" couldn't be copied The copy fails and then a new clone of the simulator is created retrying the process. And this goes on and on. What is happening? When running the tests for the iOS app on the simulator, under the hood, the OS tries to clone the simulator device. A list of folders is created, including one Shared/AppGroup. Under the AppGroup folder, the list of multiple UDIDs corresponds to the specific App Group containers created for individual app targets or extensions that are sharing data within that App Group. One of these folders contains the Realm DB files. There are a few files called named pipes which are invisible in Finder (even if you have hidden files to true). You need to list them with ls -l. So, when we select the test target and run the tests, the OS clones the permanent simulator and copies all files from its folders. All files are being copied except those pipes. When the OS attempts to copy the pipes, the operation fails with the error Code=512. Reproduction Steps Run the iOS app on a simulator (e.g. iPhone SE 3rd gen. with iOS 18.1). Quit Xcode and the iOS simulator. Reopen Xcode. Select a test target to run. Select the same simulator you previously ran the iOS app. Run the tests (CMD + U). The simulator is now stuck. Are there any workarounds available? Yes. We found that running the tests works if we first “Erase All Content and Settings” from the simulator. Another workaround is to remove all simulators and reinstall the iOS runtimes. This prevents the issue for a longer period (almost a full day), but eventually, the problem reoccurs. Alternatively, we can delete the named pipes from the App Groups directory before running the tests. Have we tried to give full disk permissions? Yes, we tried to give full disk permissions to a lot of things (Xcode, simulator, file paths, directories, etc.) but with no luck. We still get the same error. Is the issue happening on a specific version of Xcode? No, it happens for multiple Xcode versions: Xcode 15.4 Xcode 16.0 Xcode 16.1 Is the issue happening on a specific macOS Sequoia version? No, it happens on multiple macOS Sequoia versions: 15.0 Beta 8 15.0 RC 15.0 15.1 Stacktrace Oct 29 17:41:18 CoreSimulatorService[14079] <Error>: New device is stuck in creation state, deleting: Clone 712 of iPhone SE (3rd generation) (58D6DED6-2C55-4E7C-BBB4-E0D661DC41A1, iOS 18.1, Creating) Oct 29 17:41:20 CoreSimulatorService[14079] <Error>: Error Domain=NSPOSIXErrorDomain Code=22 "Invalid argument" UserInfo={NSLocalizedFailureReason=Device was allocated but was stuck in creation state. Check CoreSimulator.log for more information.} Oct 29 17:41:20 com.apple.dt.Xcode[90147] <Error>: Error Domain=NSPOSIXErrorDomain Code=22 "Invalid argument" UserInfo={NSLocalizedFailureReason=Device was allocated but was stuck in creation state. Check CoreSimulator.log for more information.} Oct 29 17:41:30 CoreSimulatorService[14079] <Warning>: Device 6C270BDB-2945-42B5-A985-884F93BFD3E1 encountered in creation state at launch. The device will be re-created. Oct 29 17:41:38 CoreSimulatorService[14079] <Error>: Failed to clone the device data path, error = Error Domain=NSCocoaErrorDomain Code=512 "“access_control.new_commit.cv” couldn’t be copied to “db.realm.management”." UserInfo={NSSourceFilePathErrorKey=[...]/db.realm.management/access_control.new_commit.cv, NSUserStringVariant=( Copy ), NSDestinationFilePath=[...]/db.realm.management/access_control.new_commit.cv, NSUnderlyingError=0x600000ffa490 {Error Domain=NSPOSIXErrorDomain Code=45 "Operation not supported"}} Oct 29 17:41:38 CoreSimulatorService[14079] <Error>: Failed to re-create device that was encountered in the creation state (Clone 713 of iPhone SE (3rd generation) (6C270BDB-2945-42B5-A985-884F93BFD3E1, iOS 18.1, Creating)): Error Domain=NSCocoaErrorDomain Code=512 "“access_control.new_commit.cv” couldn’t be copied to “db.realm.management”." UserInfo={NSSourceFilePathErrorKey=[...]/db.realm.management/access_control.new_commit.cv, NSUserStringVariant=( Copy ), NSDestinationFilePath=[...]/db.realm.management/access_control.new_commit.cv, NSUnderlyingError=0x600000ffa490 {Error Domain=NSPOSIXErrorDomain Code=45 "Operation not supported"}} Oct 29 17:41:40 CoreSimulatorService[14079] <Error>: Error Domain=NSPOSIXErrorDomain Code=22 "Invalid argument" UserInfo={NSLocalizedFailureReason=Device was allocated but was stuck in creation state. Check CoreSimulator.log for more information.} Oct 29 17:41:50 CoreSimulatorService[14079] <Warning>: Device C6DEBFBB-6EFA-4E4C-B51B-9DDA08AF9BDB encountered in creation state at launch. The device will be re-created. Oct 29 17:41:55 CoreSimulatorService[14079] <Error>: Failed to clone the device data path, error = Error Domain=NSCocoaErrorDomain Code=512 "“access_control.new_commit.cv” couldn’t be copied to “db.realm.management”." UserInfo={NSSourceFilePathErrorKey=[...]/db.realm.management/access_control.new_commit.cv, NSUserStringVariant=( Copy ), NSDestinationFilePath=[...]/db.realm.management/access_control.new_commit.cv, NSUnderlyingError=0x600000ffb000 {Error Domain=NSPOSIXErrorDomain Code=45 "Operation not supported"}} Oct 29 17:42:06 CoreSimulatorService[14079] <Warning>: Device 032BAE7E-E345-48F2-86EB-4DF1AD4D5291 encountered in creation state at launch. The device will be re-created.
0
0
989
Oct ’24
How to correctly resolve URL bookmarks for use in Simulator UI tests?
To make UI testing easier and faster, I usually create URL bookmarks during normal app usage in the Simulator so that they can be instantly resolved on app launch during UI tests. For example, one of my apps allows browsing selected folders and stores bookmarks so they can be quickly opened again on following app launches, and instead of selecting the test folder each time at the beginning of the UI test, I select it once during normal app usage so that it's available immediately during the UI test. This usually works fine, but every now and then the UI tests fail because the tested app isn't able to resolve the stored bookmark. I don't know why this happens, but usually opening and closing the app again in the Simulator and re-running the UI tests solves the issue. The problem now is that I've just tried to setup some new UI tests for Apple Vision Pro Simulator and I'm never able to resolve bookmarks. So I created a sample project that reproduces the issue, and curiously enough the bookmarks don't even resolve when using an iPad Simulator (which usually works fine with my regular UI tests). What am I doing wrong? This can be reproduced with a default iOS project, embedding the default storyboard view controller in a navigation view controller, and this code: import UIKit class ViewController: UIViewController, UIDocumentPickerDelegate { override func viewDidLoad() { navigationItem.rightBarButtonItem = UIBarButtonItem(systemItem: .add, primaryAction: UIAction(handler: { _ in let picker = UIDocumentPickerViewController(forOpeningContentTypes: [.folder]) picker.delegate = self self.present(picker, animated: true) })) if let bookmark = UserDefaults.standard.data(forKey: "bookmark") { readBookmark(bookmark) } } func readBookmark(_ bookmark: Data) { do { let label = UILabel(frame: CGRect(x: 100, y: 100, width: 600, height: 100)) label.numberOfLines = 0 var stale = false let url = try URL(resolvingBookmarkData: bookmark, bookmarkDataIsStale: &stale) if !url.startAccessingSecurityScopedResource() { fatalError() } label.text = url.path view.addSubview(label) } catch { fatalError(error.localizedDescription) } } func documentPicker(_ controller: UIDocumentPickerViewController, didPickDocumentsAt urls: [URL]) { do { let url = urls[0] if !url.startAccessingSecurityScopedResource() { fatalError() } let bookmark = try url.bookmarkData() UserDefaults.standard.set(bookmark, forKey: "bookmark") readBookmark(bookmark) } catch { fatalError(error.localizedDescription) } } } And a default UI test, which always crashes because of the fatalError() in the catch clause of readBookmark(_:): final class problemUITests: XCTestCase { @MainActor func testExample() throws { let app = XCUIApplication() app.launch() } }
0
0
479
Oct ’24
Struggling to add a test target to an existing project in Xcode
Sorry if this question is too vague, however I've tried this multiple times and see the same result. I'm pretty sure I'm doing something wrong, but don't know what. I have a Multiplatform (iOS and macOS) project that builds, and runs I add a new target of type Unit Test Bundle I click the diamond in the margin beside the XCTestCase declaration at the top of the new Test file The target project builds, then Xcode says 'Testing...' and it stays like this forever. I've also tried creating a new scheme that targets the target created in the above steps. attempting to run my tests behaves the same. The top status bar will get stuck saying 'Testing...' and never get anywhere. I'm pretty sure this is something basic. thanks, in advance for any guidance. Mike
Replies
2
Boosts
0
Views
1.7k
Activity
Nov ’24
Do I need to down load iOS 18 for testing on iPhone 11 having iOS version 17.6.1 ?
I am using two iPhone11 devices, having iOS v17.6.1 for automation testing using Seleniumv4.X-Appium v2.x. Today I updated the mac to Sequoia15 version which updated the XCode to v16. Just after this update, I could not able to run the previously running automation script. The console message is "org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Response code 500. Message: Unable to launch WebDriverAgent. Original error: xcodebuild failed with code 70. This usually indicates an issue with the local Xcode setup or WebDriverAgent project configuration or the driver-to-platform version mismatch."
Replies
3
Boosts
0
Views
1.3k
Activity
Nov ’24
UITest not launching the simulator on Xcode 15
I have latest version Xcode-15.1 & Mac-os version Sonoma-14.3.1 - while running particularly UITest target Xcode not launching the simulator, even after build succeed & test's completed without any errors. I tried cleaning project, re-starting Xcode, restarting system(Mac) - no luck It would be helpful if I get any suggestion on resolving the issue.
Replies
1
Boosts
0
Views
759
Activity
Nov ’24
Running local package's tests in main app's test plan
Hi, I work on a iOS application which includes several local swift packages containing modularised code. Each of those local swift packages has a test target defined in their respective Package.swift, and when opening a local package folder standalone in Xcode, then the tests run without issues. However I would like in the main app's test plan to add the local package test targets and have them all run together when the main app's default test plan is run. Xcode allows me to select local package test targets within the main app's test plan (also within Test portion of the main app's scheme), however attempting to build for testing throws a "Module '…' was not compiled for testing". Any ideas on how to achieve this goal (run local package tests in conjunction with main app's tests) and avoid that error? Thanks Peter
Replies
2
Boosts
5
Views
1.1k
Activity
Nov ’24
How to run Catalyst xctest bundle with xcrun
I'm trying to run an xctest bundle, built for catalyst, with xcrun. i.e. xcrun xctest path_to_my_test.xctest It fails, complaining that: The bundle "MyBundle" couldn't be loaded because it is damaged or missing necessary resources. Try reinstalling the bundle.....but incompatible platform (have 'MacCatalyst', need 'macOS') So it seems like because I'm not executing it in a sim it wants the bundle to be a macOS bundle. But I would have thought it would be possible to run a Mac Catalyst target directly on a macOS host the same as a native macOS test target. Is this not possible?
Replies
0
Boosts
0
Views
456
Activity
Nov ’24
Run test specific setup logic on different target
Our use case is that we need to run a CLI script (on Mac) to set up backend configuration before each test case (UI test on real iOS device). The setup logic is test specific, so using pre-build scripts is not an option. Is this support in any way or what are best practices around such a setup? Ideally, we would still be able to execute the tests from within Xcode.
Replies
0
Boosts
0
Views
393
Activity
Nov ’24
Run test specific setup logic on a different platform than the actual test
Our use case is that we need to run a CLI script (on Mac) to set up some backend configuration for a UITest that then runs on iOS (simulator or actual device). The setup logic is test specific so using pre-build scripts is not a an option. Is that support in any way or what would be alternatives to go about this.
Replies
0
Boosts
0
Views
280
Activity
Nov ’24
Unable to inspect SwiftUI view embedded inside UITableView
We have been trying to migrate screens that were developed using UITool Kit to SwiftUI. In the process we have some screens that have SwiftUI embedded inside the UITool kit view. Our developers have defined accessibility ids for all elements in these views and these are inspectable using the native iOS xcode inspector. However when i try inspecting it with the appium inspector i get an empty list with no elements in the hierarchy tree. Attaching a screenshot of the element when inspecting through the native xcode accessibility inspector, Attaching a screenshot of the same screen when inspected through the appium inspector, Also tried printing the XCTest UI dump using appium method, `driver().executeScript("mobile:source", Map.ofEntries(Map.entry("format","description"))) The UI tree i get is the same that i get when inspecting through the appium inspector. Requesting support from the Apple team based on this ticket, [https://github.com/appium/appium/issues/20759)
Replies
0
Boosts
0
Views
395
Activity
Nov ’24
Not seeing signposts when profiling a unit test (Xcode 16.1)
While I was recently profiling some code from a Swift library, I noticed that XCTest added in signposts for the measurement tests, which I found really helpful to "home in" on the code I wanted to profile digging around in the stack trace. I tried to add my own signposts to provide just a bit of my own markers in there, but while it compiles and profiles equivalently, the signposts just aren't showing up. This is with Xcode 16.1, macOS Sequoia (15.1) and a swift library, using XCTest and profiling within one of the unit tests. Is there something in this sequence that doesn't allow the library to set up signposts and have instruments collect them? The flow I'm using: import os let subsystem = "MyLibrary" class MyClass { let logger: Logger = .init(subsystem: subsystem, category: "fastloop") let signposter: OSSignposter init() { signposter = OSSignposter(logger: logger) } func goFast() { let signpostId = signposter.makeSignpostID() let state = signposter.beginInterval("tick", id: signpostId) // ... do a bunch of work here - all synchronous signposter.endInterval("tick", state) } } Is there something I'm doing incorrectly in using this API, or not enabling to allow those signposts to be collected by the profiler? I do see the signposts that XCTests injects into the system, just not any of the ones I'm creating.
Replies
1
Boosts
0
Views
881
Activity
Nov ’24
XCUITest Screen Recording
In WWDC23: Fix failures faster with Xcode test reports, the presenter showed off an Xcode feature called Automation Explorer, that allows playback of a screen recording corresponding to an XCUITest. For the life of me, I have not been able to find out how to enable/use this feature in Xcode 16. I am using Test Plans and have set the UI Testing configuration settings for Automatic Screen Capture to On, and keep all and Preferred Capture Format to Video, but am not seeing any screen recordings being produced after running the tests. Am I doing something wrong?
Replies
1
Boosts
0
Views
951
Activity
Nov ’24
Run identical UI/Unit tests on different build targets
I have different versions of my iOS App (written in SwiftUI). The app on the store, the App Clip, and one or two next version apps not yet released (e.g. A/B comparison). All good. But now I've started creating UI and Unit tests and I'm confused about how to get this working. Each build target has its own scheme. And in that scheme I have a Test plan for that target. E.g. The App Clip scheme has an App Clip test plan. Since all the app variants are very similar, I only have one set of unit tests and one set of UI tests so each test plan includes the same unit test target and the UI test target. Problem: When I selected a scheme (e.g. for the App Clip) and ran the tests, it turned out that all the tests ran for another build target, not the target of the scheme. I think this might be because within the definition of the test target there's a field specifying the host application. I.e. the build target. Question: How can I set up my project so that the test plan uses the relevant target build? Or do I have to duplicate all the test targets (one for each target)? Or do I have to manually change each test target before running it for a particular build target?
Replies
1
Boosts
0
Views
667
Activity
Nov ’24
Enable local network access during iOS UI test in iOS14
We are building an iOS app that connects to a device using Bluetooth. To test unhappy flow scenarios for this app, we'd like to power cycle the device we are connecting to by using an IoT power switch that connects to the local network using WiFi (a Shelly Plug-S). In my test code on iOS13, I was able to do a local HTTP call to the IP address of the power switch and trigger a power cycle using its REST interface. In iOS 14 this is no longer possible, probably due to new restrictions regarding local network usage without permissions (see: https://developer.apple.com/videos/play/wwdc2020/10110 ). When running the test and trying a local network call to the power switch in iOS14, I get the following error: Task <D206B326-1820-43CA-A54C-5B470B4F1A79>.<2> finished with error [-1009] Error Domain=NSURLErrorDomain Code=-1009 "The internet connection appears to be offline." UserInfo={_kCFStreamErrorCodeKey=50, NSUnderlyingError=0x2833f34b0 {Error Domain=kCFErrorDomainCFNetwork Code=-1009 "(null)" UserInfo={_kCFStreamErrorCodeKey=50, _kCFStreamErrorDomainKey=1}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <D206B326-1820-43CA-A54C-5B470B4F1A79>.<2>, _NSURLErrorRelatedURLSessionTaskErrorKey=("LocalDataTask <D206B326-1820-43CA-A54C-5B470B4F1A79>.<2>"), NSLocalizedDescription=The internet connection appears to be offline., NSErrorFailingURLStringKey=http://192.168.22.57/relay/0?turn=on, NSErrorFailingURLKey=http://192.168.22.57/relay/0?turn=on, _kCFStreamErrorDomainKey=1} An external network call (to google.com) works just fine in the test. I have tried fixing this by adding the following entries to the Info.plist of my UI test target: <key>NSLocalNetworkUsageDescription</key> <string>Local network access is needed for tests</string> <key>NSBonjourServices</key> <array> <string>_http._tcp</string> </array> <key>NSAppTransportSecurity</key> <dict> <key>NSAllowsArbitraryLoads</key> <true/> </dict> However, this has no effect. I have also tried adding these entries to the Info.plist of my app target to see if that makes a difference, but it doesn't. I'd also rather not add these entries to my app's Info.plist, because the app does not need local network access. Only the test does. Does anyone know how to enable local network access during an iOS UI test in iOS14?
Replies
13
Boosts
1
Views
20k
Activity
Nov ’24
Crash calling UIHostingController from a Swift Package
Hello team, We recently found a EXC_BAD_ACCESS crash when using UIHostingControllers on a SPM local Package in our application. This is happening from time to time when we run the app on simulators using Debug configurations. Also, this issue is consistent when we run application tests, there is something weird that we found after making a research... If we disable app test target and only keep packages tests, the crash is not happening, but as soon as we re-enable the app test target from the test suite the crash returns. This is the full error message: Thread 1: EXC_BAD_ACCESS (code=1, address=0xbad4017) A bad access to memory terminated the process. Is this s known issue? We've been performing explorations for some days and couldn't find any real solution for this. A basic call on UIHostingController inside of a SPM local Package is enough to make the app crash, nothing custom being done: UIHostingController(rootView: MyView())
Replies
2
Boosts
0
Views
606
Activity
Nov ’24
How to test the system extension process for coverage
I have a need to do a coverage test for the networkextension function code implemented by the system extension, but I don't know how to implement this method. For example, how do you use gtest or how do you use xctest to achieve these capabilities? If you know, please let me know. Thanks
Replies
0
Boosts
0
Views
484
Activity
Nov ’24
macOS Sequoia doesn't respect Full Disk Access for UITests-Runner
We develop and test App for macOS. We start to see system alert - "UlTests-Runner" would like to access data from other apps on each UITest run. Our test suite does cleanup of files generated by App so we need access outside of UITests-Runner sandbox. We enabled Full Disk Access for UITests-Runner at Settings -> Privacy & Security -> Full Disk Access but unfortunately still see this alert. Is there any way to permanently remove/hide this alert or remove sandbox for 'UITests-Runner' since we want to run tests on CI and having this alert is not an option? Note: everything works fine on previous versions of macOS. Environment: macOS - 15.1 (24B83) Xcode - Version 16.1 (16B40)
Replies
0
Boosts
1
Views
501
Activity
Nov ’24
XCUITest target invalid config sets both USES_XCTRUNNER and either TEST_HOST or RUNTIME_TEST_HOST
I am trying to add an XCUITest target to an existing application. When I try to run the tests, I get the error "Invalid configuration: MyProjectUITests sets both USES_XCTRUNNER and either TEST_HOST or RUNTIME_TEST_HOST". I have seen information stating that it might be because I have multiple apps built from different build configurations in the same scheme, but I could not resolve the issue with any of the suggestions I saw. I've attached a screenshot of the error message and a link to a project stripped down to just the existing configuration to help with diagnosing this issue. I would really love to be able to get some UI tests running. My Project
Replies
1
Boosts
0
Views
679
Activity
Nov ’24
Xcode not generating code coverage.
No coverage is generated for private xxxx.framework, but tests are running under this framework. This was perfectly working before and started giving this error after updating to Xcode 16. Failed to generate coverage for target xxxx.framework at paths ( "/Users/xxxxxx/Desktop/xcode/Debug-iphonesimulator/Common/xxxxxx.framework/xxxx" ): No object file for requested architecture The operation couldn’t be completed. (XCTHCoverageReportGenerationErrorDomain error 2.). (Underlying Error: No object file for requested architecture)
Replies
2
Boosts
1
Views
1k
Activity
Nov ’24
Tagging for UITests
I'm wondering if the tagging functionality which was added for Swift testing will be added to XCTest for UITests anytime soon.
Replies
1
Boosts
1
Views
481
Activity
Nov ’24
iOS simulator freezes on macOS Sequoia during test execution
Hello! 👋 We are seeing a bug on macOS Sequoia related to the test running. When attempting to run tests, the iOS simulator becomes stuck indefinitely. This can occur whether we run tests for a specific module, all unit tests, or even a single test. We narrowed down the issue and is due to the OS failing to copy some named pipes (FIFO). For example: db.realm.note db.realm.management/access_control.new_commit.cv db.realm.management/access_control.pick_writer.cv We saw in the CoreSimulator log file the following error: NSCocoaErrorDomain Code=512 ""access_control.new_commit.cv" couldn't be copied The copy fails and then a new clone of the simulator is created retrying the process. And this goes on and on. What is happening? When running the tests for the iOS app on the simulator, under the hood, the OS tries to clone the simulator device. A list of folders is created, including one Shared/AppGroup. Under the AppGroup folder, the list of multiple UDIDs corresponds to the specific App Group containers created for individual app targets or extensions that are sharing data within that App Group. One of these folders contains the Realm DB files. There are a few files called named pipes which are invisible in Finder (even if you have hidden files to true). You need to list them with ls -l. So, when we select the test target and run the tests, the OS clones the permanent simulator and copies all files from its folders. All files are being copied except those pipes. When the OS attempts to copy the pipes, the operation fails with the error Code=512. Reproduction Steps Run the iOS app on a simulator (e.g. iPhone SE 3rd gen. with iOS 18.1). Quit Xcode and the iOS simulator. Reopen Xcode. Select a test target to run. Select the same simulator you previously ran the iOS app. Run the tests (CMD + U). The simulator is now stuck. Are there any workarounds available? Yes. We found that running the tests works if we first “Erase All Content and Settings” from the simulator. Another workaround is to remove all simulators and reinstall the iOS runtimes. This prevents the issue for a longer period (almost a full day), but eventually, the problem reoccurs. Alternatively, we can delete the named pipes from the App Groups directory before running the tests. Have we tried to give full disk permissions? Yes, we tried to give full disk permissions to a lot of things (Xcode, simulator, file paths, directories, etc.) but with no luck. We still get the same error. Is the issue happening on a specific version of Xcode? No, it happens for multiple Xcode versions: Xcode 15.4 Xcode 16.0 Xcode 16.1 Is the issue happening on a specific macOS Sequoia version? No, it happens on multiple macOS Sequoia versions: 15.0 Beta 8 15.0 RC 15.0 15.1 Stacktrace Oct 29 17:41:18 CoreSimulatorService[14079] <Error>: New device is stuck in creation state, deleting: Clone 712 of iPhone SE (3rd generation) (58D6DED6-2C55-4E7C-BBB4-E0D661DC41A1, iOS 18.1, Creating) Oct 29 17:41:20 CoreSimulatorService[14079] <Error>: Error Domain=NSPOSIXErrorDomain Code=22 "Invalid argument" UserInfo={NSLocalizedFailureReason=Device was allocated but was stuck in creation state. Check CoreSimulator.log for more information.} Oct 29 17:41:20 com.apple.dt.Xcode[90147] <Error>: Error Domain=NSPOSIXErrorDomain Code=22 "Invalid argument" UserInfo={NSLocalizedFailureReason=Device was allocated but was stuck in creation state. Check CoreSimulator.log for more information.} Oct 29 17:41:30 CoreSimulatorService[14079] <Warning>: Device 6C270BDB-2945-42B5-A985-884F93BFD3E1 encountered in creation state at launch. The device will be re-created. Oct 29 17:41:38 CoreSimulatorService[14079] <Error>: Failed to clone the device data path, error = Error Domain=NSCocoaErrorDomain Code=512 "“access_control.new_commit.cv” couldn’t be copied to “db.realm.management”." UserInfo={NSSourceFilePathErrorKey=[...]/db.realm.management/access_control.new_commit.cv, NSUserStringVariant=( Copy ), NSDestinationFilePath=[...]/db.realm.management/access_control.new_commit.cv, NSUnderlyingError=0x600000ffa490 {Error Domain=NSPOSIXErrorDomain Code=45 "Operation not supported"}} Oct 29 17:41:38 CoreSimulatorService[14079] <Error>: Failed to re-create device that was encountered in the creation state (Clone 713 of iPhone SE (3rd generation) (6C270BDB-2945-42B5-A985-884F93BFD3E1, iOS 18.1, Creating)): Error Domain=NSCocoaErrorDomain Code=512 "“access_control.new_commit.cv” couldn’t be copied to “db.realm.management”." UserInfo={NSSourceFilePathErrorKey=[...]/db.realm.management/access_control.new_commit.cv, NSUserStringVariant=( Copy ), NSDestinationFilePath=[...]/db.realm.management/access_control.new_commit.cv, NSUnderlyingError=0x600000ffa490 {Error Domain=NSPOSIXErrorDomain Code=45 "Operation not supported"}} Oct 29 17:41:40 CoreSimulatorService[14079] <Error>: Error Domain=NSPOSIXErrorDomain Code=22 "Invalid argument" UserInfo={NSLocalizedFailureReason=Device was allocated but was stuck in creation state. Check CoreSimulator.log for more information.} Oct 29 17:41:50 CoreSimulatorService[14079] <Warning>: Device C6DEBFBB-6EFA-4E4C-B51B-9DDA08AF9BDB encountered in creation state at launch. The device will be re-created. Oct 29 17:41:55 CoreSimulatorService[14079] <Error>: Failed to clone the device data path, error = Error Domain=NSCocoaErrorDomain Code=512 "“access_control.new_commit.cv” couldn’t be copied to “db.realm.management”." UserInfo={NSSourceFilePathErrorKey=[...]/db.realm.management/access_control.new_commit.cv, NSUserStringVariant=( Copy ), NSDestinationFilePath=[...]/db.realm.management/access_control.new_commit.cv, NSUnderlyingError=0x600000ffb000 {Error Domain=NSPOSIXErrorDomain Code=45 "Operation not supported"}} Oct 29 17:42:06 CoreSimulatorService[14079] <Warning>: Device 032BAE7E-E345-48F2-86EB-4DF1AD4D5291 encountered in creation state at launch. The device will be re-created.
Replies
0
Boosts
0
Views
989
Activity
Oct ’24
How to correctly resolve URL bookmarks for use in Simulator UI tests?
To make UI testing easier and faster, I usually create URL bookmarks during normal app usage in the Simulator so that they can be instantly resolved on app launch during UI tests. For example, one of my apps allows browsing selected folders and stores bookmarks so they can be quickly opened again on following app launches, and instead of selecting the test folder each time at the beginning of the UI test, I select it once during normal app usage so that it's available immediately during the UI test. This usually works fine, but every now and then the UI tests fail because the tested app isn't able to resolve the stored bookmark. I don't know why this happens, but usually opening and closing the app again in the Simulator and re-running the UI tests solves the issue. The problem now is that I've just tried to setup some new UI tests for Apple Vision Pro Simulator and I'm never able to resolve bookmarks. So I created a sample project that reproduces the issue, and curiously enough the bookmarks don't even resolve when using an iPad Simulator (which usually works fine with my regular UI tests). What am I doing wrong? This can be reproduced with a default iOS project, embedding the default storyboard view controller in a navigation view controller, and this code: import UIKit class ViewController: UIViewController, UIDocumentPickerDelegate { override func viewDidLoad() { navigationItem.rightBarButtonItem = UIBarButtonItem(systemItem: .add, primaryAction: UIAction(handler: { _ in let picker = UIDocumentPickerViewController(forOpeningContentTypes: [.folder]) picker.delegate = self self.present(picker, animated: true) })) if let bookmark = UserDefaults.standard.data(forKey: "bookmark") { readBookmark(bookmark) } } func readBookmark(_ bookmark: Data) { do { let label = UILabel(frame: CGRect(x: 100, y: 100, width: 600, height: 100)) label.numberOfLines = 0 var stale = false let url = try URL(resolvingBookmarkData: bookmark, bookmarkDataIsStale: &stale) if !url.startAccessingSecurityScopedResource() { fatalError() } label.text = url.path view.addSubview(label) } catch { fatalError(error.localizedDescription) } } func documentPicker(_ controller: UIDocumentPickerViewController, didPickDocumentsAt urls: [URL]) { do { let url = urls[0] if !url.startAccessingSecurityScopedResource() { fatalError() } let bookmark = try url.bookmarkData() UserDefaults.standard.set(bookmark, forKey: "bookmark") readBookmark(bookmark) } catch { fatalError(error.localizedDescription) } } } And a default UI test, which always crashes because of the fatalError() in the catch clause of readBookmark(_:): final class problemUITests: XCTestCase { @MainActor func testExample() throws { let app = XCUIApplication() app.launch() } }
Replies
0
Boosts
0
Views
479
Activity
Oct ’24