I have filed this as FB13722352. I am sharing it here because I haven't seen it mentioned anywhere online yet and am curious if anyone else has run into it. In Xcode 15.3+, writing data to disk fails when running the tests for a Framework project in a fresh simulator. Specifically, if the selected simulator has never been launched before (i.e. is newly-created), any test execution that attempts to write data into the NSDocumentDirectory directory will fail for a period of time after the simulator is first launched (I've observed between 10s and 20s). After that period of time, the same data write action will succeed. It appears that Xcode 15.3+ is starting test execution too soon, without waiting a sufficient amount of time for the Simulator to fully boot. This issue does not occur in Xcode 15.2 or prior versions. Since the issue only appears in a fresh (never-before booted) simulator, it is likely to pop up consistently in CI test runs (where simulators are not re-used). This can cause confusion because the same test would not fail locally when re-using an existing simulator. When the issue appears, the file write API returns the following error: Domain=NSCocoaErrorDomain Code=4 "The folder “testFile” doesn’t exist." UserInfo={NSFilePath=[...]/data/Documents/testFile, NSUserStringVariant=Folder, NSUnderlyingError= {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"} } Reproduction Steps: Open Xcode 15.3 or 15.4. Make sure Simulator.app is closed. Using the "Devices and Simulators" window, create a new iPhone 15 Pro simulator with iOS 17.4 (other devices and OS versions work as well). Do not launch this new simulator. Create a new Framework project and add a test that performs and then checks the output of a data write to the Document directory (see example test code below). Select the new simulator (created in step 2) as the test run target and run the test. Here's an example test that fails in the scenario outlined above: - (void)testBasicRepro { NSString *testString = @"Hello, World!"; NSData *data = [testString dataUsingEncoding:NSUnicodeStringEncoding]; // Get documents directory NSURL *url = [[[NSFileManager defaultManager] URLsForDirectory:NSDocumentDirectory inDomains:NSUserDomainMask] lastObject]; NSURL *testFileURL = [url URLByAppendingPathComponent:@"testFile"]; // Write the data NSError *error; bool result = [data writeToURL:testFileURL options:NSDataWritingAtomic error:&error]; // Check if it was successful XCTAssertTrue(result); XCTAssertNil(error); XCTAssertTrue([[NSFileManager defaultManager] fileExistsAtPath:testFileURL.path]); } Workaround The workaround that I have come up with is to create a test that runs first (by disabling parallelization and randomization, and making sure the test class filename is alphabetically first). Alternatively, it could be called from the setUp method in any test files that are affected. This test performs a data write and checks the result in a loop in order to block until the data write succeeds (i.e. the Simulator is sufficiently booted for data write operations to complete). - (void)testWorkaroundBug { NSString *testString = @"Hello, World!"; NSData *data = [testString dataUsingEncoding:NSUnicodeStringEncoding]; NSError *error; // Get documents directory NSURL *documentsURL = [[[NSFileManager defaultManager] URLsForDirectory:NSDocumentDirectory inDomains:NSUserDomainMask] lastObject]; NSURL *testFileURL; NSDate *startTime = [NSDate date]; NSLog(@"Starting test at %@", startTime); for (int i = 0; i < 120; i++) { // Create unique URL testFileURL = [documentsURL URLByAppendingPathComponent:[NSString stringWithFormat:@"testFile-%@", @(i)]]; // Write the data BOOL success = [data writeToURL:testFileURL options:NSDataWritingAtomic error:&error]; // Check if it exists if (success && [[NSFileManager defaultManager] fileExistsAtPath:testFileURL.path]) { NSLog(@"Test file %@ was created successfully! Elapsed time %@s", @(i), @(fabs([startTime timeIntervalSinceNow]))); return; } else { NSLog(@"Test file %@ was not created. Error: %@. Sleeping for 0.5s and trying again.", @(i), error); [NSThread sleepForTimeInterval:0.5]; } } }

Hi, Recently noticed this issue on Xcode 15.2 When I select a Test Plan, disable a class, enable just 1 test within that class and refresh the test plan the entire class gets enabled again. steps to notice the issue: Select Test Plan 1 Disable a class Enable 1 test within the class Select Test Plan 2 Select Test Plan 1 Disabled class and all the tests inside the class are enabled. Note: For now the workaround for that is disabling each test of the class individually Does anyone have any idea why this is happening? If theres a way to disable a class instead of disabling each test inside the class?

Hello everyone, I looked at various methods how to Unit/UITest SwiftData but I couldn't find something simple. Is it even possible to test SwiftData? Does someone found a solution for that?

I'm working on some apps at the minute, with the intention of running a lot of automated UI tests. Xcode cloud looks great and has a lot of integrated features, but having to make sure I don't run too many hours so my subscription doesn't stop mid dev cycle, or getting a surprise large bill etc, are things I want to avoid. Even the cheapest paid plan for a year works out about the same cost as an M2 mac mini, which would probably be significantly faster than the cloud nodes. Github actions allow you to provide your own machines. But actions requires a lot more scripting, and so far they've been very slow to update Macos/Xcode versions (not sure how this will effect supplying own machine). My preference would be to deploy my own machine for Xcode cloud, similar to what we used to be able to do with Xcode server. I think this is currently impossible right? Is there any word that this might be an added feature in the near future?

To authenticate Apple Pay payment through Mobile Test Automation via Face ID and a side button click if there is a way to automate a process involving Apple Pay where authentication could be performed using a software-based solution accessible through automation. Is there any way to access the side click button programmatically, using XCTest?

device : iPhone 8 Plus version: iOS 16 xcode version: 15.0.1 mac os: 13.6.6 When XCTest start to automation running, it does not cover the device's whole window as the picture shows. And my test cases testify that it cannot inspect the element outside of the cover. Is there any solution to make it compatible?

Hi all, I ran into an XCUITest issue where my tests fail randomly with the message that my app "has not loaded accessibility" I run the tests in my scheme in random order and with every run, some other random test fails and a test that previously failed would later pass. So I know it's not my test as-such that is causing the problem. There seems to be a 60s wait for accessibility to load, and I have not found any information how I could extend the timeout or convince accessibility to load faster. I found some older posts, but no real solution was found there either. This completely blocks my test execution since I cannot get a single run in where the tests all pass (and I only have 10 tests in that scheme). I ran the scheme against an iOs sim and against a real iOS device and I get the same random accessibility errors on either execution platform. I've tried iOS17.2 & 17.4 and get the same behavior. Strange enough, the test actually keeps running (and technically passes all the checks and asserts in the actual test) after the accessibility error, but then XCTest marks it as failed because of the early accessibility error. Any ideas what I could try or what the reason could be? Thanks! -- I'm using XCode Version 15.3 (15E204a) and test iOS17.2 and iOS17.4 Test Case '-[MyScheme.MyTestClass testMyFunction]' started. t = 0.00s Start Test at 2024-04-19 01:32:09.225 t = 0.02s Set Up t = 0.02s Open com.myCompany.myApp t = 0.03s Launch com.myCompany.myApp t = 0.25s Wait for accessibility to load t = 60.30s Capturing diagnostic spindump /Users/some/path/goes/here/MyTestCase.swift:120: error: -[MyScheme.MyTestClass testMyFunction] : Application 'com.myCompany.myApp' has not loaded accessibility t = 60.34s Waiting 60.0s for "test" Button to exist t = 61.38s Checking `Expect predicate `exists == 1` for object "test" Button` t = 61.39s Checking existence of `"test" Button` ... more stuff happening in the test here ...

So I have a simple view for my SwiftUI application below: import SwiftUI struct ContentView: View { var body: some View { List { ForEach(0..<4) { index in RowView(index: index) .accessibilityIdentifier("row") } } } } struct RowView: View { let index: Int var body: some View { HStack { Image(systemName: "globe") .imageScale(.large) .foregroundStyle(.tint) Text("Hello, world!") Spacer() ButtonView(index: index) } .padding() } } struct ButtonView: View { let index: Int var body: some View { VStack { if index != 3 { Button(action: {}, label: { Text("Cancel") }) .accessibilityIdentifier("cancel_button") } Button(action: {}, label: { Text("Submit") }) .accessibilityIdentifier("submit_button") } } } My goal is to reference each cancel and submit button per row for a UI Test, which I have tried doing so here: let rowElements = app.tables.cells.matching(identifier: "row") for i in 0..<rowElements.count { let cancelButton = rowElements.element(boundBy: i).buttons["cancel_button"] let submitButton = rowElements.element(boundBy: i).buttons["submit_button"] XCTAssertTrue(cancelButton.exists, "Cancel button does not exist in row \(i)") XCTAssertTrue(submitButton.exists, "Submit button does not exist in row \(i)") } Even with the identifiers set like this my tests fail and say it cannot find the buttons labeled as they are with their specific identifiers or the rows. What is the correct way to reference elements if they are in a list or table for UI Testing? I'm specifically trying to use accessibility Identifiers to make my life easy.

This code works when I run it in the iOS Simulator with iOS 17.0.1: let passcodeInput = springboard.secureTextFields["Passcode field"] _ = passcodeInput.waitForExistence(timeout: 10) passcodeInput.tap() However if I run it on the iOS Simulator with iOS 17.4 I get this error: t = nans Checking existence of `"Passcode field" SecureTextField` t = nans Capturing debug information t = nans Requesting snapshot of accessibility hierarchy for app with pid 66943 t = nans Tap "Passcode field" SecureTextField t = nans Wait for com.apple.springboard to idle t = nans Find the "Passcode field" SecureTextField t = nans Find the "Passcode field" SecureTextField (retry 1) t = nans Find the "Passcode field" SecureTextField (retry 2) t = nans Requesting snapshot of accessibility hierarchy for app with pid 66943 <unknown>:0: error: PRCheckUITests : Failed to tap "Passcode field" SecureTextField: No matches found for Descendants matching type SecureTextField from input {( Application, pid: 66943, label: ' ' )} Did the hardcoded string "Passcode field" change for iOS 17.4? How can I access the passcode field through springboard in a test?

I have a CI setup where one job builds xctestrun files for tests and another job that uses tem to actually run tests. The setup worked fine with XCode 14 on macOS 13. Now after switching to macOS 14 and XCode 15.0.1 xcodebuild doesn't want to run tests anymore: xcodebuild -destination 'generic/platform=iOS' -derivedDataPath DerivedData_Temporary/ -resultBundlePath './fastlane/test_output/APPName-iOS.xcresult' -testPlan 'APPName-Unit-All' -xctestrun 'DerivedData_Temporary/Build/Products/APPName_APPName-Unit-All_iphonesimulator17.0-arm64-x86_64.xctestrun' test-without-building throws an error xcodebuild: error: Failed to build workspace iOS with scheme APPName-iOS.: Scheme “APPName-iOS” does not have an associated test plan named “APPName-iOS-Unit-All” Almost the same command for macOS works perfectly fine. What does this error even mean? And how to fix it?

Hello! I've been trying to automate tests using Appium/XCUITests in a React Native APP, but I'm finding many blockers. They are related to the amount of nested elements in the "DOM" in which the XCUItest can not go deeper to get all the elements we need. snapshotMaxDepth -> The XCUITest does not support more than 60 value, otherwise it returns the following error: Got response with status 404: {"value":{"error":"stale element reference","message":"The previously found element "Application 'xyz.***.***'" is not present in the current view anymore. Make sure the application UI has the expected state. Original error: Error kAXErrorIllegalArgument getting snapshot for element <AXUIElementRef 0x600003aaf750> {pid=95967} {uid=[ID:1 hash:0x0]}","traceback":"(\n\t0 CoreFoundation 0x00007fff20405604 __exceptionPreprocess + 242\n\t1 libobjc.A.dylib 0x00007fff201a4a45 objc_exception_throw + 48\n\t2 WebDriverAgentLib 0x000000010a3caa53 -[XCUIElement(FBUtilities) fb_takeSnapshot] + 723\n\t3 WebDriverAgentLib 0x000000010a3cad07 -[XCUIElement(FBUtilities) fb_snapshotWithAttributes:maxDepth:] + 183\n\t4 WebDriverAgentLib 0x000000010a37baea +[FBXPath writeXmlWithRootElement:indexPath:elementStore:includedAttributes:writer:] + 778\n\t5 WebDriverAgentLib 0x000000010a37b12c +[FBXPath xmlRep... But if I inform less than 60, the XCUITest is not able to get all the elements we need to automate: There are many threads about this, all of them the issue is in the XCUITest: https://github.com/appium/appium/issues/14825 https://discuss.appium.io/t/handling-staleelementreferenceexception/35095/11 https://github.com/appium/appium/issues/18085 https://discuss.appium.io/t/error-in-appium-desktop-refreshing-source-after-adding-snapshotmaxdepth-greater-than-62/34058 https://stackoverflow.com/questions/74235441/appium-cant-reach-elements-in-ios-source-tree-they-are-too-deep-works-fine-in Tested all possible solutions suggested in the threads, but I will have the issue.

I've a non-document based single window (mostly) SwiftUI app. In my observable app state class an undo manager is defined and changes are registered using that manager. This has not been a problem in the past. But now... 1st is a bug reported in FB13708788. I've replaced the macOS undo/redo with code that references the UndoManager I created. The menu does not reflect the current undo status. This is easily duplicated in a small app submitted with the feedback. 2nd issue is hit or miss. Sometimes selecting Undo via the menu or the keyboard invokes undo multiple times. This is rare when running the app, but it has happened. On the other hand undo is always invoked multiple times when running an XCUITests. A little logging shows the invocations are quite close together. Undo invoked at 2024-04-05 13:14:19.975570-0700 Undo invoked at 2024-04-05 13:14:19.975768-0700 Calls were only 198 microsec apart. The test makes a change, makes a second change, then tried to undo the second change. Both changes are undone. But not (usually) if I perform the exact steps by hand instead of as an XCTest. Any ideas on how I might track down the cause? Mac Studio M2 Max running 14.4.1.

I tried to unit test the method "FilterDataProvider.handleNewFlow" from the sample project https://developer.apple.com/documentation/networkextension/filtering_network_traffic I always get this below issue, Undefined symbols: Linker command failed with exit code 1 (use -v to see invocation) Could someone help with it?

Got an issue with XC UI Testing where I can't open documents without them being marked as locked (the Locked label is shown in the titlebar). The document in question is a testing asset and copied into the testing bundle for easy access. I've checked the file in bundle in Finder and the document is not marked as locked. I've also tried copying the document to a temp directory before opening it and it's still locked?! What's the trick for getting it to open without being locked?

I am trying to do the tests form the terminal and xcpretty the output. Here is the command: xcodebuild -workspace bimb-authenticate-ios.xcworkspace -scheme bimb-authenticate-ios-afx -configuration Debug -destination 'platform=iOS Simulator,name=iPhone 14 Pro,OS=16.4' -enableCodeCoverage YES -derivedDataPath Build/ clean build test | xcpretty -c "--report" "html" "--output" "../Project_report.html" However, the tests are never run. The simulator is not even opened. Here's the last items of the xcbuild logs: ... *compiling, linting and linking logs* 2024-03-27 21:42:37.151 xcodebuild[84540:2237180] [MT] IDETestOperationsObserverDebug: 264.553 elapsed -- Testing started completed. 2024-03-27 21:42:37.152 xcodebuild[84540:2237180] [MT] IDETestOperationsObserverDebug: 0.001 sec, +0.001 sec -- start 2024-03-27 21:42:37.152 xcodebuild[84540:2237180] [MT] IDETestOperationsObserverDebug: 264.554 sec, +264.553 sec -- end ▸ Test Succeeded Can anybody help me? Thanks.

I am currently using Xcode 15.2, but I am sure this issue exists on other versions too. If you have any packages made pre Xcode 14.3, i.e. ones without an automatically made test plan, then the following issue happens: Let's call the package in this example PackageX, and you have with this a corresponding test scheme, called PackageXTests. PackageXTests is a test scheme saved on disk, that is on the test scheme of PackageX, meaning that when you test the package directly, it will run the test scheme. Now, with test plans becoming the norm, we want to move away from having these test schemes on disk and instead align old packages with new and have each package's tests dictated by its own corresponding test plan, the same way that any newly made package would work. So we click 'convert to test plan', success we have a test plan, we delete the test scheme from the disk. All appears to be working fine. However, if you now swap branch and come back, the test plan will no longer be connected to that package. It will have reverted back to a test scheme. Which in git shows as a test scheme which it has added (i.e. the git diff has an uncommitted test scheme it has made). This is not a derived data issue, as I have deleted my derived data and it still occurs. Similarly with cleaning and building.

Since couple of weeks ago (probably after I've updated my macbook to Sonoma 14.3.1) my Xcode cannot load unit test. When opening Test Navigator all I see is: Loading Tests... and it stays forever in that state. This is completely new project and this is crazy. Only way I can start testing is if I use CMD+U to run all the tests which is my company's project take a lot of time. Also there's no option to click on test-diamond-shaped button to run individual tests. Also tried checking "Test Plans" but I have infinite progress views: Anyone has any idea how to fix this? I tried to remove Derived Data, to reinstall Xcode (and different versions), did "git clean -fxd" nothing helped. Also some function like: "Rename", "Edit all in scope" stopped working as well.

My project has UI tests with total length of ~50 minutes. I'm using Xcode Cloud for builds delivery, UI tests execution is the part of the workflow. When I was using Monterey as tests environment everything was fine, but now it's no longer available so i had to switch to Sonoma. Since this i wasn't able to run anything but a couple of separate UI tests, when I run all of them I'm having 2 major issues. Locally on my machine I have the same Xcode/MacOS/simulator versions and everything works perfectly, issues only appear on Xcode Cloud. Here are they: There is a huge time gap between build-for-testing is finished and test-without-building is started. The gap is more than 1 hour so the sum of build time, ui tests time and the gap results in more than 2 hours and tests execution fails with the timeout. Funny is when I try to run the a single UI test or just run unit tests this gap only lasts for like 2-6 minutes, but when I add more tests the gap goes huge and fails everything. Also I guess my company is paying for those gaps and runs of tests when they timeout because of it even tho it's not our fault. Second issue is that some UI tests just freeze in the meddle of execution. This happens when UI tests is trying to interact with text fields on modal view controllers, sometimes it happens during the text field lookup, sometimes - after it takes focus and test is trying to type things into it. This is cannot be dodged anyhow and it is not related to some specific code - I've intentionaly rewritten the whole view controller from UIKit to SwiftUI , changed the way I present it and the location of presentation. Looks like just having a text field on a modal view is enough for test to freeze. Have anyone experienced something like this? Found any solutions?

I have latest version Xcode-15.1 & Mac-os version Sonoma-14.3.1 - while running particularly UITest target Xcode not launching the simulator, even after build succeed & test's completed without any errors. I tried cleaning project, re-starting Xcode, restarting system(Mac) - no luck It would be helpful if I get any suggestion on resolving the issue.