I am writing a SPM based project for MacOS. In this project? I need to access MacOS Keychain. I am write a swift test built by SPM testTarget(). I can see it generates a bundle ./.build/x86_64-apple-macosx/debug/MyProjectTests.xctest with an executable: % file ./.build/x86_64-apple-macosx/debug/MyProjectPackageTests.xctest/Contents/MacOS/MyProjectPackageTests ./.build/x86_64-apple-macosx/debug/MyProjectPackageTests.xctest/Contents/MacOS/MyProjectPackageTests: Mach-O 64-bit bundle x86_64 This bundle file cannot be executed. How can I execute its tests? I tried with xcodebuild test-without-building -xctestrun ./.build/x86_64-apple-macosx/debug/MyProjectPackageTests.xctest -destination 'platform=macOS' without any chance. Obviously the next question is can I 'simply' add entitlement to this bundle with codesign to fix my enttilement error. My error when running the test is A required entitlement isn't present.

In my iOS project, there is an infrequent crash related to virtual memory problem. Therefore, I plan to use UITest in combination with Product/Perform Action/Profile "TestCaseName" to conduct Game Performance-type testing. This allows the automatic testing to continuously operate until the profile stops recording upon a crash. This enables me to observe the various states of the program at the time of the crash. However, I have found that the UITest using Profile is highly unstable. The UITestCase often terminates unexpectedly during execution, leading to failed tests (Instruments is still working). Sometimes, the app is terminated immediately after startup. It seems that the use of sleep() in the code can easily cause interruption issues, which do not occur during normal UI testing. I am wondering if anyone has experience using Profile for UITest and whether they have encountered the issues I described." Working Environment: XCode14.3.1, iPhone Device iOS17.2

I have a CI pipeline, which runs unit tests and exports the code coverage. I need to run the sonar-scanner tool to export the report to the sonarqube server running on a server that can only be accessed over a VPN only. How can I do this?

I am directed from https://discuss.appium.io/t/create-multiple-instances-of-the-same-app/41266/14?u=lxnm, the context is that Appium max2 driver is implementing the activating of MacOS app using XCTest API methods. There are 2 ways to implement the activating of the app, using app path (calling initWithURL) and bundle id (calling initWithBundleIdentifier). This pull request shows how the XCTest methods are called when using the Mac2 driver, specifically in the file WebDriverAgentMac/WebDriverAgentLib/Routing/FBSession.m. The problem is that I am able to launch my MacOS app(it is a company app) using bundle id, but when I launch the app with app path, I receive XcodeBuild errors: [WebDriverAgentMac] [xcodebuild] XCTExpectFailure: matcher accepted Assertion Failure: Failed to launch com.company.companyApp: You do not have permission to run the application “companyApp”. You don’t have permission. To view or change permissions, select the item in the Finder and choose File > Get Info. (Underlying Error: The operation couldn’t be completed. Operation not permitted) I followed this to enable R+W permissions to all users, but the same error is displayed.

I am currently working on a SwiftUI project and I am trying to write a UI test for a DatePicker. However, I am having trouble interacting with the DatePicker in the test. Here is the code for the DatePicker in my SwiftUI view: DatePicker("Date", selection: $date, displayedComponents: [.date]) .datePickerStyle(.graphical) .labelsHidden() .frame(width: 150) And here is the code for my UI test: func testSelectingDateChangesMainView() throws { let app = XCUIApplication() app.launch() let dateDatePicker: XCUIElement = app.windows["mainUI-AppWindow-1"].datePickers["Date"] dateDatePicker.click() let col = dateDatePicker.descendants(matching: .any) print("\(dateDatePicker.debugDescription), \(col.debugDescription)") } When I run the test, it seems like the DatePicker does not have any descendants, so I am unable to select a date. Does anyone have any suggestions on how I can interact with a .graphical style DatePicker in a UI test? Any help would be greatly appreciated. Thank you, OnlyForF1

Hi! I'm experimenting with Xcode performance testing. I'm specifically focusing on XCTMemoryMetric. I have a demo project with two performance tests that do some work wrapped with XCTestCase.measure. I can navigate to my Report navigator in Xcode and see the memory footprints of these two tests… but I only see how to view them individually: From this screen it looks like I have the ability to Export this benchmark to CSV. My question is what my options are for viewing all my XCTestCase.measure tests together. I see the Duration for both tests… but I don't see any way to view the memory benchmarks for all my tests in just one place: Ideally… I would also like to run these tests and produce one CSV file (or XML or JSON) report. I'm not opposed to using command line for this (if the option exists there) but Xcode also works for me. I'm also flexible whether or not these tests live in a Swift Package or directly in an Xcode project if that makes any difference. Any more advice about how to set up these performance tests? Thanks!

We recently updated to Xcode 15.1 and start using iOS 17.2 simulator and ran into a blocker issue with out UITests. Setting accessibility identifier on a UIButton with image no longer works. It seems iOS automatically set the label to the file name used for the button and ignore (overwrite) the id and value we set in code. iOS 17.0 with Xcode 15.1 still works. I spent 2 days on this and still cannot find a solution. Anyone had similar issue? I did file a feedback https://feedbackassistant.apple.com/feedback/13515676 Thanks!

Hi, I have created the applications (debug and runner) using xcodebuild. I have tried it both using Xcode 14.3.1 and 15.1. I used the xcodebuild build-for-testing... with -configuration set to Debug (I've also built the apps by omitting the -configuration altogether). The configurations are set correctly in Xcode, and I can run the automation from Xcode and running xcodebuild. I am using a different tool other than xcodebuild to run the tests (I'm using go-ios https://github.com/danielpaulus/go-ios). The issue I'm having is that right at the beginning of the test, I'm getting this error "Failed to retrieve test configuration from IDE". I know it's something to do with the two apps (debug and runner appp), because I've made comparison to a brand new application I had created using Xcode. I assume this is something to do with the build configurations we have in the app (Debug or Release), but they're all correctly set in Xcode, and I can also run the tests (albeit using the xctestrun file) without any problems. Do you know what could be causing this? In particular, what could cause the above error to be generated? Any help (however related or unrelated you think it could be) is greatly appreciated. Thanks in advance. Sep

I'm working on this project for the first time in a while, so I'm not sure if this issue started with Xcode 15 or what, but now when I run my UI tests I get the warning "“MyAppUITests-Runner” is from an unidentified developer and differs from previously opened versions. Are you sure you want to open it?" Code signing identity for the UITest target is set to "sign to run locally". Changing it to "Apple Development" doesn't help, and neither did a clean build. How do I need to configure it to eliminate the warning?

I've created a test plan that includes the below configurations. {"configurations": [{ ... "options": { "defaultTestExecutionTimeAllowance" : 60, "maximumTestExecutionTimeAllowance" : 60, "testTimeoutsEnabled" : true }], "defaultOptions": { "defaultTestExecutionTimeAllowance" : 60, "maximumTestExecutionTimeAllowance" : 60, "testTimeoutsEnabled" : true, .... } ... } If I write a test that intentionally will take longer than 60 seconds, executing the test locally fails as expected with the error message Test exceeded execution time allowance of 1 minute let app = XCUIApplication() app.launch() let missingButton = app.buttons.firstMatch XCTAssertTrue(missingButton.waitForExistence(timeout: 120), "When a timeout is > 60 seconds, the default max setting of 60 seconds is still enforced") However, when this test is run using this test plan in XCode Cloud, it reports that it took 124 seconds to run and there is no message about exceeding execution time allowance. How can the test step be configured in XCode Cloud to ensure that the execution time allowances are respected?

when I download the sample backyard app from wwdc 23. The files for the Unit and Ui test are missing. Where do I find the? since I would like to see how they Apple implemented unit tests with SwiftData

Hi community: I found that after Xcode update from 14.3.1 to 15.0.1, my UITests fail only on the CI (Apple Xcode Cloud) because the use of boundElementsByIndex. Seems that the use of this property stale the main thread: Failed to resolve query: Unable to perform work on main run loop, process main thread busy for 30.0s Is it a known issue? Thanks

I'm trying to write performance tests with Xcode 15, but it's not working: even though setting a baseline seems to work (baseline data files are created), Xcode seems to ignore that baseline altogether whenever the test is run. As a result, performance tests never fail, no matter how bad performance gets. The source editor persistently displays a “No baseline average for Time” message. Has anyone else encountered this? Figured out how to sidestep the issue? Of note, the issue is easily reproduced: create a new (macOS in my case) app project with tests, add a dummy performance test; run the test once and save its baseline, and despite this subsequent tests will always succeed, even if you significantly slow the dummy test down. (I've filed this as FB13330139)

For my test automation project I use addUIInterruptionMonitor to keep the tests running smoothly. On iOS 17 this function seems to never get triggered. This code snippet contains a test that passes on iOS 15 and 16, but not on iOS 17 (tested on iPhone 12, iOS 15.5, iPhone 13 Mini, iOS 16.4.1 and iPhone 14 Pro, iOS 17.0). To run the test, disable wifi and cell data on the test device first. import XCTest class Interruptions: XCTestCase { func testCatchInterruption() { // Run test on device with wifi and cell data disabled var caughtInterruption = false addUIInterruptionMonitor(withDescription: "Generic Alert Handler") { element -> Bool in element.buttons["OK"].tap() caughtInterruption = true return true } let safariApp = XCUIApplication(bundleIdentifier: "com.apple.mobilesafari") safariApp.launch() // Expect to see pop-up saying "Cellular Data is turned off" safariApp.tap() sleep(5) // Pop-up should be gone XCTAssert(caughtInterruption) } }

Xcode is throwing an error when attempting to report test coverage after running unit tests. Showing All Messages Failed to merge raw profiles in directory /{my project dir}/DerivedData/{my project name}/Build/ProfileData/A98EC493-3AB4-4B1C-B7FC-BC5D77B23EE3 to destination /{my project dir}/DerivedData/{my project name}/Build/ProfileData/A98EC493-3AB4-4B1C-B7FC-BC5D77B23EE3/Coverage.profdata: Aggregation tool '/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/llvm-profdata' failed with exit code 1: warning: /{my project dir}/DerivedData/{my project name}/Build/ProfileData/A98EC493-3AB4-4B1C-B7FC-BC5D77B23EE3/997F01E3-3735-4897-AE00-31EBB0A9E8D3-57885.profraw: invalid instrumentation profile data (file header is corrupt) error: no profile can be merged Anyone else seeing this issue?

I've written a Swift package plugin to add the colours provided in the asset bundles in an extension of the SwiftUI Color type but I'm having difficulties testing this since the bundle and/or colour that is in the asset catalogs are not yet loaded when XCTest goes to evaluate the test conditions. Here is my test code: (colours are just random colours, some derive from macOS colour picker; also, the new static method is functionality that I added and is not present in the existing Color APIs) final class TestCase: XCTestCase { func testBanana() { XCTAssertEqual(Color.banana, Color.new(from: "FEFC78")!) } func testAlexsColor() { XCTAssertEqual(Color.alexsColor, Color.new(from: "0432FF")!) } func testMaximumPowerColor() { XCTAssertEqual(Color.maximumPower, Color.new(from: "FF2600")!) } func testLongNameColor() { XCTAssertEqual(Color.reallyLongNameThatJustKeepsGoingOnAndOnAndOnAndOn, Color.new(from: "AAA000")) } } Here is an example error message that I get when I run the test: testBanana(): XCTAssertEqual failed: ("NamedColor(name: "Banana", bundle: Optional(NSBundle </Users/trevorhafner/Library/Developer/Xcode/DerivedData/TransportBase-cbagdabrompfzofwkimswvlsincu/Build/Products/Debug/TransportBaseTests.xctest/Contents/Resources/TransportBase_TransportBaseTests.bundle> (not yet loaded)))") is not equal to ("#FEFC78FF") Here is the autogenerated code that my Swift Package Plugin creates: #if canImport(SwiftUI) import SwiftUI import Foundation @available(iOS 13.0, macOS 10.15, tvOS 13.0, watchOS 6.0, *) public extension Color { static let banana = Color("Banana", bundle: .module) static let alexsColor = Color("Alex's Color", bundle: .module) static let reallyLongNameThatJustKeepsGoingOnAndOnAndOnAndOn = Color("really long name that just keeps going on and on and on and on", bundle: .module) static let maximumPower = Color("Maximum Power", bundle: .module) } #endif Does anyone know how to somehow instruct the bundle or the Color to load such that the comparison can be made correctly between the two colours?

My tests fail with "Restarting after unexpected exit, crash, or test timeout". This is what I have so far established: Does only happen when building with xcodebuild (14.3) but not when building with Xcode (14.3) Tests that are failing are in a .testTarget within a local swift package and part of a test plan together with tests that are in subprojects there are no crashlogs, so likely these tests are not crashing with Xcode 14.2 there are no problems whatsoever (xcodebuild or Xcode) Any ideas on how to debug this any further? Best Roddi

I get this error from time to time in UI tests when they are run on CI. Any ideas what might be causing this error?

The documentation states that Xcode Cloud comes with support for git lfs (https://developer.apple.com/documentation/xcode/source-code-management-setup) but when I run my unit tests in Xcode Cloud, which have many images which are stored in git lfs, the tests all fail because they don't have the images. How do I get Xcode Cloud to fetch git lfs images? Thanks

I'm new to Xcode cloud - working with a Mac OS app, build is working great. Now I am trying to add a Test action; the testing target builds but won't run, and the error indicates it can't find the testing bundle in the expected build output. There's also mention of a code signing error, but I have automatic code signing enabled with the same settings on test target as the app. I am only running the unit test (XCTest) scheme, not the UI tests. When I run it locally from the IDE it works fine, either selecting the test scheme explicitly or as the test step of the app scheme. I notice the XCTest target's scheme setup uses Debug builds and expects the test output to be in the Debug .app bundle, I thought perhaps that was the problem (in case only the release app bundle actually gets built in the Xcode Cloud environment). So I created a duplicate scheme and set the build targets to Release - again I can run this fine locally (after creating a release build), but it fails with the same error in Xcode cloud. I also tried changing the code signing certificate from "Development" to "Sign to run locally" to see if that made a difference, but I get the same error. (It's using my developer account Team, and "Automatically manage signing".) Can anyone relate the proper way to set up an XCTest scheme so that the tests will actually run in a Mac OS Xcode Cloud workflow? I'm using Xcode 14.0.1. Here's the full error output, with [AppName] and [TestTargetName] substituted for the actual: [AppName] (....) encountered an error (Failed to load the test bundle. If you believe this error represents a bug, please attach the result bundle at /Volumes/workspace/resultbundle.xcresult. (Underlying Error: The bundle “[TestTargetName]” couldn’t be loaded. The bundle couldn’t be loaded. Try reinstalling the bundle. dlopen(/Volumes/workspace/TestProducts/Debug/[AppName].app/Contents/PlugIns/[TestTargetName].xctest/Contents/MacOS/[TestTargetName], 0x0109): tried: '/Volumes/workspace/TestProducts/Debug/[TestTargetName]' (no such file), '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib/[TestTargetName]' (no such file), '/Volumes/workspace/TestProducts/Debug/[AppName].app/Contents/PlugIns/[TestTargetName].xctest/Contents/MacOS/[TestTargetName]' (code signature in <....> '/Volumes/workspace/TestProducts/Debug/[AppName].app/Contents/PlugIns/[TestTargetName].xctest/Contents/MacOS/[TestTargetName]' not valid for use in process: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?)))) Thanks!