I've written a Swift package plugin to add the colours provided in the asset bundles in an extension of the SwiftUI Color type but I'm having difficulties testing this since the bundle and/or colour that is in the asset catalogs are not yet loaded when XCTest goes to evaluate the test conditions. Here is my test code: (colours are just random colours, some derive from macOS colour picker; also, the new static method is functionality that I added and is not present in the existing Color APIs) final class TestCase: XCTestCase { func testBanana() { XCTAssertEqual(Color.banana, Color.new(from: "FEFC78")!) } func testAlexsColor() { XCTAssertEqual(Color.alexsColor, Color.new(from: "0432FF")!) } func testMaximumPowerColor() { XCTAssertEqual(Color.maximumPower, Color.new(from: "FF2600")!) } func testLongNameColor() { XCTAssertEqual(Color.reallyLongNameThatJustKeepsGoingOnAndOnAndOnAndOn, Color.new(from: "AAA000")) } } Here is an example error message that I get when I run the test: testBanana(): XCTAssertEqual failed: ("NamedColor(name: "Banana", bundle: Optional(NSBundle </Users/trevorhafner/Library/Developer/Xcode/DerivedData/TransportBase-cbagdabrompfzofwkimswvlsincu/Build/Products/Debug/TransportBaseTests.xctest/Contents/Resources/TransportBase_TransportBaseTests.bundle> (not yet loaded)))") is not equal to ("#FEFC78FF") Here is the autogenerated code that my Swift Package Plugin creates: #if canImport(SwiftUI) import SwiftUI import Foundation @available(iOS 13.0, macOS 10.15, tvOS 13.0, watchOS 6.0, *) public extension Color { static let banana = Color("Banana", bundle: .module) static let alexsColor = Color("Alex's Color", bundle: .module) static let reallyLongNameThatJustKeepsGoingOnAndOnAndOnAndOn = Color("really long name that just keeps going on and on and on and on", bundle: .module) static let maximumPower = Color("Maximum Power", bundle: .module) } #endif Does anyone know how to somehow instruct the bundle or the Color to load such that the comparison can be made correctly between the two colours?

My tests fail with "Restarting after unexpected exit, crash, or test timeout". This is what I have so far established: Does only happen when building with xcodebuild (14.3) but not when building with Xcode (14.3) Tests that are failing are in a .testTarget within a local swift package and part of a test plan together with tests that are in subprojects there are no crashlogs, so likely these tests are not crashing with Xcode 14.2 there are no problems whatsoever (xcodebuild or Xcode) Any ideas on how to debug this any further? Best Roddi

Issue: When I click run unit tests the tests get 'stuck' and in the top bar is says "Testing..." and never completes - I even left it for an hour. I have created new projects with no code written and the same thing happens. I have deleted and re-downloaded code as well. Current Xcode: 14.3 I don't believe it's a project issue since I have tried multiple but rather something with Xcode. and my CPU % for Xcode shoots to 365+ when this is happening. I have a M1 Max with 64GB. Any help would be appreciated.

App launches and crashes immediately with error Details Simulator device returned an error for the requested operation. Domain: NSPOSIXErrorDomain Code: 3 Failure Reason: No such process User Info: { DVTErrorCreationDateKey = "2023-04-06 13:10:58 +0000"; IDERunOperationFailingWorker = IDELaunchiPhoneSimulatorLauncher; SimCallingSelector = "launchApplicationWithID:options:pid:error:"; } -- Application launch for 'com.abc.example' did not return a valid pid nor a launch error. Domain: NSPOSIXErrorDomain Code: 3 Failure Reason: No such process -- Analytics Event: com.apple.dt.IDERunOperationWorkerFinished : { "device_model" = "iPhone10,4"; "device_osBuild" = "16.4 (20E247)"; "device_platform" = "com.apple.platform.iphonesimulator"; "launchSession_schemeCommand" = Run; "launchSession_state" = 1; "launchSession_targetArch" = arm64; "operation_duration_ms" = 666; "operation_errorCode" = 3; "operation_errorDomain" = NSPOSIXErrorDomain; "operation_errorWorker" = IDELaunchiPhoneSimulatorLauncher; "operation_name" = IDERunOperationWorkerGroup; "param_consoleMode" = 0; "param_debugger_attachToExtensions" = 0; "param_debugger_attachToXPC" = 1; "param_debugger_type" = 3; "param_destination_isProxy" = 0; "param_destination_platform" = "com.apple.platform.iphonesimulator"; "param_diag_MainThreadChecker_stopOnIssue" = 0; "param_diag_MallocStackLogging_enableDuringAttach" = 0; "param_diag_MallocStackLogging_enableForXPC" = 1; "param_diag_allowLocationSimulation" = 1; "param_diag_checker_tpc_enable" = 1; "param_diag_gpu_frameCapture_enable" = 0; "param_diag_gpu_shaderValidation_enable" = 0; "param_diag_gpu_validation_enable" = 0; "param_diag_memoryGraphOnResourceException" = 0; "param_diag_queueDebugging_enable" = 1; "param_diag_runtimeProfile_generate" = 0; "param_diag_sanitizer_asan_enable" = 0; "param_diag_sanitizer_tsan_enable" = 0; "param_diag_sanitizer_tsan_stopOnIssue" = 0; "param_diag_sanitizer_ubsan_stopOnIssue" = 0; "param_diag_showNonLocalizedStrings" = 0; "param_diag_viewDebugging_enabled" = 1; "param_diag_viewDebugging_insertDylibOnLaunch" = 1; "param_install_style" = 0; "param_launcher_UID" = 2; "param_launcher_allowDeviceSensorReplayData" = 0; "param_launcher_kind" = 0; "param_launcher_style" = 0; "param_launcher_substyle" = 0; "param_runnable_appExtensionHostRunMode" = 0; "param_runnable_productType" = "com.apple.product-type.application"; "param_testing_launchedForTesting" = 0; "param_testing_suppressSimulatorApp" = 0; "param_testing_usingCLI" = 0; "sdk_canonicalName" = "iphonesimulator16.4"; "sdk_osVersion" = "16.4"; "sdk_variant" = iphonesimulator; } -- System Information macOS Version 13.3 (Build 22E252) Xcode 14.3 (21812) (Build 14E222b) Timestamp: 2023-04-06T09:10:58-04:00

I am working on a XCTest UI test automation and I want to add the ability to communicate with a test hub on our local network that controls external test equipment that is being used to create test conditions that are external to the IPAD. Currently when I run the test on the simulator running on a MAC mini it works fine and communicates with the external machine but if I run on target which is a 6th Get IPAD it receives "POSIXErrorCode(rawValue: 50): Network is down" I have tried to add permissions to the test bundle to allow this to work but nothing seems to fix it. I am a novice when it comes to IOS development in general so maybe I am missing something obvious. I thought that this https://developer.apple.com/forums/thread/668729 solution would fix my problem but adding the permissions didn't help. I am using Xcode 13.4.1, the IOS on the target is version 15.5.

The documentation states that Xcode Cloud comes with support for git lfs (https://developer.apple.com/documentation/xcode/source-code-management-setup) but when I run my unit tests in Xcode Cloud, which have many images which are stored in git lfs, the tests all fail because they don't have the images. How do I get Xcode Cloud to fetch git lfs images? Thanks

I'm new to Xcode cloud - working with a Mac OS app, build is working great. Now I am trying to add a Test action; the testing target builds but won't run, and the error indicates it can't find the testing bundle in the expected build output. There's also mention of a code signing error, but I have automatic code signing enabled with the same settings on test target as the app. I am only running the unit test (XCTest) scheme, not the UI tests. When I run it locally from the IDE it works fine, either selecting the test scheme explicitly or as the test step of the app scheme. I notice the XCTest target's scheme setup uses Debug builds and expects the test output to be in the Debug .app bundle, I thought perhaps that was the problem (in case only the release app bundle actually gets built in the Xcode Cloud environment). So I created a duplicate scheme and set the build targets to Release - again I can run this fine locally (after creating a release build), but it fails with the same error in Xcode cloud. I also tried changing the code signing certificate from "Development" to "Sign to run locally" to see if that made a difference, but I get the same error. (It's using my developer account Team, and "Automatically manage signing".) Can anyone relate the proper way to set up an XCTest scheme so that the tests will actually run in a Mac OS Xcode Cloud workflow? I'm using Xcode 14.0.1. Here's the full error output, with [AppName] and [TestTargetName] substituted for the actual: [AppName] (....) encountered an error (Failed to load the test bundle. If you believe this error represents a bug, please attach the result bundle at /Volumes/workspace/resultbundle.xcresult. (Underlying Error: The bundle “[TestTargetName]” couldn’t be loaded. The bundle couldn’t be loaded. Try reinstalling the bundle. dlopen(/Volumes/workspace/TestProducts/Debug/[AppName].app/Contents/PlugIns/[TestTargetName].xctest/Contents/MacOS/[TestTargetName], 0x0109): tried: '/Volumes/workspace/TestProducts/Debug/[TestTargetName]' (no such file), '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib/[TestTargetName]' (no such file), '/Volumes/workspace/TestProducts/Debug/[AppName].app/Contents/PlugIns/[TestTargetName].xctest/Contents/MacOS/[TestTargetName]' (code signature in <....> '/Volumes/workspace/TestProducts/Debug/[AppName].app/Contents/PlugIns/[TestTargetName].xctest/Contents/MacOS/[TestTargetName]' not valid for use in process: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?)))) Thanks!

Hi, I have an extensive macOS test suite that accesses a lot of files located on an external volume (several Tbytes of databases, images,...). After moving to Ventura this test suite is broken. More specifically, when trying to access .sqlite databases I receive a SQL error 23: unauthorised access. This is just the symptom of the fact that I do not seem to have the access rights on those files from the test suite. I verified that by moving such a .sqlite file to /tmp, it opens fine and the corresponding test succeeds. I tried fixing those rights by giving Full Disk Access (FDA) to Xcode, to xctest...but the problem persists. Anyone out there having the same issue that found a fix ? I should add that this test suite is associated with a framework (Swift package) and I have no Xcode project associated with it...just a plain Package.swift file. So I can't really fiddle with project level settings. They surely must exist a way to access any file from a test suite even if it is located on an external disk, etc... Thanks Matthieu

Hello, I’ve a problem with unit tests. Whenever I run the unit tests individually, they pass. However, if I run them altogether, some fail. Since the tests are based on shared data (Database), I know this may be the problem.  In order to solve this, I clear the database on setup function in each test. Yet, this doesn’t fix the issue. So I assume the unit tests work in parallel, which leads to race condition (so some fail & some pass). My questions are how unit test order works?  Can I force unit tests to run the tests sequentially?  Can I force it not to move to the next test until the current test is done? What about setup & teardown, do they always work til the end before running test functions? Thanks! 🙏

Setup I've been experimenting with migrating to Xcode Cloud for our CI but I'm struggling with getting Snapshot Testing to work. I've been involved in some discussions here... https://github.com/pointfreeco/swift-snapshot-testing/discussions/553 And I saw this site also which explains a little... https://wojciechkulik.pl/xcode/xcode-cloud-overview-and-setup But so far I've been unsuccessful in getting it to work at all. I'm using the swift-snapshot-testing library from PointFree https://github.com/pointfreeco/swift-snapshot-testing and I have overridden the assertSnapshot function to change the snapshot testing directory. The function looks like this... func snapshotDirectory( for file: StaticString, ciScriptsPathComponent: String = "ci_scripts", relativePathComponent: String = "Tests" ) -> String { var sourcePathComponents = URL(fileURLWithPath: "\(file)").pathComponents if let indexFolder = sourcePathComponents.firstIndex(of: relativePathComponent) { sourcePathComponents.insert("resources", at: indexFolder) sourcePathComponents.insert(ciScriptsPathComponent, at: indexFolder) } var pathsComponents: [String] = sourcePathComponents.dropLast() let fileUrl = URL(fileURLWithPath: "\(file)", isDirectory: false) let folderName = fileUrl.deletingPathExtension().lastPathComponent pathsComponents.append("__Snapshots__") pathsComponents.append(folderName) let directory = String(pathsComponents.joined(separator: "/").dropFirst()) return directory } public func assertSnapshot<Value, Format>( matching value: @autoclosure () throws -> Value, as snapshotting: Snapshotting<Value, Format>, named name: String? = nil, record recording: Bool = false, timeout: TimeInterval = 5, file: StaticString = #file, testName: String = #function, line: UInt = #line ) { let failure = verifySnapshot( matching: try value(), as: snapshotting, named: name, record: recording, snapshotDirectory: snapshotDirectory(for: file), timeout: timeout, file: file, testName: testName ) guard let message = failure else { return } XCTFail("\(message) snap: \(snapshotDirectoryUrl) file: \(file) ", file: file, line: line) } Essentially this takes my test file path... repoRoot/Tests/FeatureTests/FeatureTestFile.swift. And injects some path component into it so that you end up with a directory path... repoRoot/ci_scripts/resources/Tests/FeatureTests/__Snapshots__/FeatureTestFile/. And then the snapshot file will be located in that directory using the name of the test function with a suffix of .1.txt or .2.txt (etc... for each subsequent snapshot in each function). i.e. testSnapshotStuff.1.txt, testSnapshotStuff.2.txt. Problem This all works locally. And all the files are checked into GitHub. But, when I run this on Xcode Cloud it fails the tests and tells me the files are not there. Having added some logging in it is writing new snapshot files to where I am expecting them to be so it just looks like those files are not available to the Test environment. This is where I read about putting them into the ci_scripts file at the root of the repo. Which is what I've done. Files in this directory are supposed to be copied into the test environments so that they can be accessed... but it seems that they're not being. I have tried using ci_scripts/resources and ci_scripts/Artifacts but it's always the same. The files aren't there and the Xcode Cloud tests write those files there over time. I'm running out of options of what to do with this now. I just want a way that I can access these snapshot files in the test environment on Xcode Cloud. Any help would be much appreciated. Thanks

Hello everyone. There is really annoying regression in xcode 14 and 14.1 beta. We have a large modularized workspace. In workspace there is one ios app target that hosts xctestplan that contains over 50 unit test targets. When we try to run tests with cmd+u xcode immidately freezes and force quit is required. This was not an issue with xcode 13. I tried to run tests via command line with xcodebuild. This time it stucks at prepare packages stage before building any target. After about 7-10 minutes it continues building targets. So I am guessing there is something wrong with resolving target dependencies in xcodebuild. Anyway this is super frustrating for us and it holds us back to moving xcode 14. Is there other people experiencing the same issue with xcode 14? Did you find any workaround?

Hi everybody, given the following case using SwiftUI: Button("testButton", action: noop) .accessibilityRepresentation{representation: { Slider(value: $sliderData.sliderValue) }) .accessibilityIdentifier("testSlider") I'm trying to control the slider using XCUITest via slider.adjust(toNormalizedSliderPosition: 0.2) and receive the following error message: Failed: Unable to get expected attributes for slider, found { "XC_kAXXCAttributeMaxScrubberPosition" = { X = 0; Y = 0; }; "XC_kAXXCAttributeMinScrubberPosition" = { X = 0; Y = 0; }; Similarly, print (slider.normalizedSliderPosition) leads to the error message "Failed to determine current position of slider "0 %". Is there a way to set and read the value of accessibilityRepresentation-Sliders with XCUITest? Thanks!

Hi, I'm currently researching if we are able to migrate to Xcode Cloud in our project. I encountered one issue. I run unit tests on Xcode Cloud and I see that they finish in ~150 seconds, but the whole step takes ~530s and there is no information in logs why. I even run the same command on my local computer and it doesn't take that long. Any ideas? Any way to optimize it? Also one more thing: it looks like Xcode Cloud is not running on M1 machines. Do you know guys if there are plans for Xcode Cloud to take advantage of M1/M2?

On an intel mac, my XCUITest test cases which is included swipe actions works perfectly but it doesn't work on my m1 mac. I tried running simulator with Rosetta but it doesnt work out it still gives an error when I run my cases because it clicks instead of swipe. Does anyone have a solution for swiping problem on the m1?

We are having an issue when trying to view the the coveage report in our "merged result bundle" Running xccov view --report --only-targets merged.xcresult results in: 09:52:54 Error: Error Domain=XCCovErrorDomain Code=0 "Failed to load coverage archive in scheme action '(null)' in result bundle" UserInfo={NSLocalizedDescription=Failed to load coverage archive in scheme action '(null)' in result bundle, NSUnderlyingError=0x7f8ff8714f30 {Error Domain=NSCocoaErrorDomain Code=260 "The file “Metadata.plist” couldn’t be opened because there is no such file." UserInfo={NSFilePath=/tmp/action.xccovarchive/Metadata.plist, NSUnderlyingError=0x7f8ff8714930 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}}} Checking the man page for xccov I was inspired by the idea of extracting the coverage report from the result bundle to try to understand or work around our error. The man xccov page states that If xccov is passed a result bundle directly, it will extract the report or archive implicitly, as part of the action being requested. Alternatively, the raw report/archive files can be extracted from the result bundle using xcresulttool(1) and subsequently passed to xccov. Checking man xcresulttool and trying out some stuff randomly I was not able to learn how a person might: Given an xcresult bundle use xcresulttool to create/extract an xccovreport bundle.

We got the newly issue that our Test devices keeps us asking for the pin code to "Enable UI Automation". Then it works for some hours or days, but after some time it starts again. "Enable UI Automation" is already enabled in "Settings" - "Developer" menu. The devices are located remotely and we can't access them directly, so this is a big issue for us right now. Is there any way to avoid this?

I have a SwiftUI menu Menu{ .... }, label : { Image(...).accessibility(identifier: "cardMenu") } I used to be able to bring up the menu (before upgrading to xcode 13 (ios15)) like this let app = XCUIApplication() app.launch() app.buttons["cardMenu"].tap() But now i am unable to see the identifier in app.buttons. Can't seem to find the identifier anymore. I've tried looking for the identifier in the other app fields and changing to use text instead of identifer. No luck. These tests used to work prior to the upgrade. Any help would be appreciated

Test UI and unit tests are working fine on 14.3 with XCode 13.0 beta4 but it gets stuck on 15.0 and doesn't execute any test case with no error.

We are building an iOS app that connects to a device using Bluetooth. To test unhappy flow scenarios for this app, we'd like to power cycle the device we are connecting to by using an IoT power switch that connects to the local network using WiFi (a Shelly Plug-S). In my test code on iOS13, I was able to do a local HTTP call to the IP address of the power switch and trigger a power cycle using its REST interface. In iOS 14 this is no longer possible, probably due to new restrictions regarding local network usage without permissions (see: https://developer.apple.com/videos/play/wwdc2020/10110 ). When running the test and trying a local network call to the power switch in iOS14, I get the following error: Task <D206B326-1820-43CA-A54C-5B470B4F1A79>.<2> finished with error [-1009] Error Domain=NSURLErrorDomain Code=-1009 "The internet connection appears to be offline." UserInfo={_kCFStreamErrorCodeKey=50, NSUnderlyingError=0x2833f34b0 {Error Domain=kCFErrorDomainCFNetwork Code=-1009 "(null)" UserInfo={_kCFStreamErrorCodeKey=50, _kCFStreamErrorDomainKey=1}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <D206B326-1820-43CA-A54C-5B470B4F1A79>.<2>, _NSURLErrorRelatedURLSessionTaskErrorKey=("LocalDataTask <D206B326-1820-43CA-A54C-5B470B4F1A79>.<2>"), NSLocalizedDescription=The internet connection appears to be offline., NSErrorFailingURLStringKey=http://192.168.22.57/relay/0?turn=on, NSErrorFailingURLKey=http://192.168.22.57/relay/0?turn=on, _kCFStreamErrorDomainKey=1} An external network call (to google.com) works just fine in the test. I have tried fixing this by adding the following entries to the Info.plist of my UI test target: <key>NSLocalNetworkUsageDescription</key> <string>Local network access is needed for tests</string> <key>NSBonjourServices</key> <array> <string>_http._tcp</string> </array> <key>NSAppTransportSecurity</key> <dict> <key>NSAllowsArbitraryLoads</key> <true/> </dict> However, this has no effect. I have also tried adding these entries to the Info.plist of my app target to see if that makes a difference, but it doesn't. I'd also rather not add these entries to my app's Info.plist, because the app does not need local network access. Only the test does. Does anyone know how to enable local network access during an iOS UI test in iOS14?

Sometimes when running UI tests, the tests fail on keyboard input because the "Speed up your typing" keyboard tutorial overlay obscures the keys. This happens randomly, but sometimes repeatedly. Is there a way to turn this off? Checking for this tutorial before any keyboard input would significantly complicate tests just to check for one edge case...