That's probably a bad title, let's try with specifics: we have a network extension, it has some classes / functions of its own, and they, when push comes to build, depend on (for example) NEAppProxyFlow and its subclasses. The code is written in Swift, since it is the language of the future.
If I want to do a unit test for my code, I need to provide something that at least looks like NEAppProxyFlow, since I can't otherwise create one. I thought I could provide my own NetworkExtension module for test case, but that... did not work well, and I still don't understand why.
On the other hand, I'm really bad at making unit tests, so the odds that I'm missing something fairly obvious to most other people are pretty high.
XCTest
RSS for tagCreate and run unit tests, performance tests, and UI tests for your Xcode project using XCTest.
Posts under XCTest tag
160 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
Apparently UI tests are unable to tap menu buttons but can tap regular buttons inside forms. Earlier today I was able to see in the Simulator that the UI test tries to tap the button by tapping the center of the containing form row, which works for regular buttons, but not for menu buttons. In fact, when trying in the SwiftUI preview in Xcode it seems that menu buttons have to be tapped exactly on top of them, while regular buttons can be tapped anywhere in the form row. (Now I’m not able to see touches performed by the UI test anymore in the Simulator for an unknown reason, even though I have “Show single touches” enabled in the Simulator settings.)
How can I open a menu button in a UI test?
The UI code:
struct ContentView: View {
@State private var label1 = "Menu 1"
@State private var label2 = "Menu 2"
var body: some View {
NavigationStack {
Form {
LabeledContent("Menu 1") {
Button(label1) {
label1 = "Menu 1 tapped"
}
.accessibilityIdentifier("menu1")
}
LabeledContent("Menu 2") {
Menu(label2) {
Button("Button") {
}
.accessibilityIdentifier("button")
}
.accessibilityIdentifier("menu2")
}
}
}
}
}
#Preview {
ContentView()
}
And the test:
final class problemUITests: XCTestCase {
func testExample() throws {
// UI tests must launch the application that they test.
let app = XCUIApplication()
app.launch()
app.collectionViews.element(boundBy: 0).buttons["menu1"].tap()
app.collectionViews.element(boundBy: 0).buttons["menu2"].tap()
app.collectionViews.element(boundBy: 0).buttons["button"].tap()
}
}
I have a few UI Tests in my app, and some of them access the tab bar, for instance: XCUIApplication().tabBars.element.buttons.element(boundBy: tabBarButtonCount - 1).tap()
In Xcode 16 though, when performing tests on iPad, this fails, likely due to the new tab bar. How am I supposed to perform this kind of action in iPadOS 18?
We are building an iOS app that connects to a device using Bluetooth. To test unhappy flow scenarios for this app, we'd like to power cycle the device we are connecting to by using an IoT power switch that connects to the local network using WiFi (a Shelly Plug-S).
In my test code on iOS13, I was able to do a local HTTP call to the IP address of the power switch and trigger a power cycle using its REST interface. In iOS 14 this is no longer possible, probably due to new restrictions regarding local network usage without permissions (see: https://developer.apple.com/videos/play/wwdc2020/10110 ).
When running the test and trying a local network call to the power switch in iOS14, I get the following error:
Task <D206B326-1820-43CA-A54C-5B470B4F1A79>.<2> finished with error [-1009] Error Domain=NSURLErrorDomain Code=-1009 "The internet connection appears to be offline." UserInfo={_kCFStreamErrorCodeKey=50, NSUnderlyingError=0x2833f34b0 {Error Domain=kCFErrorDomainCFNetwork Code=-1009 "(null)" UserInfo={_kCFStreamErrorCodeKey=50, _kCFStreamErrorDomainKey=1}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <D206B326-1820-43CA-A54C-5B470B4F1A79>.<2>, _NSURLErrorRelatedURLSessionTaskErrorKey=("LocalDataTask <D206B326-1820-43CA-A54C-5B470B4F1A79>.<2>"), NSLocalizedDescription=The internet connection appears to be offline., NSErrorFailingURLStringKey=http://192.168.22.57/relay/0?turn=on, NSErrorFailingURLKey=http://192.168.22.57/relay/0?turn=on, _kCFStreamErrorDomainKey=1}
An external network call (to google.com) works just fine in the test.
I have tried fixing this by adding the following entries to the Info.plist of my UI test target:
<key>NSLocalNetworkUsageDescription</key>
<string>Local network access is needed for tests</string>
<key>NSBonjourServices</key>
<array>
<string>_http._tcp</string>
</array>
<key>NSAppTransportSecurity</key>
<dict>
<key>NSAllowsArbitraryLoads</key>
<true/>
</dict>
However, this has no effect.
I have also tried adding these entries to the Info.plist of my app target to see if that makes a difference, but it doesn't. I'd also rather not add these entries to my app's Info.plist, because the app does not need local network access. Only the test does.
Does anyone know how to enable local network access during an iOS UI test in iOS14?
We are using the XCUITest Framework in one of our Apple TV Application. We have created test plans and using Jenkins for CI-CD. Once the test execution is completed, then it gets stuck and it is not stopping/completing the execution by itself.
Hello,
I am developing Java test code to test a web application on an Android and iPhone platforms using Selenium and Appium inside Eclipse IDE. Android device could be tested just fine. When it comes to instantiation of iOSDriver Object for running test script on an iPhone, the run time is complaining that it needs Xcode libraries, or Xcode tool on the class path. This makes it necessary to use a mac system for coding and install Xcode on it.
Once I use a mac and develop the test code within my Eclipse, can I deploy the code on a Windows Server? Does the code would not need Xcode during run time or just the Selenium-java client libraries and Appium alone?
Thanks a lot,
-- Prasad Nutalapati
I’m writing UI tests for biometrics in iOS. For a failing case, I encounter an alert that needs to be dismissed by the user. How can I tap a button in this alert?
XCTest has the addUIInterruptionMonitor method (https://developer.apple.com/documentation/xctest/xctestcase/1496273-adduiinterruptionmonitor). Is this what I need? If so, what description should I use? Or do I need some other functionality?
For Touch ID, I can simply simulate a failure twice, and the alert will be dismissed. However, for Face ID and Optic ID, this approach is not suitable—the alert buttons need to be tapped. Additionally, there is an old method to handle the alert using the Springboard app (with the bundle ID “com.apple.springboard”). However, this only works for iOS and iPadOS, while VisionOS does not have this bundle ID.
I want to automate tests for my iOS app and start writing UITests.
Sometimes system alerts appear, and my tests have to simulate button tapping.
In iOS and iPadOS these alerts are available via the system Springboard application:
let springboard = XCUIApplication(bundleIdentifier: "com.apple.springboard")
let cancelButton = springboard.alerts.buttons["Cancel"].firstMatch
if cancelButton.exists {
cancelButton.tap() // <-- cancel button taps, and test continue working
}
But when I launch my test in the Vision Pro simulator, the springboard is not available:
let springboard = XCUIApplication(bundleIdentifier: "com.apple.springboard")
print(springboard.exists) // <-- "false" will be printed, springboard does not exist
It means that I can't automate button tapping in system alerts.
So, my question is:
How can I access system alerts in VisionOS and tap buttons by UITests?
We have multiple Jenkins jobs running UI Tests on a Mac with a device attached.
This works most of the time but sometimes the tests don't run and the xcreport generated has an error like:
BCOVBRDCoverageUITests-Runner encountered an error in BCOVBRDCoverageUITests failed with:
Failed to install or launch the test runner. (Underlying Error: Failed to create directory on device 'Brightcove's iPhone 14 Plus' (00008110-000C54912246401E) to hold runtime profiles for application with bundle ID 'com.brightcove.BCOVBRDCoverageUITests.xctrunner': (null). (Underlying Error: The system failed to get the path on the remote device for the provided domain. (Underlying Error: The connection was interrupted.)))
Has anyone else run into this? Any ideas on how to get around it? Kicking off another job after seeing this behavior works fine. The failures seem to be random or we haven't noticed a pattern yet.
I am able to reproduce this also in a minimal example.
Here is my setup:
a Hello World app and a UI test target.
a Test Plan which executes the UI test target
test plan configures application language as German for sanity testing whether changing the test configuration via the test plan in general works -> this works perfectly
test plan configures launch arguments and Environment variables in the shared settings "Arguments passed on launch"
In the ContentView of the app I try to access these values via ProcessInfo:
print(ProcessInfo.processInfo.arguments)
print(ProcessInfo.processInfo.environment)
Now I run the test plan
Expected:
my custom launch arguments appear in arguments
my custom env vars appear in environment
Actual: none of them appear.
What am I doing wrong here?
Hello,
I have a test bundle in my application and one of the strong request of our project is to run ui tests from command line having just application build.
How to run unit tests on .app file that has a test target?
What we tried
xcrun devicectl device install app --device <device-identifier> <UITestBundle-Runner.app>
xcrun devicectl device process launch --device <device-identifier> --start-stopped <com.apptestbundle-Runner.xcrun>
It launches and is waiting pid to be attached.
In lldb:
device select <device-identifier>
device process list
Seeing process 'UITestBundle-Runner' pid.
device process --pid 'pid id'
It attaches and I can see 'debugserver' in the pid list, but it does not start testing and I need tests to start and run.
Thanks!
We currently try to run tests on 4 physically attached iPhones all on ios 17. When tests are concurrently running we occasionally get failures that look like this:
Testing failed:
Failed to install the app on the device.
SwiftUITests-Runner encountered an error (Failed to install or launch the test runner. (Underlying Error: Failed to install the app on the device. (Underlying Error: Connection with the remote side was unexpectedly closed : { count = 1, transaction: 0, voucher = 0x0, contents =
"XPCErrorDescription" => { length = 22, contents = "Connection interrupted" }
}. Unhandled error domain IXRemoteErrorDomain, code 6))
I noticed that when relying on xcodebuild to install the target app and test runner it cannot install concurrently/parallel on all 4 devices but instead does each device sequentially causing the test to hang. The same behavior can be exhibited when trying to install an app concurrently on 4 different phones with xcrun devicectl as well.
STEPS TO REPRODUCE
Trigger a test with xcodebuild on 4 different phones at the same time where xcodebuild also needs to install the target apps.
Currently using Xcode15.2 and iOS 17.5.1
Unable to see the first element in a Xamarin forms List view while running the automated UI iOS tests using xctest
Element with property in Xamarin forms "ListViewCachingStrategy.RetainElement" , the first element is missing from the list view while running the tests in iOS .
Inbetween the tests if we remove the WDA and tried the same scenario, it works as expected.
The first element was displayed when working with xamarin forms (v4.7.0.1351) and it is observed with only the current version.
Manually trying the same scenario does not have any issues , Elements are displayed as expected, issue is observed only through automation
Unable to record and play the new contacts permission system dialog.
App: https://developer.apple.com/documentation/contacts/accessing-a-person-s-contact-data-using-contacts-and-contactsui
func handleContactsAccessAlert() {
let springboard = XCUIApplication(bundleIdentifier: "com.apple.springboard")
let allowButton = springboard.buttons["Allow Full Access"]
allowButton.tap()
let access6ContactsButton = springboard.alerts["Allow full access to 6 contacts?"].scrollViews.otherElements.buttons["Allow"]
access6ContactsButton.tap()
}
func handleContactsPermissionAlert() {
let springboard = XCUIApplication(bundleIdentifier: "com.apple.springboard")
let allowButton = springboard.buttons["Continue"]
if allowButton.exists {
allowButton.tap()
sleep(5)
handleContactsAccessAlert()
}
}
func testContactPermissions() {
let app = XCUIApplication()
app.launch()
app.buttons["Request Access"].tap()
sleep(5)
handleContactsPermissionAlert()
sleep(5)
app.collectionViews/*@START_MENU_TOKEN@*/.staticTexts["Kate Bell"]/*[[".cells.staticTexts[\"Kate Bell\"]",".staticTexts[\"Kate Bell\"]"],[[[-1,1],[-1,0]]],[0]]@END_MENU_TOKEN@*/.tap()
}
This issue started to appear in the last couple of weeks. And it's annoying because Xcode Cloud actually is based on build duration. Here is a sample:
The whole build took like 10 minutes.
We can easily consume the 25 hours free tier because of these. The crash logs are completely useless. And the failing tests change from one run to the other and on different simulators. I really hope Xcode Cloud team look into this as it's frustrating and kills the whole point of Xcode Cloud (BTW, these issues won't happen on Bitrise).
I'm trying to measure app launch time in an existing Xcode project. I added a simple test that's supposed to display a "no baseline" message and a button to set a baseline, but when I run the test, it just succeeds without showing the pop-up.
It works in a new project, so I suspect there might be a specific setting in my existing project that's causing the issue. The project is about 6 years old.
Does anyone know if there's a particular setting I need to enable for launch tests to display the baseline pop-up in older projects?
I'm running the test from new project in Xcode 14.2.
Existing project in Xcode 15.2 and 14.2 both succeed with no baseline message.
Thank you!
I've written a Swift package plugin to add the colours provided in the asset bundles in an extension of the SwiftUI Color type but I'm having difficulties testing this since the bundle and/or colour that is in the asset catalogs are not yet loaded when XCTest goes to evaluate the test conditions. Here is my test code: (colours are just random colours, some derive from macOS colour picker; also, the new static method is functionality that I added and is not present in the existing Color APIs)
final class TestCase: XCTestCase {
func testBanana() {
XCTAssertEqual(Color.banana, Color.new(from: "FEFC78")!)
}
func testAlexsColor() {
XCTAssertEqual(Color.alexsColor, Color.new(from: "0432FF")!)
}
func testMaximumPowerColor() {
XCTAssertEqual(Color.maximumPower, Color.new(from: "FF2600")!)
}
func testLongNameColor() {
XCTAssertEqual(Color.reallyLongNameThatJustKeepsGoingOnAndOnAndOnAndOn, Color.new(from: "AAA000"))
}
}
Here is an example error message that I get when I run the test:
testBanana(): XCTAssertEqual failed: ("NamedColor(name: "Banana", bundle: Optional(NSBundle </Users/trevorhafner/Library/Developer/Xcode/DerivedData/TransportBase-cbagdabrompfzofwkimswvlsincu/Build/Products/Debug/TransportBaseTests.xctest/Contents/Resources/TransportBase_TransportBaseTests.bundle> (not yet loaded)))") is not equal to ("#FEFC78FF")
Here is the autogenerated code that my Swift Package Plugin creates:
#if canImport(SwiftUI)
import SwiftUI
import Foundation
@available(iOS 13.0, macOS 10.15, tvOS 13.0, watchOS 6.0, *)
public extension Color {
static let banana = Color("Banana", bundle: .module)
static let alexsColor = Color("Alex's Color", bundle: .module)
static let reallyLongNameThatJustKeepsGoingOnAndOnAndOnAndOn = Color("really long name that just keeps going on and on and on and on", bundle: .module)
static let maximumPower = Color("Maximum Power", bundle: .module)
}
#endif
Does anyone know how to somehow instruct the bundle or the Color to load such that the comparison can be made correctly between the two colours?
I have some (very old ObjC) XCUITests, in Xcode 15.2. I recently noticed programmatic screenshots are not saving. The screenshots generated automatically by XCUITest are saving fine. This is code that used to work as expected; I do not know when it broke as I haven't had need to run this in a while.
-(void)takeScreenshotWithName:(NSString *)name {
XCUIScreenshot *screenshot = XCUIScreen.mainScreen.screenshot;
XCTAttachment *attachment = [XCTAttachment attachmentWithScreenshot:screenshot];
attachment.lifetime = XCTAttachmentLifetimeKeepAlways;
attachment.name = name;
[self addAttachment:attachment];
UIImage *image = screenshot.image;
UIImageWriteToSavedPhotosAlbum(image, nil, nil, nil);
}
During testing I added the UIImageWriteToSavedPhotosAlbum call, to verify the code is running and the image is available - and it does. But I really don't want these images cluttering up my library.
The results are the same whether I run my tests from within Xcode, or via the command line.
I need these screenshots so I have a set in PNG format, with standardized naming, for archiving, searching, and comparisons. The automatic screenshots are in Xcode's format - useful when reviewing a single test, which isn't my use case here.
The documentation states that Xcode Cloud comes with support for git lfs (https://developer.apple.com/documentation/xcode/source-code-management-setup) but when I run my unit tests in Xcode Cloud, which have many images which are stored in git lfs, the tests all fail because they don't have the images.
How do I get Xcode Cloud to fetch git lfs images?
Thanks
The connection to service named com.apple.testmanagerd was invalidated: failed at lookup with error 3 - No such process.
Currently I am struggling with this problem while running tests runner
`xcrun devicectl device process launch --console --device '000000000000000000000' 'com.gl.RunnerPOC-Watch-AppUITests.xctrunner
What may be causing the issue?
Thanks in advance