Detect issues like logic failures, UI problems, and performance regressions by running tests on your app.

Posts under Testing tag

110 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

IOS18 Beta - Public (is it just me?)
From downloading Beta 18 (public) I have the following sever issues: Cellar Network dips and has weakened (wife & kids on same network with no issues.) Wi-Fi stability has become horrendous AirDrop is a nightmare zooming in on emails or other document documents cuts the page off and it becomes unmovable battery drains significantly quicker than ever before please tell me I’m not alone! @Apple - would be great if uninstalling beta was easier with out the risk and time the current process is annotated!
0
1
85
1h
Universal links & redirect not working on certain devices
We are currently doing our beta testing for our application and we are having some issues with universal links. The issues can be seen below: we are using auth0 for authentication. In this process, after users verify their email addresses they should be redirected back to the application. For some users, they are directed back to a page that shows error 404. For other users where it works, they are directed back to the application. What could be my issue? Our app-site- association file is hosted in the link below for reference. https://yourmomentshub.com/.well-known/apple-app-site-association
1
0
104
3d
Enable local network access during iOS UI test in iOS14
We are building an iOS app that connects to a device using Bluetooth. To test unhappy flow scenarios for this app, we'd like to power cycle the device we are connecting to by using an IoT power switch that connects to the local network using WiFi (a Shelly Plug-S). In my test code on iOS13, I was able to do a local HTTP call to the IP address of the power switch and trigger a power cycle using its REST interface. In iOS 14 this is no longer possible, probably due to new restrictions regarding local network usage without permissions (see: https://developer.apple.com/videos/play/wwdc2020/10110 ). When running the test and trying a local network call to the power switch in iOS14, I get the following error: Task <D206B326-1820-43CA-A54C-5B470B4F1A79>.<2> finished with error [-1009] Error Domain=NSURLErrorDomain Code=-1009 "The internet connection appears to be offline." UserInfo={_kCFStreamErrorCodeKey=50, NSUnderlyingError=0x2833f34b0 {Error Domain=kCFErrorDomainCFNetwork Code=-1009 "(null)" UserInfo={_kCFStreamErrorCodeKey=50, _kCFStreamErrorDomainKey=1}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <D206B326-1820-43CA-A54C-5B470B4F1A79>.<2>, _NSURLErrorRelatedURLSessionTaskErrorKey=("LocalDataTask <D206B326-1820-43CA-A54C-5B470B4F1A79>.<2>"), NSLocalizedDescription=The internet connection appears to be offline., NSErrorFailingURLStringKey=http://192.168.22.57/relay/0?turn=on, NSErrorFailingURLKey=http://192.168.22.57/relay/0?turn=on, _kCFStreamErrorDomainKey=1} An external network call (to google.com) works just fine in the test. I have tried fixing this by adding the following entries to the Info.plist of my UI test target: <key>NSLocalNetworkUsageDescription</key> <string>Local network access is needed for tests</string> <key>NSBonjourServices</key> <array> <string>_http._tcp</string> </array> <key>NSAppTransportSecurity</key> <dict> <key>NSAllowsArbitraryLoads</key> <true/> </dict> However, this has no effect. I have also tried adding these entries to the Info.plist of my app target to see if that makes a difference, but it doesn't. I'd also rather not add these entries to my app's Info.plist, because the app does not need local network access. Only the test does. Does anyone know how to enable local network access during an iOS UI test in iOS14?
11
1
11k
4d
TimeLimitTrait: minimal time limit way to huge
From the Documentation of .timeLimit(_:): Test timeouts do not support high-precision, arbitrarily short durations due to variability in testing environments. The time limit must be at least one minute, and can only be expressed in increments of one minute. Unit Tests are usually considered too long when they take more than 0.1 seconds. So a minimal time limit of on minute is completely useless for unit tests. Is there a similar trait in Swift Testing to handle millisecond time limits?
1
0
177
1w
How to access Test Plans launch arguments and environment in the app target during UI tests
I am able to reproduce this also in a minimal example. Here is my setup: a Hello World app and a UI test target. a Test Plan which executes the UI test target test plan configures application language as German for sanity testing whether changing the test configuration via the test plan in general works -> this works perfectly test plan configures launch arguments and Environment variables in the shared settings "Arguments passed on launch" In the ContentView of the app I try to access these values via ProcessInfo: print(ProcessInfo.processInfo.arguments) print(ProcessInfo.processInfo.environment) Now I run the test plan Expected: my custom launch arguments appear in arguments my custom env vars appear in environment Actual: none of them appear. What am I doing wrong here?
2
1
129
1w
Inconsistent User Reviews in Admin Panel: Same Content, Different IDs and Dates Issue
Issue: I am facing a situation where multiple reviews from some users display the same titles, body, and author names in my admin panel, but they have different review IDs and review submission dates. For example, User A might have a review titled "Great App!" with the same content and author name, but different review IDs (123 and 456) and submission dates (June 1st and June 15th). This inconsistency appears when users update their reviews in the app or when reviews are deleted and then resubmitted through the App Store. Reason: When users update their feedback or delete and resubmit their reviews in the app or through the App Store, each action creates a new review entry in our database due to the different review IDs assigned by the platform, leading to these discrepancies. Technical Details: Fetching Reviews: I fetch the latest reviews from the App Store using a scheduled task that retrieves up to 200 reviews per hour. These reviews are retrieved using the App Store Connect API: https://api.appstoreconnect.apple.com/v1/apps/{id}/customerReviews?include=response&sort=-createdDate&limit=200. Admin Panel Technology: My admin panel is developed using the Laravel framework (PHP). Our Concern: I want to ensure that our database stores only unique reviews and that our admin panel displays only unique reviews to avoid confusion and maintain data integrity. I'm seeking guidance on how to handle this scenario correctly within my admin panel to ensure accurate representation of user reviews. Any insights or best practices on how to manage and display user reviews under these circumstances would be greatly appreciated. Thank you.
0
0
175
1w
My app is aimed only at iPhone, but when I upload it to TestFlight, it tells me that it is for iPad, and I don't know how to remove it
Hello colleagues, this is the first time this has happened to me, and I don't know what to do, I have done research and I have not found a solution for this, I am compiling my application only for iPhone, and when it is up to send it for review, it tells me that it is also for iPhone/iPad, and I'm frustrated, because my application has never been uploaded for iPad. In my xcode, I put in the Target Device, iPhone only, and it uploads as if it were compatible with iPad. Did it happen to someone else?
0
0
187
2w
Failed to install the app on the device
We currently try to run tests on 4 physically attached iPhones all on ios 17. When tests are concurrently running we occasionally get failures that look like this: Testing failed: Failed to install the app on the device. SwiftUITests-Runner encountered an error (Failed to install or launch the test runner. (Underlying Error: Failed to install the app on the device. (Underlying Error: Connection with the remote side was unexpectedly closed : { count = 1, transaction: 0, voucher = 0x0, contents = "XPCErrorDescription" => { length = 22, contents = "Connection interrupted" } }. Unhandled error domain IXRemoteErrorDomain, code 6)) I noticed that when relying on xcodebuild to install the target app and test runner it cannot install concurrently/parallel on all 4 devices but instead does each device sequentially causing the test to hang. The same behavior can be exhibited when trying to install an app concurrently on 4 different phones with xcrun devicectl as well. STEPS TO REPRODUCE Trigger a test with xcodebuild on 4 different phones at the same time where xcodebuild also needs to install the target apps. Currently using Xcode15.2 and iOS 17.5.1
1
0
180
2w
Xcode Cloud SIGILL Crashes
This issue started to appear in the last couple of weeks. And it's annoying because Xcode Cloud actually is based on build duration. Here is a sample: The whole build took like 10 minutes. We can easily consume the 25 hours free tier because of these. The crash logs are completely useless. And the failing tests change from one run to the other and on different simulators. I really hope Xcode Cloud team look into this as it's frustrating and kills the whole point of Xcode Cloud (BTW, these issues won't happen on Bitrise).
6
0
334
2w
Need Apple Pay test cards for failed cases (insufficient balance, etc.) on sandbox environment
I am currently testing Apple Pay integration on my sandbox environment and I am in need of test cards for failed cases such as insufficient balance. Does anyone have access to or know where I can find Apple Pay test cards specifically for scenarios where transactions fail due to reasons like insufficient balance? Any help or guidance on this matter would be greatly appreciated. Thank you.
0
0
163
2w
Xcode Accessibility audit in UI Tests gives less than stellar results
I have added a UI test that uses the newish app.performAccessibilityAudit() on each of the screens in my SwiftUI app. I get a handful of test failures for various accessibility issues. Some of them are related to iOS issues (like the "legal" button in a map view doesn't have a big enough tappable are; and, I think, there is a contrast issue with selected tabs in a tab bar, or maybe it is a false positive). Two of the error type I get are around Text elements that use default font styles (like headline) but that apparently get clipped at times and apparently "Dynamic Type font sizes are unsupported". For some of the errors it supplies a screenshot of the whole screen along with an image of the element that is triggering the error. For these text errors it only provides the overall image. After looking at the video I saw that part of the test runs the text size all the way up and I could see that at the largest size or two the text was getting cut off despite using default settings for the frames on all the Text elements. After a bit of trial and error I managed to get the text to wrap properly even at the largest of sizes and most of the related errors went away running on an iPhone simulator. I switched to an iPad simulator and the errors re-appeared even though the video doesn't show the text getting clipped. Does anyone have any tips for actually fixing these issues? Or am I in false positive land here? Also, is there any way to get more specific info out of the test failures? Looking at the entire screen and trying to guess where the issue is kinda sucks.
0
2
371
3w
iOS 18 Beta 2 Carplay Music Problem
After the ios 18 beta 2 update, when listening to music in carplay, the sound quality is very bad and I can't turn down the volume of the music. it acts like a voice assistant instead of music. when turning the volume down and up, it's like adjusting the volume of the voice assistant when it should turn up the volume of the music. So instead of music, it acts like the voice assistant is activated.
16
12
1.2k
3w
App Rejected - Guideline 2.1 - Performance - App Completeness
I'm unable to reproduce an issue which only seems to occur on App Review devices. It's mainly stuck at a loading screen when trying to sign in with GameCenter. I'm using Microsoft Azure PlayFab as a game backend. I've tested my game on two iPads and three different iPhone models. They're all able to connect to the server. No debug build was installed on those devices via Xcode. I used the same build which was rejected, freshly installed via TestFlight on the internal test track. Looking at my server logs, I can't even find one single request sent from the App Review device. This can only mean that there is some connectivity issue between their device and the server (e.g. a firewall or proxy). I've also implemented a retry mechanism so in case the request times out, it would try it again and after three retries it would eventually display an error message and further details. How shall I figure this issue out, if the only thing the reviewers provided me is a screenshot of the loading screen?
0
0
245
4w
The connection to service named com.apple.testmanagerd was invalidated: failed at lookup with error 3 - No such process.
The connection to service named com.apple.testmanagerd was invalidated: failed at lookup with error 3 - No such process. Currently I am struggling with this problem while running tests runner `xcrun devicectl device process launch --console --device '000000000000000000000' 'com.gl.RunnerPOC-Watch-AppUITests.xctrunner What may be causing the issue? Thanks in advance
0
0
239
Jun ’24
UI Tests Runner application does not launch manually.
I need to run my tests bundle independently from Xcode. Previously, I used to deploy it and run as regular iOS application, it would execute test cases and display 'Automation running' label. I needed to execute tests some time after deploy and in different time of the day, so it is a problem for my team right now. So, UI Tests Runner app file does not launch and 'Automation Running' label is not displayed, is there any possible ways to run tests as it was or other ways to run tests not from Xcode.
0
0
186
Jun ’24
Approach to adoption of Swift Testing
Currently Swift Testing has much less features than XCTest, so the adoption will be very slow from our side. Notable features we miss are UI tests, performance tests and attachments. I did not want to create many issues in Swift Testing GitHub project as lots of these shortcoming are most probably tracked internally (I can see lots of references to radars in GitHub issues.) So, my question is: Is it a good idea to wait with wider adoption or should we experiment with other tools like swift Benchmarks?
3
0
373
Jun ’24
XCUIElement with zero frame but showing.
I have a bug I am writing a UI Test for and I am struggling to find a way to test the fixed UI vs the buggy UI. In the buggy version, the UI element is visible on the screen but has a 0,0 width and height. It's visible because it's not set to 'clip to bounds', so it's overlaying the rest of my UI. The app sets the element to be hidden in certain situations and I wanted to make sure in my test, that it's not on the screen in that situation. The problem is that .exists returns true in both situations (.hidden == true, .hidden == false), and .isHittable always returns false because its either no on the screen at all or has a 0,0 width, height. So I can't use .exists; I can't use .isHittable; and I can't use the .frame either. Is there simply a way to check if it's actually visible?
1
0
276
Jun ’24
Testflight and AppStore installation fails
I'm working on this Mac Safari extension https://apps.apple.com/app/id6464076425 The installation of the app via Testflight stops without any success (only for me) As well as the installation via AppStore. This happens only to me. I tried it on another computer, and it installed successfully. Here are some videos: This makes the development and release process impossible until I solve this problem. I tried all the methods I could find online, such as: Deleting the currently installed app Restarting Safari Restarting my Mac Is there anything else I can do to fix this problem?
0
0
296
Jun ’24
How to correctly use Accessibility Identifiers for Rows of Elements
So I have a simple view for my SwiftUI application below: import SwiftUI struct ContentView: View { var body: some View { List { ForEach(0..<4) { index in RowView(index: index) .accessibilityIdentifier("row") } } } } struct RowView: View { let index: Int var body: some View { HStack { Image(systemName: "globe") .imageScale(.large) .foregroundStyle(.tint) Text("Hello, world!") Spacer() ButtonView(index: index) } .padding() } } struct ButtonView: View { let index: Int var body: some View { VStack { if index != 3 { Button(action: {}, label: { Text("Cancel") }) .accessibilityIdentifier("cancel_button") } Button(action: {}, label: { Text("Submit") }) .accessibilityIdentifier("submit_button") } } } My goal is to reference each cancel and submit button per row for a UI Test, which I have tried doing so here: let rowElements = app.tables.cells.matching(identifier: "row") for i in 0..<rowElements.count { let cancelButton = rowElements.element(boundBy: i).buttons["cancel_button"] let submitButton = rowElements.element(boundBy: i).buttons["submit_button"] XCTAssertTrue(cancelButton.exists, "Cancel button does not exist in row \(i)") XCTAssertTrue(submitButton.exists, "Submit button does not exist in row \(i)") } Even with the identifiers set like this my tests fail and say it cannot find the buttons labeled as they are with their specific identifiers or the rows. What is the correct way to reference elements if they are in a list or table for UI Testing? I'm specifically trying to use accessibility Identifiers to make my life easy.
2
0
417
May ’24