Detect issues like logic failures, UI problems, and performance regressions by running tests on your app.

Posts under Testing tag

110 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

Xcode test coverage for Swift APIs that don't have explicit body
Hi all, I'm looking for a way to get test coverage for the Swift APIs that don't have explicit implementations (e.g., properties, enum cases, type names, protocols, etc). Specifically, to understand whether they are used during execution or not. It looks like Xcode 14 skips all mentioned Swift constructs in the coverage report both in Xcode UI and in the .xccov file produced from .xcresult. Is there any way to get it?
0
0
293
Oct ’23
XCode 14.3.1 - M1 runner - UI tests doesn't create clones of simulators
I'm having problems with UI tests on an M1 runner using Xcode 14.3.1. In my project, I need to build using the Rosetta simulator because I have a dependency that is not updated to support the arm simulator. This dependency relies on OpenCV." This is the command that I use when I want to run UI tests on CI: xcodebuild ARCHS=x86_64 clean test -scheme UITests -configuration Debug -destination 'id=50E9C593-9A40-4E4D-B58D-1DE437CFBE3A' -parallel-testing-enabled YES -parallel-testing-worker-count 3 -maximum-concurrent-test-simulator-destinations 4 -test-iterations 3 -retry-tests-on-failure And this code run only with 1 clone of simulator. Anyone know why?
2
0
1.8k
Oct ’23
Need to register devices of external testflight users?
Based on Apple, you can only register up to 100 devices. And you can only remove devices at the beginning of a membership renewal cycle. However in testflight, you can create up to 10,000 external testers. AFAIK, you need to register your device in order to create the provisioning profile for you to be able to install the app from testflight on your device. So question is, do external testers need to have their devices registered, to be able to install the app on their phone? Or is it not necessary to register their devices? If they need to register their devices, then what good would it do to be able to create 10,000 external testers since only 100 of them can install the app?
0
1
413
Oct ’23
How to test against non-iOS17 SDKs with Xcode15 series command line tools?
Hello. I would like to perform UnitTest using the xcodebuild command in Xcode15 series, but it does not work. Specifically, I want to run the test on an iPhone 14 with iOS 16.4 simulator environment, but for some reason it fails due to a certificate mismatch. For some reason I don't want to test on iOS17, so I specify iOS16.4. The following is a pseudo-command to run the test. Thank you! xcodebuild -workspace AppWorkspace.xcworkspace -scheme App -sdk iphonesimulator -destination "platform=iOS Simulator,name=iPhone 14,OS=16.4" -configuration TestConfiguration test
1
0
533
Oct ’23
[UI Testing] Verify Local Notifications When App Is Terminated
When my app is terminated, a warning is immediately presented to the users. I want to write a UI test for this behavior. I now have an XCUI application, I know I can terminate the app using terminate()from XCTest. After the XCUI application is terminated, a local notification should show up. Is there any way to access the notification and verify its existence and content?
1
0
439
Sep ’23
XCUIElementTypeToolbar disappears from the page source
I have a problem with test automation of my company app. We're using an external tool called Appium to automate our app, but under the hood it's using XCTest to obtain page source, and it looks like it might not work correctly, or we're doing something wrong. The problem is that since iOS 16.4 XCUIElementTypeToolbar is completely disappearing from the page source XCTest provides to Appium. What's even stranger, it's working when the app is opened for the first time after the phone restart. But when the app is simply put to background and opened again, this toolbar is no longer recognizable and instead of having XCUIElementTypeToolbar with arrows up and down and "Done" button inside it, XCTest returns only XCUIElementTypeOther there with nothing inside. And the only thing to have it found again is to restart the phone. It's happening in WebView. We are aware that since iOS 16.4 inspectable for WebViews is turned off by default (link to docs), but we have it implemented, and it did not help. It was working perfectly correct on iOS 16.3 and below, but completely stopped working on iOS 16.4 and above. Do you have any ideas if it might be anything done incorrectly on our side, or it's maybe some kind of bug in XCTest?
1
0
267
Sep ’23
Sandbox testing
In order to conduct sandbox in-app purchase testing, a testuser needs to be added to Apple Connect. This user cannot be me. It has to be someone who is NOT associated with ANY other developer account. Otherwise you'll get an error while adding test-user. So, I tried to create a new iCloud user. I confirm the email, verify phone number with an SMS, and then within 10 minutes, I'm locked out from this new account out because the "verification fails". Is Apple's logic that, in order to test an in-app purchase in sandbox, one needs to purchase a new device and create a new iCloud account under another person's name and also make a new SIM-card so the phone number would be fresh?
0
0
306
Sep ’23
SwiftUI Laggy on a Real Device
I have recently started Beta Testing my SwiftUI App on my iPhone 6s, previously I was testing it on the iPhone 12 Simulator. While Testing on the Real Device I encountered issues like: CPU going past 98% Really Laggy ScrollView Tab Bar Freezes Randomly Since My app is going to go in Production State in a few days, Since this is my first iOS App as well, I am not too sure if these issues are because of SwiftUI, my Phone or what, I have asked Apple Support but, haven't received a response so far. My App Stats: Lines of Code: ~1,500 CPU Usage on Simulator: Max. 10% Bundle Size: 2MB RAM Max. on Simulator: 32.3mb
2
0
4.3k
Sep ’23
Preboot simulators in parallel testing
We run our UI tests in parallel on 3 simulators, and I noticed, that the first one starts executing tests way faster then the rest (because others are still booting, I guess). Also we have our backend server running locally which is used in UI tests. And here comes the problem - I noticed that tests on first simulator are often failing because of the problems with local backend(requests take much more time than usual) exactly in period when other simulators are still booting(tests stop failing when all 3 simulators started running tests). When I tried running tests without parallel simulators problems with backend disappear. So my guess is that simulators booting they take some resources(CPU, memory) which interferes with backend. So my question is there some way to start executing tests only when all parallel simulators are booted?
0
0
364
Sep ’23
The wrong BLE channel calculation of iphone13 pro
I am using iphone13 pro to connect with nxp kw45(one BLE SOC chip) demo board as master, but found the KW45 side report 0x3E error(CONNECTION TIMEOUT), I record the log with Ellysis, which shows the BLE channel calculation is not right which lead to the error. The hop increment is 8, channel map is 1-3, 6-33, 36, in Frame 45093, the channel number should be 9, not 8. connection interval is 30ms. My question is it normal, any advice? Note: This problem is sporadic, not easy to reproduce.
0
0
492
Sep ’23
Headless Game Center Testing
Hello, We have an API that wraps around Game Center API and we would like to have automated tests for our API. The test would need to run in a remote data center, so there is no way to perform manual test steps. Ideally, we would want to simulate sign in and sign out, test achievements and game saves without the need for any manual input. Running in the simulator should be enough for our use case, though we could also use real devices if necessary. Is there any way to accomplish that?
0
0
427
Sep ’23
How can I effectively utilize XCUITest, , to conduct UI tests for iOS applications on the pCloudy platform?
In the context of iOS development, I've been leveraging XCUITest for UI automation. Given its native support from Apple, it's been instrumental in ensuring our app's UI consistency. Now, I'm looking to scale our testing efforts. How can I integrate XCUITest with the pCloudy platform to run these tests, and are there any specific considerations or best practices I should be aware of when doing so?
0
0
327
Aug ’23
Xcode cloud access source code after test execution
Hello, I'm trying to upload my code coverage to an external service. I've created a workflow on Xcode Cloud that build and run my test + a ci_post_xcodebuild script that uploads my Project.xcresult to an external service My problem here is that I need the repository source code the moment I upload my coverage file. I have two steps in my workflow: Build tests (Which contain the sources but not the coverage results) Run tests (Which contain the coverage result but not the sources) I found this in the Apple documentation: Lastly, it should be noted that in a test action, multiple environments are used to build and run your tests. Only the environment that is used for building your tests will have your source code cloned into it by default. The environments that run your tests won't have source code cloned into them. They'll only have the ci_scripts folder made available on them. As a result, the post-clone script won't run in these environments and your custom scripts and any of their dependencies, such as other shell scripts and small tools, must be entirely contained within the ci_scripts folder. Source: https://developer.apple.com/videos/play/wwdc2021/10269/ I feel a bit stuck here maybe Xcode cloud is not the way to go for this but I was wondering if by any way I could still check out my project sources after the test execution.
2
1
2.0k
Aug ’23