Detect issues like logic failures, UI problems, and performance regressions by running tests on your app.

Posts under Testing tag

124 Posts
Sort by:
Post marked as solved
1 Replies
180 Views
On my M1 mac, using Xcode 13.3, I created a package and displayed the code coverage bar (Editor menu –> Code Coverage). After running tests, there is no indication of code coverage at all in the source code. How do I get code coverage when testing a package?
Posted
by
Post not yet marked as solved
0 Replies
110 Views
Hello, I am developing a hybrid app and I am looking write UI tests. My scenario basically is - Launch the app Click Login button (which will open a Web page) Enter User details on the web page and click submit Now verify native components on app for successful login. What is the tool recommended by Apple to Automate above test?
Posted
by
Post not yet marked as solved
0 Replies
238 Views
Hi Apple! Since the latest iOS update, Xcode will no longer build to either of my iPhones (and they were all working great before). I have been developing for iPhones for 14 years. This is a new problem. It occurred once I updated each of my devices to the latest iOS operating system (15.4.1). Now, Xcode always shows "Failed to prepare device for development". I've restarted my Mac many times, updated to the latest Mac OS (11.6.5), restarted both phones several times, restarted Xcode. NOTHING (I've been working on iPhones for a long time and I know all the tricks) works to build to the phone. This has completely stopped my development for my two companies. Please list the steps you took to reproduce the issue: Connect iPhone to Xcode. Push run button from any iOS app. Xcode reports "Failed to prepare device for development". What did you expect to happen? App to get transferred and run on the phone. What actually happened? Development and all progress stopped with no resolution possible from us. I've submitted screenshots of two different phones, both updated to latest iOS. JUST before the iOS updates, things were working.
Posted
by
Post not yet marked as solved
0 Replies
183 Views
Context - We have E2E Testing running on the CI on the iPhone devices. The script performs the following steps - Build the Test runner app from code. Extracts the app file from the IPA. Re-codesign the app with the required Provision Profiles. Creates a .xctestrun file Uses the xcodebuild command like the following to run the test - xcodebuild -sdk iphoneos -destination id=<device_id> -xctestrun <path_to_xctestrun_file> -resultBundlePath <path_to_test_bundle_file> test-without-building -only-testing:E2ETestsBundle.xctest/<test_filter> The above command installs the runner and target app into the device and starts executing the test. The apps get installed and uninstalled at each run during a single execution. Problem - During the execution, the xcodebuild CLI tool, suddenly loses connection to the device and is not able to install the apps into the device and then results in the following error - Testing failed: E2ETestsBundle.xctest: E2ETestsBundle-Runner encountered an error (Early unexpected exit, operation never finished bootstrapping - no restart will be attempted. (Underlying Error: Test runner exited before starting test execution. If you believe this error represents a bug, please attach the log file at /var/folders/1r/0xlnbmzd0yj3m_cc7p95yb4r0000gn/T/tmpl7d9p1s6/testResults.bundle/Staging/1_Test/Diagnostics/E2ETestsBundle-Configuration-Test Scheme Action-Iteration-1/E2ETestsBundle/Session-E2ETestsBundle-2022-04-13_205520-muaMCx.log)) ** TEST EXECUTE FAILED ** Points to note - The xcodebuild CLI tool will keep on failing with the error once it occurred until recovered. Even while the xcodebuild is in a failed state, the devices are still accessible via Xcode IDE and other CLI tools like libimobiledevice and are able to install both the test runner and target applications. This is occurring intermittently across all devices models and across subversions of iOS 14 and 15. Recovery - The only way to recover is to reboot the device. Even after a reboot, the devices start facing this exception again after a few days of execution. As we can't keep on rebooting the device every time before execution, we need a solution to fix this problem. Any help would be appreciated!!
Posted
by
Post not yet marked as solved
0 Replies
148 Views
Since upgrading to Xcode 13.3.1 unit-tests doesn't start anymore with "execute in parallel" set on. "XYZ" (1234) encountered an error (Unexpected TestCaseDidStart...)
Posted
by
Post not yet marked as solved
0 Replies
158 Views
I am getting a problem where builds in my Swift Playgrounds App project are failing -> Command CompileSwiftSources failed with nonzero exit code. When I rebuild the error goes away and it is successful, but is there a way I can get to the root of the issue and stop it?
Posted
by
Post not yet marked as solved
0 Replies
158 Views
By using the UIKit views and controls, I wanted to know if the accessibility attributes (e.g. traits, value, identifier) are automatically added to the element in the screen? Or do I need to add manually the information for each attribute? If it is done automatically, will it appear in the Accessibility Inspector when looking at each element in the screen on the simulator?
Posted
by
Post not yet marked as solved
0 Replies
211 Views
When importing an executable target for testing purposes, Xcode says the casing in the import is wrong, however it is not. The module is named Swiftly, importing it as Swiftly results in the error Cannot load module 'swiftly' as 'Swiftly'. Building it using the Swift CLI however, works perfectly fine, no issues. Also the Swift build using GitHub actions works. Xcode is the only tool that throws this error and denies to build the package. I've already tried clearing all of Xcode's cache, cloning a clean version of the package, changing the executable name to use upper case, all without any results... Is this an Xcode issue? Any suggestions for getting this solved? Also note that changing the import to lower case makes the build work in Xcode, but this then breaks builds using the Swift CLI and GitHub actions, so this is not a solution. The import: @testable import Swiftly // Cannot load module 'swiftly' as 'Swiftly' Package.swift // swift-tools-version: 5.5 import PackageDescription let package = Package(     name: "Swiftly",     platforms: [         .macOS(.v10_12),     ],     products: [         .executable(             name: "swiftly",             targets: ["Swiftly"]),     ],     dependencies: [         .package(url: "https://github.com/apple/swift-argument-parser", from: "1.0.0"),         .package(url: "https://github.com/JochenBe/Shell", from: "1.0.0")     ],     targets: [         .executableTarget(             name: "Swiftly",             dependencies: [                 .product(name: "ArgumentParser", package: "swift-argument-parser"),                 "Shell"             ]),         .testTarget(             name: "SwiftlyTests",             dependencies: ["Swiftly"]),     ] )
Posted
by
Post not yet marked as solved
1 Replies
320 Views
I am trying to change the Mac OS Apps to a new locale other than English using the following arguments in XCUITest: app.launchArguments = [  "-inUITest", "-AppleLanguages",  "(nl)",  "-AppleLocale", "nl_NL" ] But I get the following message on the console: Launching macOS application via NSWorkspace while sandboxed. Launch arguments and environment variables may be dropped The app loads in English itself. Is there any other way I can change the language of such apps ? I DONT mind changing the system language as well, but it needs to be done programatically. I tried writing a zsh script to do sudo languagesetup but it requires a restart for the system language to change, which I can't do when my automation is running. If there is any other way please let me know. EDIT: I open a third party app using the bundle Identifier, like this: XCUIApplication(bundleIdentifier: "<app-bundle-name>")
Posted
by
Post not yet marked as solved
1 Replies
319 Views
tldr: Our app is randomly crashing on startup (about 1 every 10 times) but no Crash Report alert is being shown on screen nor sent to the Crash Logs tab on TestFlight. The app just simply opens and immediately closes with no apparent reason. No logs of the crash are kept inside Settings -> Privacy -> Analytics & Improvements -> Analytics Data. On the iPhone, the options Share iPhone Analytics and Share With App Developers are checked ( Settings -> Privacy -> Analytics & Improvements). No memory leaks detected using the Instruments app. More detailed explanation: Out of 6 iPhones used for testing, only 1 iPhone is presenting the crash. It's an iPhone 11 iOS 15.4 and no one has been able to replicate the crash using the other iPhones. We have an iPhone SE (2020) and an iPhone 12 Pro Max both with iOS 15.4, but we haven't been able to replicate the bug. The steps to reproduce the bug are these: Open the app, switch views, then force-close it (swipe up the app in the App Switcher) Quickly open the app again, switch views and force-close it again. Repeat the previous step and in about the 10th iteration, the app will load and immediately close (crash) I know that this the most unlikely scenario possible, but the company won't let pass the app to production until the cause of the crash is identified. Specific questions: Any idea how can we obtain a crash log in this situation? How possible is the scenario where NOT the app, but the OS or the phone itself is the responsible for the crash? How can we prove that this is the case? Is there any scenario where an App on TestFlight crashes without showing the crash dialog and without sending a crash record to the developer (Obviously assuming that you previously agreed to do it on the Privacy settings)? How does TestFlight defines and handles crashes? Under those parameters and definitions, can this behavior be considered a crash? Any idea of what can be happening? Thank you very much.
Posted
by
Post not yet marked as solved
1 Replies
519 Views
Hi everyone, I'm having a hard time solving my issue. I have a VStack with a ForEach loop that displays (Core)data info. In that ForEach loop I have a View that has a tap gesture (to open a sheet to edit the entity) and a contextMenu (to allow users to delete the selected entity). The sheet is at the VStack (parent of ForEach) level. VStack() {         ForEach(customModels) { customModel in           VStack() {             Text(customModel.title)           }           .frame(minWidth: 0, maxWidth: .infinity)           .onTapGesture(perform: {             selectedCustomModel = customModel           })           .contextMenu {             Button {               customViewModel.deleteModel(customModel)             } label: {               Label("Delete", systemImage: "trash")             }           }         }       }       .sheet(item: $selectedCustomModel) { selectedCustomModel in         EditCustomView(customModel: selectedCustomModel)       } If I long press on one of my "child" view the contextMenu appears, if I dismiss that contextMenu by tapping on another child view, then it triggers the "onTapGesture" (which seems fair), and a warning pops in the console : Attempt to present &lt;_TtGC7SwiftUI29PresentationHostingControllerVS_7AnyView_: 0x7f90939e05b0&gt; on [...] which is already presenting &lt;_UIContextMenuActionsOnlyViewController: 0x7f9093983bd0&gt;. The problem is at that point any tap on a child view does not work, the onTapGesture is like "dead" I have to tap like 5 or 10 times to to make the edit sheet appears... Is this a bug or do you see something bad in my implementation ? Thank you Jim
Posted
by
Post not yet marked as solved
2 Replies
1k Views
Hi Everyone. I'm trying to use Accessibility Inspector thats included with Xcode 13.3 and it fails to identify UI elements with Target mode... basically the entire iOS simulator is a green/unreadable element. Besides the fact that is hangs a lot is this tool completely broken now? If I try to run an audit it just throws the warning: Element has no description This SimulatorKit.SimDisplayRenderableView is missing useful accessibility information. Note that no matter what app I try to inspect, it always fails, it also fails when I try to inspect the springboard / home screen. It worked fine in 13.2.1 and now its completely broken.
Posted
by
Post marked as solved
2 Replies
217 Views
Hello, When I try to test a new non-renewing subscription in-app purchase I get a weird prompt from iOS: "Ask Permission", "A request to buy ... will be sent to your parent or guardian" (please see screenshot below) I'm using sandbox testers that can be specified at  https://appstoreconnect.apple.com/access/testers (all testers have a birth year specified at 1980 or older, so there should be now reason for Apple to consider it a child account) I already tried to set up new and different sandbox testers with different App Store regions at https://appstoreconnect.apple.com/access/testers but nothing helps. (Only a non-renewing subscription seems to be affected. I was able to complete other (non consumable) in-app purchases without any problems.) I can't complete the purchase and can't finish my app as long as this problem exists. Does someone has a solution? (nothing at all happens when I tap "ask") Thank you so much!
Posted
by
Post not yet marked as solved
0 Replies
175 Views
Our automated tester has a suite of ui tests he's been working on. When these are enabled on our release pipeline they are: Very slow While all tests pass, when building the app afterwards the build hangs indefinitely When disabling the tests the build is successful. The build machine is a 2020 M1 with 16GB ram running Xcode 13.1, I remoted in while it was running and the CPU usage / Ram seemed fine. The CI log / Fastlane gives no error indication, it just freezes trying to link a library and I have to manually cancel the build I'm wondering if the tests have exhausted something within Xcode build itself so it gives up at a certain point, is there anything obvious I can check? Thanks
Posted
by
Post not yet marked as solved
1 Replies
378 Views
I'm trying to test an API I'm writing with Structured Concurrency. I prefer to run my tests with continueAfterFailure = false so that I may establish preconditions on my tests. At the moment my test is:     func testRssUrl() async throws {         self.continueAfterFailure = false         let xml = PodcastXml.url(url)         let feed = try await xml.feed         let rssFeed: RSSFeed? = feed.rssFeed         XCTAssertNil(rssFeed) // Included for this sample         XCTAssertNotNil(rssFeed)         validate(zebraFeed: rssFeed!)     } My expectation of Swift Concurrency is that the try await should hold the method, until xml.feed resolves to a value or throws. Either of the XCTAssertNil or XCTAssertNotNil should fail (in the actual test, I'm not using NotNil). As written, between the two asserts and the try, validate(zebraFeed: ) should never be called because of continueAfterFailure. Yet it is. The test still fails because XCTAssertNil is failing, which is my actual expectation. Test execution should also be stopping there. What am I missing?
Posted
by
Post not yet marked as solved
1 Replies
774 Views
UI Tests seem to be broken when run on an iPhone running iOS 15.4. Steps to reproduce: Create a fresh app project in Xcode 13.3 including UI tests Select an attached iPhone running iOS 15.4 Run the UI test testExample which only starts the app, nothing more. Restart the attached phone Try to run testExample again. Expectation: Test succeeds again Actual: When launching the app, XCTest is internally first trying to terminate the app and gets stuck. After 60s the tests fail. Logging (app id and home folder are removed for privacy reasons): TESTUITests-Runner[341:7025] Running tests... Test Suite 'Selected tests' started at 2022-03-17 14:53:36.855 Test Suite 'TESTUITests.xctest' started at 2022-03-17 14:53:36.855 Test Suite 'TESTUITests' started at 2022-03-17 14:53:36.856 Test Case '-[TESTUITests.TESTUITests testExample]' started. t = 0.00s Start Test at 2022-03-17 14:53:36.856 t = 0.07s Set Up t = 0.07s Open <app_id>.TEST t = 0.11s Launch <app_id>.TEST t = 0.11s Terminate <app_id>.TEST:269 <UserHome>/Workspace/TEST/TESTUITests/TESTUITests.swift:28: error: -[TESTUITests.TESTUITests testExample] : Failed to terminate <app_id>.TEST:269: Failed to terminate <app_id>.TEST:0 t = 60.29s Tear Down Test Case '-[TESTUITests.TESTUITests testExample]' failed (60.294 seconds). Test Suite 'TESTUITests' failed at 2022-03-17 14:54:37.150. Executed 1 test, with 1 failure (0 unexpected) in 60.294 (60.294) seconds Test Suite 'TESTUITests.xctest' failed at 2022-03-17 14:54:37.150. Executed 1 test, with 1 failure (0 unexpected) in 60.294 (60.295) seconds Test Suite 'Selected tests' failed at 2022-03-17 14:54:37.151. Executed 1 test, with 1 failure (0 unexpected) in 60.294 (60.296) seconds If I manually start the app once, then the UI tests will run again afterwards for a while (including restarting the app). At some point they get stuck again and we are not sure why, yet. This makes it really hard to run tests in our CI because we can't keep sending someone into the server room to manually start the app... Has anybody else encountered this and found a solution? It works fine on iOS 15.1.1.
Posted
by
Post not yet marked as solved
1 Replies
160 Views
I have been developing my App with my Developer and it was all going well. Splash Screen worked, App loaded correctly. It linked to my Backend Server. All the relevant backend content was displayed. But now, I get a TestFlight or Diawi link to install the latest Beta Build of my App. It doesn't work on my iPhone 12 Pro Max. But it does work on my developers iPhone in India. I am in the UK. He has had people test the build there. It works. I have invited people to download the TestFlight build here (the UK) and it doesnt work (no splash screen, no hamburger menu, no search screen). So my Dev and I are stumped. Screen Recordings: India Install (working) on my Devs iPhone https://youtu.be/52ZuSW1GvuQ UK Install (not working) on my iPhone - No Hamburger Menu Working, No Search Button Working, No Splash on Launch https://youtu.be/SmdyG5XapBw If anyone who reads this has any comments/feedback as I am stumped. Thank you in advance. MB
Posted
by
Post not yet marked as solved
0 Replies
141 Views
Hi, i am testing with uitest our application, but tests are too slow. I cannot find any tips or how to accelerate it. Is there any option to make it quickier? Thanks
Posted
by
Post not yet marked as solved
3 Replies
1.4k Views
When I try to deploy my application on the Apple test flight I get this email from apple. While processing your iOS app, xxxxxxx 1.0.149(1), errors occurred in the app thinning process, and your app couldn’t be thinned. If your app contains bitcode, bitcode processing may have failed. Because of these errors, this build of your app will not be able to be submitted for review or placed on the App Store. For information that may help resolve this issue, see Tech Note 2432. But the problem was I did deploy this application day before without any issues using Transporter. I spend more than 20hr searching for answers for this issue but had no luck. Can someone guide me on this issue? Thanks.
Posted
by
Post not yet marked as solved
5 Replies
541 Views
I am performing some tests that involves the app extension and an external app connected through XPC I get the following when I try to connect to an external app from the safari app extension with sandbox enabled connection to service named "com.test.sample"was invalidated: failed at lookup with error 159 - Sandbox restriction." UserInfo={NSDebugDescription=The connection to service named com.test.sample was invalidated: failed at lookup with error 159 - Sandbox restriction. I have tried: Disabling the sandbox for the app extension target Adding the entitlements file path in the test target Both of which still give me the same result. How can I proceed with this? Does the entitlement for the sandbox only work when Safari is running the appex?
Posted
by