Detect issues like logic failures, UI problems, and performance regressions by running tests on your app.

Posts under Testing tag

111 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

Test Plans: application data for macOS apps
In the test plan settings, it's possible to select an .xcappdata bundle for a set of tests: I failed to find any documentation about it in the context of macOS apps testing. What I'm trying to achieve is having a sandboxed app container (/Users/{user}/Library/Containers/{bundle-id)/Data/) replaced before running a test suite. Is it something possible to achieve using the latest Xcode? What should be the .xcappdata structure to make it work on macOS?
0
0
102
5d
Testing a planned price increase for an existing subscription
Hello, I have a few questions about increasing a price increase for an existing subscription on an iOS application: 1.) Is it possible to test the price increase in the sandbox environment? If so do I have to plan the scheduled increase first or can I create this scenario in the sandbox environment without any scheduling? I ask because id like to test the new price update in some of the local code first without having the risk of a user getting notified that a price increase is coming before the app is ready to display it properly. 2.) When a price increase is scheduled say two weeks out, but then its determined that we need to cancel it by deleting it from app store connect. Are users that were potentially notified of the price increase via the consent form notified a second time that it has been cancelled? Thanks for any help!
1
0
148
1w
Suggestion to Add Performance Metrics for SwiftUI in XCTest
As SwiftUI adoption grows, developers face challenges in effectively measuring and optimizing SwiftUI app performance within automated tests. Currently, the only viable approach to analyzing SwiftUI performance is through Profiling (Instruments), which, while powerful, lacks the ability to be incorporated into automated testing workflows. It would be incredibly valuable if XCTest could introduce new performance metrics specifically tailored for SwiftUI, allowing us to write tests against common performance bottlenecks. Suggested Metrics: View Body Evaluation Count – Tracks how often a SwiftUI body is recomputed to detect unnecessary re-renders. Slow View Bodies – Flags SwiftUI views that take a significant amount of time to compute their body. These metrics would help developers identify inefficiencies early, enforce performance best practices through CI/CD pipelines, and ensure a smooth user experience. I believe adding these performance metrics would be a game-changer for SwiftUI development. Thank you for your time and consideration!
1
0
212
2w
Shared dependencies between test and production code creates library duplication
When test support code relies on production code, a diamond can occur. If this occurs across packages, it can lead to duplicated symbols and incorrect behavior at runtime despite no warnings at build time. This only occurs in Xcode and top-level application testing. This doesn't occur when the code being tested is in a separate package. I'm trying to understand how to correctly manage shared test support code which needs to access production code, or if this is the correct way and it is an Xcode bug. For a minimized example project, see https://github.com/rnapier/SupportCode. The setup includes three packages: Dependencies, which manages all the dependencies for the app and tests; Core which contains core logic and test support; and Feature, which relies on Core an contains feature-related logic and test support. Building this system causes Core.framework to show up three times in DerivedData: ./App.app/PlugIns/AppTests.xctest/Frameworks/Core_59974D35D_PackageProduct.framework ./App.app/Frameworks/Core_59974D35D_PackageProduct.framework ./PackageFrameworks/Core_59974D35D_PackageProduct.framework When unit tests are run, there is a collision on the symbol for Keychain: objc[48914]: Class _TtC8Keychain8Keychain is implemented in both /Users/ornapier/Library/Developer/Xcode/DerivedData/App-grdjljgevqofhqgflgtrqvhvbtej/Build/Products/Debug-iphonesimulator/PackageFrameworks/Core_59974D35D_PackageProduct.framework/Core_59974D35D_PackageProduct (0x100a98118) and /Users/ornapier/Library/Developer/CoreSimulator/Devices/216C441E-4AE5-45EC-8E52-FA42D8562365/data/Containers/Bundle/Application/7197F2F2-EB26-42FF-B7DB-67116159897D/App.app/PlugIns/AppTests.xctest/AppTests (0x1011002c0). One of the two will be used. Which one is undefined. This is not a benign warning. There are two distinct copies of _TtC8Keychain8Keychain and test cases will access one and the app will access a different one. This leads to mismatches when accessing static instances such as .shared. I believe this dependency graph should work, and it does work as long as the top-level testing system is a Swift module. But if it is the application, it builds successfully, but behaves incorrectly in subtle ways.
0
4
307
2w
B2B Beta Testing with Managed Apple IDs - TestFlight Incompatible - Seeking Solutions
I'm seeking advice on distributing a beta iOS app to a business partner for internal testing, but we're encountering a challenge due to their organization's use of Managed Apple IDs. Problem We are developing an iOS app for B2B use and need to provide our partner organization with a beta version for testing. However, this partner organization strictly utilizes Apple Managed IDs for all their employees. Their devices are configured in a way that employees cannot use personal Apple IDs on their company devices. This effectively prevents the use of TestFlight, as TestFlight requires personal Apple IDs for testers. We understand TestFlight is the standard recommendation for beta distribution, but it seems incompatible with organizations heavily invested in Managed Apple IDs and their associated security policies. Question What are the recommended alternative methods for distributing our iOS beta app to this organization for testing, considering these constraints? Are there any official Apple-recommended solutions for B2B beta testing in this scenario when TestFlight is not viable due to Managed Apple ID restrictions? Any insights, suggestions, or alternative approaches would be greatly appreciated. We want to ensure we're following best practices and respecting the partner organization's IT policies while still enabling them to effectively test our application. Thank you in advance
0
1
205
2w
Cannot launch simulated executable: no file found at /Applications/Warenwirtschaft.app
Hallo zusammen, ich habe seit einiger Zeit ein Problem, dass ich eine App 'Warenwirtschaft' aus Xcode heraus nicht mehr im Simulator oder auf meinem IPad starten kann. Ein Compilieren und Verteilen über TestFlight funktioniert weiterhin ohne Probleme. Leider klappt das Debugger nun nicht mehr. Anbei die Fehlermeldung von Xcode. Ich hoffe ihr könnt mir helfen. Grüsse Mark Cannot launch simulated executable: no file found at /Applications/Warenwirtschaft.app Domain: IDEFoundationErrorDomain Code: 1 User Info: { DVTErrorCreationDateKey = "2025-02-05 10:10:06 +0000"; IDERunOperationFailingWorker = IDELaunchiPhoneSimulatorLauncher; } Event Metadata: com.apple.dt.IDERunOperationWorkerFinished : { "device_identifier" = "D572897F-2ECB-4109-9DFC-C7D0FB145C1C"; "device_model" = "iPad16,6"; "device_osBuild" = "18.2 (22C150)"; "device_platform" = "com.apple.platform.iphonesimulator"; "device_thinningType" = "iPad16,6-A"; "dvt_coredevice_version" = "397.28"; "dvt_coresimulator_version" = "993.7"; "dvt_mobiledevice_version" = "1759.81.1"; "launchSession_schemeCommand" = Run; "launchSession_state" = 1; "launchSession_targetArch" = arm64; "operation_duration_ms" = 562; "operation_errorCode" = 1; "operation_errorDomain" = IDEFoundationErrorDomain; "operation_errorWorker" = IDELaunchiPhoneSimulatorLauncher; "operation_name" = IDERunOperationWorkerGroup; "param_debugger_attachToExtensions" = 0; "param_debugger_attachToXPC" = 1; "param_debugger_type" = 3; "param_destination_isProxy" = 0; "param_destination_platform" = "com.apple.platform.iphonesimulator"; "param_diag_113575882_enable" = 0; "param_diag_MainThreadChecker_stopOnIssue" = 0; "param_diag_MallocStackLogging_enableDuringAttach" = 0; "param_diag_MallocStackLogging_enableForXPC" = 1; "param_diag_allowLocationSimulation" = 1; "param_diag_checker_tpc_enable" = 1; "param_diag_gpu_frameCapture_enable" = 0; "param_diag_gpu_shaderValidation_enable" = 0; "param_diag_gpu_validation_enable" = 0; "param_diag_guardMalloc_enable" = 0; "param_diag_memoryGraphOnResourceException" = 0; "param_diag_mtc_enable" = 1; "param_diag_queueDebugging_enable" = 1; "param_diag_runtimeProfile_generate" = 0; "param_diag_sanitizer_asan_enable" = 0; "param_diag_sanitizer_tsan_enable" = 0; "param_diag_sanitizer_tsan_stopOnIssue" = 0; "param_diag_sanitizer_ubsan_enable" = 0; "param_diag_sanitizer_ubsan_stopOnIssue" = 0; "param_diag_showNonLocalizedStrings" = 0; "param_diag_viewDebugging_enabled" = 1; "param_diag_viewDebugging_insertDylibOnLaunch" = 1; "param_install_style" = 2; "param_launcher_UID" = 2; "param_launcher_allowDeviceSensorReplayData" = 0; "param_launcher_kind" = 0; "param_launcher_style" = 0; "param_launcher_substyle" = 0; "param_runnable_appExtensionHostRunMode" = 0; "param_runnable_productType" = "com.apple.product-type.application"; "param_structuredConsoleMode" = 1; "param_testing_launchedForTesting" = 0; "param_testing_suppressSimulatorApp" = 0; "param_testing_usingCLI" = 0; "sdk_canonicalName" = "iphonesimulator18.2"; "sdk_osVersion" = "18.2"; "sdk_variant" = iphonesimulator; } System Information macOS Version 15.3 (Build 24D60) Xcode 16.2 (23507) (Build 16C5032a) Timestamp: 2025-02-05T11:10:06+01:00
3
0
203
1w
Unit tests and persistent tokens
I'd like to implement unit tests that exercise keys made available via a persistent token interface. However, when attempting to list available tokens by passing kSecAttrAccessGroupToken as the kSecAttrAccessGroup to SecItemCopyMatching from a unit test, -34018 is returned. It succeeds without the kSecAttrAccessGroup, which makes sense given the unit test binary does not have com.apple.token Keychain Group. The Xcode UI indicates "Capabilities are not supported" for the unit test binary when attempting to add a Keychain Sharing capability to enable use of persistent tokens. This feels like a dead end but begs the question is there any way to implement unit tests to exercise a persistent token interface? It seems like the only path may be write unit tests that drive an independent app that handles the interactions with the persistent token.
1
0
203
2w
FinanceKit Mock Data
Hello, I'm building an expense management app and have the necessary FinanceKit entitlements. However I'm based in India and hence do not have access to an Apple Card. Is there anyway to test FinanceKit with some sort of mock data? I have tried following the developer documentation and built a minimal implementation to share via Testflight to my users. However it's failing to get any transaction data. I'm unable to debug the code myself and if anyone here has valid entitlements along with Apple Card, I'd appreciate if you could debug an example project I made below: https://github.com/tanmays/FinanceKitExample Feedback #FB14136552
0
0
190
3w
MFi - ATS iAP2 Session Test Crash
We just updated our ATS to the latest 8.3.0 version and tried to run the iAP2 Session Test via BPA100 Bluetooth Analyzer and we are experiencing this EXC_BAD_INSTRUCTION. This same test still seems to work on ATS version 6. Please advise. Process: ATS [1782] Path: /private/var/folders/*/ATS.app/Contents/MacOS/ATS Identifier: com.apple.ATSMacApp Version: 8.3.0 (1826) Build Info: ATSMacApp-1826000000000000~2 (1A613) Code Type: X86-64 (Native) Parent Process: launchd [1] User ID: 501 Date/Time: 2025-01-27 11:05:21.1334 -0800 OS Version: macOS 15.2 (24C101) Report Version: 12 Bridge OS Version: 9.2 (22P2093) Anonymous UUID: 098E2BB5-CB98-CA1C-CEFE-188AF6EFE8CF Time Awake Since Boot: 9700 seconds System Integrity Protection: enabled Crashed Thread: 2 com.apple.ATSMacApp.FrontlineFrameworkInterface Exception Type: EXC_BAD_INSTRUCTION (SIGILL) Exception Codes: 0x0000000000000001, 0x0000000000000000 Termination Reason: Namespace SIGNAL, Code 4 Illegal instruction: 4 Terminating Process: exc handler [1782]
0
0
224
4w
Apple Pay Test Environment - The Right Way
Hi team, We were wondering what's the correct way of configuring a test environment with Apple Pay. Not sure if this is explicitly mentioned in the documentation, but in order to avoid having the same certificates shared between test and production, should we have a different merchant identifier (and pair of certificates) for test purposes only? The above is the main question. However, two follow up questions: Do you know if payment processors usually allow the merchant ID to be configured, so that only payments generated with the prod certificates can be accepted? Is there any risk of someone getting hold of the certificates generated for the test environment (which are usually less safe than production) and using that to process payments in production?
2
0
249
2w
Choosing the Right Test Scope in Swift: UI, Unit, or End-to-End?
Hello, everyone! I'm currently working on creating tests for a study project in Swift. My current task is to create a test to check if a file is saved correctly. The workflow in the app is as follows: Launch the app. Open a file within the app. Modify the file. Save it inside the app. Save it to the Files app. I need to verify if the saved file in the Files app is identical to a base file stored in the app bundle. Initially, I thought I could solve this by creating a UI test, which I've already implemented up to a certain point. The UI test successfully navigates through the workflow steps until it's time to compare the saved file with the base file. The problem is that I cannot open a saved file from the Files app in a UI test because it operates in a sandboxed environment and cannot interact with external app scopes. So, my question is: What should I do in this case? Would it be better to create a unit test specifically for testing the save function and ensure the UI test only verifies if the expected filename exists in the Files app? I would prefer an end-to-end (E2E) test that covers the entire workflow, but it seems Swift splits tests into Unit and UI test groups, making this approach less straightforward. Any suggestions or best practices would be greatly appreciated!
0
2
237
Jan ’25
Identifying SwiftUI list cells in XCUITest
In my macOS app I have a SwiftUI list that starts like this: List(selection: $selection) { HStack { Label("Staging", systemImage: "arrow.up.square") Spacer() WorkspaceStatusBadge(unstagedCount: model.statusCounts.unstaged, stagedCount: model.statusCounts.staged) } (where WorkspaceStatusBadge is a custom view that just contains a Text) I'm trying to set the accessibility ID of that first cell so I can find it in XCUITest. If I apply the accessibilityIdentifier() modifier to the HStack, it instead sets the ID of the two static text elements inside it, and the cell still has no ID. I could find the cell based on the ID of the child staticText, but I have some other cases where this doesn't work as well. If I use .accessibilityElement() on the HStack, then XCUI sees a cell containing a Group element with the ID. This might be workable, but it's certainly not ideal. So how do I set the ID of the cell itself?
3
0
235
Jan ’25
Testflight app doesn't load on iPads with ios 17, 16, 15
Hi, Testflight app is showing "Couldn't Load Apps. TestFlight is currently unavailable. Try again." error each time when i trying to open it last 2 days. I'm tried to uninstall/re-install TestFlight app, but error still remain. Can anyone confirm this error? Devices which is affected with the same error: iPad 6 (2018) / iOS 17.7, iPad Pro / iOS 16.7.9, iPad Air 2 / iOS 15.8.2
5
7
789
Jan ’25
Xcode Cloud unable to execute iOS tests
All tests build and run locally just fine. Xcode Cloud can build my code, but always fails to run tests with the below errors. How can I even begin to troubleshoot this? I think my setup is pretty straightforward. Just a simple test plan that execute some unit tests with Swift Testing. Run command: 'xcodebuild test-without-building -destination 'platform=iOS Simulator,id=1EB80431-1A0B-4AD8-8EA6-968EA09C3F23' -resultBundleVersion 3 -resultBundlePath /Volumes/workspace/resultbundle.xcresult -resultStreamPath /Volumes/workspace/tmp/resultBundleStream679e7ce9-a095-4be5-8dfa-4c9df982e547.json -IDEPostProgressNotifications=YES -DTDKDisableSymbolCopying=YES -test-timeouts-enabled YES -maximum-test-execution-time-allowance 1800 -hideShellScriptEnvironment -maximum-parallel-testing-workers 8 -testProductsPath /Volumes/workspace/TestProducts.xctestproducts' (6378) encountered an error (Early unexpected exit, operation never finished bootstrapping - no restart will be attempted. (Underlying Error: Test crashed with signal ill before starting test execution.)) May be related to: https://developer.apple.com/forums/thread/725660
1
0
272
Jan ’25
Development and testing on different machines
Hi developers, I'm searching for a kind of way of working to develop my apps on a different machine than testing and final building. For development I have a MacBook Pro m4 and for testing I want to outsource this to a Mac mini m1. I was searching for a solution and also contacted the support, but the answer wasn't really helpful. Any ideas how to setup this configuration to automate this kind of tests? Thanks a lot!
0
0
174
Jan ’25
Sandbox/Wallet/Apple Pay hacker
So this is a long shot. My apple device has been hacked. In March, my credit card, which is in apple pay, started to get used, until May. It was used for thousands of dollars. In July i discovered installed and generally “invisible” files in the files app. I’ve called All IPhone that was a contact file. when i started deleting everything from my phone, all messages and photos and accounts, it became obvious i was signed into the app store in a sandbox account i never signed up for or into. I believe this is one method by device as compromised through apple. does anyone have any ideas on if it would be possible for a hacker to use my apple pay through some scam developer who had initiated a testing agreement between myself and them?
1
0
266
Jan ’25