Swift Testing

RSS for tag

Swift Testing is a framework with expressive and intuitive APIs that make testing your Swift code a breeze.

Posts under Swift Testing tag

32 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

Requesting permission for MusicKit in Xcode Cloud
I am experimenting with Swift Testing and Xcode Cloud and would like to write some tests that require to use MusicKit functionality. For example I'd like to fetch an album via MusicCatalogRessourceRequest to test an initializer of another struct. However this test fails because the permission to access the music library is not granted. Once the permission is granted, the test works as expected. Things I have tried: Add NSPrivacyAccessedAPITypes to the Info.plist. This did not show any effect. Below is the corresponding snippet Trying to tap the button programmatically. Once again this did not show any effect. The Info.plist snippet: <key>NSPrivacyAccessedAPITypes</key> <array> <string>NSPrivacyAccessedAPIMediaLibrary</string> </array> The code snippet to tap the button: let systemAlerts = XCUIApplication(bundleIdentifier: "com.apple.springboard") let allowButton = systemAlerts.buttons["Allow"] if allowButton.exists { allowButton.tap() } What am I doing wrong here? I need access to MusicKit functionalities to write meaningful tests. Thank you
0
0
109
4d
Swift Testing environment differences from regular executable
I am working on a Swift package which uses CoreAudio, and includes some tests in a testTarget which use the Testing framework, and a couple of executableTarget targets which exercise the same code. I'm using Xcode 16.2 on macOS 15.3.1. One of the things I do in the test code is create a HAL plugin, then find that plugin using the kAudioHardwarePropertyTranslateUIDToDevice. Finding the plugin that I just created always fails from within a Swift Testing test, unless I run the test which creates the plugin individually first, then separately, run the test which finds the plugin, by clicking on the little arrows next to the function names. If I put the tests in a serialized suite (so creation always happens first, then finding), running the suite always fails - it creates the plugin, but can't find it. If I run the 'find my plugin' test again manually, it is always found. If I call the same functions from a regular executable (the thing created by a "executableTarget" in my .package.swift file), the just-created plugin is always found. Is there a way to mimic the runtime environment of a regular executable in a Swift Testing target, or am I misunderstanding something? this my be related to this issue: https://github.com/swiftlang/swift/issues/76882 but I don't understand it well enough to be sure.
4
0
183
5d
Swift Test Parameterized Test
Is anyone seeing flaky results when using parameterized test with pairs of (input, result) data? I have several (5) tests for a given method. I create a zip sequence by zip([in1, in2, in3, in4, in5],[out1, out2, out3, out4, out5]) Sometimes, the test only runs 4 of the tests and fails to report a failure even though I deliberately place data that should cause a failure. Sometimes, even though I only select one test to run, the test explorer goes crazy into a loop and I have to clear test results to get it to stop. Following a suggestion, I disabled running tests in parallel. Xcode 16.2 / OSX 14.7 (Sonoma) / Mac mini M2 Pro
3
0
147
4d
SwiftTesting
I am using Swift Testing for TDD my project. I have a struct that contains an array of 65536 UInt8's along with other properties. I am testing methods that modify the properties of the struct including the contents of the array. In just one of my test files, any test fails (there are several in the file), the entire contents of the struct, including the array, are printed to the console. In another test file they are not. I'm don't think there are any differences to the test setup in the two files. Any idea what's going on? I'm able get the program to skip the dump to the console by copying the properties I want to test to local let constants and testing those. Thanks
1
0
140
6d
Unable to see data from a production environment
I am trying to test using Testflight and have set up a test with a user on an account I also own which is different to me developer account. The app I believe is running in production on a separate device and is working from a user point of view, however I am not able to query the data via the console. As I said I know the user id and password as tey are mine so even when I use the Act as user service it logs in but the query is empty. I'm assuming I'm not doing anything wrong its possibly an security issue that is preventing me accessing this account. My question to the group then is how do I verify the data that is being tested?
1
0
244
1w
Choosing the Right Test Scope in Swift: UI, Unit, or End-to-End?
Hello, everyone! I'm currently working on creating tests for a study project in Swift. My current task is to create a test to check if a file is saved correctly. The workflow in the app is as follows: Launch the app. Open a file within the app. Modify the file. Save it inside the app. Save it to the Files app. I need to verify if the saved file in the Files app is identical to a base file stored in the app bundle. Initially, I thought I could solve this by creating a UI test, which I've already implemented up to a certain point. The UI test successfully navigates through the workflow steps until it's time to compare the saved file with the base file. The problem is that I cannot open a saved file from the Files app in a UI test because it operates in a sandboxed environment and cannot interact with external app scopes. So, my question is: What should I do in this case? Would it be better to create a unit test specifically for testing the save function and ensure the UI test only verifies if the expected filename exists in the Files app? I would prefer an end-to-end (E2E) test that covers the entire workflow, but it seems Swift splits tests into Unit and UI test groups, making this approach less straightforward. Any suggestions or best practices would be greatly appreciated!
0
2
237
Jan ’25
Xcode Cloud unable to execute iOS tests
All tests build and run locally just fine. Xcode Cloud can build my code, but always fails to run tests with the below errors. How can I even begin to troubleshoot this? I think my setup is pretty straightforward. Just a simple test plan that execute some unit tests with Swift Testing. Run command: 'xcodebuild test-without-building -destination 'platform=iOS Simulator,id=1EB80431-1A0B-4AD8-8EA6-968EA09C3F23' -resultBundleVersion 3 -resultBundlePath /Volumes/workspace/resultbundle.xcresult -resultStreamPath /Volumes/workspace/tmp/resultBundleStream679e7ce9-a095-4be5-8dfa-4c9df982e547.json -IDEPostProgressNotifications=YES -DTDKDisableSymbolCopying=YES -test-timeouts-enabled YES -maximum-test-execution-time-allowance 1800 -hideShellScriptEnvironment -maximum-parallel-testing-workers 8 -testProductsPath /Volumes/workspace/TestProducts.xctestproducts' (6378) encountered an error (Early unexpected exit, operation never finished bootstrapping - no restart will be attempted. (Underlying Error: Test crashed with signal ill before starting test execution.)) May be related to: https://developer.apple.com/forums/thread/725660
1
0
272
Jan ’25
EXC_BAD_ACCESS crash in some Swift Testing test on a macro
Hi, I'm trying to test a view model that conforms to ObservableObject with swift testing. My test looks like @Test("When a filter is selected the 'save search' button is enabled") func testWhenAFilterIsSelectedTheSaveButtonIsEnabled() { let viewModel = GxClassSearchFormViewModel( filterSelection: .init(), currentSavedSearch: nil ) // when viewModel.filterSelection.clubs.select(hasle) // then #expect(viewModel.savedSearchState == .active) #expect(viewModel.isSavedSearchButtonDisabled) } Interestingly, this test crashes on the iOS 16.2 Simulator. I'm using Version 16.2 (16C5032a) It crashes at the macro level like: And I don't understand what's wrong really. Maybe a concurrency issue? not sure. Xcode stops the execution and the debug navigator shows and incredibly long stack trace. I add images from the top and bottom of it as it's too long to add it all Has anyone experience a similar issue? It has happened to me in other tests too, sometimes when using #required
1
0
336
Dec ’24
Swift Testing tests in enum (namespace) extensions are acting strange
Using Xcode 16.1 (16B40). When adding a test suite with tests in an extension to an enum: The tests can be compiled and executed as expected (by clicking the diamond), and e.g. "Reveal in Test Navigator" works as expected. But after the tests have been executed, there is no check or cross diamond icons in the Test Navigator, If I right click the empty diamond in the margin for one of the tests and click "Jump to report", Xcode shows an alert saying "No test found matching the identifier MyEnumNameSpace/SomeTests/example()", which is very strange. Steps to reproduce: Create a new iOS app project using SwiftUI and Swift Testing. Add a file in the app target defining an enum namespace: public enum NamespaceDefinedInSomeApp {} Replace the content of the already created …Tests.swift file with: import Testing @testable import SomeApp // This works as expected: @Suite struct SomeTests { @Test func example() async throws { #expect(true) } } // This compiles and executes but acts strange: extension NamespaceDefinedInSomeApp { @Suite struct SomeTests { @Test func example() async throws { #expect(true) } } } // This compiles and executes but acts strange: enum AnotherNamespace {} extension AnotherNamespace { @Suite struct SomeTests { @Test func example() async throws { #expect(true) } } }
1
1
420
Dec ’24
How to Create a Designated Keychain for Testing Purposes?
I wrote a Keychain controller that add, delete and fetch keychain items using SecItemAdd(_:_:)and related APIs with data protection keychain enabled (kSecUseDataProtectionKeychain). I am using it in a macOS Cocoa app. I am using Swift Testing to write my tests to ensure that the controller works as expected. As I understand, I should create my own keychain for testing rather than use the actual keychain in macOS. Currently, I created a separate keychain group (e.g. com.testcompany.testapp.shared) and added it to myapp.entitlements file so that the tests pass without failing because of the missing entitlement file. SecKeychainCreate(_:_:_:_:_:_:) and SecKeychainDelete(_:) API are deprecated with no alternative provided in the documentation. I noticed SecKeychain class but documentation doesn't explain much about it. How should I test my keychain controller properly so that it does not use the actual macOS keychain, which is the "production" keychain?
3
0
464
Dec ’24
Timeout Issues with Async Polling in Xcode Cloud
Hi, Our team uses Xcode Cloud to run unit tests, and since around November, we’ve been frequently experiencing timeouts with the following type of process: @Test func hogeTest() async { // do something hoge.do() // wait until done await wait(condition: { hoge.isDone }) // test result #expect(hoge.isSucceeded) } private func wait(condition: () async -> Bool, timeout: TimeInterval = 0.5, pollingInterval: TimeInterval = 0.01) async throws { let deadline = Date().addingTimeInterval(timeout) while Date() < deadline { if await condition() { return } try await Task.sleep(nanoseconds: UInt64(round(pollingInterval * TimeInterval(NSEC_PER_SEC)))) } Issue.record("timeout") } Although sleep is supposed to wait for only a few milliseconds, there are cases where it takes more than 10 seconds, leading to a timeout. What could be causing this instability in the sleep duration? Additionally, if there are other recommended ways to implement polling and waiting in Swift Testing, I would appreciate it if you could share them. A feedback report (FB15899163) has already been submitted. Best regards,
2
1
415
Nov ’24
Using resources with Swift Testing
Hey guys. I’m working on a project where I’m using the SwiftTesting framework instead of XCTest to run my unit tests. I have a file (test.png) located in my test resources folder: PackageName > Tests > PackageNameTests > Resources > test.png I’m trying to access this file in my tests, but I’m running into issues when trying to load it dynamically. Here’s what I’ve tried so far: Using Bundle.module.path(forResource:ofType:): This approach didn’t work, as Bundle.module seems to be unsupported or returns nil in Swift Testing. Using #file Macro for Relative Paths: I tried constructing a path based on #file and navigating to the resources directory, but it also didn’t yield the correct path. Has anyone successfully loaded test resources in the Swift Testing framework? Is there a recommended way to access resource files in Swift Testing, especially for projects where Bundle.module isn’t available? I've gone through the Apple Docs for Swift Testing, but I can't seem to find anything that answers my question. Thanks in advance guys!
4
0
763
Dec ’24
Run identical UI/Unit tests on different build targets
I have different versions of my iOS App (written in SwiftUI). The app on the store, the App Clip, and one or two next version apps not yet released (e.g. A/B comparison). All good. But now I've started creating UI and Unit tests and I'm confused about how to get this working. Each build target has its own scheme. And in that scheme I have a Test plan for that target. E.g. The App Clip scheme has an App Clip test plan. Since all the app variants are very similar, I only have one set of unit tests and one set of UI tests so each test plan includes the same unit test target and the UI test target. Problem: When I selected a scheme (e.g. for the App Clip) and ran the tests, it turned out that all the tests ran for another build target, not the target of the scheme. I think this might be because within the definition of the test target there's a field specifying the host application. I.e. the build target. Question: How can I set up my project so that the test plan uses the relevant target build? Or do I have to duplicate all the test targets (one for each target)? Or do I have to manually change each test target before running it for a particular build target?
1
0
398
Nov ’24
Testing the content of a `Task` in a non-async method
Hi, Considering this method I'd like to test: public func play(_ soundFileName: String, shouldLoop: Bool) { Task { await dataSource.play(soundFileName, shouldLoop: shouldLoop) } } Previously, with XCTest we could use an expectation and wait for it to be fulfilled: func test() sut.play("", shouldLoop: false) wait(for: [mockedAudioPlayerDataSource.invokedPlayExpectation]) XCTAssertEqual(mockedAudioPlayerDataSource.invokedPlayCount, 1) With Swift Testing, I am unsure what a unit test looks like.
4
0
633
Oct ’24
Seriously? I can't deploy my demo to iPad cus swift 6?
Okay everyone who click into or just pass by this post ,to cut a farrrirly long story short ,here is the thing... I was devloping my app in the playground on my Mac , which was building for the iPad, In the early time of real machine commissioning(through the Xcode ),everything is okay everything is okay.. "well,mabe it's time to deploy my app to the playground in ipad"! Then,to top it all off🤣,then playground tell me (In case that you can't understand chinese and creat ambiguity ) emm that's too tight...,so I can only got back to the Xcode changing my language version to swift 5 then..... Noting change .... and I think err.. in order to properly deploy app to iPad ,I had to rework my code with swift 5! So, I almost fell on my knees ,can somebody who in charge of the playground ,can somebody who in charge of the playground ,just ,just! shove swift 6 in the playground (as long as it make my code run anyway), otherwise there's no way to debug it! please please plase!
1
2
661
Oct ’24
Unit Test Execution in Xcode Cloud Too Slow Since Xcode 16 Update
Hello Developers, I regularly use Xcode Cloud to run unit tests, but since the release of Xcode 16, the testing process has become significantly slower. I am seeking advice on how to improve the situation. [ Details ] With Xcode 15.4, unit tests took about 1 hour to complete. However, since upgrading to Xcode 16, the tests have not finished even after 2 hours. The blog post linked below also mentions significant overhead in running unit tests on Xcode Cloud, even with Xcode 15.4: https://dwango.github.io/articles/2024-07_nicoiphone_xcode_cloud/#%E3%83%86%E3%82%B9%E3%83%88%E5%AE%9F%E8%A1%8C%E6%99%82%E9%96%93%E3%81%8C%E5%A2%97%E5%8A%A0%E3%81%97%E3%81%A6%E3%81%97%E3%81%BE%E3%81%86%E5%95%8F%E9%A1%8C According to the blog, while reviewing the xcodebuild-test-without-building.log, it was observed that the time from start to finish was about 4 minutes. However, the overall “Run xcodebuild test-without-building” process took 23 minutes, with around 20 minutes of apparent idle time. It seems that the idle time has increased even further with Xcode 16, resulting in more than twice the testing time. [ Question/Request ] Is there any way to reduce the overhead when running unit tests on Xcode Cloud, especially with the increased overhead in Xcode 16? Any advice or solutions for improving the performance would be greatly appreciated. Best regards,
2
2
802
Oct ’24