Unexpected performance penalty attributed to first Swift Testing test

I have a Swift package with a test suite that contains some tests implemented with Swift Testing. Locally, they run quickly, but when I run them on Semaphore CI (https://semaphore.io), the first Swift Testing test to execute incurs a performance penalty.

I'm running the tests with xcodebuild on iOS Simulator:

xcodebuild test \
-scheme "Scheme" \
-workspace Workspace.xcworkspace \
-destination "platform=iOS Simulator,name=iPhone 16,OS=18.2"

The scheme is configured to use a test plan that has parallelization disabled.

Here's an excerpt from the output showing what I'm seeing:

Test Suite 'All tests' started at 2025-04-03 07:47:37.328.
◇ Test run started.
↳ Testing Library Version: 102 (arm64-apple-ios13.0-simulator)
◇ Iteration 1 started.
◇ Suite <redacted> started.
Test foo() started.
✔ Test foo() passed after 23.063 seconds.

When foo() is not the first test it runs in under 100 ms.

The reason that I have parallelization disabled is that I was initially seeing all of the tests in this suite incur a performance hit. But now it's clear that there must be some startup cost.

Things I'm wondering:

  1. What is this startup penalty?
  2. Why don't I encounter it locally?
  3. Why is it attributed to the first test? (this seems like a bug)

My wild guesses around 1 so far have been…

  • maybe some simulator clone is booting. I've tried to rule that out by disabling parallelization, but maybe there's still something there.
  • maybe swift testing is getting loaded lazily and there's some kind of dynamic linking cost

Thoughts on 2…

  • maybe there's some one-time penalty when using swift testing that I've already incurred locally but that has not yet been incurred in the CI image

Guidance welcome!

x-posted:

Answered by DTS Engineer in 833166022

It looks like you’re connected with the right folks via your Swift Forums thread.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Accepted Answer

It looks like you’re connected with the right folks via your Swift Forums thread.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Unexpected performance penalty attributed to first Swift Testing test
 
 
Q