Parallelization by Test Target

Question

Created Feb ’23

Replies 2

Boosts 0

Views 3.5k

Participants 2

I'm trying to use Apple's tools to speed up testing by running tests in parallel. One way to do this is to enable "Execute in parallel (if possible)" from the Test Plan UI.

Unfortunately, when I try that, some number of our tests end up failing intermittently.

It's likely this has something to do with how our test code is set up, but it seems to be related to how the parallelization is optimized by pulling test classes out of test targets. For our test suite, we found that as long as the test targets are not broken up, the tests run fine.

So what we'd like to be able to do is to run the test targets in parallel while keeping the classes and tests in each target bundled together during test execution.

That is, suppose we have 3 test targets in a Test Plan:

Target1
Target2
Target3

Each test target has some number of test classes and tests.

What I'd like to be able to do is to use Apple's Parallelization tools to execute these tests in parallel, but while keeping all the test classes together.

So, for example, if we ran this configuration with 2 parallel shards, it might look like:

Shard1: Target1, Target2
Shard2: Target3

I haven't been able to find a way to accomplish this. Is there a known solution or workaround?

Boost

Answer 1

Developer Tools Engineer OP

Apple

Feb ’23

Parallelization runs the code within each target in parallel. If you include all of the code you want to run synchronously in one individual test method, it can be run in parallel with other test methods.

Maybe there is a way to reorganize your code to consolidate into a few synchronous test methods that can be run in parallel, if you would like to not split up your code into smaller fragments?

0

Answer 2

evapierce OP

Feb ’23

Thanks for the response!

Our test targets have thousands of test methods in each of them. It's important for us to be able to record results of each test method as separate units. Grouping them together into a small number of mega-test-methods would obfuscate that information, and make it far more challenging to debug issues and maintain the test suites. I think that approach will be a non-starter for us and most other engineering teams.

It sounds like it is not currently possible to run some test targets synchronously while running others in parallel using Apple's tooling. Is there any chance this kind of functionality is on the roadmap for a future release? If not, I'd like to suggest it as a feature request.

2