Incredible huge .xcresult is generated by Xcode 13.2.1 in case of failure

Question

We use .xcresult files on our CI and recently noticed that size of file significantly increased. So previously we had files ~ 50 MB but now they are ~ 300 MB.

After investigation of .xcresult contents we found that it contains macOS Logs - test-session-systemlogs-2022.01.05_13-13-07-+0000

In testmanagerd .log we see in a last line: Moved logarchive from /var/tmp/test-session-systemlogs-2022.01.05_13-13-07-+0000.logarchive to ../Staging/1_Test/Diagnostics/My Mac_4203018E-580F-C1B5-9525-B745CECA79EB/test-session-systemlogs-2022.01.05_13-13-07-+0000.logarchive

Issue is reproducible on a new project. Steps:

File -> New -> Project
Select macOS -> App
Enable include test
For simplicity disable(comment) all the UITests except ResultSampleUITests::testExample
Add XCTAssertTrue(false) after app.launch()
Run tests from console and specify path to resultBundlePath

xcodebuild -project ResultSample.xcodeproj -scheme ResultSample -resultBundlePath ~/Downloads/2/all-tests -test-timeouts-enabled YES test
7. When tests are finished navigate to resultBundlePath and check size of all-tests.xcresult.

It has size 51.6 MB for me.

If change XCTAssertTrue(false) to XCTAssertTrue(true) then size of all-tests.xcresult is decreased to 31 KB.

Any idea how to disable/delete macOS diagnostic logs or decrease a size of .xcresult

Xcode
XCTest

3.8k

Posted by

myurik2

Reply

Add a Comment

Answer 1

You can stop XCTest from collecting test diagnostics when it encounters a failure or error in 1 of 2 ways:

From within the test plan. You can set the Collect test diagnostics on failure setting to Never
Or from the Command-line, with the flag -collect-test-diagnostics never

The command line setting overrides the test plan setting, so opposing values have a deterministic behavior. For example, if you set the test plan to Always but the commandline to Never, then test diagnostics are not collected.

Posted by

Developer Tools Engineer

Neat!

—
eskimo
For those reading along at home, this seems to be new in Xcode 14.

—
eskimo
Confusing. 1) no help docs from xcodebuild. 2)

xcodebuild test -project MyProject.xcodeproj -scheme ConfigTests -destination platform='iOS Simulator,name=iPhone 14,OS=16.1' -collect-test-diagnostics never

xcodebuild: error: option -collect-test-diagnostics requires one of two values: on-failure or never

xcodebuild test -project MyProject.xcodeproj -scheme ConfigTests -destination platform='iOS Simulator,name=iPhone 14,OS=16.1' -collect-test-diagnostics never Xcode14.1Workaround=YES

—
benblechman

Answer 2

Opened Feedback Assistance - FB9835135

Posted by

myurik2

Add a Comment

Answer 3

Opened Feedback Assistance - FB9835135

Thanks for that.

Any idea how to disable/delete macOS diagnostic logs

This looks like it’s capturing a copy of the system log. You can clear that log using:

% sudo log erase

I don’t know if there’s a way to disable log archive collection within XCTest. It seems like this is something that should be configurable in your test plan, but I can’t see any obvious knobs to twiddle there.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by

eskimo

Add a Comment

Answer 4

@eskimo, thank you for reply.

I tried sudo log erase but it decreases a size of xcresult on about 20%. XCResult still has a ton of system logs attached.

Unfortunately, I haven't found options where to disable logs in test plan or test target. Also xcodebuild -help doesn't have options to disable log.

I see following in diagnostic logs:

13:18:54.350 xcodebuild[1380:26502] Initiating control session with IDE capabilities: <XCTCapabilities: 0x7fa80b480e70>: {
}
13:18:54.351 xcodebuild[1380:26502] XPC connection for control was invalidated.
13:18:54.363 xcodebuild[1380:26502] Got reply to control session initiation request (result:error): <XCTCapabilities: 0x7fa80b2ad320>: {
    "authorize test session capability" = 1;
    "initiate daemon control session capability" = 4;
    "initiate test runner session capability" = 2;
    "report crashes capability" = 1;
    "request log archive capability" = 1;
    "request spindump capability" = 2;
}: (null)

I tried to dig what's XCTCapabilities but didn't find enough info. Looks like it's some private API. I wonder is any way to configure these capabilities. request log archive capability looks very promising.

Posted by

myurik2

Add a Comment

Answer 5

it decreases a size of xcresult on about 20%.

So, it’s helps, but not enough.

I wonder is any way to configure these capabilities.

I don’t know. I had a cursory look at this last week and couldn’t find anything that’d be obviously helpful. The next step is to open a DTS tech support incident and see if our tool specialist can help you out.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by

eskimo

Add a Comment

Answer 6

I've encountered the same issue, it severely impacts my ability to store the test results for introspection.

As the temporary measure I'm no longer keeping .xcresult bundle, but processing it with https://github.com/XCTestHTMLReport/XCTestHTMLReport into an HTML file.

Hoping for a solution coming from tool specialists or an ability to customise logs archiving behaviour in next Xcode version.

Posted by

siejkowski_proton

Add a Comment

Answer 7

I don’t know. I had a cursory look at this last week and couldn’t find anything that’d be obviously helpful. The next step is to open a > DTS tech support incident and see if our tool specialist can help you out.

Not sure If I want to submit DTS since I have only one ticket left. Will it be credited back for this report?

As the temporary measure I'm no longer keeping .xcresult bundle, but processing it with https://github.com/XCTestHTMLReport/XCTestHTMLReport into an HTML file.

Good choice. Our workaround: save critical data outside .xcresult and delete .xcresult if it greater than 10MB

Posted by

myurik2

@myurik2 You can try allure reports for a better test summary. https://github.com/eroshenkoam/xcresults

—
AbhishekBedi1432

Add a Comment

Answer 8

Will it be credited back for this report?

Standard DTS policy is that we bill an incident if the DTS engineer spends a significant amount of time on it. Seeing as that’s the whole point of this exercise, it’s likely you’ll be billed.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by

eskimo

Add a Comment

Answer 9

Wondering if you were able to solve this as we're facing a similar behavior of .xcresult files getting huge. Weird thing on our side is that we have several projects in our workspace and only one is affected.

Posted by

benrudhart

I suppose the affected project is the only one running on macOS. All others running in a simulator? I see the issue with failing unit tests on a macOS framework. The xcresult bundle is only a few kilobytes on success but is at least 60 megabytes when there was a failure. The reason is definitely the test-session-systemlogs-*.logarchive file in the result bundle. It contains data from all running processes of the machine.

—
pi80223

Add a Comment

Answer 10

You can stop XCTest from collecting test diagnostics when it encounters a failure or error in 1 of 2 ways:

From within the test plan. You can set the Collect test diagnostics on failure setting to Never
Or from the Command-line, with the flag -collect-test-diagnostics never

The command line setting overrides the test plan setting, so opposing values have a deterministic behavior. For example, if you set the test plan to Always but the commandline to Never, then test diagnostics are not collected.

Posted by

Developer Tools Engineer

Neat!

—
eskimo
For those reading along at home, this seems to be new in Xcode 14.

—
eskimo
Confusing. 1) no help docs from xcodebuild. 2)

xcodebuild test -project MyProject.xcodeproj -scheme ConfigTests -destination platform='iOS Simulator,name=iPhone 14,OS=16.1' -collect-test-diagnostics never

xcodebuild: error: option -collect-test-diagnostics requires one of two values: on-failure or never

xcodebuild test -project MyProject.xcodeproj -scheme ConfigTests -destination platform='iOS Simulator,name=iPhone 14,OS=16.1' -collect-test-diagnostics never Xcode14.1Workaround=YES

—
benblechman

Incredible huge .xcresult is generated by Xcode 13.2.1 in case of failure

Accepted Reply

Replies