Yes, I was taking screenshots using the hardware buttons, the sleep/wake button and volume button, to take a screenshot. As long as the app appeared to be stuck loading, the image of the screenshot would have the correct clock time in it, but when I open the exif data, it has the timestamp of the exact second when all the network requests succeed at resuming, some minutes later. Another of our developers said while his app was stuck, the screenshots did not appear in his Photos app until either the app got unstuck, or he force-quit the app. The images are also saved out of order, meaning they each have a unique sequential number in the file name, but sometimes screenshots with minute "58" in them have a later number than screenshots with minute "59" in them. Our only interaction with the photos library would be if the user goes to create new content, we would use a PHPickerViewController to select media items, but for the bugs I'm describing, the user has not gotten that far in the app. Thanks for the lead on the disc space profile, I'll give it a shot.

How I eventually alleviated the symptoms, shocker it had nothing to do with the disc: I had believed the MainActor was not getting blocked, because I was still able to use the UI and the watchdog timer was not killing the app after minutes. However, in several places we had code for a Task that would await an asynchronous fetch from the network and then await MainActor.run to send the changes to the data model for the UI. I changed all of those await MainActor.run to Task { @MainActor in. The theory was the containing Task may have been trying to keep using the network thread to run on the MainActor preventing the network thread from going back to work on other urlsesison tasks, and that by creating a sub Task, I would work around that particular issue. I also found one spot where the Task that awaited the network data was not even await MainActor.run when calling navigationController?.pushViewController, but instead was relying on the Task to implicitly capture the MainActor from the calling code, which was probably a programmer error. It also init'd a UIViewController subclass while not explicitly calling await MainActor , then pushed that onto the nav stack. So I put both the VC init call and the push explicitly into a Task { @Mainactor in. The theory being that these functions somehow required explicit redirection to the MainActor, and stalled out never getting it, blocking the network thread from continuing, but somehow not blocking other code that was explicitly running on the main actor. After making these two changes, after only a couple days of testing, people who had been having the issue frequently are no longer having an issue. So I conclude one or both of these issues prevented the task / thread associations at the system level from resuming completing work. The measurement of disc access events taking too long must also have been a result of task / thread contention stalling the starting of the actual disc reading, and the exif timestamps in the screenshots getting delayed until the apps would resume working correctly must also have come from system-level task / thread contention. The fact that an app's "threading" bug can cause a system-wide task scheduling stall for minutes on end is disturbing however, I though apple moved away from that when they introduced preemptive multitasking in macOS x? But perhaps it's got more to do with the screenshot code being run as a background process that isn't given priority somehow. As to why the OS is budgeting GB's of disc data against us, we still haven't solved that one. I'm aware of the SFSafariViewController DataStore cache clearing, that's not it. I'm aware we have some code that isn't managing to turn off the URLCache, but that's not it either. And I actually wrote code to just enumerate all our directories in our various app group containers and get their file sizes, and I haven't been able to get FileManager to account for even GB's of data on disc. The image cache code was working to keep itself down to the size we had set. As to why the app would not exhibit the main actor / network thread bug when the user had deleted the app and reinstalled, it may have been some kind of race condition regarding the timing of fetching images vs. checking them from the disc cache, or it may have been related to the data on disc that we don't know what it is.

Eskimo, thanks for your answers. There are several follow up questions I wanted to ask after I did some testing, especially around the concept that the resumeData becomes invalid when I restart the request, for cases where our app crashes after resuming an upload and thus having only the previously saved resume data and not a fresh version. Unfortunately, when I went to go test some of these scenarios, I discovered that as of last week, CloudFlare's proxy service no longer allows the 104 intermediate response to go through, and thus I'm never getting any resumeData. We've been working with our CloudFlare representative last week and this week, but don't yet have an answer on why that happened, or if they could allow the 104s again. (They never fully worked, because cloudflare always timed out the connection about a minute after the 104s came through, but we were cancelling the upload as soon as we got the 104 so we could get some resume data and save it on disk, and then restarting from resumeData immediately, and that was working fine for months during feature development.) Do you know of any proxy services which do the support Apple's implemented variant of the tusd protocol (i.e. with the intermediate 104 response)? Can Apple somehow influence CloudFlare to support Apple's implemented variant of the tusd protocol? It's been released on iOS devices for over a year. I'm happy to go sit on the sidewalk across the street from their headquarters holding up a sign that says "Support 104s" if it will help. Is there a way to force the resume manually if the device can't get the 104 intermediate response from the server? We're doing PUTs not POSTs and we already have a unique identifier, so from our perspective we could provide the Location header value pre-calculated when we start the upload initially, but I don't know how to synthesize resumeData with that value. It looks like you've carefully neutered NSKeyedArchiver's ability to read the data blob which I could maybe fix, but it also looks like you maybe even encrypted the original request headers in the data blob, so I'm assuming we wouldn't be able to install the correct encryption keys to provide you with a synthesized resumeData? Is there a way to get resume data if the 104 response is final, and not intermediate? In our testing the answer seemed to be 'no'.

sorry when writing then I should be able to reach the server and get a server response I meant to say then I shouldn't be able to reach the server and get a server response