Background URLSession uploads stalled by scheduler budget policies

Our app supports large-file uploads (20 GB+) and uploads of many files. We use a background URLSession so transfers can continue if the user backgrounds the app.

In separate affected-device captures, Console logs and a sysdiagnose show dasd declining upload activities under Energy Budget Policy or Data Budget Policy. Upload tasks are created and resumed, but no bytes are transferred. Subsequent uploads can also remain queued, including after earlier uploads are cancelled and the app is relaunched.

The issue is intermittent but can persist: one device recovered after approximately 24 hours; another remained affected for several days. We have not established what causes recovery or how the relevant budget eligibility is computed or replenished.

Foreground and power observations

For the energy-budget reproduction, the user kept the app foregrounded. Application lifecycle logs and RunningBoard running-active-Visible notifications corroborate foreground/active state around the affected task creation and resume operations.

The rejection repeatedly reports:

Energy Budget Policy
[energyBudget]: Required:1.00, Observed:0.00
Decision: MNP

Battery samples were approximately 68%, with Low Power Mode off. New uploads using the same session succeeded while the device was connected to external power and were rejected again after disconnection.

Discretionary configuration versus daemon classification

Our upload-session factory does not set isDiscretionary to true. In a separate, successful reproduction, instrumentation read isDiscretionary from the actual owning URLSession.configuration at task creation:

10:35:04.234749-0500 Frame.io:
Upload task 1 created in session com.frameio.backgroundSession.diagnostics, configured isDiscretionary: false

The corresponding daemon entry was:

10:35:04.245137-0500 nsurlsessiond:
NDSession <8BA476D9-E790-46B6-B1DA-0246E14BBDAF> Task <1B16494D-BBE8-448F-9297-9DB8772ADD92>.<1> current discretionary status for com.frame.FrameIO is discretionary (opt-in: 0)

All five tasks in that successful upload showed this combination. We do not yet have the instrumented configuration readout from the failed energy-budget reproduction.

Implementation notes

Large files are split into smaller files to support resumable uploads through our backend API. Each part is submitted as a file-backed upload task through the same background session, using a stable session identifier across launches.

Goal

User-initiated uploads make progress while the app is foregrounded and the network is available, and continue when possible after backgrounding. We understand background execution is system-controlled, but need to avoid uploads remaining stalled for days with no actionable recovery.

We are considering switching between foreground and background URLsessions when the app moves between the foreground and background, but have concerns about that the bookkeeping on that path could be a brittle pattern.

We have also considered using a BGContinuedProcessingTaskRequest for the entire upload, but have concerns about running into new scheduling issues there.

Questions

  1. What session configuration and task-submission pattern does Apple recommend for this requirement? Is a background URLSession appropriate for both foreground and background uploading, or should we use a different architecture?

  2. Is it expected that tasks created and resumed while the app is foreground/active, with the owning session reporting isDiscretionary = false, can be blocked by Energy Budget or Data Budget Policy? If so, what supported approach allows these user-initiated uploads to progress while the app remains foregrounded?

  3. When this restriction persists across cancellation, new uploads, and app relaunch, what supported recovery mechanism should we implement? Can the app detect the restriction and take an effective action, rather than requiring external power or waiting an unknown amount of time?

  4. For 20 GB+ files and large upload batches split into file-backed tasks, are there recommended chunk sizes, outstanding-task limits, or session-management practices that prevent this failure mode?

  5. If the captured foreground behavior is unexpected, is this a known OS issue with an available fix or supported workaround? What additional evidence is needed to obtain a concrete remediation?

Answered by DTS Engineer in 908301022

There are lots of questions here that I can’t answer because:

  • iOS’s power management policy is largely considered an implementation detail. It has changed in the past and it’ll likely change in the future. And the actual behaviour is very complex, varying radically based on the specific circumstances of a given device.
  • There’s no single set of best practices here. The best approach is likely going to depend on the specifics of your app. Moreover, it may require tweaking as you gain real world experience.

I have a bunch of general advice on this topic in Moving to Fewer, Larger Transfers. I recommend you read that and then come back here with your remaining questions, keeping in mind the caveats I’ve outlined above.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

There are lots of questions here that I can’t answer because:

  • iOS’s power management policy is largely considered an implementation detail. It has changed in the past and it’ll likely change in the future. And the actual behaviour is very complex, varying radically based on the specific circumstances of a given device.
  • There’s no single set of best practices here. The best approach is likely going to depend on the specifics of your app. Moreover, it may require tweaking as you gain real world experience.

I have a bunch of general advice on this topic in Moving to Fewer, Larger Transfers. I recommend you read that and then come back here with your remaining questions, keeping in mind the caveats I’ve outlined above.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thank you, Quinn. I appreciate the response.

The article is a good writeup and helpful. It makes sense why the power and data management policies are intentionally opaque. For additional context, the app is in market and we do refine our implementation and behavior for uploads based on our observability metrics and customer feedback.

We switched to primarily background uploads from foreground uploads about year ago because we could see in our analytics that people were starting uploads and then backgrounding the app, regularly. We would restart the uploads if they returned but that could lead to uploads processing in short bursts. Moving to background uploads on balance improved our reliability; however, we have had recent reports of people getting into a wedged state where uploads cease working. After investigation, we've traced it the issues outlined in this post.

Sharpening my questions a bit:

  1. The behavior that surprises me the most is that a task which was created (or resumed) while the app is in the foreground and reporting isDiscretionary = false can be blocked by Energy Budget or Data Budget Policy even while the app remains in the foreground. This seems to be at odds with the documentation for the field. Is this expected?
  2. It looks like we could reduce pressure on the system by using resumable uploads and not partitioning large files and sending them as separate requests. Is there an effective maximum size of supported uploads? For example, we've seen memory issues using a file mapped Data instance when segmenting large files whereas an InputStream was effectively unlimited.

My overall, take away so far is that if we want to be able to take responsibility for quality of service and support backgrounding the app, we should take a closer look at BGContinuedProcessingTaskRequest for the places where we can use it which would let us model the actual network work as foreground requests.

The behavior that surprises me the most is that a task which was created (or resumed) while the app is in the foreground

Indeed. IIRC we’ve flipped flopped on that behaviour over the years. Originally the foreground state of the app had no effect, then we changed the system so we boosted the priority of tasks in background sessions from apps in the foreground, and then we reverted that behaviour. Honestly, I’m not sure whether that was deliberate or not.

I think it’d be reasonable for you to file a bug about this, in that we should either change the system or the documentation.

Please post your bug number, just for the record.

It looks like we could reduce pressure on the system by using resumable uploads and not partitioning large files and sending them as separate requests.

Yes.

Although I definitely recommend that you prototype this first, before committing to that path. Resumable uploads are not exactly the well-trodden path.

Is there an effective maximum size of supported uploads?

For resumable uploads? You start those with a file, so the only limit is the available disk space.

For example, we've seen memory issues using a file mapped Data instance when segmenting large files whereas an InputStream was effectively unlimited.

Right. That’s definitely something to watch out for when you’re not using a background session, but background sessions only support uploading from a file.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"