NSPersistentCloudkitContainer Memory Leak -> Crash? (iOS 15 beta 4 & 5)

I am seeing very similar behavior on iOS 15 in my app. Installing it fresh on iOS 14 allows it to upload the initial data and start syncing, but doing the same on the latest iOS 15 beta causes the app to crash while in the background using 100% CPU, and it never manages to upload any data.

I've submitted this as FB9489988, which also references an issue I've seen on both iOS 14 and iOS 15 around cpu_usage_fatal crashes when syncing with iCloud.

Here is a screenshot of the call tree from Instruments and what's chewing up the RAM. There is something broken with the NSCloudKitMirroringDelegate since iOS15 beta 4 (and still not working in beta 6).

Please file a feedback report with your app container (which will include your store file) and this instruments trace.

Same issue with my app with iOS 15. After a fresh installation, after a couple of saves, the data stops syncing with iCloud. This can be seen on the iCloud Dashboard. No data is sent or received. They are saved only locally. The most interesting thing is that the size of the application data in the system settings stops changing. I can delete and add data - the displayed size remains the same (it is unclear where the changes occur at all). My application does not crash. Only there is no synchronization with iCloud. On iOS 14 everything syncs perfectly and quickly.

I bought a new iPad and using my app (downloaded from the App Store, not via Xcode) I can confirm that:

• Data created/modified on iOS 14.7.1 successfully syncs bidirectionally on iOS 14.7.1 devices
• Data created/modified on iOS 14.7.1 successfully syncs down only to iOS 15 (beta 4, 5 or 6) devices
• Data created/modified on iOS 15 (beta 4, 5 or 6) does not upload to iCloud (and eventually crashes the App)^

In order to see the iOS 14.7.1 -> iOS 15 (beta) changes appear I have to reload the app as the download seems to occur before the upload. Once the upload task 'starts' (it never really actually uploads anything), it prevents any changes from downloading.

I've made no changes to my code, data model or NSPersistentCloudkitContainer related methods and it was something that abruptly appeared after installing iOS 15 beta 4.

^ basically any changes made on the device after iOS 15 (beta 4) was installed have not ever been sent to iCloud.

Found this potentially related thread: https://developer.apple.com/forums/thread/688792?page=1#686082022

I’ve not hit these errors with the macOS version (not tried it), but it looks like it’s the same issue with NSCloudKitMirroringDelegate gobbling up available resources and crashing.

Just installed the RC versions of iOS 15 & Xcode 13 and it's still broken. After ingesting external data into my Core Data store & saving the context, I get messages in the console that suggests something will happen... then the memory & CPU use keeps climbing until it crashes. I've tried

• Resetting the Development Environment
• Erasing & Resetting the Simulator

At the end of my data import (from an external backup file), which saves to Core Data just fine, the console shows this:

CoreData: CloudKit: CoreData+CloudKit: -[PFCloudKitExportContext processAnalyzedHistoryInStore:inManagedObjectContext:error:]_block_invoke_3(251): Finished processing analyzed history with 30421 metadata objects to create, 0 deleted rows without metadata.

2021-09-15 07:27:25.213430+1000 Somnus[37796:7522408] [BackgroundTask] Background Task 99 ("CoreData: CloudKit Export"), was created over 30 seconds ago. In applications running in the background, this creates a risk of termination. Remember to call UIApplication.endBackgroundTask(_:) for your task in a timely manner to avoid this.

No syncing occurs and all that happens is CPU & Memory consumption goes through the roof as it has done since iOS 15 beta 4. Everything you see here is 100% NSCloudKitMirroringDelegate - it's insane, especially considering the data that needs to sync is only about 500MB. No code of mine is running at all - the App is just idling!!

This is totally unacceptable for a RC version and it's breaking my existing App Store application so when my users upgrade from iOS 14 -> 15 it's going to stop syncing (and god knows how out-of-sync it's going to get). My App is in use by medical professionals and they're not going to be very happy with me. I so disappointed with this I've effectively stopped prepping the next update of my app. This is just awful.

Apple definitely did something with CloudKit. In these threads there are also some strange issues with iOS14/15 support: https://developer.apple.com/forums/thread/682925, https://developer.apple.com/forums/thread/690044 Framework Engineer wrote this: "We made a number of changes to Core Data this year to isolate CloudKit classes from non-cloudkit clients". I think it's all somehow connected with each other.

FB9412346 should be fixed in the next iOS 15.1 beta seed.

Sorry, Apple... this is still broken in iOS 15.1 (beta 2)

The CPU & RAM usage is still excessive and no syncing occurs at all. I've replied to my TSI in the hope I can get someone to look into this as a matter of urgency. I now have users running iOS 15 who are no longer seeing their data syncing via iCloud, resulting in a terrible user experience, using my app downloaded from the App Store which has been unchanged since iOS 14.

Whatever Apple did in iOS 15 beta 4 to NSPersistentCloudKitContainer, they broke it badly. Maybe users who don't have complex databases aren't seeing this problem, but the whole point of NSPersistentCloudKitContainer is to sync a Core Data store with complex relationships (and besides, it works just fine on iOS 13 & iOS 14!).

My testing setup

• Monterey (beta 8)
• Xcode 13 (13A233) non-beta
• Testing on iPhone 13 Pro with iOS 15.1 (beta 2)

Preparation

• Deleted iCloud data for my app
• Deleted my app from the device (App Store download)
• Rebooted device
• Reset the iCloud Container Development Environment
• Built & run my app using Xcode on my iPhone 13 Pro

Results

I ingested an external backup file into my app, which saved to Core Data perfectly without any errors. This backup file restores the user database which contains lots of many-to-many and one-to-many relationships (the entire POINT of using Core Data).

After the import and the context was saved, the console shows this:

CoreData: debug: CoreData+CloudKit: -[NSCloudKitMirroringDelegate managedObjectContextSaved:](2530): <NSCloudKitMirroringDelegate: 0x2806a4dd0>: Observed context save: <NSPersistentStoreCoordinator: 0x2816b0070> - <NSManagedObjectContext: 0x2806ac1a0>
CoreData: debug: CoreData+CloudKit: -[NSCloudKitMirroringDelegate managedObjectContextSaved:](2530): <NSCloudKitMirroringDelegate: 0x2806a4dd0>: Observed context save: <NSPersistentStoreCoordinator: 0x2816b0070> - <NSManagedObjectContext: 0x2806ac1a0>
CoreData: debug: CoreData+CloudKit: -[NSCloudKitMirroringDelegate managedObjectContextSaved:](2530): <NSCloudKitMirroringDelegate: 0x2806a4dd0>: Observed context save: <NSPersistentStoreCoordinator: 0x2816b0070> - <NSManagedObjectContext: 0x2806ac1a0>
CoreData: debug: CoreData+CloudKit: -[NSCloudKitMirroringDelegate managedObjectContextSaved:](2530): <NSCloudKitMirroringDelegate: 0x2806a4dd0>: Observed context save: <NSPersistentStoreCoordinator: 0x2816b0070> - <NSManagedObjectContext: 0x2806ac1a0>
CoreData: debug: CoreData+CloudKit: -[NSCloudKitMirroringDelegate managedObjectContextSaved:](2530): <NSCloudKitMirroringDelegate: 0x2806a4dd0>: Observed context save: <NSPersistentStoreCoordinator: 0x2816b0070> - <NSManagedObjectContext: 0x2806ac1a0>
CoreData: CloudKit: CoreData+CloudKit: -[PFCloudKitExportContext processAnalyzedHistoryInStore:inManagedObjectContext:error:]_block_invoke_3(251): Finished processing analyzed history with 19785 metadata objects to create, 0 deleted rows without metadata.

Then the CPU & Memory usage starts to rise as before, although not as dramatically, so it takes longer to crash, giving the impression that this is 'fixed', yet absolutely no data is being transmitted from the device - zero syncing is occurring. After several minutes of no activity (other than rising RAM consumption) iOS kills the app for 'using too much memory' - awesome...

What Now?

So the question is what on earth do I do now?

It's been months for this to be acknowledged and now we're being told it's 'fixed' when it's just as broken. What was changed in iOS 15 beta 4 that totally wrecked this... months ago?

If this won't be fixed then I have no choice but to abandon my apps and stop development. I really hope this isn't how it's going to end...

This is all very confusing to me...

...and it's quite possible that this 'bug' is something I'm doing wrong when ingesting data from a backup file, despite it all appearing to save to Core Data just fine. But why now though? Why has it been working just fine for almost 2 years without modification?

Backup Restore Process

The backup file that my users create is a serialised dictionary file, which includes encoded data for images (if present). When I restore it, I use a private context to ingest it on a background thread, with a progress bar being updated on the main thread. As I do not want partially imported data to be incorporated into the user's Core Data store I only save the context at the end of the import when successful.

This has been working flawlessly for the past 2 years with NSPersistentCloudKitContainer, with the data syncing after the import finishes and is saved to Core Data... but was it only working by accident all this time!? Should I not be using the private context to save imported data??

Is it me? I'm quite happy to eat humble pie if it's my problem which as only appeared now because Apple has fixed something at their end and I was 'getting away with it' until now. That's why I filed the TSI (780063461) and why I really want to have a one-on-one with someone!

Update

While the issues is not yet resolved in iOS 15.1 beta 2 (just downloading beta 3 now), I have been very happy with the Apple Developer Technical Support Representative I contacted (via a TSI) they have been the 'intermediate' to liaise with the Framework Engineers.

Had I been trying to troubleshoot this with Apple via messages here, or through the Feedback Assistant (no two-way communication ever occurred there), it would have been a very long & tedious process as we've run into several hurdles. We've been exchanging several emails per day.

So far I've run many tests and uploaded tens of gigabytes of .trace data, etc. (some files are 60GB or more and tests that take hours to run) to get to the bottom of this. They can see their is an issue with CPU & RAM consumption and they're looking into it. This is great news. While the bug (if it's a bug) is a big concern concern, the communication aspect of resolving it was far more important to me. I no longer feel like I'm 'yelling into the void'.

It might take time to fix (that's fine) but there does appear to be something going on when trying to sync lots of relationships - either an initial sync of 'default data' or 'backup data' or when adding objects to an already large & complex (relationship-wise) database.

Syncs with fewer relationships between objects (but the same data model) seem to sync just fine, although sometimes it takes while before any data is transmitted. On earlier versions of iOS syncing would start pretty quickly, but relationships would often be the last to complete to 'finish' the sync and have the data appear correct on other devices.

It's as though it's trying to sync everything in one hit so you don't see any incomplete relationships as objects sync. In iOS 14 this wasn't the case and it took time before the data all synced and looked correct. Perhaps this is a bug related to attempting to stop this from occurring as it might confuse the end user?

The problem is that when the number of metadata objects required to sync exceeds a thousand or so (from my crude test runs), it ends up using so much CPU & RAM, it crashes the app and the actual syncing of data never has a chance to start. It also results in serious battery drain, listing my app as the cause of it!

This will be my last post on this until I hear back from Apple regarding a resolution, which might take some time. Here is a screenshot from App Store Connect showing crashes over time with my App. I last updated it 4 months ago.

You can see the moment my users started installing iOS 15 😬

The same Issues appeared in my app. Are there any Updates regarding this?

Anyone tried with 15.2 beta 1 yet? :)

The problem may even result in fully crashed CoreData stack with data loss (iOS 15.1).

I submitted a case (FB9744735) including sysdiagnose and crash logs. From my point of view an absolutely critical bug.

Quick Update

(Note: I've not received any official update from Apple yet)

I've just installed iOS 15.2 (beta 2 19C5036e) today and have noticed that data now seems to be uploading from my App (App Store version) when previously it would just sit there... then crash. If I look under iCloud -> Manage Storage -> MyApp I can see the figure for data stored going up slowly.

Better yet, I'm seeing data come DOWN to my other devices now.

I'm not near my development setup so I can't test this on a physical device to see what the console output shows, but I'll try and do this tonight on my iPad. I'll ingest my large external backup file into Core Data and see if it uploads to iCloud and report back.

There is no new update for Xcode yet so I don't think we can test with iOS 15.2 (beta 2) in a simulator environment. I'm not going to count my chickens just yet...

Seem to finally sync correctly on iOS 15.2.
But I am seeing a really annoying rate limiting in development environment, is this rate limiting also applying in production environment ? Because if that's the case initial sync will take an enormous amount of time considering how much I am storing in core data and a rate limiting of 70-80 seconds...

To date with the official release of iOS 15 & macOS 12 using Xcode 13.2.1 this is still an issue. The problem is still reflected with the mirror delegate as the source of the leaks on the macOS where the app is terminated during a cloud sync when it goes over 120 Gigs of ram and on the phone.

broken again in ios16b1. doesn't crash but same error message and doesn't sync

test in iOS 16.4.1, This BUG still exists... ;(

This issue might be back when using SwiftData and CloudKit. On iOS 17.5.1 I'm getting multiple warnings before the app crashes after idling for a while.

Could this be a regression?

Background Task 1793 ("CKScheduler-com.apple.coredata.cloudkit.activity.export.F7AC7CB7-21C8-4443-BB52-49B93441C5E3"), was created over 30 seconds ago. In applications running in the background, this creates a risk of termination. Remember to call UIApplication.endBackgroundTask(_:) for your task in a timely manner to avoid this.