How to iterate a collection of NSManagedObject items concurrently for boosting performance?

Question

Created Nov ’20

Replies 1

Boosts 0

Participants 2

Here is my use case:

I need to export a large core data store into some format (e.g., CSV, JSON) which requires fetching all objects of the main entity, and then iterating each object and serializing it to the desired format. Here is my code:

Code Block NSError *error = nil;NSFetchRequest *request = [NSFetchRequest fetchRequestWithEntityName:@"MyEntity"];NSArray<NSManagedObject *> *allItems = [managedObjectContext executeFetchRequest:request error:&error];for (NSManagedObject *item in allItems) {    [self exportItem:item];}

Since the for-loop code is running synchronously and in a single thread, it might take a long time to complete. This is especially true when dealing with a large database containing thousands of records.

I wonder if there is a way to iterate the array concurrently, in a way that will take full advantage of the multiple cores available on the iOS device (rather than using only a single core). This would probably boost the performance significantly.

I was thinking in the direction of using the following code to replace the for-loop code above:

Code Block [allItems enumerateObjectsWithOptions:NSEnumerationConcurrent usingBlock:^(NSManagedObject* item) {     [self exportItem:item]; }

However, this would obviously crash the app due to violating core data concurrency rules...

Any ideas how to target this use case?

Boost

Answer 1

Engineer OP

Apple

Dec ’20

You'd need to have different NSManagedObjectContexts each executing different fetch requests that have a predicate to specify the subrange that worker will export. The NSPeristentStoreCoordinator will run NSPersistentStoreConnectionPoolMaxSizeKey operations concurrently

However this kind of approach is pretty rare because you're more I/O bound than CPU bound and more cores won't help with that.

Instead you might consider a single fetch, like you have above, with batched fetching. And for each batch you pull the results out and asynchronously dispatch the property values to serialize to a separate JSON exporter.

More of a heterogeneous pipeline of concurrent actors than a group of homogenous ones.

0