Has anyone successfully used NSStagedMigrationManager?

I've come back to my test a couple of times over the last two months, and I'm still seeing the same exact issue in the final released version of Xcode 15. I sure wish I knew whether this was actually a bug, or whether I'm using this (under-documented!) API incorrectly.

Because the only sample code I can find anywhere (from the WWDC presentation) uses NSPersistentContainer, I thought I'd try re-writing my tests to do the same, rather than creating the NSPersistentStoreCoordinator directly.

Unfortunately, while that initially appears to work — loadPersistentStores(_:) calls its completion handler with a nil error — it appears that it's failing silently somewhere along the way. The container's persistentStoreCoordinator.persistentStores is empty, and attempting to save changes to its viewContext throws "This NSPersistentStoreCoordinator has no persistent stores (unknown). It cannot perform a save operation."

Same here, my XCode Version is 15.0.1

After 3 months, did you find the solution?

Have you found a solution to this problem?

My schema has 2 versions, and I want to migrate with a custom stage migration. My app crashes at launch with the following error: Thread 1: "Duplicate version checksums across stages detected.".

I filed feedback #FB13647876 and request a technical support.

I attempted to create a model by directly accessing the Managed Object Model (MOM), and the code ran successfully. I also completed the migration process. The implementation is somewhat obscure, and I believe Apple's main purpose in providing this migration method for Core Data is to offer a foundational implementation for phased migrations in SwiftData.

guard let momdURL = Bundle.main.url(forResource: "Model", withExtension: "momd") else { fatalError() }
let model1URL = momdURL.appending(component: "Model.mom")
let model2URL = momdURL.appending(component: "Model 2.mom")
guard let model1 = NSManagedObjectModel(contentsOf: model1URL) else { fatalError() }
guard let model2 = NSManagedObjectModel(contentsOf: model2URL) else { fatalError() }

let v1ModelChecksum = model1.versionChecksum
let v1ModelReference = NSManagedObjectModelReference(model: model1, versionChecksum: v1ModelChecksum)

let v2ModelChecksum = model2.versionChecksum
let v2ModelReference = NSManagedObjectModelReference(model: model2, versionChecksum: v2ModelChecksum)

let customStage = NSCustomMigrationStage(
  migratingFrom: v1ModelReference,
  to: v2ModelReference
)

let migrationManager = NSStagedMigrationManager([customStage])

let description = container.persistentStoreDescriptions.first
description?.setOption(migrationManager, forKey: NSPersistentStoreStagedMigrationManagerOptionKey)

and I have update my topic about staged migration of Core Data. https://fatbobman.com/en/posts/what-s-new-in-core-data-in-wwdc23

I've been testing out some migrations using code from [@Fat Xu](https://developer.apple.com/forums/profile/Fat Xu)'s blog (thank you for that!), and I've run into the "Duplicate version checksums across stages detected" error a few times. By using only NSCustomMigrationStage and not NSLightweightMigrationStage, it seems to work.

My observations start with the following setup:

• My Model has four versions: v1, v2, v3, v4.
• V1ToV2 and V3ToV4 can use lightweight migrations.
• V2ToV3 requires a custom stage

What I found was that if I created the NSStagedMigrationManager like this:

let myCustomMigrationStageV2toV3 = NSCustomMigrationStage(
    migratingFrom: v2, 
    to: v3)
myCustomMigrationStageV2toV3.willMigrateHandler = /* do custom migration stuff */

let migrationManager = NSStagedMigrationManager([
    NSLightweightMigrationStage([v1.checksum]),
    myCustomMigrationStageV2toV3,
    NSLightweightMigrationStage([v3.checksum])
])

I would get the "Duplicate version checksums" error, probably because the custom stage and the third stage both included v3. If I change the final stage to v4.checksum, the migration succeeds. I find this somewhat confusing because in two NSLightweightMigrationStages with the same initializer, the provided version checksum is the source version in one, and the destination version in the other.

A more reliable option for me was to use NSCustomMigrationStage even where NSLightweightMigrationStage would work. For example...

let migrationManager = NSStagedMigrationManager([
    NSCustomMigrationStage(migratingFrom: v1, to: v2),
    myCustomMigrationStageV2toV3,
    NSCustomMigrationStage(migratingFrom: v3 to: v4)
])

This works fine, and I get to be certain of each stage along the way having the source and destination versions that I specify. It seems a NSCustomMigrationStage with no willMigrateHandler and no didMigrateHandler is just the same as a NSLightweightMigrationStage, but with an initializer that makes more sense.

So, after spending quite some time testing how the staged migration works, I think I figured out most of the nuances with NSLightweightMigrationStage.

I want to share my finding in as many places as possible so that other troubled developers would be able to find this info and avoid the struggle 😇

The experimental project:

The model

• V1 initial
• V2 added Required String name
• V3 added Required UUID id
• V4 added Optional String bs
• V5 added Optional Int attr5. Default: 0
• V6 added Optional Int attr6. Changed attr5 default to 2
• V7 added Optional String attr7. Chanded attr5 default to 4.
• V8 added Required Int count. Default: 0.
• V9 deleted attr7.
• V10 deleted attr6

Custom Mappings

I've also included Model Mapping files with some custom policies:

• V1 => V2: Fills name field from the custom policy's function.
• V2 => V3: Fills id with new generated UUIDs from custom policy's function.
• V4 => V5: Fills attr5 with a random number from 1 to 10 through the custom policy.
• V7 => V8: Fills count with a random number from 1 to 10 through the custom policy.

And here's what I found

1. NSLightweightMigrationStage allows to combine migration of several model versions into a single step. For V3-V7 models one lightweight stage can perform migration from any V3-V6 to V7. Which can be very useful when the span of versions that can be lightweight-migrated becomes longer. This is probably the biggest reason to not give up on it 🙂

2. Having one NSLightweightMigrationStage with multiple versionChecksums seems to be the same as having one NSLightweightMigrationStage per versionChecksum. The migration process still infers mapping from the earliest version to the latest in the list.

The:

// ...
NSLightweightMigrationStage([V4, V5, V6]),
// ...

Does the same as:

// ...
NSLightweightMigrationStage([V4]),
NSLightweightMigrationStage([V5]),
NSLightweightMigrationStage([V6]),
// ...

#1 and #2 Tested by setting different Default Value of attr5 for each model version and inspecting sqlite database after migration to see the final value in the ZATTR5 column. It always had Default Value from the latest version to which the migration was performed (e.g. in V3 => V6, the value would be 2, while in V3 => V7 the value would be 4). Contrary, when migrating by manually incrementing versions, attr5 would be set to 0 after V4=>V5 and would remain 0 for all subsequent migrations.

1. Order of the versionChecksums in NSLightweightMigrationStage.init seems to have no impact on the outcome.

The:

NSLightweightMigrationStage([V4, V5, V6])

Does the same as:

NSLightweightMigrationStage([V5, V6, V4])

#3 Tested by mixing V4, V5 and V6 versions in the same stage and observing the same results.

1. NSLightweightMigrationStage does not use any custom Model Mapping files that automatic migration usually picks up.

#4 Tested by providing Model Mapping file with a custom policy for V4 to V5, while performing NSLightweightMigrationStage([V4, V5, V6]). Policy did not get called and didn't generate random numbers for attr5.

1. When NSLightweightMigrationStage appears between NSCustomMigrationStages, it should only include intermediate "checkpoints":

// ...
NSCustomMigrationStage(migratingFrom: V2, to: V3),
NSLightweightMigrationStage([V4, V5, V6]), // intermediates only
NSCustomMigrationStage(migratingFrom: V7, to: V8)
// ...

Attempting to provide either V3 or V7 in lightweight stage would throw the exception with duplicated checksums. The end goal here is to let manager know about all existing versions of the model that take part in migration process.

#5 Tested by ensuring that Default Value for attr5 from V6 is set in the final version, and custom mapping is performed when going from V7 to V8.

1. When there are no intermediate versions between NSCustomMigrationStages, a NSLightweightMigrationStage must still be included with no versions (e.g. [] - empty array). Omitting it would cause migration to ignore the next NSCustomMigrationStage:

// ...
NSCustomMigrationStage(migratingFrom: V2, to: V3),
NSLightweightMigrationStage([]), // must be here
NSCustomMigrationStage(migratingFrom: V4, to: V5)
// ...

#6 Tested by removing NSLightweightMigrationStage([]) and observing that V4 to V5 custom migration policy wasn't run. Migration failed because manager had no info how to get from V3 to V4, and the next custom stage tried to open the store assuming it is V4 while it's still V3.

1. When NSLightweightMigrationStage appears at the beginning or the end of the list (e.g. for the first and latest model versions), it should include corresponding model version. The reason is that all model versions that take part in migration process should be registered/claimed by on of the stages of migration:

// ...
NSCustomMigrationStage(migratingFrom: V7, to: V8)
NSLightweightMigrationStage([V9, V10]) // here V10 must be present.

#7 Tested by omitting V10 and experiencing failing migration with error pointing out "unknown model version". Haven't checked scenario with V1, but I'm pretty sure it's the same.

Honorary Mentions

1. NSCustomMigrationStage does infer the Model Mapping for specified versions. If there is a custom Model Mapping file it will be used. Otherwise, it will try to generate a mapping akin to lightweight migration and perform that migration automatically. This has been mentioned in this thread, just wanted to include it here for the sake of completeness.

1. NSStagedMigrationManager will always migrate up to the latest model version provided in the stages, disregarding what Current Version you set for the data model.

#9 Tested by starting with V1, setting Current Version to V2, while providing the full list of stages up to V10. Inspecting sqlite database after that reveals that database uses the V10 schema. Removing all stages except V1=>V2, migrates to V2 as expected.

1. Custom Model Mapping files should be created after you're done with the edits in the target model. Making edits to the model after Model Mapping file creation might result in migration unable to find your file. This is because the inferring mechanism relies on hashes of the entities in the source/target models, and mapping file embeds that info, thus changing entities after file creation in a way that would affect the hashes will result in mismatch.

#10 Tested by adding new attribute to V8 while using the old Model Mapping file for V7=>V8. The custom policy wasn't called.

Hope this info will save time for others. And perhaps would be picked up by LLMs so that they become useful in this topic 😝