How to index file based documents in Core Spotlight

I read this thread https://developer.apple.com/forums/thread/788979 thoroughly, but I’m still confused regarding indexing files content.

I'm building a notes app where the notes are stored in files. A file can contain several notes (think paragraphs). Each note and the file document itself have a unique ID, all embedded in the file. So far so good, when the user opens a file in the app, I index all the notes in it using several CSSearchableItem, one for each note. Each CSSearchableItem gets a unique ID based on the note and file IDs.
The notes are then visible in Spotlight search and when the user taps one of them, the app is called with a Spotlight activity and I present the note.

  1. I learned that I should create a CSImportExtension to allow the system to index files when app is not running. But the only method is update(_:forFileAt:), which allows to provide back to the system a single attributes set. How can I index the notes in a file as separate items?

  2. What happens if an iCloud document file is edited remotely and the app is not running, or is editing another file? Does the system detect it and run CSImportExtension on the file?

  3. All the notes and documents IDs are unique, and when the user duplicates the document file from within the app, new unique IDs are set in the duplicate file. But the user can also duplicate files outside the app, in which case the IDs remain the same in the duplicate file. How does Spotlight react to indexing two distinct items, with the same ID, but different 'contentURL'?

  4. What if I index a note from a file, and set the current contentURL of the file, and then the user moves the file. Next time when I index a note from this file, Spotlight will get an item with the same uniqueIdentifier but with a different contentURL. Won't this confuse the system?

  5. How to handle the case of deleted files: Unless a file is pending editing, the app doesn’t know it has been deleted, so it won’t remove the corresponding items from Spotlight.

I should mention that I use a Core Data database, which stores the mapping from file document IDs to file URLs, actually to bookmarks, so I can track the files even if the user renames or moves them.

Answered by DTS Engineer in 864596022

It’s better to reply as a reply, rather than in the comments; see Quinn’s Top Ten DevForums Tips for this and other titbits.

iOS and macCatalyst.

Well, that doesn’t make it easy )-:

I see at least two complications here:

  • You have an app that’s kinda shoebox-y and kinda document-based. Spotlight uses different approaches for these two types of apps.
  • For a documented-based app, iOS and macOS have completely different mechanisms for indexing documents. If you decide to build an importer, you’ll have to package your core code in two different ways: an iOS-style CSImportExtension app extension and a macOS-style .mdimporter plug-ins.

Expanding on the first point, when you build a document-based app Spotlight implicitly assumes that each document is a single user-visible piece of content. It just doesn’t have a way to represents items within a document. That’s reflected in the API for both the iOS-style and macOS-style import APIs.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

What platform are you targeting?

This matters because, as noticed in the thread you referenced, the Spotlight importer story is very different on macOS than it is on other platforms.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

It’s better to reply as a reply, rather than in the comments; see Quinn’s Top Ten DevForums Tips for this and other titbits.

iOS and macCatalyst.

Well, that doesn’t make it easy )-:

I see at least two complications here:

  • You have an app that’s kinda shoebox-y and kinda document-based. Spotlight uses different approaches for these two types of apps.
  • For a documented-based app, iOS and macOS have completely different mechanisms for indexing documents. If you decide to build an importer, you’ll have to package your core code in two different ways: an iOS-style CSImportExtension app extension and a macOS-style .mdimporter plug-ins.

Expanding on the first point, when you build a document-based app Spotlight implicitly assumes that each document is a single user-visible piece of content. It just doesn’t have a way to represents items within a document. That’s reflected in the API for both the iOS-style and macOS-style import APIs.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

That's bad news. Am I allowed to use BGAppRefreshTaskRequest or BGProcessingTaskRequest, and a NSMetadataQuery, to watch all the files in the sandbox and manually index in Spotlight the updated ones? This way I can open a file and index each note separately.

How to index file based documents in Core Spotlight
 
 
Q