How to index file based documents in Core Spotlight

App & System Services Core OS
myintruder OP
Created 4d
Replies 6
Boosts 0
Views 165
Participants 2

I read this thread https://developer.apple.com/forums/thread/788979 thoroughly, but I’m still confused regarding indexing files content.

I'm building a notes app where the notes are stored in files. A file can contain several notes (think paragraphs). Each note and the file document itself have a unique ID, all embedded in the file. So far so good, when the user opens a file in the app, I index all the notes in it using several CSSearchableItem, one for each note. Each CSSearchableItem gets a unique ID based on the note and file IDs.
The notes are then visible in Spotlight search and when the user taps one of them, the app is called with a Spotlight activity and I present the note.

  1. I learned that I should create a CSImportExtension to allow the system to index files when app is not running. But the only method is update(_:forFileAt:), which allows to provide back to the system a single attributes set. How can I index the notes in a file as separate items?

  2. What happens if an iCloud document file is edited remotely and the app is not running, or is editing another file? Does the system detect it and run CSImportExtension on the file?

  3. All the notes and documents IDs are unique, and when the user duplicates the document file from within the app, new unique IDs are set in the duplicate file. But the user can also duplicate files outside the app, in which case the IDs remain the same in the duplicate file. How does Spotlight react to indexing two distinct items, with the same ID, but different 'contentURL'?

  4. What if I index a note from a file, and set the current contentURL of the file, and then the user moves the file. Next time when I index a note from this file, Spotlight will get an item with the same uniqueIdentifier but with a different contentURL. Won't this confuse the system?

  5. How to handle the case of deleted files: Unless a file is pending editing, the app doesn’t know it has been deleted, so it won’t remove the corresponding items from Spotlight.

I should mention that I use a Core Data database, which stores the mapping from file document IDs to file URLs, actually to bookmarks, so I can track the files even if the user renames or moves them.

Answered by DTS Engineer in 864596022

It’s better to reply as a reply, rather than in the comments; see Quinn’s Top Ten DevForums Tips for this and other titbits.

iOS and macCatalyst.

Well, that doesn’t make it easy )-:

I see at least two complications here:

  • You have an app that’s kinda shoebox-y and kinda document-based. Spotlight uses different approaches for these two types of apps.
  • For a documented-based app, iOS and macOS have completely different mechanisms for indexing documents. If you decide to build an importer, you’ll have to package your core code in two different ways: an iOS-style CSImportExtension app extension and a macOS-style .mdimporter plug-ins.

Expanding on the first point, when you build a document-based app Spotlight implicitly assumes that each document is a single user-visible piece of content. It just doesn’t have a way to represents items within a document. That’s reflected in the API for both the iOS-style and macOS-style import APIs.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Replies  6
Boosts  0
Views  165
Participants  2
DTS Engineer OP
Apple
3d

What platform are you targeting?

This matters because, as noticed in the thread you referenced, the Spotlight importer story is very different on macOS than it is on other platforms.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

0
DTS Engineer OP
Apple
2d
Accepted Answer
Recommended

It’s better to reply as a reply, rather than in the comments; see Quinn’s Top Ten DevForums Tips for this and other titbits.

iOS and macCatalyst.

Well, that doesn’t make it easy )-:

I see at least two complications here:

  • You have an app that’s kinda shoebox-y and kinda document-based. Spotlight uses different approaches for these two types of apps.
  • For a documented-based app, iOS and macOS have completely different mechanisms for indexing documents. If you decide to build an importer, you’ll have to package your core code in two different ways: an iOS-style CSImportExtension app extension and a macOS-style .mdimporter plug-ins.

Expanding on the first point, when you build a document-based app Spotlight implicitly assumes that each document is a single user-visible piece of content. It just doesn’t have a way to represents items within a document. That’s reflected in the API for both the iOS-style and macOS-style import APIs.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

0
myintruder OP
2d

That's bad news. Am I allowed to use BGAppRefreshTaskRequest or BGProcessingTaskRequest, and a NSMetadataQuery, to watch all the files in the sandbox and manually index in Spotlight the updated ones? This way I can open a file and index each note separately.

0
DTS Engineer OP
Apple
1d

“Allowed” is a tricky word, because it covers both business and technical limitations. I don’t work for App Review and thus can’t offer definitive answers about the former. I recommend that you review their published guidelines.

But ignoring that, the technical aspects are also tricky. The basic idea of updating your Core Spotlight index in the background should be feasible. The sticking point is getting access to the documents.

The cool thing about Spotlight importers is that they’re tightly sandboxed. So the system is able to find all the user’s documents and present them to the importer. The importer is sandoxed so info can’t leak to the app, and the app never gets access to the user’s entire file system. Neat-o!

You lose that if you do this indexing yourself. You will be able to access any documents in your app’s container, and any documents which you were previously granted access to by the system [1], but you won’t necessarily be able to see all documents.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] On macOS you retain this access using a bookmark.

0
myintruder OP
22h

What happens if I index the note items from a file by calling CSSearchableIndex.default().indexSearchableItems(allNoteItems) from within the update(_:forFileAt:) of the CSImportExtension, when the system calls it. Will Spotlight register or discard these separate items.

0
DTS Engineer OP
Apple
15h
Will Spotlight register or discard these separate items.

No idea, sorry. My concern, however, is that this is not using the API in which it was intended to be used.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

0
How to index file based documents in Core Spotlight
First post date Last post date
 
 
Q