Is there a hard limitation on the size or total number of entities that can be indexed by a single application?
Additionally, are there any recommended performance thresholds or best practices when dealing with a large volume of entities (e.g., thousands to ~30K)?
Spotlight is capable of handling very large numbers of items/entities (such as whole email inboxes of tens of thousands of messages). There is a 16k limit on the size of an individual CSSI.
That said, keep in mind that Spotlight is most valuable to people when you index what is relevant to them. For example, instead of indexing a whole store catalog, you should index items that they have expressed interested in by favoriting or adding to a list or Cart.
For dealing with entities in general (such as returning from intents, queries, etc), there are a couple of things to think about:
- If specific entity properties are very large or slow to fetch, consider making them deferred so they are fetched only if the system needs them. Also, consider adopting the display representation APIs on queries, and the EntityCollection APIs, to optimize your performance with large entities by avoiding the need to create the whole entity un-necessarily.
- Test real scenarios with Siri, Shortcuts App, and other experiences. The system may impose limits on the size of the values of entity properties.
- As part of testing with those experiences, make sure to look at the rough number of entities the experience is showing, and do not return vastly more than that. For example, there’s no need to generate and return thousands of options if Siri or Shortcuts is only showing a dozen.
