Should static domain documentation live in on-device RAG (local embeddings + FM), while time-sensitive data always comes from network tools — and are there practical size/latency budgets for on-device embedding indexes?
Static domain documentation makes sense for the on-device index, with item donations under 100k considered 'small'.
Large catalog content from a server is only recommended in the cases where:
By default, all donations expire out of the index within 30 days, unless a custom expiration date is set. This is a great way to manage index content size.
One of the most important tips for performance, however, is to ensure your content is only donated when necessary (eg. use batch donation with client state and updates to avoid re-donating the same item unnecessarily).
