Pretty much as per the title and I suspect I know the answer. Given that Foundation Models run on device, is it possible to use Foundation Models framework inside of a DeviceActivityReport? I've been tinkering with it, and all I get is errors and "Sandbox restrictions". Am I missing something? Seems like a missed trick to utilise on device AI/ML with other frameworks.
Yeah, I see the following error when trying to prompt the on-device foundation model from within a device activity report extension (DeviceActivityReportExtension), which I believe is the same as what you mentioned:
establishment of session failed with Unrecognized underlying error: Underlying connection was invalidated. Reason: failed at lookup with error 159 - Sandbox restriction
ModelManagerError: got unrecognized error Underlying connection was invalidated. Reason: failed at lookup with error 159 - Sandbox restriction
I am pretty sure that is as-designed.
To give you more context, when responding your prompt, the model needs to communicate with the model manager, which is a system daemon that manages the inference requests. To protect the user’s privacy, however, the system doesn't allow a device activity report extension to move data outside the extension’s address space, and that prevents the model from creating a connection to the model manager.
Having said that, if you have a compelling case that needs to use the foundation model from within your device activity report extension, please feel free to file a feedback report.
Best,
——
Ziqiao Chen
Worldwide Developer Relations.