Does Foundation Models ever do off-device computation?

Question

Created 23h

Replies 2

Boosts 0

Participants 3

I want to use Foundation Models in a project, but I know my users will want to avoid environmentally intensive AI work in data centers.

Does Foundation Models ever use Private Compute Cloud or any other kind of cloud-based AI system?

I'd like to be able to assure my users that the LLM usage is relatively environmentally friendly. It would be great to be able to cite a specific Apple page explaining that Foundation Models work is always done locally.

If there's any chance that work can be done in the cloud, is there a way to opt out of that?

Boost

Answer 1

MB-Researcher OP

19h

Good news! The answer is a clear and simple no.

Foundation Models only uses the on-device ~3 billion parameter foundation model. That's 100% on-device. No behind-the-scenes calls to a server model.

Bonus: the on-device model is designed to be as power-efficent as possible to preserve your device battery life.... but of course, saving electricity makes it more eco-friendly too.

0

Answer 2

Jimscard OP

5h

For the reference you asked for on this - it’s clearly stated that Foundation Models give access to the on-device models here: https://developer.apple.com/documentation/FoundationModels

0