“MLX”

I have a Claude Pro subscriptions and tried this with a Claude Agent and it worked just like in the video. But if I use the free OpenAI GPT-5 Chat then I don't get the links. There is a significant capability difference between an agent and just a plain chat. The agents have tools that make them much smarter and have access to your file system, but other than running an open source agent on your own hardware, you will need a subscription. I got an open source chat model running on my laptop last night using MLX. I have yet to get the OpenCode Agent working yet. Cheers.

I am doing some experiments in past few days, it's not easy to convert my PyTorch model to coreML models. I have to modify the python code to avoid conversation error, even if the conversation succeeds, it may crash when running in Xcode; finally I can manage to convert the model successfully and run it in Xcode, but it only uses the CPU to compute, only if I use fixed input shape, it can use the NPU to compute, but I really need to use non-fixed input shape to deal with different images. So I'm searching other solutions, mlx? libTorch? I don't know.

Hi @fbalancin, The Foundation Models framework gives developers access to Apple's on-device large language model that ships with the user's operating system. One advantage of using it is that your app is accessing the model that's already on-device and you don't have to include a language model in your bundle, which takes up storage space and download bandwidth. That said, you can indeed use your own model in your app. Our new framework MLX is one tool for training your own, it's optimized for Apple Silicon and has tight integration with Hugging Face. There are also other third-party AI models that have slimmed down versions of their models that can be included in your app and run on-device depending on your needs. We also have frameworks like CoreML for integrating machine learning and you can explore a lot of what's offered for developers in our documentation. Best, -J

[quote='892460022, Vision Pro Engineer, /thread/833038?answerId=892460022#892460022'] there's value in learning a DCC like Blender [/quote] Blender is awesome! Totally worth learning, open source and very much programmable with python (scripts or cli). I'd look into available MCP's to help, but I think to some degree it's worth knowing the basics... Also there are several extensions and plugins (plus you can code[x] your own if needed;) that could help you leverage your workflow. I really love python, BTW, would love to learn more about how I could use it on Xcode for iOS dev in general, including 3D, LiDAR, BIM, Vision, ML, MLX workflows, but I'm still somewhat a newb on Apple platforms and Swift... Any particular tips for my journey towards my first and yet-to-be-published iOS app?

For apps using the 'MLX distributor' or local adapters I'm not sure what you mean here, can you elaborate? are there any specialized background processing entitlements needed to ensure inference isn't killed by the OS during long-running tasks In general, if you want to allow a model inference to continue running in the background, you should look at the Background Tasks framework to achieve this: https://developer.apple.com/documentation/BackgroundTasks . For models, it's especially important to consider the memory footprint, so make sure your model is small enough to fit on your targeted devices without any issues. If your model uses GPU in the background, you may also need to add the com.apple.developer.background-tasks.continued-processing.gpu entitlement: https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.developer.background-tasks.continued-processing.gpu Hope that helps!

Update on LocalLM Lab: 1.0.0-beta.1 adds a model layer to the SDK. This means one API across Apple's on-device FoundationModels model, Claude, and a locally-run open-weight model via MLX (a separate runtime, LocalLMLabSDKInference), with routing and model residency (which model stays warm, how many at once) owned by the SDK rather than something every app has to build itself. This is entirely SDK-side, not a FoundationModels API change. SystemModelProvider still wraps SystemLanguageModel under the hood the same way it always has. A fourth provider, PCCModelProvider (Apple's Private Cloud Compute), ships in this release but isn't functional yet in 1.0.0-beta.1. Check out https://developer.apple.com/private-cloud-compute/ for more info on this. Requires macOS 27 beta, Apple Intelligence enabled, Xcode 27 beta for SDK development. SDK guide: https://github.com/ancientcomputing/locallm/blob/main/docs/sdk-guide.md Feature page: thisbrain.ai/locallm/1.0.0-beta

I don't think Apple will do anything about providing a one click option, and that's for many reasons. AMD-ROCm are responsible for making the GPU support AI use on an operating system (either macOS or Linux). Current 1-click solution (after ROCm installation) is provided by Ollama. It works on macOS, but due to lack of ROCm support on macOS, ollama only uses CPU+RAM on macOS. Apple's focus is on Apple Silicon (& MLX models?) Mac Pro 2019 is officially discontinued. We're lucky to get hardware repairs in the coming few years. On another note, regarding the AMD Radeon PRO W6900X being supported on Linux or with ROCm, I can confirm that it works, with the single change of adding the kernal flag iommu=pt in grub. Keep in mind that ROCm's documentation does not mention the W6800X or W6800X Duo; but only mentions the W6800. However, the W6800X & Duo work right out of the box. (assumingly due to being an almost exact replica to the standard W6800 on the inside) Ollama even goes so far as to explicit

Hopefully I can help clarify! As of WWDC 2026, Foundation Models framework covers both on-device foundation models and server-based models.... and both Apple Foundation Models as well as any other LLMs. So foundation models can mean a bunch of different things and a bunch of possible models, which is part of the reason why there isn't currently a clean device-capability flag. The full list of Apple Intelligence requirements (for Apple Foundation Models) can be found here https://support.apple.com/en-us/121115 and include a combination of regional and hardware requirements. Models from other sources can be used with Foundation Models using MLX or CoreAI, so you can still reach users with hardware that can't run Apple's on-device foundation model. So... what can you do? Run an availability check as soon as you launch your app. For example https://developer.apple.com/documentation/foundationmodels/systemlanguagemodel/isavailable for the on-device model. Availability can tell you additional information a

You can add your own model with MLX. I've used MLX_LM.lora to 'upgrade' Mistral-7B-Instruct-v0.3-4bit with my own content. The 'adapter' folder that is created can be added to the original model with mlx_lm.fuse and tested with mlx_lm.generate in the Terminal. I then added the resulting Models folder into my macOS/iOS Swift app. The app works on my Mac as expected. It worked on my iPhone a few days ago but now crashes in the simulator. I don't want to try it my phone until it works (again) on the simulator. The models folder contains the files created by the fuse command. I'm not sure that I changed anything in the last few days but it seems to be crashing in the loadmodel function with a lot of 'strange' output that I don't understand. My loadModel func `func loadModel() async throws -> Void { // return // one test for model not loaded alert // Avoid reloading if the model is already loaded let isLoaded = await MainActor.run { self.model != nil } if isLoaded { return } let modelFactory = LLMModel

(Continued) Can developers train their own adapters (LoRAs) to be used with the Foundation models? Yes, Apple has released a Foundation Models Adapter Training for ML practitioners to train custom adapters. This is useful for specialized datasets and can reduce the amount of prompting needed. However, you are responsible for retraining the adapter whenever Apple updates the base model in a new OS. Adapters are approximately 150-160 MB in size, and can be integrated using the background asset delivery APIs. Can you talk a bit about the improvements that were made this year in existing Vision Models? Improvements have been made increasing accuracy and speed. A new RecognizeDocumentRequest has been added for analyzing documents like receipts and nutrition labels. There are also new lens smudge detection and hand pose models. I’m developing an educational app about some ancient books, how can how can I load all the books’ text into an Apple model? Due to the context window limit, you'll need to get creative. Cons