“MLX”

On this talk: Bring an LLM provider to the Foundation Models framework URL: https://developer.apple.com/videos/play/wwdc2026/339/ on the coding examples a very peculiar framework is shown: import MLXFoundationModels However I am not able to find it nowhere, there is even a code section with this framework as part of an example. Where is this framework, there are no BETA branches on the MLX framework either. Thanks!

Hi, I'm building ANF (Autonomous Native Forge) — a cloud-free, 4-agent autonomous software production pipeline running on local hardware with local LLM inference. No middleware, pure Node.js native. Currently running on NVIDIA Blackwell GB10 with vLLM + DeepSeek-R1-32B. Now porting to Apple Silicon. Three technical questions: How production-ready is mlx-lm's OpenAI-compatible API server for long context generation (32K tokens)? What's the recommended approach for KV Cache management with Unified Memory architecture — any specific flags or configurations for M4 Ultra? MLX vs GGUF (llama.cpp) for a multi-agent pipeline where 4 agents call the inference endpoint concurrently — which handles parallel requests better on Apple Silicon? GitHub: github.com/trgysvc/AutonomousNativeForge Any guidance appreciated.

I’m evaluating a third-party Swift-based “orchestration layer” for enterprise AI workloads on Apple Silicon — it claims to handle job queuing, scheduling, batching, memory management, monitoring, auditing, and failover on top of on-device inference. Given the Core AI framework’s device-specialization step and InferenceFunction pipeline (and MLX’s unified-memory model), how much of this kind of orchestration is already handled natively versus something a developer would still need to build themselves? Specifically: 1. Does Core AI’s inference pipeline provide any built-in job queuing/batching across multiple concurrent requests, or is that entirely app-side? 2. Is there native failover/monitoring tooling for on-device inference, or would a developer need to build that themselves (e.g., via os_log, MetricKit, custom retry logic)? 3. For memory management across CPU/GPU/ANE, does unified memory in MLX/Core AI eliminate most of the manual management a custom orchestration layer would otherwise n

It seems to be that Swift has more APIs implemented than the C++ interface (especially APIs found in the MLXNN and MLXOptimize folders). Is there any intention to implement more APIs for neural networks and training them in the future?

I'm downloading a fine-tuned model from HuggingFace which is then cached on my Mac when the app first starts. However, I wanted to test adding a progress bar to show the download progress. To test this I need to delete the cached model. From what I've seen online this is cached at /Users/userName/.cache/huggingface/hub However, if I delete the files from here, using Terminal, the app still seems to be able to access the model. Is the model cached somewhere else? On my iPhone it seems deleting the app also deletes the cached model (app data) so that is useful.

We are developing a standalone AI avatar application for hospital reception kiosks using Mac mini (M2/M4). The app runs on SwiftUI + RealityKit, displays on a 75-inch monitor, and utilizes a USB-connected 4K camera and external sensors (LiDAR/mmWave). We have several technical concerns regarding the transition from iPadOS to macOS. Could you please provide insights on the following? ARKit/Vision Framework on macOS with External Camera On iPadOS, ARKit provides robust Face Tracking. On macOS with an external USB 4K camera: Can we achieve real-time face tracking (expression/gaze/depth) with Vision framework or ARKit comparable to iPadOS performance? Are there any specific limitations for accessing the Neural Engine via Vision framework for real-time 4K video analysis on macOS? Accessing External Hardware (LiDAR/Sensors) in Sandbox We plan to connect external LiDAR and mmWave sensors (e.g., Akara) via USB/Bluetooth. Is it feasible to communicate with these custom drivers/devices within the App Sandbox environmen