I'm building an app that uses PrivateCloudComputeLanguageModel as the primary inference tier with SystemLanguageModel as the fallback. The app is entitled (com.apple.developer.private-cloud-compute, granted and provisioned) and generations serve normally. My question is how a client should decide to fall back because in extended measurement, no public signal ever reflects the blocked state I actually hit. What I measured (macOS 27.0 beta, 26A5416b / Xcode 27 beta 27A5237l, entitled signed bundle constructing PrivateCloudComputeLanguageModel directly): Serving stopped mid-run with no leading signal: request N served normally (1.4 s), request N+1 threw LanguageModelError.rateLimited 494 ms later, at cumulative generation 786 for the day. 100% served → 100% refused between consecutive calls. Every quota signal read healthy the entire time: before, during, and after the block. Across 1,517 readings in a single day: quotaUsage.status = belowLimit, isApproachingLimit = false, isLimitReached = false, resetDate = nil, availability = .available. A preflight on these APIs cannot see the condition. The refusal is enforced locally after first contact: rejections return in ~230 ms vs ~0.9–1.4 s for served calls, so the client appears to cache the verdict rather than ask the server per-request. The trigger is a cumulative ledger, not a request rate: 501 generations at 33/min in one 15-minute sitting was fine, and a later arm sustained 39.7/min; two bursts of 16 concurrent at 5.0 and 5.2 req/s served 32/32; the count that tripped survived a process restart and a 4.9-hour idle gap. But it's not a fixed daily number either. 501 fast was fine earlier the same day; the trip came 285 requests later. A rolling window on the order of hours-to-a-day is consistent with this, but nothing here measures its length. Recovery: still blocked at +41 minutes (probes at +1/2/5/10/20/40 min all refused); fully recovered by +20 h with no intervention and no upgrade. Next day served normally from the first request. quotaLimitReached never occurred: not once in ~800 generations plus the blocked period. The wall is typed as the transient error while carrying what the documentation describes as daily quota semantics ("a person either waits for their usage quota to refresh or they upgrade"). limitIncreaseSuggestion is presence-constant: nil at process start, non-nil on every reading after first PCC contact (identical while fully serving and while fully blocked) so its presence can't gate an upsell affordance. The same signals-read-healthy-while-refusing divergence also reproduces against the developer-tool pool (fm serve), which I've reported separately (FB24273854 covers quota exhaustion surfacing there as a generic server_error/500 while /health reports the model available). Questions: Is attempt-and-classify the intended contract? Given that no preflight can observe the blocked state, should a client simply issue the request, treat the typed error as authoritative, and route to SystemLanguageModel? And is the ~230 ms local fail-fast on the blocked path contractual (cheap and safe to probe) or incidental? This is the one that decides how I ship; the rest are diagnostics behind it. What does quotaUsage actually track, and at what granularity? I have driven the entitled app-tier path to a hard block and the developer-tool pool to exhaustion, and no field ever moved. Is there any consumption pattern that moves isApproachingLimit / isLimitReached / resetDate? If the intended answer is "only the per-person daily quota, which these volumes never approached," what is the wall I am hitting at ~786 cumulative, and why does it surface as rateLimited? Should rateLimited and quotaLimitReached drive different client behavior — and which one is the daily allowance in practice? The documentation distinguishes rate limiting ("wait a period and retry") from daily exhaustion ("wait for refresh or upgrade"), but what I observe is the transient-typed error carrying the multi-hour ledger semantics. Concretely: what retry cadence is recommended after rateLimited (my measured recovery horizon was somewhere between 41 minutes and 20 hours. My current design stays on the on-device model and re-probes PCC at a low fixed interval rather than per-request)? And under what condition is resetDate ever populated, given it was nil even while blocked? (Smaller, design guidance): my app can generate a few hundred requests as one feature batch (quiz generation over a user's imported document). Measured: 501 in a sitting was fine, cumulative 786 in a day was not. Since this allowance belongs to the person and is shared with every Apple Intelligence feature, is a several-hundred-request batch a reasonable use of it, or should features like this generate on demand? (I'm aware of the existing feature request for richer quota reporting (FB23378161); this is a narrower design question.) I can attach the measurement driver and timestamped JSONL logs. The divergence is reproducible on a fresh day, though reaching the wall took ~800 cumulative generations.

A suspected RTL performance erratum in the Apple Neural Engine throttles DRAM weight streaming throughput down to 17–19 GB/s from the nominal 45–60 GB/s, whenever the total weight size is an integer multiple of 1 MiB. This was confirmed on M3 Macbook Air (2024) 24GB. 1 MiB is a common transfer size, and currently affects 7 of ANEMLL’s 15 baseline models. Avoiding the suspected problematic path in the kernel DMA's prefetch ring by splitting 1 MiB transfers into non-multiples of 1 MiB, directly increased Llama 3.2 1B token throughput from 10.0 to 24.3 tokens/s (DRAM usage from 24.7 to 60.0 GB/s), and Qwen3-8B from 1.36 to 2.97 tokens/s (DRAM usage from 22.4 to 48.7 GB/s). Please see writeup here: https://eiln.github.io/posts/ane-dma.html I'm looking to confirm that this behavior is reproducible on your side, and will provide the scripts in my plots.

Hi all :) I tried to raise this in the group lab and was pointed here. I’m seeing a flat per-app memory ceiling of about 6 GB on iPhone, even on devices with more physical RAM and with com.apple.developer.kernel.increased-memory-limit. Measured with os_proc_available_memory() plus task_vm_info.phys_footprint, the total process budget stays around 6144 MB on both: iPhone 16 Pro Max, 8 GB RAM iPhone 17 Pro Max, 12 GB RAM This came up while running Gemma 4 multimodal support in mlx-swift-lm (PR #343). The model loads at about 4.4 GB resident, leaving roughly 1.7 GB for inference/prefill. Reducing a GPU buffer cache from 512 MB to 64 MB recovered enough headroom to avoid jetsam and allowed a full image + video + audio multimodal test to complete, so the measurement seems to reflect a real per-process limit rather than free system memory. I re-measured the ceiling on the 12 GB phone with these capabilities: increased-memory-limit only: ~6144 MB increased-memory-limit + extended-virtual-addressing: ~6144 MB, no change increased-memory-limit + increased-debugging-memory-limit: ~6656 MB I have also observed that 12 GB iPad devices expose more memory to an app than 12 GB iPhone devices but I didn't measure specifically and no longer have the device to hand. Is the ~6 GB per-process tier on Pro iPhones expected, even with increased-memory-limit? Is there any supported way for a shipping app to access more of the available RAM on 12 GB iPhone models? FB23183521

This year, new APIs were introduced such as l Metal Performance Shaders which provide access to high-performance Metal kernel. But this is still GPU. Are there APIs or plans to provide API and language to access ANE layer?

I’m delighted with the introduction of new color folders. Although, I can’t help but wonder why we still need both color folders and tags. Aren’t the color folders sufficient for our needs?

https://github.com/pytorch/pytorch/issues/132624 this fp8 dtypes unsupport issue has been existed for 2 years, does mlx have any plan to it?

Area: CoreML / Machine Learning Describe the issue: On iOS 26.4, calling MLModel(contentsOf:configuration:) to load an .mlpackage model hangs indefinitely and eventually kills the app via watchdog. The same model loads and runs inference successfully in under 1 second on iOS 26.3.1. The hang occurs inside eort_eo_compiler_compile_from_ir_program (espresso) during on-device AOT recompilation triggered by MLE5ProgramLibraryOnDeviceAOTCompilationImpl createProgramLibraryHandleWithRespecialization:error:. A C++ exception (__cxa_throw) is thrown inside libBNNS.dylib during the exception unwind, which then hangs inside __cxxabiv1::dyn_cast_slow and __class_type_info::search_below_dst. Swift's try/catch does not catch this — the exception originates in C++ and the process hangs rather than terminating cleanly. Setting config.computeUnits = .cpuOnly does not resolve the issue. MLE5ProgramLibrary initialises as shared infrastructure regardless of compute units. Steps to reproduce: Create an app with an .mlpackage CoreML model using the MLE5/espresso backend Call MLModel(contentsOf: modelURL, configuration: config) at runtime Run on a device on iOS 26.3.1 — loads successfully in <1 second Update device to iOS 26.4 — hangs indefinitely, app killed by watchdog after 60–745 seconds Expected behaviour: Model loads successfully, or throws a catchable Swift error on failure. Actual behaviour: Process hangs in MLE5ProgramLibrary.lazyInitQueue. App killed by watchdog. No Swift error thrown. Full stack trace at point of hang: Thread 1 Queue: com.apple.coreml.MLE5ProgramLibrary.lazyInitQueue (serial) frame 0: __cxxabiv1::__class_type_info::search_below_dst libc++abi.dylib frame 1: __cxxabiv1::(anonymous namespace)::dyn_cast_slow libc++abi.dylib frame 2: ___lldb_unnamed_symbol_23ab44dd4 libBNNS.dylib frame 23: eort_eo_compiler_compile_from_ir_program espresso frame 24: -[MLE5ProgramLibraryOnDeviceAOTCompilationImpl createProgramLibraryHandleWithRespecialization:error:] CoreML frame 25: -[MLE5ProgramLibrary _programLibraryHandleWithForceRespecialization:error:] CoreML frame 26: __44-[MLE5ProgramLibrary prepareAndReturnError:]_block_invoke CoreML frame 27: _dispatch_client_callout libdispatch.dylib frame 28: _dispatch_lane_barrier_sync_invoke_and_complete libdispatch.dylib frame 29: -[MLE5ProgramLibrary prepareAndReturnError:] CoreML frame 30: -[MLE5Engine initWithContainer:configuration:error:] CoreML frame 31: +[MLE5Engine loadModelFromCompiledArchive:modelVersionInfo:compilerVersionInfo:configuration:error:] CoreML frame 32: +[MLLoader _loadModelWithClass:fromArchive:modelVersionInfo:compilerVersionInfo:configuration:error:] CoreML frame 45: +[MLModel modelWithContentsOfURL:configuration:error:] CoreML frame 46: @nonobjc MLModel.__allocating_init(contentsOf:configuration:) GKPersonalV2 frame 47: MDNA_GaitEncoder_v1_3.__allocating_init(contentsOf:configuration:) frame 48: MDNA_GaitEncoder_v1_3.__allocating_init(configuration:) frame 50: GaitModelInference.loadModel() frame 51: GaitModelInference.init() iOS version: Reproduced on iOS 26.4. Works correctly on iOS 26.3.1. Xcode version: 26.2 Device: iPhone (model used in testing) Model format: .mlpackage

We’ve encountered what appears to be a CoreML regression between macOS 26.0.1 and macOS 26.1 Beta. In macOS 26.0.1, CoreML models run and produce correct results. However, in macOS 26.1 Beta, the same models produce scrambled or corrupted outputs, suggesting that tensor memory is being read or written incorrectly. The behavior is consistent with a low-level stride or pointer arithmetic issue — for example, using 16-bit strides on 32-bit data or other mismatches in tensor layout handling. Reproduction Install ON1 Photo RAW 2026 or ON1 Resize 2026 on macOS 26.0.1. Use the newest Highest Quality resize model, which is Stable Diffusion–based and runs through CoreML. Observe correct, high-quality results. Upgrade to macOS 26.1 Beta and run the same operation again. The output becomes visually scrambled or corrupted. We are also seeing similar issues with another Stable Diffusion UNet model that previously worked correctly on macOS 26.0.1. This suggests the regression may affect multiple diffusion-style architectures, likely due to a change in CoreML’s tensor stride, layout computation, or memory alignment between these versions. Notes The affected models are exported using standard CoreML conversion pipelines. No custom operators or third-party CoreML runtime layers are used. The issue reproduces consistently across multiple machines. It would be helpful to know if there were changes to CoreML’s tensor layout, precision handling, or MLCompute backend between macOS 26.0.1 and 26.1 Beta, or if this is a known regression in the current beta.

While analyzing system power on an M4 Max under GPU-heavy compute workloads, I noticed that the the GPU power reported by powermetrics does not come anywhere close to total system DC power reported by the SMC counter PDTR (as used by utilities like mactop). For example, in a heavy GPU workload, powermetrics would report a 65W idle-load delta on the GPU, but at the same time system DC power would rise by 179W, leaving 114W or nearly 2/3 of total system DC power on a Mac Studio M4 Max unexplained. From measurements, the difference appears to correlate with the amount of on-chip data movement (for example, varying bytes-per-FLOP in the workload changes the observed gap). Using SMC and IOReport, I was able to reverse engineer an energy model for the GPU that explains almost all of the energy flow with less than 2% error on the workload I studied. The result is a simple two-term energy roofline model: P_GPU (GPU_combined term in the plot) ≈ a * bytes + b * FLOPs with: ~5 pJ/byte for SRAM movement ~2.7 pJ/FLOP for compute. Has anyone observed similar behavior, or is there guidance on how GPU power reported by IOReport/powermetrics should be interpreted relative to total system power? In particular, I’m interested in whether certain classes of GPU activity may not be attributed to the GPU component in IOReport. Full details with the methodology and results are available here: https://youtu.be/HKxIGgyeISM

In macOS Tahoe 26.2 an RDMA capability was added for Thunderbolt-5 interfaces. This has been demonstrated to significantly decrease the latency and maintain bandwidth for "clustered" Apple Silicon devices with TB5. What is the ideal and the maximum RDMA burst width for transfers over RDMA-enabled Thunderbolt-5 interfaces?

Environment: Hardware: Mac M4 OS: macOS Sequoia 15.7.4 TensorFlow-macOS Version: 2.16.2 TensorFlow-metal Version: 1.2.0 Description: When using the tensorflow-metal plug-in for GPU acceleration on M4, the ReLU activation function (both as a layer and as an activation argument) fails to correctly clip negative values to zero. The same code works correctly when forced to run on the CPU. Reproduction Script: import os import numpy as np import tensorflow as tf # weights and biases = -1 weights = [np.ones((10, 5)) * -1, np.ones(5) * -1] # input = 1 data = np.ones((1, 10)) # comment this line => GPU => get negative values # uncomment this line => CPU => no negative values # tf.config.set_visible_devices([], 'GPU') # create model model = tf.keras.Sequential([ tf.keras.layers.Input(shape=(10,)), tf.keras.layers.Dense(5, activation='relu') ]) # set weights model.layers[0].set_weights(weights) # get output output = model.predict(data) # check if negative is present print(f"min value: {output.min()}") print(f"is negative present? {np.any(output < 0)}")

Subject: Technical Report: Float32 Precision Ceiling & Memory Fragmentation in JAX/Metal Workloads on M3 To: Metal Developer Relations Hello, I am reporting a repeatable numerical saturation point encountered during sustained recursive high-order differential workloads on the Apple M3 (16 GB unified memory) using the JAX Metal backend. Workload Characteristics: Large-scale vector projections across multi-dimensional industrial datasets Repeated high-order finite-difference calculations Heavy use of jax.grad and lax.cond inside long-running loops Observation: Under these conditions, the Metal/MPS backend consistently enters a terminal quantization lock where outputs saturate at a fixed scalar value (2.0000), followed by system-wide NaN propagation. This appears to be a precision-limited boundary in the JAX-Metal bridge when handling high-order operations with cubic time-scale denominators. have identified the specific threshold where recursive high-order tensor derivatives exceed the numerical resolution of 32-bit consumer architectures, necessitating a migration to a dedicated 64-bit industrial stack. I have prepared a minimal synthetic test script (randomized vectors only, no proprietary logic) that reliably reproduces the allocator fragmentation and saturation behavior. Let me know if your team would like the telemetry for XLA/MPS optimization purposes. Best regards, Alex Severson Architect, QuantumPulse AI

Hi everyone, I am developing a benchmarking tool to measure memory latency (L1/L2/DRAM) on Apple Silicon. I am currently using Xcode Instruments (CPU Counters) to validate my results. In my latest run for a 128 MB buffer with random access, Instruments shows: Latency (cycles): ~259 cycles (derived from LDST_UNIT_OLD_L1D_CACHE_MISS / L1D_CACHE_MISS_LD). Manual Timer Result: ~80 ns. To correlate these two values, I need the exact CPU Frequency (GHz) at the time of the sample. My Questions: Is there a recommended way to programmatically fetch the current frequency of the Performance cores (p-cores) during a benchmark run? Does Apple provide a "nominal" frequency value for M-series chips that we should use for cycle-to-nanosecond conversions? In Instruments, is there a hidden counter or "Average Frequency" metric that I can enable to avoid manual math? Hardware/Software Environment: Tool: Instruments 26.3+ (CPU Counters Template). Chip: A19, iPhone 17 pro. OS: 26.3.

After exerting a custom model with nms=True. In Xcode, the outputs show as: confidence: MultiArray (0 × 5) coordinates: MultiArray (0 × 4) I want to set fixed shapes (e.g., 100 × 5, 100 × 4), but Xcode does not allow editing—the shape fields are locked. The model graph shows both outputs come directly from a NonMaximumSuppression layer. Is it possible to set fixed output dimensions for NMS outputs in CoreML?

what is the diff between INST_ALL and Instructions(FIXED_INSTRUCTIONS)? also CORE_ACTIVE_CYCLE VS Cycles(FIXED_CYCLES)

Hi everyone, I believe I’ve encountered a potential bug or a hardware alignment limitation in the Core ML Framework / ANE Runtime specifically affecting the new Stateful API (introduced in iOS 18/macOS 15). The Issue: A Stateful mlprogram fails to run on the Apple Neural Engine (ANE) if the state tensor dimensions (specifically the width) are not a multiple of 32. The model works perfectly on CPU and GPU, but fails on ANE both during runtime and when generating a Performance Report in Xcode. Error Message in Xcode UI: "There was an error creating the performance report Unable to compute the prediction using ML Program. It can be an invalid input data or broken/unsupported model." Observations: Case A (Fails): State shape = (1, 3, 480, 270). Prediction fails on ANE. Case B (Success): State shape = (1, 3, 480, 256). Prediction succeeds on ANE. This suggests an internal memory alignment or tiling issue within the ANE driver when handling Stateful buffers that don't meet the 32-pixel/element alignment. Reproduction Code (PyTorch + coremltools): import torch.nn as nn import coremltools as ct import numpy as np class RNN_Stateful(nn.Module): def __init__(self, hidden_shape): super(RNN_Stateful, self).__init__() # Simple conv to update state self.conv1 = nn.Conv2d(3 + hidden_shape[1], hidden_shape[1], kernel_size=3, padding=1) self.conv2 = nn.Conv2d(hidden_shape[1], 3, kernel_size=3, padding=1) self.register_buffer("hidden_state", torch.ones(hidden_shape, dtype=torch.float16)) def forward(self, imgs): self.hidden_state = self.conv1(torch.cat((imgs, self.hidden_state), dim=1)) return self.conv2(self.hidden_state) # h=480, w=255 causes ANE failure. w=256 works. b, ch, h, w = 1, 3, 480, 255 model = RNN_Stateful((b, ch, h, w)).eval() traced_model = torch.jit.trace(model, torch.randn(b, 3, h, w)) mlmodel = ct.convert( traced_model, inputs=[ct.TensorType(name="input_image", shape=(b, 3, h, w), dtype=np.float16)], outputs=[ct.TensorType(name="output", dtype=np.float16)], states=[ct.StateType(wrapped_type=ct.TensorType(shape=(b, ch, h, w), dtype=np.float16), name="hidden_state")], minimum_deployment_target=ct.target.iOS18, convert_to="mlprogram" ) mlmodel.save("rnn_stateful.mlpackage") Steps to see the error: Open the generated .mlpackage in Xcode 16.0+. Go to the Performance tab and run a test on a device with ANE (e.g., iPhone 15/16 or M-series Mac). The report will fail to generate with the error mentioned above. Environment: OS: macOS 15.2 Xcode: 16.3 Hardware: M4 Has anyone else encountered this 32-pixel alignment requirement for StateType tensors on ANE? Is this a known hardware constraint or a bug in the Core ML runtime? Any insights or workarounds (other than manual padding) would be appreciated.

I am running some experiments with WebGPU using the wgpu crate in rust. I have some Buffers already allocated in the GPU. Is it possible to use those already existing buffers directly as inputs to a predict call in CoreML? I want to prevent gpu to cpu download time as much as possible. Or are there any other ways to do something like this. Is this only possible using the latest Tensor object which came out with Metal 4 ?

Hello, I am currently working on a research project under ENINCA Consulting, focused on advanced diagnostic tools for pseudorandom number generators (structural metrics, multi-seed stability, cross-architecture reproducibility, and complementary indicators to TestU01). To validate this diagnostic framework, I prototyped a small non-linear 64-bit PRNG (not as a goal in itself, but simply as a vehicle to test the methodology). During these evaluations, I observed something interesting on Apple Silicon (Mac M1): • bit-exact reproducibility between M1 ARM CPU and M1 Metal GPU, • full BigCrush pass on both CPU and Metal backends, • excellent p-values, • stable behaviour across multiple seeds and runs. This was not the intended objective, the goal was mainly to validate the diagnostic concepts, but these results raised some questions about deterministic compute behaviour in Metal. My question: Is there any official guidance on achieving (or expecting) deterministic RNG or compute behaviour across CPU ↔ Metal GPU on Apple Silicon? More specifically: • Are deterministic compute kernels expected or guaranteed on Metal for scientific workloads? • Are there recommended patterns or best practices to ensure reproducibility across GPU generations (M1 → M2 → M3 → M4)? • Are there known Metal features that can introduce non-determinism? I am not sharing the internal recurrence (this work is proprietary), but I can discuss the high-level diagnostic observations if helpful. Thank you for any insight, very interested in how the Metal engineering team views deterministic compute patterns on Apple Silicon. Pascal ENINCA Consulting

果敢腾 龙企业有限公司，薇---184-7933-278成立于2006年8月12日，下面给大家介绍一下公司主要位置和主要经营那些行业，腾龙公司位置靠于云南省临沧市边联的一个小城市，名称果敢老街，实时位置果敢城市中心双峰塔附近，公司主要是经营，旅游业，酒店服务行业，建筑业，科技游戏，餐饮，等等，这个有名的小城市虽然不是很大，但有着纸醉金迷小澳门的名誉之称呼，今天给大家介绍的就这些，如果你有想旅游的心，也欢迎来这个小城市旅游，来感受一下这里的民族风情。

Hello All, I’m a mobile-app developer working with iOS 26+ and I’m using BGContinuedProcessingTask to perform background work. My app’s workflow includes the following business logic: Loading images via PHImageRequest. Using a CLIP model to extract image embeddings. Using an .mlmodel-based model to further process those embeddings. For both model inferences I set computeUnits = .cpuAndNeuralEngine. When the app is moved to the background, I observe that the same workload(all three workload) becomes on average 4-5× slower than when the app is in the foreground. In an attempt to diagnose the slowdown, I tried to profile with Xcode Instruments, but since a debugger was attached, the performance in background appeared nearly identical to foreground. Even when I detached the debugger, the measured system resource metrics (process CPU usage, system CPU usage, memory, QoS class, thermal state) showed no meaningful difference. Below are some of the metrics I captured: Process CPU: 177% (Foreground) → 153% (Background) → ~-24.1% Still >1.5 cores of work. System CPU: 56.1% → 38.4% → ~-17.7% Process Memory: 244.8 MB → 218.1 MB QoS Class: userInitiated in both cases Thermal State: nominal in both cases Given these results, I’m finding it hard to pinpoint why the overall latency is so much worse when the app is backgrounded, even though the obvious metrics show little variation. I suspect the cause may involve P-core vs E-core scheduling, or internal hardware throttling/limit of Neural Engine usage, but I cannot find clear documentation or logging to confirm this. My question is: Does anyone know why a CPU (and Neural Engine)-intensive job like this would slow down so dramatically when using BGContinuedProcessingTask in the background on iOS 26+, despite apparent similar resource-usage metrics? Are there internal iOS scheduling/hardware-allocation behaviors (e.g., falling back to lower-performing cores when backgrounded) that might explain this? Any pointers to Apple technical notes, system logs, or instrumentation I might use to detect which cores or compute units are being used would be greatly appreciated. Thank you for your time and any guidance you can provide. Best regards,