Metal Performance Shaders produces from bf16/fp16 softmax for large attention tensors on , forcing the entire local generative-AI ecosystem to fall back to fp32. This is a . Environment
Minimal reproduction Decomposed softmax on MPS — diffs = x - maxes produces NaN even though every element is identical (result should be all zeros):
Real-world impact Running ComfyUI (the dominant local generative-AI UI) on M5 Max:
Root cause (per PyTorch MPS maintainers) PyTorch maintainers (@drisspg, @albanD) have traced this to in the MPS/MPSGraph fused kernels. The NaN originates in the x - maxes subtraction inside softmax, then propagates through the attention block and the entire network. PyTorch cannot fix this — it is in the Metal/MPS kernel layer. Not isolated
The M4-era fix ("fixed on macOS 15.1") did not survive onto M5, indicating the MPS fused-kernel precision fix was either reverted or not ported to the M5 GPU architecture. Request