ML Compute

RSS for tag

Accelerate training and validation of neural networks using the CPU and GPUs.

Posts under ML Compute tag

27 Posts

Post

Replies

Boosts

Views

Activity

Using the Apple Neural Engine for MLTensor operations
Based on the documentation, it appears that MLTensor can be used to perform tensor operations using the ANE (Apple Neural Engine) by wrapping the tensor operations with withMLTensorComputePolicy with a MLComputePolicy initialized with MLComputeUnits.cpuAndNeuralEngine (it can also be initialized with MLComputeUnits.all to let the OS spread the load between the Neural Engine, GPU and CPU). However, when using the Instruments app, it appears that the tensor operations never get executed on the Neural Engine. It would be helpful if someone can guide me on the correct way to ensure that the Nerual Engine is used to perform the tensor operations (not as part of a CoreML model file). based on this example, I've created a simple code to try it: import Foundation import CoreML print("Starting...") let semaphore = DispatchSemaphore(value: 0) Task { await withMLTensorComputePolicy(.init(MLComputeUnits.cpuAndNeuralEngine)) { let v1 = MLTensor([1.0, 2.0, 3.0, 4.0]) let v2 = MLTensor([5.0, 6.0, 7.0, 8.0]) let v3 = v1.matmul(v2) await v3.shapedArray(of: Float.self) // is 70.0 let m1 = MLTensor(shape: [2, 3], scalars: [ 1, 2, 3, 4, 5, 6 ], scalarType: Float.self) let m2 = MLTensor(shape: [3, 2], scalars: [ 7, 8, 9, 10, 11, 12 ], scalarType: Float.self) let m3 = m1.matmul(m2) let result = await m3.shapedArray(of: Float.self) // is [[58, 64], [139, 154]] // Supports broadcasting let m4 = MLTensor(randomNormal: [3, 1, 1, 4], scalarType: Float.self) let m5 = MLTensor(randomNormal: [4, 2], scalarType: Float.self) let m6 = m4.matmul(m5) print("Done") return result; } semaphore.signal() } semaphore.wait() Here's what I get on the Instruments app: Notice how the Neural Engine line shows no usage. Ive run this test on an M1 Max MacBook Pro.
2
4
778
Mar ’25
Troubleshooting Apple Vision Framework Errors
When working on the project "Analyzing a Selfie and Visualizing Its Content" from Apple's documentation, I downloaded the project and opened it in Xcode. However, I encountered the following error: VTEST: error: perform(_:): inside 'for await result in resultStream' error: internalError("Error Domain=com.apple.Vision Code=9 \"Could not create inference context\" UserInfo={NSLocalizedDescription=Could not create inference context}") VTEST: error: DetectFaceRectanglesRequest was cancelled. VTEST: error: DetectFaceRectanglesRequest was cancelled. Error Domain=com.apple.Vision Code=9 "Could not create inference context" UserInfo={NSLocalizedDescription=Could not create inference context} How can I resolve this issue? Thanks in advance!
1
0
388
Feb ’25
The yolo11 object detection model I exported to coreml stopped working in macOS15.2 beta.
After updating to macOS15.2beta, the Yolo11 object detection model exported to coreml outputs incorrect and abnormal bounding boxes. It also doesn't work in iOS apps built on a 15.2 mac. The same model worked fine on macOS14.1. When training a Yolo11 custom model in Python, exporting it to coreml, and testing it in the preview tab of mlpackage on macOS15.2 and Xcode16.0, the above result is obtained.
6
1
1.4k
Feb ’25
Broken compatibility in tensorflow-metal with tensorflow 2.18
Issue type: Bug TensorFlow metal version: 1.1.1 TensorFlow version: 2.18 OS platform and distribution: MacOS 15.2 Python version: 3.11.11 GPU model and memory: Apple M2 Max GPU 38-cores Standalone code to reproduce the issue: import tensorflow as tf if __name__ == '__main__': gpus = tf.config.experimental.list_physical_devices('GPU') print(gpus) Current behavior Apple silicone GPU with tensorflow-metal==1.1.0 and python 3.11 works fine with tensorboard==2.17.0 This is normal output: /Users/mspanchenko/anaconda3/envs/cryptoNN_ml_core/bin/python /Users/mspanchenko/VSCode/cryptoNN/ml/core_second_window/test_tensorflow_gpus.py [PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')] Process finished with exit code 0 But if I upgrade tensorflow to 2.18 I'll have error: /Users/mspanchenko/anaconda3/envs/cryptoNN_ml_core/bin/python /Users/mspanchenko/VSCode/cryptoNN/ml/core_second_window/test_tensorflow_gpus.py Traceback (most recent call last): File "/Users/mspanchenko/VSCode/cryptoNN/ml/core_second_window/test_tensorflow_gpus.py", line 1, in <module> import tensorflow as tf File "/Users/mspanchenko/anaconda3/envs/cryptoNN_ml_core/lib/python3.11/site-packages/tensorflow/__init__.py", line 437, in <module> _ll.load_library(_plugin_dir) File "/Users/mspanchenko/anaconda3/envs/cryptoNN_ml_core/lib/python3.11/site-packages/tensorflow/python/framework/load_library.py", line 151, in load_library py_tf.TF_LoadLibrary(lib) tensorflow.python.framework.errors_impl.NotFoundError: dlopen(/Users/mspanchenko/anaconda3/envs/cryptoNN_ml_core/lib/python3.11/site-packages/tensorflow-plugins/libmetal_plugin.dylib, 0x0006): Symbol not found: __ZN3tsl8internal10LogMessageC1EPKcii Referenced from: <D2EF42E3-3A7F-39DD-9982-FB6BCDC2853C> /Users/mspanchenko/anaconda3/envs/cryptoNN_ml_core/lib/python3.11/site-packages/tensorflow-plugins/libmetal_plugin.dylib Expected in: <2814A58E-D752-317B-8040-131217E2F9AA> /Users/mspanchenko/anaconda3/envs/cryptoNN_ml_core/lib/python3.11/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so Process finished with exit code 1
3
3
1.7k
Feb ’25
Code with Swift Assist
Hello, I would like to inquire about the release date of Swift Assist’s beta version. Apple has stated that it will be released later this year, but they have not provided a specific date or time. Could you please provide information on the beta version’s release date? Additionally, is there a trial version available? If so, when was it released? Thank you for your assistance.
2
1
2.5k
Jan ’25
Segmentation Fault in np.matmul on macOS 15.2 with Accelerate BLAS
I'm encountering a segmentation fault when using np.matmul with relatively small arrays on macOS 15.2. The issue only occurs in specific scenarios and results in a crash with the following error: Exception Type: EXC_BAD_ACCESS (SIGSEGV) Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000110 Termination Reason: Namespace SIGNAL, Code 11 Segmentation fault: 11 Full error log: Gist link The crash consistently occurs on a specific line where np.matmul is called, despite similar np.matmul operations succeeding earlier in the same script. The issue cannot be reproduced in a separate script that contains identical operations. When I build the NumPy wheel using OpenBLAS, this issue no longer arises, which leads me to believe that it is related to a problem with Accelerate. Environment NumPy Version: 2.1.3 Python Version: 3.12.7 OS Version: macOS 15.2 BLAS Configuration: Build Dependencies: blas: detection method: system found: true include directory: unknown lib directory: unknown name: accelerate openblas configuration: unknown pc file directory: unknown version: unknown lapack: detection method: system found: true include directory: unknown lib directory: unknown name: accelerate openblas configuration: unknown pc file directory: unknown version: unknown Compilers: c: commands: cc linker: ld64 name: clang version: 15.0.0 c++: commands: c++ linker: ld64 name: clang version: 15.0.0 cython: commands: cython linker: cython name: cython version: 3.0.11 Machine Information: build: cpu: aarch64 endian: little family: aarch64 system: darwin host: cpu: aarch64 endian: little family: aarch64 system: darwin
1
0
471
Jan ’25
How to confirm whether CreatML is training
I am currently training a Tabular Classification model in CreatML. The dataset comprises 30 features, including 1,000,000 training data points and 1,000,000 verification data points. Could you please estimate the approximate training time for an M4Max MacBook Pro? During the training process, CreatML has been displaying the “Processing” status, but there is no progress bar. I would like to ascertain whether the training is still ongoing, as I have often suspected that it has ceased.
1
0
653
Jan ’25