Low level API to take control of Neural Engine

Question

joel2001k OP

Created Feb ’21

Replies 4

Boosts 3

Views 8.6k

Participants 3

Hi, I would love to code with the Neural Engine on my macbook pro M1 2020.

Is there any low-level API to create my very own work-loads?

I am working with audio and MIDI. As well sound synthesis and mixing.

Can I use the Neural Engine to offload the CPU? I am especially interested in parallelism using threads.

My programming lanuage of choice is ANSI C and Objective C.

Answered by joel2001k in 661559022

This seems to be what I need. Vector operations and such.

https://developer.apple.com/library/archive/documentation/Performance/Conceptual/vDSP_Programming_Guide/Introduction/Introduction.html

and

https://developer.apple.com/documentation/accelerate/veclib

The Neural Engine looks versatile.

Boost

Answer 1

joel2001k OP

Feb ’21

Accepted Answer

This seems to be what I need. Vector operations and such.

https://developer.apple.com/library/archive/documentation/Performance/Conceptual/vDSP_Programming_Guide/Introduction/Introduction.html

and

https://developer.apple.com/documentation/accelerate/veclib

The Neural Engine looks versatile.

0

Answer 2

c0key OP

Apr ’21

Does vecLib use the Neural Engine, or it executes on the GPU?

0

Answer 3

Fee_Lynn OP

Aug ’21

I feel you have a bit wrong idea what Neural Engine is. It have nothing common to CPU or GPU - it's just a machine learning accelerator with very limited area of application - even not every layer type in your model can be used. Look at This FAQ for better understanding what it is and what it can.

Q: Is there any low-level API to create my very own work-loads?

A: Yes and No. Low level AppleNeuralEngine.framework is private to Apple and you can't use it.

But:

take a look at ANE Tools - compiler and decompiler for Neural Engine. Also there is coremltools - this will help to interface with TensorFlow and PyTorch

Q: Can I use the Neural Engine to offload the CPU? I am especially interested in parallelism using threads.

A: Basically - No. ANE can't execute CPU/GPU code and don't have threads. It operates with layer connectivity map and net weights.

One more thing - vDSP and veclib DONT use ANE.

1

Answer 4

Fee_Lynn OP

Aug ’21

UPDATE: Famous reverse engineer Geohot exposed some examples of using AppleH11ANEInterface

Also you can look at Wish Wu presentation on BlackHat Asia 2021 about ANE internals. Sorry, Apple not allows to paste it's link or attach file here - use google.

Based on these two sources, you can conclude that only thing ANE can do well - convolutions. Perhaps it was intentionally designed as convolution accelerator. Other AI/ML things, even so easy as ReLU is 20 times slower than CPU.

1