Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.

The NUMA error message for an Apple Silicon computer is benign and can be ignored. Apple silicon memory is UMA (unified memory architecture) not NUMA

Experiencing the same issue February 2023, running MacOS Ventura 13.2 on 2021 M1 Pro.

Same here M1 Pro Ventura 13.1

For me the fix was matching the current version of the tensoflow-macos with tensorflor-metal If you scroll down here -> https://developer.apple.com/metal/tensorflow-plugin/ You'll reach the "Release" section And then the compatible tensorflow-macos and metal plugin are there showed, for example in order to use the metal plugin with the version 0.5 you've to manually set the version of tensorflow as 2.9.0

python -m pip install tensorflow-macos==2.90
pip install tensorflow-metal==0.5.0

New tensorflow-metal 0.7.1 was released on 8 Feb 2023 !

It is with regret to tell you that it solves nothing, the annoying and useless NUMA etc. warnings are still there, and TF training fails on "OP_REQUIRES failed at xla_ops.cc".

[SOLUTION]

You should uninstall tensorflow-macos and tensorflow-metal because the main reason for this problem is the version conflict between these. After uninstalling, you should reinstall tensorflow-macos with target version 2.9 and install tensorflow-metal with target version 0.5.0 to fix this issue.

pip uninstall tensorflow-macos
pip uninstall tensorflow-metal
python -m pip install tensorflow-macos==2.9
pip install tensorflow-metal==0.5.0

Thanks 😇

Thanks, xrois, I did that, but tensorflow still fails.

Here's the result;

I'm trying to follow the script in Jupyter Notebooks from VSCode, which many people must also be trying, as they learn VSCode...and tensorflow fails.

Here's the script from code.visualstudio.com: https://code.visualstudio.com/docs/datascience/data-science-tutorial#_prepare-the-data

But when I run this, following each step, tensorflow on my MacBook Air, M1, 2020, FAILS.

I followed xrois's suggestions to install 2.9 and 0.5.0, but it still failed. I'll reinstall the tensor-flow metal 0.7.1 just released, but I don't expect any resolution.

I'll report if it works.

Anyone have any success running this?

How can I report this in a verbose mode to tensorflow? Do they listen?

-------Here's the sequence----------

!python3 -m pip install tensorflow-macos==2.9 !pip3 install tensorflow-metal==0.5.0

• Run !pip3 list | tensor

tensorboard 2.9.1 tensorboard-data-server 0.6.1 tensorboard-plugin-wit 1.8.1 tensorflow-estimator 2.9.0 tensorflow-macos 2.9.0 tensorflow-metal 0.5.0

• Then set up model:

#run model again from keras.models import Sequential from keras.layers import Dense model = Sequential() model.add(Dense(5, kernel_initializer = 'uniform', activation = 'relu', input_dim = 5)) model.add(Dense(5, kernel_initializer = 'uniform', activation = 'relu')) model.add(Dense(1, kernel_initializer = 'uniform', activation = 'sigmoid'))

• Everything looks good:

Model: "sequential_3"

Layer (type) Output Shape Param #

dense_3 (Dense) (None, 5) 30

dense_4 (Dense) (None, 5) 30

dense_5 (Dense) (None, 1) 6

================================================================= Total params: 66 Trainable params: 66 Non-trainable params: 0

• then, FAILURE

• now, compile the model: will accuracy be %61?

model.compile(optimizer="adam", loss='binary_crossentropy', metrics=['accuracy']) model.fit(X_train, y_train, batch_size=32, epochs=50)

• RESULT: And it goes on with many code failures

Epoch 1/50 WARNING:tensorflow:AutoGraph could not transform <function Model.make_train_function..train_function at 0x298e87910> and will run it as-is. Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: closure mismatch, requested ('self', 'step_function'), but source function had () To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert WARNING: AutoGraph could not transform <function Model.make_train_function..train_function at 0x298e87910> and will run it as-is. Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: closure mismatch, requested ('self', 'step_function'), but source function had () To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert Unexpected exception formatting exception. Falling back to standard exception

Guys, look into this article about the UMA: https://www.cgdirector.com/apple-unified-memory-guide/ Just like some of the people said, the UMA is the edge not causing any problem if we implement our codes correctly. In fact, the performance shall be even better on a condition, you have enough memory for your project.

I'm getting the warning after running tensorflow.keras.model's Sequential() (that is, "...could not identify NUMA node on GPU...")

Still, TensorFlow seems to be working in Jupyter Notebook. running on a Apple M2, Ventura 13.5.1

For example, next I can compile the model from the Sequential call, and fit it:

history = model.fit(X_train, y_train, batch_size=100, epochs=10, verbose=2, validation_data=(X_test, y_test))

I do get a warning on the first two epochs but the subsequent others continue on without the warnings:

` ... 25/25 - 2s - loss: 0.6618 - accuracy: 0.5917 - val_loss: 0.6444 - val_accuracy: 0.7475 - 2s/epoch - 73ms/step Epoch 2/10

2023-09-22 15:01:03.984920: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled. ... `

My versions according to conda list shows:

python 3.11.4

tensorboard 2.13.0 tensorboard-data-server 0.7.1

tensorflow 2.13.0
tensorflow-estimator 2.13.0
tensorflow-macos 2.13.0 tensorflow-metal 1.0.1

Hope it helps.

Apple M1/M2 processors are endowed with an unified memory architecture (UMA) i.e. memory shared by all components including GPUs. They are not NUMA (Non-Unified Memory Architecture) where you will find different memory spaces e.g. RAM for CPUs/GPUs --> https://en.wikipedia.org/wiki/Apple_M1

Switching from python 3.09 to python 3.11 works for me in a M2 Pro