Environment:
Hardware: Mac M4
OS: macOS Sequoia 15.7.4
TensorFlow-macOS Version: 2.16.2
TensorFlow-metal Version: 1.2.0
Description:
When using the tensorflow-metal plug-in for GPU acceleration on M4, the ReLU activation function (both as a layer and as an activation argument) fails to correctly clip negative values to zero. The same code works correctly when forced to run on the CPU.
Reproduction Script:
import os
import numpy as np
import tensorflow as tf
# weights and biases = -1
weights = [np.ones((10, 5)) * -1, np.ones(5) * -1]
# input = 1
data = np.ones((1, 10))
# comment this line => GPU => get negative values
# uncomment this line => CPU => no negative values
# tf.config.set_visible_devices([], 'GPU')
# create model
model = tf.keras.Sequential([
tf.keras.layers.Input(shape=(10,)),
tf.keras.layers.Dense(5, activation='relu')
])
# set weights
model.layers[0].set_weights(weights)
# get output
output = model.predict(data)
# check if negative is present
print(f"min value: {output.min()}")
print(f"is negative present? {np.any(output < 0)}")
2
0
454