tensorflow-metal ReLU activation f… | Apple Developer Forums

tensorflow-metal ReLU activation fails to clip negative values on M4 Apple Silicon

Environment:

Hardware: Mac M4

OS: macOS Sequoia 15.7.4

TensorFlow-macOS Version: 2.16.2

TensorFlow-metal Version: 1.2.0

Description:

When using the tensorflow-metal plug-in for GPU acceleration on M4, the ReLU activation function (both as a layer and as an activation argument) fails to correctly clip negative values to zero. The same code works correctly when forced to run on the CPU.

Reproduction Script:

import os
import numpy as np
import tensorflow as tf

# weights and biases = -1
weights = [np.ones((10, 5)) * -1, np.ones(5) * -1]

# input = 1
data = np.ones((1, 10))

# comment this line => GPU => get negative values
# uncomment this line => CPU => no negative values
# tf.config.set_visible_devices([], 'GPU') 

# create model
model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(10,)),
    tf.keras.layers.Dense(5, activation='relu')
])

# set weights
model.layers[0].set_weights(weights)

# get output
output = model.predict(data)

# check if negative is present
print(f"min value: {output.min()}")
print(f"is negative present? {np.any(output < 0)}")