It's clearly stated at https://developer.apple.com/metal/tensorflow-plugin/ that multiple GPUs are still not supported by Tensorflow-mac, but I've toying around with some examples and benchmarking it's performance against Colab and Intel/NVIDIA setups and got this testing a NLP classification task (time per epoch while training, smaller is better):
My question is: can we expect, once multi-GPU is available for the M1, an increase in performance - maybe close to 8x if the 8 GPU cores become available, and would the GPU cores will be seen as a single GPU as the NVIDIA cards or we will need to use a distribution strategy to be able to use them in parallel?
Thanks!
