Apple Remote Desktop is completely unstable under 10.11.4 beta. Before this was using 10.11.3 Beta2 with 10.11.2 and was getting the same results. Before 10.11.3 Beta 2 things seemd to be a bit more stable.
So the issue isn't when connecting on a local network it's whenever connecting remotely and it has to rely on icloud to connect through. That's when ARD says the system is offline. I created a new partition on rMBP and mini and installed 10.11.2 and everything works perfectly fine.