WatchConnectivity Session becomes unreachable but shouldn't

So for context I am building an app where the Apple Watch establishes a WatchConnectivity session with the parent iPhone app and streams audio data from the watch Mic to the iPhone for processing on Deepgram to perform STT.

This works very well, unless I tilt my wrist and the display goes to sleep. What I find strange is that due to the mic being in use on my watch, the app is still showing on the always on display and is still trying to send the audio data to my phone in the background (which is perfect!) but the iPhone does not seem to be responding.

So my watch code is:

private func sendData(_ data: Data) {
        let dict: [String: Any] = ["audioData": data]
        session.sendMessage(dict, replyHandler: nil, errorHandler: { error in
            print("Failed to send data: \(error.localizedDescription)")
        })
    }

and my Xcode logs Failed to send data: WatchConnectivity session on paired device is not reachable.

So the watch is still running the process but because session.isReachable is false the audio data is not sent which it should be!

Is there any way to keep the connection established and data sharing when the display dims?