NWConnection (web-socket) does not disconnect when server disconnects gracefully

Here is the code we use for reading data continuously on web socket. We only stop reading only when we encounter an error.

func listen() {
        connection?.receiveMessage { [weak self] (data, context, _, error) in
            guard let self = self else {
                return
            }

            if let data = data, !data.isEmpty, let context = context {
                self.receiveMessage(data: data, context: context)
            }

            if let error = error {
                self.reportErrorOrDisconnection(error)
            } else {
                self.listen()
            }
        }
    }

Hi @eskimo,

After our discussion last evening, we did some deliberations and now we are able to get FIN from server to client. However, we still notice that NWConnection does not close the connection after receiving FIN and subsequent writes are being sent to server. Is there any reason NWConnection is not closing web-socket after receiving FIN and ACKing to the FIN?

Our server has separate management of read and write web sockets. They are sending FIN after closing their write socket while keeping their read socket open for a little while longer so that client can write any last message or send FIN and close.

Could there be any reason/configuration for NWConnection to not disconnect by writing client FIN after receiving server FIN? In Windows client, we see client side FIN after server FIN and then connection closes completely.

The packet capture screenshot with server FIN can be seen https://ibb.co/M6cLz85 since image upload here was failing.

we are able to get FIN from server to client

Cool.

As to you follow-up questions, it’s probably best to open a DTS tech support incident for this so that I can allocate time to look into it in depth.

Note DTS is officially closed this week for the US Thanksgiving holiday. If you open an incident I won’t get it until next week.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Sure, I will open a TSI next week. Meanwhile, continuing my investigation, I looked into console logs and I can see SSL3_RT_ALERT, description: close notify. Which means Mac client is receiving TLS close but is not closing the connection. The screenshot for console log is available here: https://ibb.co/bXYrPT4

The server side calls SSL_shutdown which only closes the write direction from server side while keeping read side open.

SSL_shutdown() only closes the write direction. It is not possible to call SSL_write() after calling SSL_shutdown(). The read direction is closed by the peer.

So, as per the OpenSSL docs, server is doing right thing but NWConnection is not responding back with notify close, to finish the connection from both sides.

Finally I was able to solve this problem. Here are few things we learned and fixed:

1. At first we were not receiving the FIN,ACK message from server when connection was closed from server side. When we looked at server packet capture, we could see that server was sending FIN,ACK but was not receiving ACK from client. So, server was retransmitting FIN,ACK. Similarly, client was not aware of server closing the connection and when trying to send something to server was also not being ACKed by server and was being retransmitted. This was happening after 5 or more minutes of idle. We narrowed it down to a NAT issue where NAT device was losing the mapping of client IP after certain period of idle connection. So, after losing the mapping, both server and client were not able to reach each other and were in re-transmission. We fixed this by sending TCP keep-alive every one minute during idle connection.

2. After fixing that we started getting FIN,ACK from server but were not able to receive it in NWConnection callback. SSL_shutdown from server closes the write direction and sends a FIN,ACK to client. On client side, this is indicates by isFinal property of NWConnection.ContentContext received in receiveMessage completion handler. Our mistake was that we were expecting either data or error in the completion handler to be non-nil. Otherwise, we were discarding the call-back. But when FIN,ACK is sent by server, error and data, both are nil and isFinal in context is set. This gives the hint that server has closed the write channel and client side should also send any remaining message and close the connection. Once we implemented that, we were able to fix complete issue.

Thanks @eskimo for your help and suggestions. Let me know if you see any issue with above fixes or any improvements that can be made.

@dispatchMain , @eskimo could you please explain how we handle isFinal flag? Does that always indicate that server has closed the connect? I have noticed that whenever isFinal is set to true, we also receive [connection] nw_flow_add_read_request already delivered final read, cannot accept read requests event which indicates the server has closed the connection. Which is proved by another event we receive nw_socket_handle_socket_event Socket SO_ERROR [54: Connection reset by peer] exactly after 1 minute (i'm not sure why is there a delay of exactly 1 minute) so what i'm wondering is:1.

1. What are the cases when we want to set receive listener again and when we don't. Is isFinal: true when we don't want to listen again?
2. Why is nw_socket_handle_socket Socket SO_ERROR event sent with 1 minute delay when server has closed the connection 1 minute ago?

Any ideas?