URLSession connection upgrade to stream task loses early data

Hi there. I'm doing some initial prototyping of using URLSession's connection upgrade mechanism to stream some arbitrary data back and forth between a client and server. Ultimately we're using this to get MQTT data through http proxies, with additional websocket framing on top because nginx has some built in support for that. The specific framing/protocol is kind of an implementation detail for the purposes of this question though; I'm just including for a bit of background.


The two main references I'm using as a starting point here are the WWDC video from a few years back where the connection upgrade stuff was first introduced, https://developer.apple.com/videos/play/wwdc2015/711/, which talks specifically about upgrading to a stream task, and https://developer.apple.com/documentation/foundation/nsurlsessionstreamtask which links specifically to the web sockets and tls-upgrade rfcs.


I have a dummy iOS app (which I can attach if required) which kind of works, but I'm running into an issue with losing some of the early bytes after upgrading the connection.


What I'm doing right now:


* Creating a URLSession data task, with httpShouldUsePipelining turned off on the session configuration

* Setting a few headers so the server will recognise this and upgrade the connection on its end (e.g. "Upgrade: websocket", and a few others)

* Calling resume() on the data task to kick off the initial request

* Handing the urlSession(_:dataTask:didReceive:completionHandler:) to handle the point we get the initial http response back, and returning .becomeStream to upgrade to a stream task

* Handling the subsequent call to urlSession(_:dataTask:didBecome:) which gives me the URLSessionStreamTask to read/write from.

* Immediately calling read on the stream task with a minimum of 1 byte and a large max


The server is sending a 101 upgrade response which ultimate is just


HTTP/1.1 101 Switching Protocols
Upgrade: WebSocket
Connection: Upgrade

Though I don't believe URLSession really cares about the specifics of this.


After this http response, the server is immediately writing some websocket data down to the client. This seems to be causing a problem though: these first early bytes are getting dropped somewhere and the read() call on the stream task is never completing. Though this is inconsistent - sometimes a few of the trailing bytes will get through, sometimes not.


If I use netcat as the server and paste in the http response myself, wait a second, then start typing some stuff, the bytes are not missed and the stream read() calls return all the data as expected.


So it seems like some of the first bytes after the HTTP response are getting swallowed up. I thought perhaps URLSession was expecting a body as part of the initial data task response and swallowing the bytes up for that reason, but if I simulate a response with a Content-Length: 1 and a dummy byte before writing the websocket data, this doesn't help and we still miss stuff.


So a couple of questions:


* Fundamentally, is what I'm doing reasonable and supported?

* Any idea why I might be missing these first few bytes? It seems like a timing issue of some sort with the upgrade mechanism.

off topic and format

After wiresharking it appears that any data included in the same packet as the HTTP response gets dropped / swallowed up by the URLSession internals, and is not available for reading through the resulting stream task.


I think I'll submit a bug report and sample project for this. It's a shame this doesn't seem to be used by anybody and all the open source implementations of websockets end up implementing HTTP response parsing and proxy support themselves (and often in very broken ways).


There is also another issue in that URLSession doesn't let you set or add a "Connection" header with the value "upgrade" which is required by a lot of server-side websocket implementations. It always overrides it to be "keepalive". Someone posted a similar thread for this issue years ago on https://forums.developer.apple.com/message/128666


Feels to me like this is kind of broken and not used.

I think I'll submit a bug report and sample project for this.

Thanks! Please post the bug number, just for the record.

Feels to me like this is kind of broken and not used.

Agreed. While converting to a stream task seems like an easy way to implement WebSocket, it turns out that there are critical parts missing that make this impossible. Most disappointing.

You can find more background about this on this thread on the MacNetworkProg mailing list.

One thing to note here is that the Network framework, part of the current OS betas, includes a lot of the smarts used by

NSURLSession
to deal with TLS, proxies, and so on. That’ll provide better infrastructure for a WebSocket implementation (especially when compared to
CFSocketStream
, which is what most folks use) but you still end up needing to format and parse HTTP messages (although
CFHTTPMessage
helps with that).

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"
  • Submitted rdar://42892506 for the problem with upgrading to stream tasks dropping data in the same packet as the HTTP response.
  • Submitted rdar://42892813 for the inability to set the Connection header to Upgrade
URLSession connection upgrade to stream task loses early data
 
 
Q