iOS 11 Beta renders chunk of application inoperable

I filed Radar 33741310 …

Thanks for that.

Looking at the analysis of the bug so far it seems that:

• The bug is a real regression and is being treated as such

• The bug doesn’t seem to have a widespread impact; there’s something about your app that’s triggering it

• That “something” is likely to be your use of polled I/O

Most folks using NS/CFStream use it in one of two modes:

• Async mode, where they set up a delegate and wait for events to be delivered via that delegate

• Sync mode, where they just call

  read(_:maxLength:)

  and block waiting for data

Note Neither of these options every require you to call the

hasBytesAvailable

property, and I generally recommend that you avoid that property at all times. Likewise for

hasSpaceAvailable

.

In contrast, your test app is using the stream synchronously but trying to avoid blocking by polling for data (using

hasBytesAvailable

). This is what’s triggering the bug you’ve encountered.

Before going deeper into this I have a quick test for you to run. It seems like your ultimate goal is to do I/O using the

Stream

API. If that’s the case, why are you using

URLSession

to create your streams? You can open streams directly using

getStreamsToHost(withName:port:inputStream:outputStream:)

. This is a lot more direct, both from your perspective and from the perspective of CFNetwork.

Note That API is a little tricky to call from Swift. At the end of this response you’ll find the standard wrapper that I use.

Please switch to creating your streams with

getStreamsToHost(…)

and then repeat your test to see if that changes things.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

import UIKit

extension Stream {
    static func streamsToHost(name hostname: String, port: Int) -> (inputStream: InputStream, outputStream: OutputStream) {
        var inStream: InputStream? = nil
        var outStream: OutputStream? = nil
        Stream.getStreamsToHost(withName: hostname, port: port, inputStream: &inStream, outputStream: &outStream)
        return (inStream!, outStream!)
    }
}

Quinn,

Thanks so much for responding and confirming that this is a regression.

We switched over to using NSURLSession in 2015 after viewing the latter segment of WWDC 2015's “Networking with NSURLSession” (Session 711). We did so for several reasons:

1. Dealing with SSL and proxies was a historical pain point. SSL for us (getting it configured properly) and proxies for our users (as users encountered new proxies, we’d get support calls regarding connectivity). NSURLSession promised to take care of that for us and has done so since we began using it.
2. We could use NSURLSession to obtain streams as we had historically done without major disruption to our existing algorithm.
3. We send a standard HTTP GET request when we initially open the connection to inform an F5 which server resource we’re contacting. NSURLSession makes this quite easy.

Using “Stream.getStreamsToHost(…)” does work, but we lose the benefits of the items above.

I altered the sample application provided with the radar to avoid using “hasBytesAvailable”. I’ll attach the updated sample application to the radar. The thread still blocks. The crux of the issue is this:

1. the stream has 59 bytes available
2. we call “inputStream.read(&buffer, maxLength: 4)”
3. we receive 4 bytes and interpret those bytes as a 32bit Int (whose value is 55).
4. we call “inputStream.read(&buffer, maxLength: 55)”
5. the thread blocks

It appears to us as if reading less than the available bytes, followed by reading the remaining available bytes will cause the thread to block. Is that expected?

This is the stack trace when the thread blocks at step 5:

Thread 5 Queue : Anonymous

#0 0x0000000113e3a31e in __ulock_wait ()

#1 0x0000000113e5faff in _os_ulock_wait ()

#2 0x0000000113e5f7d0 in _os_nospin_lock_lock_slow ()

#3 0x000000010f643bef in __NSCFTCPIOReadStream::_streamImpl_Read(__CFReadStream*, unsigned char*, long, CFStreamError*, unsigned char*) ()

#4 0x000000010fc4b54c in CFReadStreamRead ()

#5 0x000000010e628ee9 in static FourByteHeaderMessageProcessor.readFromInputStream(_:atMost:) at /...Sanitized.../StreamFailure/ClientCommon/FourByteHeaderMessageProcessor.swift:43

#6 0x000000010e629d23 in FourByteHeaderMessageProcessor.readBytesWith4ByteHeader(timeoutTimeInterval:maxMessageLength:) at /...Sanitized.../StreamFailure/ClientCommon/FourByteHeaderMessageProcessor.swift:88

#7 0x000000010e621b3a in closure #1 in static URLSessionStreamTaskManager._urlSessionRead() at /...Sanitized.../StreamFailure/ClientCommon/URLSessionStreamTaskManager.swift:74

#8 0x000000010e625075 in URLSessionStreamTaskManager.urlSession(_:streamTask:didBecome:outputStream:) at /...Sanitized.../StreamFailure/ClientCommon/URLSessionStreamTaskManager.swift:208

#9 0x000000010e6256db in @objc URLSessionStreamTaskManager.urlSession(_:streamTask:didBecome:outputStream:) ()

#10 0x000000010f6ce97e in __88-[NSURLSession delegate_streamTask:didBecomeInputStream:outputStream:completionHandler:]_block_invoke ()

#11 0x000000010e9689b7 in __NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ ()

#12 0x000000010e96881a in -[NSBlockOperation main] ()

#13 0x000000010e966cd6 in -[__NSOperationInternal _start:] ()

#14 0x00000001139ab43c in _dispatch_client_callout ()

#15 0x00000001139b0af4 in _dispatch_block_invoke_direct ()

#16 0x00000001139ab43c in _dispatch_client_callout ()

#17 0x00000001139b0af4 in _dispatch_block_invoke_direct ()

#18 0x00000001139b0884 in dispatch_block_perform ()

#19 0x000000010e962ce4 in __NSOQSchedule_f ()

#20 0x00000001139ab43c in _dispatch_client_callout ()

#21 0x00000001139b1856 in _dispatch_continuation_pop ()

#22 0x00000001139afc86 in _dispatch_async_redirect_invoke ()

#23 0x00000001139b71f9 in _dispatch_root_queue_drain ()

#24 0x00000001139b6e97 in _dispatch_worker_thread3 ()

#25 0x0000000113e6e5a2 in _pthread_wqthread ()

#26 0x0000000113e6e07d in start_wqthread ()

It appears to us as if reading less than the available bytes, followed by reading the remaining available bytes will cause the thread to block. Is that expected?

No. If your using the stream in blocking mode it should behave like a socket (in the BSD Sockets sense), that is:

• If there’s any data on the stream you should get what’s there, which might be less than you’d asked for

• It only blocks if there’s no data at all on the stream

Using

Stream.getStreamsToHost(…)

does work, but we lose the benefits of the items above.

Fair enough. But does it help with this specific problem? The reason I ask is that CFNetwork is in the process of transitioning to the newer stream infrastructure, which is likely to be the source of this bug, and I’m curious whether

getStreamsToHost(…)

also puts you on the failing path.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

"Stream.getStreamsToHost()" does prevent us from getting blocked. The updated sample application associated with the Radar is set up to demonstrate that.

That said, we've tested eliminating the call to "URLSessionStreamTask.captureStreams()" by using its "readData(:Int,:Int,:TimeInterval:(Data?,Bool,Error?)->Void)" and "writeData(:Data,:TimeInterval,:(Error?)->Void)" methods directly. Our code is significantly simplified. Unit tests however, have broken in an unfortunate way - it's harder to mock URLSessionStreamTask than it was to mock a Stream.

At this point, we're going to continue to plow forward using "URLSessionStreamTask.readData(...)" and "URLSessionStreamTask.writeData(...)" for future releases of our app. They eliminate the need to call URLSessionStreamTask.captureStreams() and prevent us from touching the underlying Stream objects altogether. Given the typical cadence of iOS releases, we anticipate having enough time to get a new build through our QA and onto the store by the time iOS 11 is released. Knowing our customer base though, I would anticipate that our help desk will get a number of calls from folks that have upgraded to iOS 11, but not upgraded our app. Thus, if the CFNetwork team can fix the regression prior to the official release of iOS 11, it would be appreciated. We'd be happy to test any fixes in a very timely manner.

Stream.getStreamsToHost()

does prevent us from getting blocked.

Well, that is interesting.

The updated sample application associated with the Radar is set up to demonstrate that.

Thanks.

That said, we've tested eliminating the call to

URLSessionStreamTask.captureStreams()

by using its

readData(:Int,:Int,:TimeInterval:(Data?,Bool,Error?)->Void)

and

writeData(:Data,:TimeInterval,:(Error?)->Void)

methods directly. Our code is significantly simplified.

Cool.

Unit tests however, have broken in an unfortunate way - it's harder to mock

URLSessionStreamTask

than it was to mock a

Stream

.

Indeed. I’ve been struggling with this issue myself recently (with regards

NWTCPConnection

, but it’s API is very similar to

URLSessionTask

).

At this point, we're going to continue to plow forward using

URLSessionStreamTask.readData(…)

and

URLSessionStreamTask.writeData(…)

for future releases of our app.

Best of luck with that.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"