Swift NSOutputStream not outputting full base64 string

You’re not using NSStream correctly. NSStream supports two modes of operation:

• async, via the

  -stream:handleEvent:

  callback scheduled on a run loop (

  -scheduleInRunLoop:forMode:

  ) or a dispatch queue (

  CF{Read,Write}StreamSetDispatchQueue

  )

• sync, the default

Most folks use it async but it’s OK to use it sync under specific circumstances (more on this below). I’m going to assume that’s true in your case and take things from there.

Which brings us to this code:

var readByte :UInt8 = 0 
while inputStream.hasBytesAvailable { 
    inputStream.read(&readByte, maxLength: 1) 
}

This is definitely wrong. You’re mixing up the notion of whether bytes are available (

hasBytesAvailable

) with the notion of whether the stream has hit EOF (that is, no more bytes can become available). These are different things.

The correct approach depends on how your data is structured on the ‘wire’:

• If the remote peer sends you data and then closes its end of the TCP connection, you can use EOF to denote the end of your data.

• If not, then the data stream you get consists of records and you have to work out how to determine your record boundaries. How you do this is up to you, but the most common options are:

  • start the record with a record length

  • use some sort of delimiter (like a CR LF pair) to mark the end of a record

  Whatever you do, keep in mind that TCP does not preserve record boundaries on the wire. So, if the remote peer writes 2 x 256-byte records, you might receive 1 x 512-byte chunk, or 512 x 1-byte chunks, or anything in between. That’s why you need your own scheme for working out record boundaries.

  IMPORTANT This is a feature of TCP, not a feature of any specific TCP API, and thus you have to deal with it no matter what API you use.

It sounds like you’re going for the first option, in which case you just need to read until you hit EOF. The correct way to do this is as follows:

repeat {
    let bytesRead = inputStream.read(…)
    if bytesRead > 0 {
        … append bytesRead bytes of data to buffer …
    } else if bytesRead < 0 {
        … handle error …
    } else {
        // EOF
        break
    }
} while true

NSStream is not particularly unusual here; this approach is very common in TCP APIs, inherited from the original BSD Sockets API.

ps The specific situation where it’s illegal to use any networking API, including NSStream, is on the main thread on iOS. QA1693 Synchronous Networking On The Main Thread discusses this in detail.

Doing networking on the main thread on macOS is not great either, but there are situations where it makes sense (like in a command line tool).

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thankyou so much for your reply Eskimo!! I was begginning to think i would never be able to figure this out haha. It look like i have a lot of work to do on the input front. I'll change up my code with the code you have provided and do some more testing. For the moment i will take out the input code i think and try and worry about the output. If i was to get rid of the:

var readByte :UInt8 = 0 
while inputStream.hasBytesAvailable { 
    inputStream.read(&readByte, maxLength: 1) 
}

entirely for now; can you see any reason why the output to the client wouldn't all be coming through on this line:

let OStreamCount = outputStream.write(base64String, maxLength: (base64String.characters.count))

From the original question if i'm checking on netcat with the string that is coming through and writing that to a file. Do you know why that output file would hold a string less than the actual transmitted string?

Thanks so much again for your help. I'm quite new to this but i'm getting my head around it.

write(_:maxLength:)

returns the number of bytes that were accepted by the kernel for later transmission. What value do you get back?

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

So with this example string it should send with length 510368 however .write returns only 490272 characters.

Here is a picture to show what i mean listening with netcat:

http://i1380.photobucket.com/albums/ah162/chrislukelyndon/Screen%20Shot%202016-08-05%20at%2012.57.28%20PM_zpsam5lcizx.jpg

So with this example string it should send with length 510368 however .write returns only 490272 characters.

This is allowed behaviour per the

write(_:maxLength:)

documentation. The correct response (for a caller that’s using NSStream synchronously) is to look at how many bytes were sent and then call

write(_:maxLength:)

again to transmit the remaining bytes. This API design, which admittedly is a little weird, was inherited from the underlying UNIX system call,

write

.

IMPORTANT This behaviour explains why the second parameter is named

maxLength

, not

length

.

In many cases

write

is all or nothing. However, when dealing with TCP connections you have to allow for “short writes”. What’s happening under the covers here is that

write

has copied data into the kernel socket buffer up to the point that it’s become full. At that point

write

has two choices:

• it could block waiting for more space to become available in the kernel socket buffer (that is, for data to go over the wire and be ACK’d by the remote peer) (A)

• it could return a short write (B)

It turns out that, for obscure UNIX-y reasons (namely the interaction with UNIX signals and the semantics of

EINTR

), option B is the only workable option.

Most folks who use

write

synchronously create a loop like this:

import Darwin

func write(fd: Int32, _ data: UnsafePointer<UInt8>, _ length: Int) -> Bool {
    var cursor = data
    var bytesRemaining = length
    repeat {
        let bytesWritten = Darwin.write(fd, cursor, bytesRemaining)
        if bytesWritten < 0 {
            if errno != EINTR {
                return false
            }
        } else {
            cursor += bytesWritten
            bytesRemaining -= bytesWritten
            if bytesRemaining == 0 {
                return true
            }
        }
    } while true
}

This function returns true if all the bytes were written and false otherwise (leaving

errno

set to the error in that case).

WARNING While this code compiles (in Xcode 7.3.1), I haven’t actually tested it.

As

write(_:maxLength:)

has the same semantics of

write

, if you want to use it synchronously then you’ll need a similar loop. Keep in mind that most folks use NSStream asynchronously, and thus don’t have to deal with this complexity (although using it asynchronously add other complexities).

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thanks Eskimo!!! It's now sending the entire string. I didn't realise that was how the write function worked. I thought it would essentially do what your write function does by itself. I did what you said and this is the code i came up with just in case anyone stumbles on this thread later:

func writer(NSO: NSOutputStream, _ data: String, _ length: Int) -> Bool {
    var cursor = data
    var bytesRemaining = length
    repeat {
        let bytesWritten = NSO.write(cursor, maxLength: bytesRemaining)
        print("bytes written: ", bytesWritten)
        if bytesWritten < 0 {
            if errno != EINTR {
                return false
            }
        } else {
            print("cursor len before: ", cursor.characters.count)
            cursor = cursor.substringWithRange(Range<String.Index>(start: cursor.startIndex.advancedBy(bytesWritten), end: cursor.endIndex))
            print("cursor len after: ", cursor.characters.count)
            bytesRemaining -= bytesWritten
            if bytesRemaining == 0 {
                return true
            }
        }
    } while true
}

Now i just need to figure out how i can run this function in another while loop in a way that the client receives that full string and isnt blocked by the function calling again. IE the client will always get the full string but wont be blocked so it can use the string to do other things. Any ideas / tips / sites i should read up on?

Any ideas / tips / sites i should read up on?

I recommend that you use NSStream asynchronously. This requires you to do a bit of a re-think about how you use the API, but it has a couple of key advantages:

• it lets your main thread continue to do work while network I/O is happening

• it avoids the need for secondary threads, which keeps your code simpler

There’s a bunch of sample code for dealing with streams asynchronously on our web site. A good place to start would be RemoteCurrency.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"