Framer protocol for STARTTLS

It looks like BoringSSL does not know know which version of TLS you are wanting to use, because the connection may not know to use TLS. It looks like you are passing NWProtocolTLS into prependApplicationProtocol when you want to pass in here a custom frame protocol definition.

When creating a custom framing protocol it's best to start with a definition so that there is a structure for how frames are setup and for how to parse the reads and writes. Next, for TLS, your can configure the TLS options as part of the NWParameters used when an NWConnection is setup. That way after you initiate a NWConnection you can set application frame protocol as well.

There is an example of how to do this very thing in the Building a Custom Peer-to-Peer Protocol sample. Checkout the entire GameProtocol definition and the NWParameters extension.

Keep in mind this is configured for peer-to-peer with PSK authentication, but this should provide a good starting place:

convenience init(passcode: String) {
  // Customize TCP options to enable keepalives.
  let tcpOptions = NWProtocolTCP.Options()
  tcpOptions.enableKeepalive = true
  tcpOptions.keepaliveIdle = 2

  // Create parameters with custom TLS and TCP options.
  self.init(tls: NWParameters.tlsOptions(passcode: passcode), tcp: tcpOptions)

  // Enable using a peer-to-peer link.
  self.includePeerToPeer = true

  // Add your custom game protocol to support game messages.
  let gameOptions = NWProtocolFramer.Options(definition: GameProtocol.definition)
  self.defaultProtocolStack.applicationProtocols.insert(gameOptions, at: 0)
}

Matt Eaton

DTS Engineering, CoreOS

meaton3 at apple.com

Thanks for the response.

I may not have explained what I am trying to do very clearly: the preferred way for a client to use TLS in NNTP, IMAP, etc is, apparently, to open a plain TCP connection and at some later point issue a STARTTLS command (having ascertained that the server supports the command). The TLS negotiation is supposed to happen after the server's response to the STARTTLS command is received. This requires TLS support to be added dynamically to the protocol stack at this point. I believe this is not possible after a protocol has been marked'ready'.

The hint given in the WWDC presentation was, as I understand it, to write a framing protocol based on plain TCP which returns '.willMarkReady' from its start() method, checks whether STARTTLS is supported and if so issues that command and reads the response. At this point it should dynamically add TLS support to the protocol stack and mark the protocol as ready. The code excerpt I supplied was of my framing protocol trying to do that.

Have I misunderstood how to implement STARTTLS? Does anyone have a working example they can share?

No, I was just confused on why you had to explicitly use STARTTLS.

| Have I misunderstood how to implement STARTTLS?

For example, when connecting to nntp.aioe.org I am able to use NWConnection with TLS to setup an initial TLS 1.2 connection and then run a CAPABILITIES command with a response. I did not setup a framing protocol around this, but one could be added during connection setup like so:

init(with host: String) {
    
    let tlsOptions = NWProtocolTLS.Options()
    
    let tcpOptions = NWProtocolTCP.Options()
    tcpOptions.enableKeepalive = true
    tcpOptions.keepaliveIdle = 2
    
    let connectionParams = NWParameters(tls: tlsOptions, tcp: tcpOptions)
    
    // Setup your custom CommandProtocol frame definition here and add it to
    //let commandOptions = NWProtocolFramer.Options(definition: CommandProtocol.definition)
    //connectionParams.defaultProtocolStack.applicationProtocols.insert(commandOptions, at: 0)
    
    guard let port = NWEndpoint.Port("443") else { return }
    
    let connectionEndpoint = NWEndpoint.hostPort(host: NWEndpoint.Host(host), port: port)
    connection =  NWConnection(to: connectionEndpoint, using: connectionParams)
    
}


// Sample Usage
nntpConnection = NNTPConnection(with: "nntp.aioe.org")
nntpConnection?.start()

...

let command = "CAPABILITIES\r\n"
nntpConnection?.sendDataOnConnection(command: command)

Wrapping a framing protocol definition around the data been read and written to the connection would be dependent on how your specific application works though.

Matt Eaton

DTS Engineering, CoreOS

meaton3 at apple.com

No, I was just confused on why you had to explicitly use STARTTLS.

| Have I misunderstood how to implement STARTTLS?

Well, according to RFC4642 - Using Transport Layer Security (TLS) with Network News Transport Protocol (NNTP):

Hello Colin, Matt

I have been having similar issues with getting StartTLS working in a framer for communicating with IMAP/SMTP servers

I am working in obj-c so my code structure is a little different but the end result is the same, I have followed what guidance I can find to no avail and am wondering if I am doing things incorrectly or missing a step of if this mechanism of STARTTLS is simply not working.

Here a link to my discussion: https://forums.developer.apple.com/message/409391.

Thanks for any insights you can give

Hi,

For some reason I am unauthorised to see your discussion. Maybe it's because I'm a cheapskate and don't have a paid Apple Developer Account.

I've just come across swift-nio (https://github.com/apple/swift-nio), swift-nio-ssl and swift-nio-examples on GitHub. I think swift-nio is based on the Network framework and the examples repository includes an SMTP example that implements StartTLS so there may be some clues in there.

Yeah, the NIOSMTP example project does a great job of integrating the secure transport infrastructure from swift-nio-ssl with the transport services work from swift-nio-transport-services and Network framework. Using this example should be a good base to investigate your requirements of using port 119 with NNTP. One thing to keep in mind as you dig in here is where you will be running this project; for example, Swift-NIO is designed to run on macOS and Linux, but if you need the to run this on iOS you will need to integrate swift-niotransport-services as with Network Framework.

Matt Eaton

DTS Engineering, CoreOS

meaton3 at apple.com

For the record: my little test protocol framer is now successfuly enabling TLS on a plain TCP connection to port 119 of an NNTP server, using NNTP's STARTTLS command. Here is an updated version of the code excerpt I supplied previously:

fileprivatefunc getStartTlsResponse(framer: NWProtocolFramer.Instance) {
        if let startResponse = String(data: receivedLine, encoding: .utf8) {
            print(startResponse)
            if startResponse.starts(with: "382 ") {
                // add TLS to protocol stack and mark protocol ready
                do {
                    let tlsOpts = NWProtocolTLS.Options()
                    try framer.prependApplicationProtocol(options: tlsOpts)
                    print("AutoTlsProtocol: prepended TLS to protocol stack")
                } catch {
                    print("AutoTlsProtocol: error prepending TLS to protocol stack: \(error)")
                    framer.markFailed(error: nil)
                    return
                }
            }
        }
        framer.passThroughInput()
        framer.passThroughOutput()
        framer.markReady()
    }

I'm embarrassed to point out that I had failed to set input and output to 'passthrough' before calling markReady().

THANK YOU SO MUCH!!!!!

I cannot tell you how much time I have wasted in the last week over this and it boils down to 2 lines of code! (passing through input and output)

How did you find this out? -- I have scoured numerous documents and examples. I was actually at the point of wondering if I was going to have to write TLS handshaking code or giving up on NetworkFramework all together and going back to (eep) sockets.

Again. Thank you so much

Scott

It was just an "I wonder ..." moment. I didn't find any document that said "remember to ...".