URLSession works for request but not NWConnection

I am trying to convert a simple URLSession request in Swift to using NWConnection. This is because I want to make the request using a Proxy that requires Authentication. I posted this SO Question about using a proxy with URLSession. Unfortunately no one answered it but I found a fix by using NWConnection instead.

Working Request

func updateOrderStatus(completion: @escaping (Bool) -> Void) {
let orderLink = "https://shop.ccs.com/51913883831/orders/f3ef2745f2b06c6b410e2aa8a6135847"
guard let url = URL(string: orderLink) else {
completion(true)
return
}
let cookieStorage = HTTPCookieStorage.shared
let config = URLSessionConfiguration.default
config.httpCookieStorage = cookieStorage
config.httpCookieAcceptPolicy = .always
let session = URLSession(configuration: config)
var request = URLRequest(url: url)
request.httpMethod = "GET"
request.setValue("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", forHTTPHeaderField: "Accept")
request.setValue("none", forHTTPHeaderField: "Sec-Fetch-Site")
request.setValue("navigate", forHTTPHeaderField: "Sec-Fetch-Mode")
request.setValue("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.0.1 Safari/605.1.15", forHTTPHeaderField: "User-Agent")
request.setValue("en-US,en;q=0.9", forHTTPHeaderField: "Accept-Language")
request.setValue("gzip, deflate, br", forHTTPHeaderField: "Accept-Encoding")
request.setValue("document", forHTTPHeaderField: "Sec-Fetch-Dest")
request.setValue("u=0, i", forHTTPHeaderField: "Priority")
// make the request
}

Attempted Conversion

func updateOrderStatusProxy(completion: @escaping (Bool) -> Void) {
let orderLink = "https://shop.ccs.com/51913883831/orders/f3ef2745f2b06c6b410e2aa8a6135847"
guard let url = URL(string: orderLink) else {
completion(true)
return
}
let proxy = "resi.wealthproxies.com:8000:akzaidan:x0if46jo-country-US-session-7cz6bpzy-duration-60"
let proxyDetails = proxy.split(separator: ":").map(String.init)
guard proxyDetails.count == 4, let port = UInt16(proxyDetails[1]) else {
print("Invalid proxy format")
completion(false)
return
}
let proxyEndpoint = NWEndpoint.hostPort(host: .init(proxyDetails[0]),
port: NWEndpoint.Port(integerLiteral: port))
let proxyConfig = ProxyConfiguration(httpCONNECTProxy: proxyEndpoint, tlsOptions: nil)
proxyConfig.applyCredential(username: proxyDetails[2], password: proxyDetails[3])
let parameters = NWParameters.tcp
let privacyContext = NWParameters.PrivacyContext(description: "ProxyConfig")
privacyContext.proxyConfigurations = [proxyConfig]
parameters.setPrivacyContext(privacyContext)
let host = url.host ?? ""
let path = url.path.isEmpty ? "/" : url.path
let query = url.query ?? ""
let fullPath = query.isEmpty ? path : "\(path)?\(query)"
let connection = NWConnection(
to: .hostPort(
host: .init(host),
port: .init(integerLiteral: UInt16(url.port ?? 80))
),
using: parameters
)
connection.stateUpdateHandler = { state in
switch state {
case .ready:
print("Connected to proxy: \(proxyDetails[0])")
let httpRequest = """
GET \(fullPath) HTTP/1.1\r
Host: \(host)\r
Connection: close\r
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.0.1 Safari/605.1.15\r
Accept-Language: en-US,en;q=0.9\r
Accept-Encoding: gzip, deflate, br\r
Sec-Fetch-Dest: document\r
Sec-Fetch-Mode: navigate\r
Sec-Fetch-Site: none\r
Priority: u=0, i\r
\r
"""
connection.send(content: httpRequest.data(using: .utf8), completion: .contentProcessed({ error in
if let error = error {
print("Failed to send request: \(error)")
completion(false)
return
}
// Read data until the connection is complete
self.readAllData(connection: connection) { finalData, readError in
if let readError = readError {
print("Failed to receive response: \(readError)")
completion(false)
return
}
guard let data = finalData else {
print("No data received or unable to read data.")
completion(false)
return
}
if let body = String(data: data, encoding: .utf8) {
print("Received \(data.count) bytes")
print("\n\nBody is \(body)")
completion(true)
} else {
print("Unable to decode response body.")
completion(false)
}
}
}))
case .failed(let error):
print("Connection failed for proxy \(proxyDetails[0]): \(error)")
completion(false)
case .cancelled:
print("Connection cancelled for proxy \(proxyDetails[0])")
completion(false)
case .waiting(let error):
print("Connection waiting for proxy \(proxyDetails[0]): \(error)")
completion(false)
default:
break
}
}
connection.start(queue: .global())
}
private func readAllData(connection: NWConnection,
accumulatedData: Data = Data(),
completion: @escaping (Data?, Error?) -> Void) {
connection.receive(minimumIncompleteLength: 1, maximumLength: 65536) { data, context, isComplete, error in
if let error = error {
completion(nil, error)
return
}
// Append newly received data to what's been accumulated so far
let newAccumulatedData = accumulatedData + (data ?? Data())
if isComplete {
// If isComplete is true, the server closed the connection or ended the stream
completion(newAccumulatedData, nil)
} else {
// Still more data to read, so keep calling receive
self.readAllData(connection: connection,
accumulatedData: newAccumulatedData,
completion: completion)
}
}
}
I posted this SO Question about using a proxy with URLSession.

It’s weird that you’d cite you that post when I answered your original proxy question here on DevForums: Use proxy for http request from iOS device. But, yeah, authenticating with a custom proxy is a well-known pain point for URLSession.

As a workaround, writing your own HTTP protocol is… well… challenging. How you should proceed here depends on your expectations of your HTTP implementation. Are you expecting it to work with arbitrary web servers? Or are you targeting one specific web server that you control, and if it only works with that you’ll be happy?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

@DTS Engineer I am targeting multiple web servers that I don't have control over, but each thread/task will only target one specific server, so there is no need to manage cookies for multiple sites at the same time. These web-servers use a shared service so all the request/responses will be similar across. I could use the NWConnection solution that I already have which works but just unsure how to get the Set Cookies from responses and add them to next requests.

How important is performance here?

The reason I ask is that there’s an option that, if it works for you, would be significantly easier. Rather than doing your own HTTP implementation, run a local SOCKS proxy and tell URLSession to use that proxy. Your proxy can then talk to the real proxy, taking care of the authentication in a way that’s transparent to URLSession.

The advantage of this approach is that you can continue to use the HTTP implementation within URLSession. The main disadvantage is the potential performance impact. Data will end up going in to and out of the networking stack twice.

ps I didn’t respond earlier because I didn’t see your comment. It’s better to reply as a reply; see Quinn’s Top Ten DevForums Tips for this and other titbits.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

URLSession works for request but not NWConnection
 
 
Q