General:
TN3151 Choosing the right networking API
Networking Overview document — Despite the fact that this is in the archive, this is still really useful.
TLS for App Developers DevForums post
Choosing a Network Debugging Tool documentation
WWDC 2019 Session 712 Advances in Networking, Part 1 — This explains the concept of constrained networking, which is Apple’s preferred solution to questions like How do I check whether I’m on Wi-Fi?
TN3135 Low-level networking on watchOS
Adapt to changing network conditions tech talk
Foundation networking:
DevForums tags: Foundation, CFNetwork
URL Loading System documentation — NSURLSession, or URLSession in Swift, is the recommended API for HTTP[S] on Apple platforms.
Network framework:
DevForums tag: Network
Network framework documentation — Network framework is the recommended API for TCP, UDP, and QUIC on Apple platforms.
Network Extension (including Wi-Fi on iOS):
See Network Extension Resources
Wi-Fi Fundamentals
Wi-Fi on macOS:
DevForums tag: Core WLAN
Core WLAN framework documentation
Wi-Fi Fundamentals
Secure networking:
DevForums tags: Security
Apple Platform Security support document
Preventing Insecure Network Connections documentation — This is all about App Transport Security (ATS).
Available trusted root certificates for Apple operating systems support article
Requirements for trusted certificates in iOS 13 and macOS 10.15 support article
About upcoming limits on trusted certificates support article
Apple’s Certificate Transparency policy support article
Technote 2232 HTTPS Server Trust Evaluation
Technote 2326 Creating Certificates for TLS Testing
QA1948 HTTPS and Test Servers
Miscellaneous:
More network-related DevForums tags: 5G, QUIC, Bonjour
On FTP DevForums post
Using the Multicast Networking Additional Capability DevForums post
Investigating Network Latency Problems DevForums post
Local Network Privacy FAQ DevForums post
Extra-ordinary Networking DevForums post
Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"
Network
RSS for tagNetwork connections send and receive data using transport and security protocols.
Posts under Network tag
200 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
I am trying to programmatically block some egress and ingress connections using bsd packet filters. My program writes rules in a file and this file is loaded using an anchor in /etc/pf.conf (main ruleset) . Rules work as intended. But when there is network change like turn on/off wifi , and change in wifi nw the main ruleset is getting flushed and i have to reapply (pfctl -q -f /etc/pf.conf) to get the rules back in place.
Looking for guidance to keep the main ruleset intact irrespective of system changes.
Reproduce:
Download live-caller-id-lookup-example
Add
let url = URL(string: "http://another-macbook.local:80")!
let task = URLSession.shared.dataTask(with: url) {(data, response, error) in
guard let data = data else { return }
print(String(data: data, encoding: .utf8)!)
}
task.resume()
anywhere in the code
run PIRService target in xcode
Result: no dialogue, host is unreachable
Works fine when launching same binary from terminal
We have encountered an issue with the app where it fails to progress beyond the splash screen. Interestingly, the problem seems to resolve itself after either reinstalling the app or restarting the phone. This behavior is observed specifically in version 18.3.1.
Could this issue be related to a network connection or initialization problem in this version? Any insights or recommendations would be appreciated.
Thank you for your assistance!
I am currently developing an app using WKWebView.
In iOS 17, Apple introduced ProxyConfiguration, and I have two questions regarding its usage:
Is there a way to use ProxyConfiguration to distinguish between HTTP and HTTPS traffic?
I have reviewed the documentation but couldn’t find a clear method for this.
As an alternative approach, I am attempting to determine the request scheme within WKWebView and switch proxyConfigurations just before loading the request.
My current implementation sends requests to hoge_a by default and switches to hoge_b only for HTTP traffic.
if #available(iOS 17.0, *) {
if URLString.contains("http:") {
webView.configuration.websiteDataStore.proxyConfigurations = [hoge_b]
}
}
webView.load(request)
if #available(iOS 17.0, *) {
if URLString.contains("http:") {
webView.configuration.websiteDataStore.proxyConfigurations = [hoge_a]
}
}
2. Since I modify proxyConfigurations every time webView.load(request) is called, are there any potential issues or concerns with this approach?
I would appreciate any insights you may have on either of these questions.
When installing a new version the app while a tunnel is connected, seemingly the old packet tunnel process gets stopped but the new one does not come back up. Reportedly, a path monitor is reporting that the device has no connectivity. Is this the expected behavior?
When installing an update from TestFlight or the App store, the packet tunnel instance from the old tunnel is stopped, but, due to the profile being on-demand and incldueAllNetworks, the path monitoring believes the device has no connectivity - so the new app is never downloaded. Is this the expected behavior?
During development, the old packet tunnel gets stopped, the new app is installed, but the new packet tunnel is never started. To start it, the user has to toggle the VPN twice from the Settings app. The tunnel could be started from the VPN app too, if we chose to not take the path monitor into account, but then the user still needs to attempt to start the tunnel twice - it only works on the second try. As far as we can tell, the first time around, the packet tunnel never gets started, the app receives an update about NEVPNStatus being set to disconnecting yet NEVPNConnection does not throw.
The behavior I was naively expecting was that the packet tunnel process would be stopped only when the new app is fully downloaded and when the update is installed, Are we doing something horribly wrong here?
When setting up a packet tunnel with a profile that has includeAllNetworks set to true, we seemingly cannot send any traffic inside the tunnel using any kind of an API. We've tried using BSD sockets, as we ping a host only reachable within the tunnel to establish whether we have connectivity - this does not work. When using NWConnection from the Network framework and specifying the required interface via virtualInterface from the packet tunnel, the connection state never reaches ready. Our interim solution is to, as ridiculous as it sounds, include a whole userspace networking stack so we can produce valid TCP packets just to send into our own tunnel. We require a TCP connection within our own tunnel to do some configuration during tunnel setup. Is there no better solution?
I must be missing something. How can an iphone that is in lockdown mode, using ONLY data, no Bluetooth connected and only one singular iPhone have seven UNLISTED items on the local network in privacy and settings?
Hi folks,
I would like to ask for clarification regarding Local Network Policy. I found 2 cases where I think the behaviour differs from the documentation.
1. Use case
In a CI environment, we have multiple services (LaunchAgents) which require Local Network Access. We are fine by manually approving the Local Network Permission once (per service), but we also require these services to be able to self-update. Self update results in downloading the a binary with an (obviously) different UUID, which seems to result in re-triggering the Local Network Consent prompt. Strange thing: If I don't click either buttons (Allow of Reject), just restart macOS, it will result in an enabled entry in Privacy & Security > Local Network.
I read a reply somewhere on this forum by an Apple engineer that the approval process is a mix of Bundle ID + UUID + other components, so I would expect a new binary with the same properties (but different UUID) to be already whitelisted.
Is this behaviour intended?
2. Use Case
Given the first issue, I decided to do this in the "right way".
I was happy to read this sentence in the documentation:
If you ship a launchd agent that’s not installed using SMAppService, make macOS aware of the responsible code by setting the AssociatedBundleIdentifiers property in your launchd property list.
I have a properly setup (and code signed) Application, for which I have enabled Local Network permission in Privacy & Security.
I have a standalone LaunchAgent, which runs a long running binary from a user directory. The agent is managed with launchd, and in this sense it is "independent" from the main Application Bundle. I have setup AssociatedBundleIdentifiers in the Agent plist, to associate it with the Application. The TeamIdentifier of the 2 binaries are the same. Based on the documentation, this should be enough for my agent to signal macOS that the responsible code is the Application Bundle (which is already enabled from Local Network POV).
Instead, once the LaunchAgent starts, the Local Network Consent popup appears for the binary. I would expect the Application to be the responsible code, thus no more Consent popup.
Is this behaviour intended?
I need this service to run as user, so I cannot just circumvent the Consent popup by running as a Daemon or Root. Nor I would like to manage the Agent with ServiceManagement. What do you guys think, have I misunderstood the documentation?
I'm working on a game that uses NWBrowser and NWListener to create a connection between an iOS and tvOS app.
I've got the initial networking up and running and it works perfectly when running in the simulator(s). However, when I run on-device(s), I've found that browseResultsChangedHandler gets called multiple times for what is ostensibly the same service.
My browser handler (which runs on iOS) looks like this:
browser.browseResultsChangedHandler = { [weak self] results, changes in
if let result = browser.browseResults.first {
self?.onPeerConnected?(PeerConnection(endpoint: result.endpoint))
}
}
The first time it gets called, the interface in the NWBrowser.Result is en0, but the 2nd time it gets called, it is en0 AND awdl0.
Because my current handling is so naive, this re-invocation ends up with two connections being made to the remote server (the Apple TV).
Now, I know that this handler, by its very name, is designed to be called multiple times as things change, so I'm curious as to what strategies I might employ here.
Is there any value in tearing down any previous connections and re-connecting using the latest one? Should I just kill the browser as soon as I handle the first one? Just ignore subsequent ones?
I'm sure that, to a degree, the answer is probably "it depends"... but I'm curious to see if there might be at least some high-level strategies like "whatever you do, don't do xxxx" or "most apps do yyyy" :-)
Thanks.
We use URLSessionWebSocketTask for web socket connection. When get error we reconnect - recreate new URLSessionWebSocketTask.
Test case: off wifi on iOS device; get error(s) URLError.notConnectedToInternet. When on wifi correct create new task with connect.
This working on iOS 12, 14, 15, 16, 17. But on iOS 18 we get error URLError.notConnectedToInternet without correct connection.
class WebSocketManager {
...
func openConnection() {
webSocketTask?.cancel(with: .goingAway, reason: nil)
webSocketTask = urlSession?.webSocketTask(with: urlRequest)
webSocketTask?.resume()
listen()
}
func closeConnection() {
webSocketTask?.cancel(with: .goingAway, reason: nil)
webSocketTask = nil
}
private func listen() {
webSocketTask?.receive { [weak self] result in
guard let self else { return }
switch result {
case .failure(let error):
delegate?.webSocketManager(self, error: error)
case .success(let message):
switch message {
case .string(let text):
delegate?.webSocketManager(self, message: .text(text))
case .data(let data):
delegate?.webSocketManager(self, message: .data(data))
@unknown default:
fatalError()
}
listen()
}
}
}
}
Delegate:
func webSocketManager(_ webSocketManager: WebSocketManagerType, error: Error) {
webSocketManager.openConnection()
}
I am currently operating an app using an embedded web server that communicates over local HTTP.
Recently, when opening Safari, I started encountering the following error message:
"Safari cannot open the page. Error: Failed to navigate to an HTTP URL with HTTPS-only mode enabled."
However, I am currently in a situation where switching to HTTPS is difficult. Are there any solutions to resolve this issue besides using HTTPS?
Thank you.🙏
Hi, I’m working on a case in our organisation where we encounter regular wireless network disconnections - 30 and 60 minutes. ~ 1800 sec session is widely seen across hundreds if not thousand Macbooks.
We excluded internal wireless configuration issue and vendor specific problem as the disconnections happen on both Cisco and Ubiquiti Access Points.
Wireless Controller debugging most often show EAP timeout error.
Sniffer shows that the disassociation is initiated by Macbook.
MacOS logs show wifianalyticsd performing some actions before the disconnection (generated with sudo log show --debug [time]):
2025-02-04 14:16:31.219169+0100 0xc01342 Default 0x0 86459 0 wifianalyticsd: [com.apple.wifi.analytics:Default] -[WAEngine gatherConsecutiveDatapathReadings:forProcessToken:andReply:]_block_invoke::4238:SDNS: WiFi Not Primary - setting suppressedReason kSymptomsDnsWiFiInterfaceNotPrimary
2025-02-04 14:16:31.219192+0100 0xc01342 Default 0x0 86459 0 wifianalyticsd: [com.apple.wifi.analytics:Default] -[DPSQuickRecoveryRecommendationEngine updateTimeSincePreviousTriggerForStudy:msgKey:dictKey:]::913:msgkey:WFAAWDWASDS_symptomsDnsTimeSincePreviousTriggerMinutes dictKey:dps_lastSymptomsDpsTrigger previous_TS:(null) current_TS:Tue
Feb 4 14:16:31 2025 difference:0
2025-02-04 14:16:31.219704+0100 0xc01342 Default 0x0 86459 0 wifianalyticsd: [com.apple.wifi.analytics:Default] -[DPSQuickRecoveryRecommendationEngine updateScreenState]::198:DPS Fast Reset Recommendation Engine: (screenON & foreGrnd traffic) is DETECTED
2025-02-04 14:16:31.219713+0100 0xc01342 Default 0x0 86459 0 wifianalyticsd: [com.apple.wifi.analytics:Default] -[DPSQuickRecoveryRecommendationEngine recommendSymptomsDpsRecovery:symptomsDnsStats:awdlState:currentSymptomsCondition:isLANPingSuccessful:appUsage:averageCCA:]::966:PeerDiagnostics: Data not received from peerDiagnostics
2025-02-04 14:16:31.219714+0100 0xc01342 Default 0x0 86459 0 wifianalyticsd: [com.apple.wifi.analytics:Default] -[DPSQuickRecoveryRecommendationEngine checkForPriorityNetwork]::256:Priority Network Check Disabled: NO IsPriorityNetwork: YES
2025-02-04 14:16:31.219732+0100 0xc01342 Default 0x0 86459 0 wifianalyticsd: [com.apple.wifi.analytics:Default] -[DPSQuickRecoveryRecommendationEngine isResetAllowedForKey:forPrefSelector:]::330:key:symptomsDps_lastScreenOnRecoveryWD previousWD_TS:(null) currentWD_TS:Tue Feb 4 14:16:31 2025 recommendation:YES
2025-02-04 14:16:31.219735+0100 0xc01342 Default 0x0 86459 0 wifianalyticsd: [com.apple.wifi.analytics:Default] -[DPSQuickRecoveryRecommendationEngine updateSymptomsDPSRecoveryWDStatsForKey:]::210:Added key: symptomsDps_numRecommendedScreenOnRecoveryWD value:1 dict:(null)
2025-02-04 14:16:31.219737+0100 0xc01342 Default 0x0 86459 0 wifianalyticsd: [com.apple.wifi.analytics:Default] -[DPSQuickRecoveryRecommendationEngine recommendSymptomsDpsRecovery:symptomsDnsStats:awdlState:currentSymptomsCondition:isLANPingSuccessful:appUsage:averageCCA:]::1023:PeerDiagnostics: Recommendation for DNS Symptoms Recovery: Reassoc
2025-02-04 14:16:31.219740+0100 0xc01342 Default 0x0 86459 0 wifianalyticsd: [com.apple.wifi.analytics:Default] -[WAEngine gatherConsecutiveDatapathReadings:forProcessToken:andReply:]_block_invoke::4276:PeerDiagnostics: Triggering Reassoc for symptoms-dps
2025-02-04 14:16:31.219741+0100 0xc01342 Default 0x0 86459 0 wifianalyticsd: [com.apple.wifi.analytics:Default] -[WAEngine gatherConsecutiveDatapathReadings:forProcessToken:andReply:]_block_invoke::4277:SDNS: Recommendation - kSymptomsdDPSReassoc, triggering reassoc wiith reason ReassociateOnDNSSymptoms
2025-02-04 14:16:31.220001+0100 0xc01342 Default 0x0 86459 0 wifianalyticsd: (IO80211) Apple80211SetWithIOCTL:11858 Processing APPLE80211_IOC_REASSOCIATE_WITH_CORECAPTURE
2025-02-04 14:16:31.387985+0100 0xc01342 Default 0x0 86459 0 wifianalyticsd: (IO80211) Apple80211SetWithIOCTL: Processed APPLE80211_IOC_REASSOCIATE_WITH_CORECAPTURE Ioctl error:0
WAEngine and DPSQuickRecoveryRecommendationEngine functionalities (?) play significant role in here recommending Reassociation
We can see that reassociation is being triggered because of DNS symptoms - why and where can I find them?
Recommendation for DNS Symptoms Recovery: Reassoc
I have some concerns related to shortening the lifetime of certificates, as per
https://support.apple.com/en-gb/102028
Does this apply to Private CA root certificates?
And if yes:
does it apply if I use ATS and higher level API like URLSession
does it apply it I carry my root CA cert in my app payload and use low level libraries without ATS support?
We have several apps that our business uses to connect to internal private HTTP sites. We noticed in IOS 18.3 we are getting SSL errors to the web server and noticed the issue in the Chrome Browser as well. Our team is looking at the Application Transport Security layer exceptions in our apps Info.Plist. We do notice the browser forcing HTTPS. Any insight on what could be the issue?
I am trying to connect an iPhone 16 (iOS 18.3) to a Wi-Fi device with the SSID "DIRECT-DR_6930_KP201128", but every time, without being able to enter the Wi-Fi password, the message "Unable to join the network 'DIRECT-DR_6930_KP201128'" is displayed. Below are the system logs from the connection failure. Could you please tell me the cause of the connection failure?
By the way, an iPhone SE 2nd (iOS 18.2.1) can connect to the same Wi-Fi device without any issues.
System Logs:
・Jan 31 19:18:14 900-iPhone-16-docomo Preferences(WiFiKit)[351] : {ASSOC-} association finished for DIRECT-DR_6930_KP201128 - success 0
・Jan 31 19:18:14 900-iPhone-16-docomo runningboardd(RunningBoard)[33] : Assertion 33-351-4412 (target:[app<com.apple.Preferences(DE1AB487-615D-473C-A8D6-EAEF07337B18)>:351]) will be created as inactive as start-time-defining assertions exist
・Jan 31 19:18:14 900-iPhone-16-docomo Preferences(WiFiKit)[351] : association failure: (error Error Domain=com.apple.wifikit.error Code=12 "Unknown Error" UserInfo={NSDebugDescription=Unknown Error, NSUnderlyingError=0x303307660 {Error Domain=com.apple.corewifi.error.wifid Code=-3938 "(null)"}})
・Jan 31 19:18:14 900-iPhone-16-docomo Preferences(WiFiKit)[351] : dismissing credentials view controller for DIRECT-DR_6930_KP201128
Hi,
We are working on an app that communicates over a UDP connection on the local network. In our testing, we have a Python UDP client running on the same network, which responds when we send a message to a broadcast IP (255.255.255.255). This setup works as expected on an iOS 15 device.
However, when we test the same scenario on an iOS 18 device, the UDP communication doesn't seem to reach the Python UDP client. We've verified that the UDP client and the iOS device are on the same network, and the Python client is responding correctly.
Has Apple introduced any restrictions or changes regarding UDP broadcast behavior in iOS 18? Is broadcasting to 255.255.255.255 still supported, or has this functionality been limited in recent iOS versions?
I’m currently porting a Chrome Extension to Safari and integrating it with native messaging in a Safari Web Extension. As part of this, I’m building a proxy to forward messages between the web extension and a socket in another application, both ways. Additionally, the socket occasionally broadcasts messages that also need to be sent to the web extension.
The issue I’m facing is that the app extension terminates whenever I call context.completeRequest(returningItems: nil), which prevents me from listening for incoming messages from the socket (I'm using the Network Framework). To work around this, I’ve tried not calling context.completeRequest(returningItems: nil), which keeps the app extension running. However, I’m unsure if this is the right approach—currently, I’m simply ignoring the response and relying entirely on SFSafariApplication.dispatchMessage.
According to the documentation, the app extension lifecycle ends when the system terminates it, but I need to keep the socket listener active.
Has anyone encountered a similar issue, or does anyone have suggestions for maintaining the socket connection while adhering to the app extension lifecycle?
Any insights would be greatly appreciated!
Hi,
I have been reviewing some previous discussions around networking in macOS, and I’d like to clarify my understanding of the differences between the kernel-space network stack and user-space network stack and validate this understanding based on the information shared in earlier threads.
I’m also curious about how these differences impact macOS applications, particularly those requiring maintaining many simultaneous network connections.
Understanding Kernel-Space vs User-Space Network Stack
Kernel-Space Network Stack (BSD Sockets):
The kernel-space networking stack refers to the traditional networking layer that runs in the kernel and handles network communication via BSD sockets. This stack is lower-level and interacts directly with the operating system's networking drivers and hardware. All network connections managed through this stack require a socket (essentially a file descriptor) to be opened, which places limits on the number of file descriptors that can be used (for example, the default 64K limit for sockets). The kernel network stack is traditionally used on macOS (and other UNIX-based systems) for networking tasks, such as when you use system APIs like BSD sockets.
User-Space Network Stack (Network Framework): The user-space network stack in macOS (via the Network framework) allows applications to handle networking tasks without directly using the kernel. This provides more flexibility and performance benefits for certain types of network operations, as the networking stack is managed in user space rather than kernel space. This approach reduces overhead and allows more control over networking configurations. In theory, with user-space networking, the application wouldn't be bound by kernel-level socket limits, and it could handle many more simultaneous connections efficiently.
In previous posts on that thread, Quinn mentioned that the Network framework in macOS can rely on the user-space stack (by default) for network operations, but there are still cases where macOS falls back to using the kernel stack (i.e., BSD sockets) under certain conditions. One key example is when the built-in firewall is enabled. This prevents user-space networking from functioning as expected, and the system defaults to using the kernel's BSD sockets for network communication.
In the same discussion, it was also highlighted that NECP (Network Extension Control Plane) could place further limitations on user-space networking, and eventually, systems may run into issues like ENOSPC errors due to excessive simultaneous network flows. This suggests that while user-space networking can offer more flexibility, it's not immune to limits imposed by other system resources or configurations.
Given the information above, I wanted to confirm:
Is the above understanding correct and does the macOS Network framework still use the user-space networking stack in macOS 14 and beyond?
Under what conditions would the system fall back to using the kernel stack (BSD sockets) instead of the user-space stack? For example, does enabling the firewall still disable user-space networking?
What is the practical impact of this fallback on applications that require many simultaneous network connections? Specifically, are there any limitations like the 64K socket limit that developers should be aware of when the system uses the user space stack, and what are the best practices to manage large numbers of connections?
In our iOS networking related app for the app store (with network extension using packet tunnel provider), we are supposed to read the list of nameservers for the network. We use res_ninit function.
This function returns only 3 items (but in reality the network has more dns servers. In my case 5. Some ipv4 and some ipv6)
Looking at the header file at iOS 18.2 -> user/include/resolve.h, it shows that the __res_state struct has a maximum limit of 3 for the nsaddr_list array.
It seems that the reason the res_ninit function returns only 3 values is because of this. For our code to work correctly, it needs to know all the dns servers, but we only get partial results.
Is there any other api that can get us all the dns servers ?
I am working on adding RFC4217 Secure FTP with TLS by extending Mike Gleason's classic libncftp client library. I refactored the code to include an FTP channel abstraction with FTP channel abstraction types for TCP, TLS, and TCP with Opportunistic TLS types. The first implementation of those included BSD sockets that libncftp has always supported with the clear TCP channel type.
I first embarked on extending the sockets implementation by adding TCP, TLS, and TCP with Opportunistic TLS channel abstraction types against the new, modern Network.framework C-based APIs, including using the “tricky” framer technique to employ a TCP with Opportunistic TLS FTP channel abstraction type to support explicit FTPS as specified by RFC4217 where you have to connect first in the clear with TCP, request AUTH TLS, and then start TLS after receiving positive confirmation. That all worked great.
Unfortunately, at the end of that effort, I discovered that many modern FTPS server implementations (vsftpd, pure-ftpd, proftpd) mandate TLS session reuse / resumption across the control and data channels, specifying the identical session ID and cipher suites across the control and data channels. Since Network.framework lacked a necessary and equivalent to the Secure Transport SSLSetPeerID, I retrenched and rewrote the necessary TLS and TCP with Opportunistic TLS FTP channel abstraction types using the now-deprecated Secure Transport APIs atop the Network.framework-based TCP clear FTP channel type abstraction I had just written.
Using the canonical test server I had been using throughout development, test.rebex.net, this Secure Transport solution seemed to work perfectly, working in clear, secure-control-only, and secure-control+data explicit FTPS operation.
I then proceeded to expand testing to include a broad set of Microsoft FTP Service, pure-ftpd, vsftpd, proftpd, and other FTP servers identified on the Internet (a subset from this list: https://gist.github.com/mnjstwins/85ac8348d6faeb32b25908d447943300).
In doing that testing, beyond test.rebex.net, I was unable to identify a single (among hundreds), that successfully work with secure-control+data explicit FTPS operation even though nearly all of them work with secure-control-only explicit FTPS operation.
So, I started regressing my libncftp + Network.framework + Secure Transport implementation against curl 8.7.1 on macOS 14.7.2 “Sonoma":
% which curl; `which curl` --version
/usr/bin/curl
curl 8.7.1 (x86_64-apple-darwin23.0) libcurl/8.7.1 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.61.0
Release-Date: 2024-03-27
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL threadsafe UnixSockets
I find that curl (also apparently written against Secure Transport) works in almost all of the cases my libncftp does not. This is a representative example:
% ./samples/misc/ncftpgetbytes -d stderr --secure --explicit --secure-both ftps://ftp.sjtu.edu.cn:21/pub/README.NetInstall
which fails in the secure-control+data case with errSSLClosedAbort on the data channel TLS handshake, just after ClientHello, attempts whereas:
% curl -4 --verbose --ftp-pasv --ftp-ssl-reqd ftp://ftp.sjtu.edu.cn:21/pub/README.NetInstall
succeeds.
I took an in-depth look at the implementation of github.com/apple-oss-distributions/curl/ and git/github.com/apple-oss-distributions/Security/ to identify areas where my implementation was, perhaps, deficient relative to curl and its curl/lib/vtls/sectransp.c Secure Transport implementation. As far as I can tell, I am doing everything consistently with what the Apple OSS implementation of curl is doing. The analysis included:
SSLSetALPNProtocols
Not applicable for FTP; only used for HTTP/2 and HTTP/3.
SSLSetCertificate
Should only be relevant when a custom, non-Keychain-based certificate is used.
SSLSetEnabledCiphers
This could be an issue; however, the cipher suite used for the data channel should be the same as that used for the control channel. curl talks about disabling "weak" cipher suites that are known-insecure even though the default suites macOS enables are unlikely to enable them.
SSLSetProtocolVersionEnabled
We do not appear to be getting a protocol version negotiation error, so this seems unlikely, but possible.
SSLSetProtocolVersionMax
We do not appear to be getting a protocol version negotiation error, so this seems unlikely, but possible.
SSLSetProtocolVersionMin
We do not appear to be getting a protocol version negotiation error, so this seems unlikely, but possible.
SSLSetSessionOption( , kSSLSessionOptionFalseStart)
curl does seem to enable this for certain versions of macOS and disables it for others. Possible.
Running curl with the --false-start option does not seem to make a difference.
SSLSetSessionOption( , kSSLSessionOptionSendOneByteRecord)
Corresponds to "*****" which seems defaulted and is related to an SSL security flaw when using CBC-based block encryption ciphers, which is not applicable here.
Based on that, further experiments I attempted included:
Disable use of kSSLSessionOptionBreakOnServerAuth: No impact
Assert use of kSSLSessionOptionFalseStart: No impact
Assert use of kSSLSessionOptionSendOneByteRecord: No impact
Use SSLSetProtocolVersionMin and SSLSetProtocolVersionMax in various combinations: No impact
Use SSLSetProtocolVersionEnabled in various combinations: No impact
Forcibly set a single cipher suite (TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, known to work with a given server): No impact
Employ a SetDefaultCipherSuites function similar to what curl does (filtering out “weak” cipher suites): No impact
Notably, I can never coax a similar set of cipher suites that macOS curl does with that technique. In fact, it publishes ciphers that aren’t even in <Security/CipherSuite.h> nor referenced by github.com/apple-oss-distributions/curl/curl/lib/vtls/sectransp.c.
Assert use of kSSLSessionOptionAllowRenegotiation: No impact
Assert use of kSSLSessionOptionEnableSessionTickets: No impact
Looking at Wireshark, my ClientHello includes status_request, signed_certificate_timestamp, and extended_master_secret extensions whereas macOS curl's never do--same Secure Transport APIs. None of the above API experiments seem to influence the inclusion / exclusion of those three ClientHello additions.
Any suggestions are welcomed that might shine a light on what native curl has access to that allows it to work with ST for these FTP secure-control+data use cases.