Post not yet marked as solved
I maintain a large production app which uses background-configured URLSession to upload large numbers of files concurrently to our servers using PUT requests. These are file based uploads (https://developer.apple.com/documentation/foundation/urlsession/1411550-uploadtask) as required by background URLSessions to be support. We concurrently upload up to 16 files at once, although the configuration specifies only configuration.httpMaximumConnectionsPerHost = 4
When our servers migrated to Http2, I noticed that users who were uploading concurrently were returned many errors by URLSession. Specially through the delegate method urlSession(session: task: didCompleteWithError:), the error returned was NSURLErrorCannotParseResponse which has little documentation or public discussion.
When our servers reverted this Http2 change, back to regular Http1.1, this stopped happening. No change in the actual server response was made. Since iOS negotiates the protocol the connection uses without our applications involvement, I cannot seem to choose a network protocol to force http1.1. I also, cannot see any other details from this NSError (no underlying error or other obvious issues). All I can do is log the network protocol via task metrics.
It seems that when this error occurs, its as if the urlsessiond has crashed or the network stack just falls apart and I get many NSURLErrorCannotParseResponse all at once before users can manually retry these failures.
This URLSession is configured by
let configuration = URLSessionConfiguration.background(withIdentifier: backgroundIdentifier)
configuration.waitsForConnectivity = true
configuration.allowsExpensiveNetworkAccess = true
configuration.httpMaximumConnectionsPerHost = 4
This is not a good experience and I have no other way to solve these errors. Does anyone have advice for this scenario?
Set up Charles well, and can monitor the network requests on the app, when the app is in foreground.
The app receive a background notification, will trigger to make a api call.
But the api call cannot be shown in Charles when the app in backgroud.
Do ios do some disguise for background network request, so that cannot be shown in Charles? Or it should can be monitored, additional settings required? thx for anyone could share related experience
I have some code that used to work but is failing to bind to a local UDP port saying it is already in use.
lsof does not show that the port is being used.
Below is my receive function that fails. bindResult is "-1" when calling bind, and the error printed out contains "Address already in use (48)". I am trying to bind to 127.0.0.1:6000.
This has been working for a very long time, but I've not used it for one or two months, so not sure if a macOS upgrade or something else broke it.
func receive(handleRxData: @escaping (UdpSocket, IpAddress, ArraySlice<UInt8>) -> Void,
withError error: (_ msg: String) -> Void) {
self.handleRxData = handleRxData
var cfSocketContext = CFSocketContext(version: 0, info: nil, retain: nil, release: nil, copyDescription: nil)
cfSocketContext.info = Unmanaged.passRetained(self).toOpaque()
cfSock = CFSocketCreate(kCFAllocatorDefault,
PF_INET,
SOCK_DGRAM,
IPPROTO_UDP,
CFSocketCallBackType.readCallBack.rawValue,
{ (socket: CFSocket?, callBackType: CFSocketCallBackType, address: CFData?, data: UnsafeRawPointer?, info: UnsafeMutableRawPointer?) -> Void in
let udpSocket = Unmanaged<UdpSocket>.fromOpaque(info!).takeUnretainedValue()
udpSocket.receiveCallback()
},
UnsafeMutablePointer<CFSocketContext>(&cfSocketContext))
let sock = CFSocketGetNative(cfSock)
// Create ipv4 addr struct:
var sin = sockaddr_in()
if (local.type == .ipv4) {
sin.sin_len = __uint8_t(MemoryLayout.size(ofValue: sin))
sin.sin_family = sa_family_t(AF_INET)
sin.sin_addr.s_addr = local.ipv4.bigEndian
sin.sin_port = local.port.bigEndian
}
let bindResult = withUnsafeMutablePointer(to: &sin) {
$0.withMemoryRebound(to: sockaddr.self, capacity: 1) {
bind(sock, UnsafeMutablePointer<sockaddr>($0), socklen_t(MemoryLayout<sockaddr_in>.size))
}
}
if bindResult < 0 {
error("Could not bind to socket \(sin) ( \(String(cString: strerror(errno)!)) (\(errno)).")
return
}
// Change socket to non-blocking:
let flags = fcntl(sock, F_GETFL);
let fcntlResult = fcntl(sock, F_SETFL, flags | O_NONBLOCK);
if (fcntlResult < 0) {
print("Could not change socket to non-blocking ( \(String(cString: strerror(errno)!)) (\(errno)).")
}
// Add to run loop:
let rls = CFSocketCreateRunLoopSource(nil, cfSock, 0);
if (rls == nil) {
error("Could not get run loop source.")
return
}
CFRunLoopAddSource(CFRunLoopGetCurrent(), rls, CFRunLoopMode.commonModes)// CFRunLoopMode.defaultMode);
}
Post not yet marked as solved
I wanted to implement a retry mechanism for a NSURLSessionDataTask. In android I am seeing that we can simply set a retry policy for the volley request as such
myRequest.setRetryPolicy(new DefaultRetryPolicy(
(int) TimeUnit.SECONDS.toMillis(200),
DefaultRetryPolicy.DEFAULT_MAX_RETRIES,
DefaultRetryPolicy.DEFAULT_BACKOFF_MULT));
Ref: https://afzaln.com/volley/com/android/volley/DefaultRetryPolicy.html
I tried finding something similar for NSURLSessionDataTask but havent been able to find it yet. Is there any iOS SDK support for this?
Or do we need to implement something like below pseudocode for retries with backoff?
self.session = [NSURLSession sessionWithConfiguration:[NSURLSessionConfiguration defaultSessionConfiguration]];
[self callServiceWithRetries:3 timeAtStart:[NSDate date] andTimeOut:200];
id callServiceWithRetries:(int)retries
timeAtStart:(NSDate)timeAtStart
andTimeOut:(int)timeout{
__weak __typeof(self) weakSelf = self;
NSURLRequest *request = //create request
NSDate *startDate = [NSDate date];
NSURLSessionDataTask *task = [self.session dataTaskWithRequest:request completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
NSDate *endDate = [NSDate date];
NSInterval timeIntervalSinceStart = [endDate timeIntervalSinceDate:timeAtStart];
NSInteger timeSinceStart = ((timeIntervalSinceStart % 1) * 1000);
if (error) {
if(retries > 0 && timeSinceStart < timeOut){
// add delay/backoff here before making the request. To calculate the delay we can
// use the current number of retries made. Something like exponential back off here
[weakSelf callServiceWithRetries:retries - 1 timeAtStart:timeAtStart andTimeout:timeout];
}
else{
//failure callback
}
return;
}
if(non 2xx error){
if(retries > 0 && timeSinceStart < timeOut){
// add delay/backoff here before making the request. To calculate the delay we can
// use the current number of retries made. Something like exponential back off here
[weakSelf callServiceWithRetries:retries - 1 timeAtStart:timeAtStart andTimeout:timeout];
}
else{
//failure callback
}
}
//success callback
}
}
Post not yet marked as solved
Is there a way for iOS NSURLSession to control dns resolution by itself, similar to the ability of the libcurl library (https://curl.se/libcurl/c/CURLOPT_RESOLVE.html). Because I found that my app's domain name was hijacked in the public network.
Good day,
I have a use case that I am currently facing problems with.
I am currently implementing a downloader service that uses some c++ code to download in the foreground, and when the user backgrounds the app, NSURLSession kicks in.
Due to chunking, we have a bunch of endpoints that contain ~1mb worth of data for each chunk, and these chunks make up a full file when processed. Small files would be downloaded as is.
As such, there might potentially be ten thousands worth of endpoints that NSURLSession might need to download from in the background.
From what I've read around here, there are four things to take note of.
Resume Rate Limiter
Number of tasks queued into nsurlsessiond
Number of concurrently running tasks
NSUrlSessionDownloadTask creation
My current implementation is to create a bunch of tasks in applicationDidEnterBackground(_:) and then starting them all at once. However, NSUrlSessionDownloadTask creation is very expensive. As a reference, it takes 3.7 seconds to create 400 tasks on my iPhone 7+.
Upwards of 500 task creation and it runs into the territory of the OS terminating my app as I did not return out of applicationDidEnterBackground(_:) in five seconds.
Creation of new tasks when the first batch ends is also not an option because of resume rate limiter. According to here, the delay is really heavy and is something I would like to avoid.
Are there any viable solutions that would solve my problem of downloading many files in the background? Thanks in advance!
Post not yet marked as solved
Directly set the mapping relationship between domain name and IP, let my app use it with NSURLSession, Is there a related interface?
After the system was upgraded to iOS15.1, our App crashed with AFSecurityPolicy. But I checked the code in the project and there is no problem, please tell me what happened to iOS15.1?
Crash Log
My App uses MQTT Client, but NSStream Input crashes, I don't know why. Can you tell me how to solve this problem? Attached is my full crash log:
Crash Log
Post not yet marked as solved
My App uses MQTT Client, but NSStream Input crashes, I don't know why. Can you tell me how to solve this problem? Attached is my full crash log.
Crash Log
Hi,
I'm writing application which use NSURLSession for https communication.
It was all good until I change my app to run as root (using sudo).
It is failing on the connection (SSL Error) and I suspect the fault is that it is not able to validate the server certificate.
I install the relevant root certificate in the system keychain (beside login) but it did not help.
What I should do for root user to be able to use the certificates in system keychain ?
b.t.w - I'm running on macOS BigSur
Post not yet marked as solved
Hi developers,
I'm facing a weird issue with URLSessionUploadTask.
I created a URLSession using background config:
let config = URLSessionConfiguration.background(withIdentifier: identifier)
config.httpMaximumConnectionsPerHost = 1
config.allowsCellularAccess = true
config.sessionSendsLaunchEvents = true
config.waitsForConnectivity = true
let backgroundURLSession = URLSession(configuration: config, delegate: self, delegateQueue: OperationQueue.main)
Started a task using this code:
let task = backgroundURLSession?.uploadTask(with: urlRequest, fromFile: fileURL)
task?.taskDescription = "someIdentifier"
task?.resume()
Now when I'm trying to cancel this task (whenever the user presses the cancel button on UI), the cancel() method is not calling the delegate method as mentioned in the documentation.
urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?)
Code used to cancel the task:
backgroundURLSession?.getAllTasks { tasks in
for task in tasks where task.taskDescription == identifier {
task.cancel()
break
}
}
While debugging I can see that task.cancel() is getting called and the state of the task is getting changed from 0 i.e. running to 2 i.e.canceling.
Any help would be appreciated. Thanks in advance!
Post not yet marked as solved
Note Much of this content has been rolled into URL Loading System documentation, but I’m leaving this doc here for my own reference.
NSURLSession background sessions are optimised for transferring a small number of large resources. Moreover, for download transfers it’s best if the transfer is resumable. This design makes the best use of client device resources and the available network bandwidth. If your app runs a lot of tasks in a background session, you should rethink its design. Below you’ll find a number of options you might consider.
Most of these options require server-side support. If your server does not have this support, and you can’t add it — perhaps you’re writing a client app for some fourth-party server — you won’t be able to implement these options directly. In that case consider creating your own server that sits between your app and the final server and implements the necessary smarts required to optimise your app’s network usage.
If that’s not possible, a final option is to not use a background session but instead take advantage of the BackgroundTasks framework. See BackgroundTasks Framework, below.
Basics
The basic strategy here is to have the sender (the server for a download, your app for an upload) pack the data into some sort of archive, transfer that archive over the network, and then have the receiver unpack it. There are, however, a number of complications, as described in the subsequent sections.
Archive Format
The obvious choices for the archive format are zip and tar. macOS has lots of options for handling these formats but none of that support is present on iOS (r. 22151959). OTOH, it’s easy to find fourth-party libraries to fill in this gap.
Incremental Transfers
It’s common to have a corpus of data at one end of the connection that you need to replicate at the other. If the data is large, you don’t want to transfer the whole thing every time there’s an update. Consider using the following strategies to deal with this:
Catalogue diff — In this approach the receiver first downloads a catalogue from the sender, then diffs its current state against that catalogue, then requests all the things that are missing. Alternatively, the receiver passes a catalogue of what it has to the sender, at which point the sender does the diff and returns the things that are missing. The critical part is that, once the diff has been done, all of the missing resources are transferred in a single archive.
The biggest drawback here is resume. If the sender is working with lots of different receivers, each of which has their own unique needs, the sender must keep a lot of unique archives around so it can resume a failed transfer. This can be a serious headache.
Versions — In this approach you manage changes to the data as separate versions. The receiver passes the version number it has to the sender, at which point the sender knows exactly what data the receiver needs.
This approach requires a bit more structure but it does avoid the above-mentioned problem with resume. The sender only needs to maintain a limited number of version diffs. In fact, you can balance the number of diffs against your desire to reduce network usage: Maintaining a lot of diffs means that you only have to transfer exactly what the receiver needs, while maintaining fewer diffs makes for a simpler server at the cost of a less efficient use of the network.
Download versus Upload
The discussion so far has applied equally to both downloads and uploads. There is, however, one key difference: NSURLSession does not support resumable uploads. When doing an upload you have to balance the number of tasks you submit to the session against the negative effects of a transfer failing. For example, if you do a single large upload then it’s annoying if the transfer fails when it’s 99% complete. On the other hand, if you do lots of tiny uploads, you’re working against the NSURLSession background session design.
It is possible to support resumable uploads with sufficient server-side support. For example, you could implement an algorithm like this:
Run an initial request to allocate an upload ID.
Start the upload with that upload ID.
If it completes successfully, you’re done.
If it fails, make a request with the upload ID to find out how much the server received.
Start a new upload for the remaining data.
(r. 22323347)
BackgroundTasks Framework
If you’re unable to use an NSURLSession background session effectively, you do have an alternative, namely, combining a standard session with the BackgroundTasks framework, and specifically the BGProcessingTaskRequest. This allows you to request extended processing time from the system. Once you’ve been granted that time, use it to run your many small network requests in a standard session.
The main drawback to this approach is latency: The system may not grant your request for many hours. Indeed, it’s common for these requests to run overnight, once the user has connected their device to a power source.
Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"
Change history:
18 Aug 2015 — First written.
24 Mar 2018 — Added the BackgroundTasks Framework section. Other editorial changes.
31 Jan 2022 — Fixed the formatting and tags. Added a link to the official docs.
Post not yet marked as solved
Hi all,
How do I, Ensure ATS is enabled within the iOS mobile application so that confidential information sent between the application and the back end
servers are secured and not to be intercepted by man-inthe-middle style attacks.
Below is what I already set but still Fails for the penetration test.
Post not yet marked as solved
Hi,
I implemented the NSPinnedDomains according to https://developer.apple.com/news/?id=g9ejcf8y "Identity Pinning: How to configure server certificates for your app".
This is my config (I added a wrong SHA256 hash so I can test if it works):
keyNSAppTransportSecurity/key
dict
keyNSPinnedDomains/key
dict
keyjsonplaceholder.typicode.com/key
dict
keyNSPinnedCAIdentities/key
array
dict
keySPKI-SHA256-BASE64/key
stringr/mIkG3eEpVdm+u/ko/cwxzOMo1bk4TyHIlByibiA5E=/string
/dict
/array
keyNSIncludesSubdomains/key
true/
/dict
/dict
/dict
With NSURLSessions it properly fails when I try to load https://jsonplaceholder.typicode.com/todos/2
it prints following error:
An SSL error has occurred and a secure connection to the server cannot be made
But when I try to load the URL from JavaScript in WKWebView, it succeeds.
Is WKWebView not supported? Or am I doing something wrong?
Thanks and kind regards,
Mika
Objc code:
NSString *urlString = @"https://jsonplaceholder.typicode.com/todos/2";
NSURL *url = [NSURL URLWithString:urlString];
NSURLSessionDataTask *downloadTask = [[NSURLSession sharedSession]
dataTaskWithURL:url completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
}];
[downloadTask resume];
Web code (can be included in any html page inside WKWebView):
script
document.addEventListener("DOMContentLoaded", () = {
fetch("https://jsonplaceholder.typicode.com/todos/2")
.then(res = res.json())
.then(res = console.log(res));
});
/script
Post not yet marked as solved
Hi everyone,
I hope I can provide enough context to receive helpful insight!
I found that the release build of my iOS app (iOS 15) seems to be able to connect to my local dev server (running on localhost) without any changes to Info.release.plist. This is fairly surprising, although I found this link that describes how App Transport Security (ATS) differs between iOS 9/macOS 10.11 and iOS 10+/macOS 10.12+. Namely, "no longer need[ing] an exception for" .local domains, IP addresses, and unqualified domains.
The above explains why Release builds can connect to local IP addresses, but doesn't explain the following questions:
If connections to local IP addresses are allowed by ATS, why can't our Testflight build connect to a local IP address?
What is different between the release build and Testflight build with regards to the above? They are being pulled from App Store and TestFlight respectively, and are configured by the same Info.release.plist from my observation.
Forum thread 69591 appears to be related, but does not address these questions. Any insight and/or resources would be greatly appreciated! Cheers
Post not yet marked as solved
In a production app, I am setting the httpMaximumConnectionsPerHost property to 1 on a URLSessionConfiguration object when creating a URLSession. This allows me to queue a number of downloads, but only download 1 file at a time. Recently I noticed that all queued files download simultaneously on iOS 10 devices. Has anyone else noticed this change? Is this intentional or a bug?Note: I am setting the value of httpMaximumConnectionsPerHost both before and after creating the session, because of a post I found months ago that suggested that some properties of URLSessionConfiguration must be set after creating a URLSession in order for them to take effect. let configuration = URLSessionConfiguration.background(withIdentifier: sessionIdentifier)
configuration.httpMaximumConnectionsPerHost = 1
session = URLSession(
configuration: configuration,
delegate: self,
delegateQueue: nil
)
session.configuration.httpMaximumConnectionsPerHost = 1
Post not yet marked as solved
I am testing an app with a small group of people with iPhones, none of whom have xcode or know how to use it. The app is connecting to a server with an expired SSL certificate, and it will be some time before it can be renewed due to workplace obstacles. We would like to start testing now, rather than waiting for the certificate issue to be fixed, but when I put the app into TestFlight it goes into release mode and will not connect to the expired cert.
Is there a way for me to easily share this app in debug mode so that we can start testing now?
Post not yet marked as solved
Hello,
I'm having an unusual problem dealing with web sockets from Swift. I adopted the URLSessionWebSocketDelegate for my session extension and there are times when after the webSocketTask:didOpenWithProtocol: method is called that I then get errors indicating that the web socket is not connected. This all happens in the foreground with airplane mode off, without putting my app in the background.
I get the connection errors simultaneously in 2 places:
urlSession(_ session:task: didCompleteWithError error: )
webSocketTask?.receive(completionHandler: @escaping (Result<URLSessionWebSocketTask.Message, Error>)
Forgive me as I have 2 rather ****** questions:
Is the web socket open state the same as the web socket's connected state?
If it isn't the same how do I know if a web socket is connected, using Apple's URLSessionTask extension?
Thanks. I'd rather stick with Apple's implementation and not use a third party web socket library. Pretty sure the error is mine.
Post not yet marked as solved
Hi all,
I'm facing a very strange problem in my application. In particular, using the application when a request receives a timeout message, all the following ones receive a timeout too (see screenshot).
Error Domain=NSURLErrorDomain Code=-1001 "The request timed out." UserInfo={NSUnderlyingError=0x28113a550 {Error Domain=kCFErrorDomainCFNetwork Code=-1001 "(null)" UserInfo={_kCFStreamErrorCodeKey=-2102, _kCFStreamErrorDomainKey=4}}, NSErrorFailingURLStringKey=https://MY_SERVER_URL, NSErrorFailingURLKey=https://MY_SERVER_URL, _kCFStreamErrorDomainKey=4, _kCFStreamErrorCodeKey=-2102, NSLocalizedDescription=The request timed out.}
I'm not sure if this happens when the application enters the foreground after a background state since it's quite difficult to replicate.
Based on what exposed,
do you any clue why this can happen?
additionally, is there a way to replicate a similar behavior in a consistent way?
Waiting for a reply I thank you for your attention.
Thanks,
Lorenzo