URLSession can't complete task when server is not found

URLSession starts waiting for connectivity when network is available, but requested domain doesn't not exist.

What is your question based on the information provided?

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

how can I receive completion of URLSession dataTask get called if request fails

Well, if the domain does exist and your app is in low data mode and is in a constrained network, such as cellular, you can get in a situation where your request goes into taskIsWaitingForConnectivity and will not leave this state unless network conditions improve.

Now, to improve this and to get a callback response for your dataTask you could try to set a timeout on your URLRequest for something that is under 60 seconds.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

if the domain does exist and your app is in low data mode and is in a constrained network, such as cellular

The behaviour i described appears in normal conditions, battery life is fine, both wi-fi and cellular were tested, and both on simulator and on device.

The behaviour i described appears in normal conditions, battery life is fine, both wi-fi and cellular were tested, and both on simulator and on device.

Okay.

Regarding:

but requested domain doesn't not exist

Okay, I did try this with a test bed project I have and I was able to reproduce what you have described with a bogus URL for https://apassdadasdsa.com/ . I seen the dataTask go into the delegate method for taskIsWaitingForConnectivity and just hang. Now, there are a few things you can do here to eliminate this situation; first always make sure you are able to use a domain that does resolve itself to a reachable server. Next, you can set a timeoutInterval on the URLRequest. If you use something like:

class NetworkSessionManager: NSObject {
    
    // MARK: - Private Variables
    
    private var urlSession: URLSession?
    
    // MARK: - Private Constants
    
    private let config: URLSessionConfiguration = URLSessionConfiguration.default
    
    // Initialize the class with a delegate to send updates on.
    override init() {
        super.init()
        config.waitsForConnectivity = true
        config.allowsCellularAccess = true
        config.timeoutIntervalForResource = 15
        urlSession = URLSession(configuration: config, delegate: self, delegateQueue: .main)
    }

    func dataTask(with url: String) {
        guard let url = URL(string: url),
            let unwrappedURLSession = urlSession else { return }
        
        var urlRequest = URLRequest(url: url, timeoutInterval: 15.0)
        urlRequest.httpMethod = "GET"
        
        let task = unwrappedURLSession.dataTask(with: urlRequest, completionHandler: { (data, response, error) in
            if let error = error {
                print("Error: \(error.localizedDescription)")
            }

            if let httpResponse = response as? HTTPURLResponse {
                print("HTTP Response: \(httpResponse.description)")
            }
        })
        task.resume()
    }
}

extension NetworkSessionManager: URLSessionDelegate {
    func urlSession(_ session: URLSession, taskIsWaitingForConnectivity task: URLSessionTask) {
        print("session: \(session.debugDescription)")
    }
}

extension NetworkSessionManager:  URLSessionDataDelegate {
    
    func urlSession(_ session: URLSession, task: URLSessionTask, didFinishCollecting metrics: URLSessionTaskMetrics) {
        for metric in metrics.transactionMetrics {
            print("Metric: \(metric)")
            
            switch metric.resourceFetchType {
            case .localCache:
                print("Resource was fetched from the local cache")
            case .networkLoad:
                print("Resource was fetched from the network")
            case .serverPush:
                print("Resource was a server push")
            case .unknown:
                print("Resource was unknown")
            @unknown default:
                print("Resource was unknown")
            }
        }
    }
}

You will get something like:

session: <__NSURLSessionLocal: 0x11fd0c0b0>
2021-08-28 11:08:39.441021-0700 URLSessionExample[6478:3825943] Task <135016DD-A5E9-4AF7-BADD-52DAB6CCEA36>.<1> finished with error [-1001] Error Domain=NSURLErrorDomain Code=-1001 "The request timed out." UserInfo={NSErrorFailingURLStringKey=https://apassdadasdsa.com/, NSErrorFailingURLKey=https://apassdadasdsa.com/, _kCFStreamErrorDomainKey=4, _kCFStreamErrorCodeKey=-2103, NSLocalizedDescription=The request timed out.}
Error: The request timed out.
Metric: (Request) <NSMutableURLRequest: 0x283610030> { URL: https://apassdadasdsa.com/ }
(Response) (null)
(Fetch Start) 2021-08-28 18:08:24 +0000
(Domain Lookup Start) (null)
(Domain Lookup End) (null)
(Connect Start) (null)
(Secure Connection Start) (null)
(Secure Connection End) (null)
(Connect End) (null)
(Request Start) (null)
(Request End) (null)
(Response Start) (null)
(Response End) (null)
(Protocol Name) (null)
(Proxy Connection) NO
(Reused Connection) NO
(Fetch Type) Network Load
(Request Header Bytes) 0
(Request Body Transfer Bytes) 0
(Request Body Bytes) 0
(Response Header Bytes) 0
(Response Body Transfer Bytes) 0
(Response Body Bytes) 0
(Local Address) (null)
(Local Port) (null)
(Remote Address) (null)
(Remote Port) (null)
(TLS Protocol Version) 0x0000
(TLS Cipher Suite) 0x0000
(Cellular) NO
(Expensive) NO
(Constrained) NO
(Multipath) NO

Hope this helps.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

first always make sure you are able to use a domain that does resolve itself to a reachable server.

In project i'm working on i have no knowledge about validity of resource before making request. Is there a way to validate resource somehow? I tried to search some possible ways to check resource programmatically earlier, but didn't find a way.

you can set a timeoutInterval on the URLRequest

The problem with timeoutIntervalForResource property is that i am loosing network management behaviour provided by system in situations when there are really no available network connection.

I was able to reproduce what you have described with a bogus URL for https://apassdadasdsa.com/

I was hoping that it is not how things should work, and it's some kind of bug, don't understand why does it works that way..

In project i'm working on i have no knowledge about validity of resource before making request. Is there a way to validate resource somehow?

Yeah, there are ways to validate that the hostname you are trying to reach is a viable endpoint, here are a few ways:

1. If you remove waitsForConnectivity on your URLSessionConfiguration the request will immediately fail.

2. You could drop down to BSD sockets and try to resolve your hostname, but this seems like way more trouble than it's worth.

Regarding:

I was hoping that it is not how things should work, and it's some kind of bug, don't understand why does it works that way..

This is absolutely not a bug. I suspect the part that is causing you confusion here is the waitsForConnectivity piece.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

You could drop down to BSD sockets and try to resolve your hostname, but this seems like way more trouble than it's worth.

I will try to make some research on BSD sockets, this things is new for me.

I suspect the part that is causing you confusion here is the waitsForConnectivity piece.

You are probably right, but if i'll disable it, i'll lose that system provided network availability handling, and not only that, but also, waitsForConnectivity allows to perform many concurrent requests at a time. Let's say i have hundreds of addresses i need to request, then URLSession would handle this process concurrently by making suitable decisions for itself internally. That is also why timeoutIntervalForResource did not worked for me, it fails all those concurrent tasks which URLSession place to "wait in queue".

You are probably right, but if i'll disable it, i'll lose that system provided network availability handling, and not only that, but also, waitsForConnectivity allows to perform many concurrent requests at a time. Let's say i have hundreds of addresses i need to request, then URLSession would handle this process concurrently by making suitable decisions for itself internally.

Can you tell me more about why you need to make hundreds of concurrent requests?

Regarding:

That is also why timeoutIntervalForResource did not worked for me, it fails all those concurrent tasks which URLSession place to "wait in queue".

This sounds like it would be a beneficial thing though because if these hostnames do not resolve to anything then they should fail and your app should move on. If you are in a poor connectivity state or on a constrained network then you could add this request back into a retry queue after the timeout takes place.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

Can you tell me more about why you need to make hundreds of concurrent requests?

I'm requesting bunch of rss feeds, and as soon as each request complete, i display it to user.

This sounds like it would be a beneficial thing though because if these hostnames do not resolve to anything then they should fail and your app should move on. If you are in a poor connectivity state or on a constrained network then you could add this request back into a retry queue after the timeout takes place.

When URLSession starts so many tasks at once, most of those tasks would just wait in queue for others to complete, and when timeoutIntervalForResource is set, then all those tasks in queue would be cancelled. Looks like manually management of queuing restricted amount of tasks for URLSession could be an option. If so, those tasks with non existed addresses mostly would still fall to waiting mode, and then would be cancelled by timeoutIntervalForResource. Seems like i still lose network availability handling which can retry requests for me.

But what i've noticed, is that if waitsForConnectivity is disabled, then i still can start all tasks concurrently, and they actually fails with proper error. I was wrong saying:

waitsForConnectivity allows to perform many concurrent requests at a time

So maybe the easiest way would be to just retry manually failed requests, and with proper errors i even would be able to retry specific requests, for example: requests where error is not A server with the specified hostname could not be found. If so, then your suggestion with removing waitsForConnectivity was actually an option for me. I'll try it out.

Still not clear for me though, why URLSession do not fail request when waitsForConnectivity enabled, and A server with the specified hostname could not be found. error can be thrown, if URLSession can understand that when waitsForConnectivity is disabled.

But what i've noticed, is that if waitsForConnectivity is disabled, then i still can start all tasks concurrently, and they actually fails with proper error.

Yep.

Regarding:

So maybe the easiest way would be to just retry manually failed requests, and with proper errors i even would be able to retry specific requests, for example: requests where error is not A server with the specified hostname could not be found. If so, then your suggestion with removing waitsForConnectivity was actually an option for me. I'll try it out.

Cool!

Regarding:

Still not clear for me though, why URLSession do not fail request when waitsForConnectivity enabled, and A server with the specified hostname could not be found. error can be thrown, if URLSession can understand that when waitsForConnectivity is disabled.

My take is that the URLSession object is in a state for the data task will wait as long as it needs to until a valid connection to a hostname is possible and that is why you are seeing this. However, that is just my take and it would be better to open an enhancement request to get this situation documented.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com