iOS 11.3: WKWebView cookie synchronizing stopped working

Thank you!

I've been struggling with an intermittent dropping of session cookies for ages, and using a single WkProcessPool as per your first piece of code appears to have solved it (I haven't seen a drop in 2 weeks now)

Hi guys, I am seeing something similar in an App that I am working on where there seems to be an intermittent problem with the expected Cookies being available in the WKWebView's cookie store. Is there a consensus that forcing the default WKWebsiteDataStore class into action by calling a method on it might avoid problems caused by lazy initialisation ?

Hi Quinn,

I’ve been tracing the cookie issues for over a month now and I am
able to reproduce and remedy all test cases 100%.

I do have a dedicated test environment with utility buttons in the UI
that let me e.g. log the cookieStore, empty the cookieStore etc.

All WKWebview delegate methods and cookieStore observers are
in place and log to the console when they fire.


So the 2 take away test scenarios for you are these:


———————————————————
SCENARIO 1:
———————————————————


• starting the test environment
• totally resetting the cookieStore by deleting all cookies
• logging the cookieStore shows that it is empty
• quitting the test environment


• re-loading the test environment
• environment is loading a test webpage from the internet with:
— myWebview.(URLRequest)
• the page loads correctly
• the response-header’s ‘Set-Cookie’ field requests 3 cookies to be set
• the WKWebview sets 3 cookies (as requested in the response-header’s ‘Set-Cookie’ field)
• the didChangeCookieStore callback fires 3 times
• logging the cookieStore shows that there are 3 cookies
• the 3 cookies are:
— 1 permanent cookie (with expiration date set) and
— 2 temporary session cookies (without expiration date set) -> [csrfToken, CAKEPHP] tokens


———————————————————
Problems:
———————————————————


NONE !!!


———————————————————
Observations:
———————————————————


Unfortunately the logged response header DOESN’T display the ‘Set-Cookie’ field !!!


=====> WK-WEB-VIEW: decidePolicyFor navigation RESPONSE

<WKNavigationResponse: 0x160a13df0; response = <NSHTTPURLResponse: 0x160a1a990>
{ URL:https://testURL }
{ Status Code: 200, Headers {...}}


———————————————————
SCENARIO 2:
———————————————————


• totally resetting the cookieStore by deleting all cookies
• logging the cookieStore shows that it is empty
• quitting the test environment


• re-loading the test environment
• environment is loading a test webpage from the internet with
— URLSession.shared.dataTask(with: URLRequest) { data, response, error in ... }
• the response-header’s ‘Set-Cookie’ field requests 3 cookies to be set
• the didChangeCookieStore callback fires 3 times
• logging the cookieStore shows that there is only 1 cookie
• the 1 cookie is:
— 1 permanent cookie (with expiration date set)
— NONE of the 2 expected temporary session cookies (without expiration date set) have been set -> [csrfToken, CAKEPHP] tokens


———————————————————
Problems:
———————————————————


Of course, if you now load the mime ‘text/HTML’ payload
(by casting the response’s data object into a string) with

• myWebview.loadHTMLString()

all user actions on that HTML page like e.g. triggering hidden requests
in POST forms that rely on the [csrfToken, CAKEPHP] tokens etc., are
denied by the server because of the missing cookies.

Equally, if there is e.g. an additional ajax HTTP request in this HTML’s
onLoad() function, I can see in the server logs, that this additional request
comes without the 2 [csrfToken, CAKEPHP] session cookies attached
(only the single permanent cookie is present) – and so the server re-sends
a new pair of [csrfToken, CAKEPHP] cookies with the new response.

As this ajax request/response only updates some minor data in the UI, but
the DOM is still the the same from the initial page request, you can imagine
that the new [csrfToken, CAKEPHP] cookies do not match up with all the
references in the HTML either, because of ‘token mismatches’.

The page is still broken!


———————————————————
Observations:
———————————————————


As the problems only seem to affect temporary session cookies maybe the bug
trace should focus on that area. Remember - every scenario was started with a
completely wiped out empty cookie store!


———————————————————


Fortunately the logged response header DOES display the ‘Set-Cookie’ field !!!


=====> WK-WEB-VIEW: decidePolicyFor navigation RESPONSE

RESPONSE: <NSHTTPURLResponse: 0x110172520> { URL: https://testURL } { Status Code: 200, Headers {

    "Set-Cookie" =     (

        "CAKEPHP=bjhbajljh9f78vm9mko836kk2h; path=/; secure; HttpOnly",

        "interfaceLanguage=deu; expires=Sat, 15-May-2021 16:21:10 GMT; Max-Age=2591999; path=/",

     "csrfToken=591d618d10632e05f1aa79e003e82a6c5da73781f55466782885...; path=/"

    );
} }


———————————————————
Miscellaneous:
———————————————————


When the navigationAction.didFinish delegate fires, thecookieStore isn’t
properly set yet, despite the didChangeCookieStore observerver having
fired long before.

You have to wait about 1.75 seconds in the best case scenario, but often
up to 2.5 seconds. So right now 3 seconds seems to be a safe setting.

I would rather expect the didFinish delegate to fire when even the
asynchronous cookieStore is in a valid state, or at least have an additional
delegate method for when really everything is completed...


Needless to say, that I did all cookieStore checks manually with a utility
button in my test environment UI, waiting at least 5 seconds to be sure
that all asynchronous tasks have finished.


———————————————————
REMEDIES:
———————————————————


One remedy for the missing cookie issue is to stash away the ‘Set-Cookie’
header from the initial request, extract the tokens from the signature,
instantiate some custom cookies with the respective values and store
them into the cookieStore.

But that is a little bit of a management nightmare!


———————————————————
GENERAL OBSERVATION AND
ENHANCEMENT REQUEST
———————————————————

Unfortunately the system doesn’t inform you when the server responds with a
REDIRECT 302 status. Neither with a <decidePolicyFor navigation RESPONSE>
nor with a dedicated callback. The system rather decides to issue a new request
automatically and you only find yourself with an additional invocation of
<decidePolicyFor navigation ACTION> wondering where that came from,
because there is no mention of any status 302 in the whole processing and
logging chain.

If I hadn’t had access to the server logs I would have never known about the
302, and kept on wondering, what in my code triggers this additional request.

However, I have seen in the WKNavigationAction.h file that there is a property
called ’isRedirect’ just along with ‘navigationType’ etc. that one can access via

• navigationAction.value(forKeyPath: "isRedirect")

In the case of the automatic REDIRECT that the system performs, this value is
set to 1 for the following <decidePolicyFor navigation ACTION>.

It would be really helpful if the navigationAction.description would make this
information available along with

• <WKNavigationAction: 0x112076910;
— navigationType = -1;
— syntheticClickType = 0;
— position x = 0.00 y = 0.00

— isRedirect = 1

— request = ...


without having to explicitly poll for it.


———————————————————

ADDENDUM:

One thing that would clarify this problem, however not solve it, is the fact that a plain

• myWebview.load(URLRequest)

is using Mozilla/5.0 as the agent, which is confirmed by the server log.

However

• URLSession.shared.dataTask(with: URLRequest) { data, response, error in ... }

uses a CFNetwork executionExtension as the agent, also confirmed by the server log.


So if the executionExtension spins up its own URLSession that writes the cookies to
the cookieStore, it could be that after the dataTask’s completion handler the session
gets deallocated and hence the temporary session cookies get removed from the
cookieStore right away.

The question then is, how do you get a copy of those short-lived cookies before the
extension closes, because even

• myWebview.configuration.websiteDataStore.httpCookieStore.getAllCookies { cookies in ...}

is asynchronous and too slow to grab hold of them. And even when
loading the response’s mime ‘text/HTML’ payload with

• myWebview.loadHTMLString()

right in the completion handler, it is too slow to find the cookies it would
need to e.g. attach to an ajax request in the onLoad() function of the
loaded HTML page.

Not to mention the general out-of-sync situation of all the forms in the
HTML that rely on corresponding tokens stored in cookies in order to
make POST requests etc.



———————————————————

WKWebView is a asynchronous component, UIWebView is a synchronous component....is there any option to force synchronous ?