Enrolled devices stop responding to push notifications

We have a few hundred iOS devices enrolled to our MDM server. They were working perfectly till about October this year. Since October, we've been having random problems with push notifications used to wake up the devices for MDM commands. A few devices simply stop responding to the push notifications and continue to do so till unenrolled and re-enrolled. Push notifications for other applications seem to get through. App Store also seems to work fine. A device restart doesn't seem to help. Push Notification tokens are properly updated in the server too, till the point where it lost contact.


What could be the reason? Any suggestions?


Note: This is not the case where they stop responding after an OS update. That seems to go fine.

Post not yet marked as solved Up vote post of theRealBatman Down vote post of theRealBatman
1.9k views

Answers

The device has most likely updated its push token. When this happens it is supposed to send the new push token to the mdm server.

Im not entirely sure on what happens if the device is unable to contact the mdm server when its push token updates, maybe the device only retries once, retries randomly, never retries, who knows?


Personally I would think it would be nice if the device wants to update its push token it should generate a temp new push token and send it to the server and get verification that the server saved the new info with a simple 200 response or something and then it can start using the new token, however If the device fails to communicate with the server then the device should carry on using the old push token for the time being and retry later, but I doubt that is the case


In the mdm protocol documentation it says:

"Warning: The PushMagic or UnlockToken fields of subsequent TokenUpdate commands may

be identical to those in previous commands, or different. If different, the server should update its

record for the device to the new value provided by the command. Failure to do so will result in the

server being unable to send Push notifications or perform passcode rests."


The only way to get the device to talk with the server again if the token is wrong on the server is to reenroll the device, I dont think you can read the active push token from the device to update your device on the mdm server as that would be a security risk

Hi Guys.


We are observing a similar behavior on our MDM implementation. Did you guys ever manage to root cause the issue and what was the workaround you applied.


Based on our investigation we dont see an instance of the device reporitng a new push magic and it looks like it stops responding to the old one.


Thanks

Krush

Same thing here. AirWatch told me to reset the network settings and see if that would help?? Haven't had the opportunity yet. I've had to reenroll over a dozen this week already. I work for a school system so it’s not uncommon for these things to not be seen for some time, especially holiday, and teachers don't drop what they are doing to tell me about an iPad that’s not got the exact same screen. They have no idea that the device could just slide right into a kid’s book bag never to be seen again, it makes it worse that they feel like they have a safety net because the teachers know I'm able to track them (if the token's a match that is) What's making this problem even more of an issue is that for whatever reason to reenroll a device as the same user account that the device was using before I can't. I get an invalid profile upon enrollment. DEP enrollment. Agent enrollment does seem to work with the same user. Only way to reenroll those devices that have lost contact for me is to delete the old device object, old user object, recreate user object, reassign the user back to the same groups, and then finally reenroll the device. Making matters EVEN worse is that I'm not even able to add the user back to its old user groups because they appear to be gone! Only way I know they didn’t just drop off the planet is that if you look at the current user groups associated with one of those users you'll see those very groups that aren't found anywhere else. So now I’ve got two sets of user accounts some new some old, two sets of user groups some new some old, trying to combine them into the same smart groups so that I don’t have to go through 500 VPP apps and assign them to a second smart group. I've had this ticket open with AirWatch since March, and a previous ticket for another month. God help me if come September these things won't talk at all and I’ve got to start all over I may just have to have that two weeks’ notice in my back pocket by the end of summer.

Have the same issue where a couple of devices stopped responding after precisely two years.

No TokenUpdate messages has been sent from the devices. We log all of those. FIERCELY.

The only thing we are wondering is:


Can a TokenUpdate message ever be piggy backed to another response?

Example:

The last successful command we se for one of the devices is a ClearPasscode command.

It does respond with Acknowledged. Then it is never heard from again. (I have the physical device, it has unrestricted network access etc.)


Could that last Acknowledged response have a "TokenUpdate" message piggy backed to it?

That is the only way I can see us missing it. Our logging is rigorous since TokenUpdate is critical.

No as the TokenUpdate is sent to a completely separate endpoint on your mdm server so the response from the clear passcode command would not have had that token info in it

As part of the device enrollment the 'com.apple.mdm' payload will define the CheckInURL and the ServerURL


CheckInURL:

This is where your tokenupdate data will be sent. This can include the new token, a new pushmagic, and/or a new unlocktoken

The unlocktoken is only used by the clear passcode command


ServerURL:

This is where the device will check if there are any available commands like clearing passcode, locking device, installing apps, etc


Possibly: because you last sent a clear passcode command, once that completed, it might have triggered ios to try and send a new unlocktoken and/or a new token to the CheckInURL but thats just a guess as its not documented on when tokenupdates actually happen


So are you really monitoring the CheckInURL fiercely or just the ServerURL as they are different urls used for different things?


btw I dont think this is only a problem with certain mdm vendors it seems more like an ios issue. I have mostly seen it with ios 10 recently, and on some devices we upgraded them to ios 11 after they lost connection to the mdm server and I saw as part of the upgrade it posted a new token to mdm and we could communicate with it again. That however did not work for all devices but did work for a lot

MDM Vendor here.

We receive multiple complaints from our customers lately, regarding the connectivity of their devices. We read a device log while pushing and the apnsd did not even receive the push, although the apns server responded with an http 200 on our push request.
 
We made sure, that we did not receive any TokenUpdate request from the device.

We use the HTTP2 APNS API to push devices.