Live Caller ID on iOS does not work - client requests not reaching backend

Question

Created Nov ’24

Replies 6

Boosts 3

Participants 2

I'm reaching out to see if anyone else is experiencing issues with the Live Caller ID feature on iOS. We recently encountered a problem where the feature stopped working entirely.

Here's a brief overview of the situation:

We were monitoring test traffic on our backend and noticed everything came to a halt around 1:00 AM UTC on November 15th.
After this time, any attempts to reach our backend through calls failed completely.
I tested this across multiple devices running iOS 18.2 and iOS 18.0.
I used both TestFlight builds and development builds via Xcode, which should communicate directly with our backend.
I experienced the problem on our main application as well as a dedicated test app.
To troubleshoot further, I even set up a local server on localhost and tried directing requests there, but the requests did not reach the local server when a call was received.

Further debugging in Console.app revealed the following error:

identity request returned error: Error Domain=com.apple.CipherML Code=400 "Error Domain=com.apple.CipherML Code=401 "Unable to request data by keywords batch: failed to fetch token issuer directory"

However, when I manually tried to hit our server endpoint using curl, the request successfully reached the server:

curl https://our_server/something
hb_method=GET hb_uri=/something [Hummingbird] Request -- log on backend

This suggests that while our backend is responsive, the requests from the iOS client side are simply not being initiated.

Boost

Answer 1

DTS Engineer OP

Apple

Nov ’24

Have you filed a bug on this and, if so, what's the feedback ID?

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

0

Answer 2

glebfann OP

Nov ’24

@DTS Engineer Hello! Here is a feedback with all the information and sysdiagnose - FB15878121

0

Answer 3

DTS Engineer OP

Apple

Nov ’24

@DTS Engineer Hello! Here is a feedback with all the information and sysdiagnose - FB15878121

So, the engineering team was not aware of any large scale outage/issue and is still looking into what might have caused you failure. Clarifying things, is this still broken or did it start working again?

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

0

Answer 4

glebfann OP

Nov ’24

Clarifying things, is this still broken or did it start working again?

Yes, the issue is still ongoing. The situation is quite peculiar - while we can see some requests reaching our server (likely from beta users, rare cases), they are limited to only config and PrivacyPass-related endpoints. We don't see any actual data requests coming through. And development team has been unable to successfully make any requests reach the server through the phone calls.

During my investigation in Console.app, I found some interesting logs indicating authentication errors with the token issuer directory:

default 17:31:16.594584+0300 com.apple.CallKit.CallDirectory received Data useCase: <bundle_id>.identity
default 17:31:16.594651+0300 com.apple.CallKit.CallDirectory sending action: fetch payload: ["extensionIdentifier": <bundle_id>, "identity_fetch_error": 400]

And this one (I'm not sure if it's exactly relevant, but it looks like):

[..., bundle id: <bundle_id>, url hash: <url_hash>, definite, attribution: developer] cancelled
 [... ip1<->ip2]
 Connected Path: satisfied (Path is satisfied), viable, interface: en0[802.11], ipv4, dns, uses wifi
 Privacy Stance: Proxied
 Duration: 0.116s, QUIC @0.068s took 0.001s
 bytes in/out: 2443/5082, packets in/out: 8/9, rtt: 0.045s, retransmitted bytes: 0, out-of-order bytes: 0
 ecn packets sent/acked/marked/lost: 0/0/0/0

Notably, there's an interesting detail "Privacy Stance: Proxied". This leads to a hypothesis that all requests might be being proxied through Apple's Oblivious HTTP. However, the dev build was used here.

I'm ready to provide any additional logs, data, or engage in any form of collaboration needed to help identify and resolve this issue as quickly as possible. Please let me know what other information would be helpful.

2

Answer 5

glebfann OP

3w

@DTS Engineer Hello! Are there any updates on the issue?

1

Answer 6

glebfann OP

1w

We've recently noticed that around the same timeframe, there was a significant increase in requests from Apple to our OHTTP gateway configuration endpoint (/ohttp-configs). This leads us to hypothesize that Apple might have switched to an OHTTP scheme (even for dev version from XCode) and is attempting to fetch OHTTP configs from our gateway, and for some reason, the subsequent requests in the flow are not happening.

Could this be related to the Live Caller ID issues we're experiencing? I've filed a separate feedback (FB16056172) with details about these OHTTP configuration endpoint requests.

Would be grateful if you could confirm whether these events are connected.

2