Archived app failing to get root certificates for SSL websocket connection

I've had a Unreal Engine project that uses libwebsocket to make a websocket connection with SSL to a server. Recently I made a build using Unreal Engine 5.4.4 on MacOS Sequoia 15.5 and XCode 16.4 and for some reason the websocket connection now fails because it can't get the local issuer certificate. It fails to access the root certificate store on my device (Even though, running the project in the Unreal Editor works fine, it's only when making a packaged build with XCode that it breaks)

I am not sure why this is suddenly happening now. If I run it in the Unreal editor on my macOS it works fine and connects. But when I make a packaged build which uses XCode to build, it can't get the local issuer certificate. I tried different code signing options, such as sign to run locally or just using sign automatically with a valid team, but I'm not sure if code signing is the cause of this issue or not.

This app is only for development and not meant to be published, so that's why I had been using sign to run locally, and that used to work fine but not anymore.

Any guidance would be appreciated, also any information on what may have changed that now causes this certificate issue to happen.

I know Apple made changes and has made notarizing MacOS apps mandatory, but I'm not sure if that also means a non-notarized app will now no longer have access to the root certificate store of a device, in my research I haven't found anything about that specifically, but I'm wondering if any Apple engineers might know something about this that hasn't been put out publicly.

Answered by DTS Engineer in 864769022

The way you sign your app does not, in general, affect how it does HTTPS server trust evaluation. It’s hard to say exactly what’s going on here because you neck deep in third-party tools and libraries, but my best guess is that this is a build issue. That is, your code tries to get access to this root certificate but fails, either because the code is doing that incorrectly or because it’s not in the right place.

If you were using Apple APIs I’d be able to suggest more specific paths for debugging this. I can’t do that here because I’m not familiar with the libraries you’re using. That leaves me with a couple of suggestions:

  • You could dump the contents of your built app’s bundle to see if the certificate is where you expect it to be.
  • You could track down the code that uses this root certificate and see how it’s failing. Is it using the right path to access it? Is the file actually there? Or is it failing to load it?

You might have more luck asking this via the support channel for your third-party tooling.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

The way you sign your app does not, in general, affect how it does HTTPS server trust evaluation. It’s hard to say exactly what’s going on here because you neck deep in third-party tools and libraries, but my best guess is that this is a build issue. That is, your code tries to get access to this root certificate but fails, either because the code is doing that incorrectly or because it’s not in the right place.

If you were using Apple APIs I’d be able to suggest more specific paths for debugging this. I can’t do that here because I’m not familiar with the libraries you’re using. That leaves me with a couple of suggestions:

  • You could dump the contents of your built app’s bundle to see if the certificate is where you expect it to be.
  • You could track down the code that uses this root certificate and see how it’s failing. Is it using the right path to access it? Is the file actually there? Or is it failing to load it?

You might have more luck asking this via the support channel for your third-party tooling.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hello @DTS Engineer , thanks for the response.

Just to clarify, my app do send HTTPS requests and those work fine. It's when it tries to do a websocket connection with SSL that it fails to get the root certificate. The specific error would read something like "SSL error: unable to get local issuer certificate (preverify_ok=0;err=20;depth=2)"

I am using Unreal Engine's built-in libwebsocket module to make a connection to a secure websocket server. I have built this project in the past with the same code and it used to work. I actually still have an old build that I can run and see it working. But any new build I am making with the same exact codebase, is now failing. The only things that have changed are updates to the build environment, we upgraded XCode at some point (I don't remember exactly when) to 16.4 - That is why my first suspicion was that it is caused by a change in the build environment. I'm kind of stumped honestly, this issue came out of nowhere and we made no code changes to our project. The built project was also tested on more than one Mac machine, so that eliminates the possibility that it is the root certificate that isn't in the right place.

I will continue to dig deeper, and will also ask in the Unreal Engine forums. But I still suspect this might be an XCode/MacOS issue, at least for now.

I am using Unreal Engine's built-in libwebsocket module

Understood.

Do you have access to that library’s source code? If you do, you could rummage around in that code to see how it’s doing server trust evaluation. If you don’t, that brings me back to my initial suggestion, that is, you’ll need to consult the resources for that library.

That is why my first suspicion was that it is caused by a change in the build environment.

Yep. As with most debugging problems, you can approach this from above or below:

  • If you want to approach this from ‘above’, you can look at the resulting built app to see what’s different in the Xcode 26 version.
  • Alternatively, to approach this from ‘below’ you can find the code that’s rejecting the server’s certificate and debug that.

Both are valid, and it’s hard to say which will yield quicker results.


Lemme explain how I’d approach this if you were Network framework, our preferred WebSocket API. By default, Network framework uses the system’s built-in HTTPS server trust evaluation. Developers can override this when they configure the connection by calling the sec_protocol_options_set_verify_block. So, if you were having this problem I’d suggest:

  1. Search your code for sec_protocol_options_set_verify_block.
  2. If you don’t find any calls to that, something weird is happening.
  3. Presuming that you do, you can step through the code to see where it’s going wrong.

I’m presuming that libwebsocket has similar facilities, but I just don’t know what they are )-:

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thank you for this response.

I believe I now understand what was happening but still don't fully understand why it suddenly started happening.

Ultimately what's happening is that the packaged build from Unreal (which is then also built through XCode) is not packaging the cacerts with it. I don't think it previously did either, but I wonder if my built app is now being sandboxed and no longer has access to the devices root certs?

The solution was essentially to ensure that a cacert.pem file (with the latest root certs) is packaged with my app so it could access that to verify my servers certs.

I can also see that root certificates got updated since Nov 4th here: https://curl.se/docs/caextract.html (which is the same day I made this post, coincidence?) - I am thinking this might have something to do with why I am suddenly seeing this issue.

If there is no sandboxing and my app has access to my devices certs, then maybe my device did not get the latest certs update (but my websocket server did) and therefore can't verify the servers cert?

In any case I thank you for the information you've shared! I would like to know however if it is possible that my MacOS did not have the latest certs available? And if so, what would be the best way to ensure that an Apple device has the latest root certificates?

Accepted Answer

I’m glad to hear you’re making progress.

no longer has access to the devices root certs?

To summarise TN3151’s [advice on this front][ref]:

  • Don’t use your own TLS implementation.
  • If you do that anyway, use the system’s built-in trust evaluation.

This ensures that your app’s TLS works as close as possible to the system’s built-in one.

[ref]: <TN3151: Choosing the right networking API | Apple Developer Documentation#TLS-best-practices

Many apps and libraries ignore this advice, which can result in odd behaviour. I don’t know if that’s the case here. If you want to dig deeper into this, you have to see whether this library uses a system trust object (SecTrust) to do server trust evaluation. If it does, that’s definitely something I can talk you through.

If there is no sandboxing and my app has access to my devices certs,

You’re making some invalid assumptions here:

  • You’re assuming that the system trust store is represented on-disk as a group of PEM files, which is not the case.
  • You’re assuming that your app needs file-system access to the trust store, which is not the case.

Rather, the system trust object uses IPC to communicate with a system daemon (currently trustd) which has access to the trust store. Your app should not, and for iOS and its child platforms, cannot access the on-disk representation of the system trust store.

what would be the best way to ensure that an Apple device has the latest root certificates?

You should bookmark this doc: Available root certificates for Apple operating systems. Each of the root store docs that it links to explains how to get the trust store version on iOS (and it’s child platforms) and macOS.

But all of this is only relevant if your TLS library is using a system trust object. If it’s doing it’s own thing, you’ll have to dig into its code.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thanks a lot that is very helpful information.

I'm going to continue investigating Unreals libwebsocket module to better understand how it attempts to access root certs.

I have noticed on my build machine that I had a macOS security update which I hadn't installed, I assume this security update contains updated root certs and this is what I would have needed to have my app build use the correct certs.

Appreciate the help!

Archived app failing to get root certificates for SSL websocket connection
 
 
Q