AI framework usage without user session

You are definitely breaking new ground here, so I’m gonna to do some research and get back to you.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I’m still working on this. It’s proving to be challenging, not helped by the fact that WWDC is looming large on everyone’s calendar.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

We ran CoreML inference from a launch daemon (no user session) for about a year — it works but with caveats. ANE access is unreliable without a session, so you'll likely fall back to CPU/GPU compute units. Vision framework calls that touch CoreGraphics can deadlock if there's no window server connection. Our workaround was forcing .cpuOnly for the daemon path and keeping the GPU/ANE path for the user-facing XPC.

eddiewangyw, Thanks for sharing your experience with this.

ANE access is unreliable without a session

Yeah, that was one of my concerns. I’m also concerned about the GPU itself, because it’s not clear to me whether that’ll be affected by the window server lifecycle.

Earlier I wrote:

I’m still working on this.

I’m starting to make some progress here, but I’ve still nothing concrete to share.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thanks Quinn and eddiewangyw for the updates. One thing I guess worth clarifying also is what is meant by "user session". Our service(s) have no connection to CGS since they are root session only but other Apple daemons/services may—and I assume that's the real crux of the issue may make a difference. But if how a user is logged in—via physical console or SSH or VNC etc makes a difference could also be interesting.

In our testing, the ANE and GPU were reliable when a user was logged in at the physical console. It's much harder to trigger these specific flows without some GUI console access (e.g. SSH would be hard) but over VNC it's definitely possible. Then I began to question if there's a difference with somebody logged in but at loginwindow, etc.

I guess worth clarifying also is what is meant by "user session".

Right. I generally try to follow the terminology from TN2083 Daemons and Agents. A system extension [1] is effectively a launchd daemon and thus runs in the global context. It can’t reliably access the window server. And any XPC service it calls upon will be similarly restricted. However, the GPU lifecycle isn’t tied directly to the window server lifecycle, so there is some wiggle room there, and that’s the point I’m still working to clarify internally.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Non-DEXT system extensions. DEXTs are weird (-:

Hey Quinn, is there any update here? I see Golden Gate made some more changes here especially on iOS but they're listed in the macOS release notes:

iOS 27 includes Neural Engine improvements for Apple Intelligence capable devices. The system now restricts background access to the Neural Engine, similar to GPU usage restrictions. Large model loading (over 1 GB) performance is improved on the Neural Engine. Neural Engine memory usage is now attributed to your app process instead of the system, and appears in the Allocations instrument. (174796039)

is there any update here?

Thanks for the ping. And sorry about the lack of follow up. You can consider this thread to be roadkill on the highway to WWDC (-:

they're listed in the macOS release notes

Unfortunately that iOS advice isn’t directly applicable to macOS. iOS doesn’t have the equivalent of the global execution context we use for launchd daemons on the Mac.

The discussions I had with folks prior to WWDC made it clear that… well… things aren’t clear. If the framework you’re using guarantees that this process will get done on the CPU, then I don’t anticipate any problems. If it might get done on the GPU or NE [1], I’m much less confident. You’re straying into the world of implementation details, where things might work but that could change in different contexts, or over time.

Probably my best advice right now is to file a bug against the docs for the specific API you’re using, asking that they clarify whether that API is expected to work from the global context in which you’re running (so either launchd daemon or system extension).

Please post your bug number, just for the record.

@eddiewangyw, I recommend that you do the same.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] This is a momentous occasion, being the first time I’ve used NE to mean something other than Network Extension (-: Hmmm, maybe I should adopt ANE instead.

Thanks Quinn. I've filed 2 bugs requesting clarification, and one for a bug we've already found.

FB23746715 FB23746638 FB23746569

FB23746715 FB23746638 FB23746569

Thanks.

Oh, one last more thing. In Bug Reporting: How and Why? I explain how you can identify your bug as coming from your team rather than from you individually. I recommend that you do that for all three of these. Search for team on that page to find the specific instructions.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"