XPC Service Cleanup and Freeing Memory

I’m confused. About at least two things |-:

You wrote:

I use dispatch_source_create () followed by dispatch_resume() to start the listener.

Say what? How do Dispatch sources come into this?

If you’re building a launchd daemon that vends a named XPC endpoint, you generally don’t need a Dispatch source. Rather, the XPC listener object calls your code directly.

I want to make sure that the XPC server is shutdown gracefully

Under what circumstances does your launchd daemon need to shut down an XPC listener gracefully?

In most causes the XPC listener for your launchd daemon should run for the lifetime of the daemon. And most daemons support transactions [1], so launchd can kill the daemon via SIGKILL, without running any code in it.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] XPC messaging opens a transaction for the duration of the message, so you don’t have to worry about this while you’re actively handling a message.

How do Dispatch sources come into this?

I have a launchd daemon(written in golang) that does some tasks. I want it to communicate with an executable, so I used XPC for communication.

When I started the daemon, the XPC server kept crashing. I assumed it had something to do with dispatch_main(), so I used a dispatch source with "DISPATCH_SOURCE_TYPE_MACH_RECV". This solved my issue. (do you know if there is a better way to do this?)

Under what circumstances does your launchd daemon need to shut down an XPC listener gracefully?

I am thinking of cases when the system shuts down, restarts, or the daemon crashes.

When I started the daemon, the XPC server kept crashing.

How are you starting your daemon?

The correct way to do this is to load the daemon configuration into launchd, either via a property list or by SMAppService, and then either:

• Have launchd start the daemon on demand.

• Set KeepAlive or RunAtLoad, so that it starts as soon as you load it.

If you’re trying to start your daemon some other way — I see a lot of folks try to start a daemon from Terminal using sudo — things will end badly on the XPC front. XPC requires that your named XPC endpoints be known to launchd. I talk about this in detail in XPC and App-to-App Communication, linked to from the XPC Resources post.

I am thinking of cases when the system shuts down, restarts, or the daemon crashes.

Let’s start with the last case, namely the daemon crashing. Your daemon is managed by launchd and that owns the named XPC endpoint. When your daemon checks in with launchd — by calling one of the XPC listener APIs — it temporarily takes over responsibility for the endpoint. If the daemon crashes, that responsibility passes back to launchd.

What happens in that case is determined by two things:

• If your daemon is configured to run all the time, via KeepAlive, launchd will start it again. There’s a rate limited on this to prevent the system glowing red hot if your daemon crashes on start.

• If not, launchd will monitor your on-demand sources, including this named XPC endpoint, and launch you when there’s demand.

Coming back to shut down and restart, the process is the same for both. The ideal pattern is for your daemon to support transactions. In that case launchd will simply SIGKILL your daemon when it wants it to stop. Or, if there’s a transaction open, it’ll SIGTERM it and wait for the daemon to exit, resorting to SIGKILL if that doesn’t happen promptly.

If your daemon doesn’t support transactions then launchd always starts with the SIGTERM.

In neither case do you have to do anything special with your XPC listener. If, for example, you were expecting to tidy things up after dispatch_main returns, you’re in for a disappointment because that never returns (-: Note the DISPATCH_NORETURN attribute on its declaration.

Finally:

written in golang

Please make sure your Go project is set up to use the Apple linker. Failing that, make sure the final product:

• Uses libSystem rather than making system calls directly

• Has a main executable build UUID, per TN3178 Checking for and resolving build UUID problems

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hey Quinn,

I dont think I was clear regarding my implementation of the daemon. So I am attaching a link to the code. Its super small and easy to go through. Please have a look. I still have a few questions-

1. XPC Lifecycle

My Golang daemon is continuously running in the background. I added <key>MachServices</key> field in my daemon's plist file to use XPC. I have C code that contains XPC logic, which I call from Golang(using Cgo).

My question: In my implementation I am using Golang to call C code, how is the XPC lifecycle affected by this? Is the XPC listener launched on demand, or does it stay running as long as the daemon?

2. freeing up variables

I am using dispatch_resume that doesnt return, so how can I free the dispatch source, queue and the XPC connection object variables ?

3.

Your daemon is managed by launchd and that owns the named XPC endpoint.

Do you mean launchd owns the endpoint?

4.

The ideal pattern is for your daemon to support transactions.

Do you mean using xpc_transaction_begin and xpc_transaction_end?

In my implementation I am using Golang to call C code, how is the XPC lifecycle affected by this?

It isn’t, at least not from a launchd perspective.

I’m presuming that your C code is linked into your executable and thus running in the same process as your Go code. In that case, this split doesn’t affect launchd or the XPC API:

• launchd just starts a process that runs your executable. It doesn’t case what language you write this in.

• Likewise, the XPC API doesn’t care about the language you use for your main function.

Is the XPC listener launched on demand, or does it stay running as long as the daemon?

OK, so an XPC listener isn’t launched. Rather, it’s something that you start within your process by calling XPC APIs.

Your launchd daemon is launched. Whether that’s on demand on not depends on how you’ve configured your launchd property list.

If you list a named XPC endpoint in your MachServices array then you must start an XPC listener for that endpoint when your daemon starts and keep that XPC listener alive until your process terminates. That listener checks in with launchd and assumes responsibility for handling XPC requests on your named endpoint.

Do you mean launchd owns the endpoint?

Well, yeah, that’s pretty much what I wrote (-:

I go into this in some detail in XPC and App-to-App Communication. It’s really important that you read that.

I am using dispatch_resume that doesn’t return…

I’m still not sure why you’re calling dispatch_resume at all. XPC listeners don’t need that in general. [goes to look at your test project] Oh, weird.

Anyway, that timer you create is pointless. You should get rid of it.

As to why your listener isn’t working, I’m not 100% sure but I suspect it’s combination of two things:

• You should maintain a persistent reference to your listener. Right now it’s only stored in a local variable, listener, which is released by ARC when you return from runXPCServer. You need to persist that somehow.

  One way to do that is to store it in a global. That’s acceptable because the listener is really a global resource, based on its connection to your MachServices entry.

  Alternatively, you could return it to your Go code and have that persist the reference. I don’t know enough about Go to advise you on how to do that.

• In ListenerHandler you’ve set an event handler on the new connection but not queue. XPC will default to its own queue for this, but I think it makes more sense for you to use the queue you created, xpcQueue.

Finally, my general advice with XPC is that you do your bring up using a loopback connection. See TN3113 Testing and debugging XPC code with an anonymous listener. The code snippets in that technote are for NSXPCConnection, but there was a recent thread here on the forums that explain how to get the same technique working with the low-level C API.

how can I free the dispatch source, queue and the XPC connection object variables?

You don’t. Your listener should run, and service requests, until your process terminates. If you shut your listener down without terminating, that kinda breaks your XPC named endpoint. Any clients that try to talk to that endpoint will be opening connections and sending messages with no hope of a reply.

Do you mean using xpc_transaction_begin and xpc_transaction_end?

Well, kinda.

You enable transaction via the EnableTransactions property in your launchd property list. You can then manually call xpc_transaction_begin and xpc_transaction_end to start and end a transaction. However, in many cases that’s not necessary. XPC starts a transaction when it delivers a request to your connection and ends it when you complete that request. So, if you only handle XPC requests, the transaction support is fully automatic.

The transaction API is for cases where that automatic support doesn’t work. For example, if you want to reply to a request and then do some housekeeping, you’d start a transaction before you reply and then end that transaction once the housekeeping is done.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I’m still not sure why you’re calling dispatch_resume at all. XPC listeners don’t need that in general.

I am assuming that I still need to call dispatch_main() or do something similar.

I came across this post which says to set RunLoopType property to NSRunLoop. (I am using C APIs for my XPC code, will NSRunLoop work in this case?) If this is something I can use, do I need to make any changes in my XPC listener code?

Is there any other alternative if I dont use dispatch_main()?

I am assuming that I still need to call dispatch_main() or do something similar.

Well, you need to do something with the main thread. If the main thread returns from its main function then the process will exit, which is obviously problematic.

What you should do depends on the nature of your code:

• If your code relies on the main queue, you need either dispatch_main or a run loop, with the former being preferred.

• If your code relies on the main thread, you need a run loop.

• Otherwise, it’s fine to block the main thread however you want. For example, it’s not uncommon for a Unix-y daemon to block in a select loop or on a kqueue.

I came across this post which says to set RunLoopType property to NSRunLoop.

Note the context of that thread, which was about running AppKit in an XPC service. AppKit depends on main thread, hence the need for a run loop. And RunLoopType is the way to get a run loop in an XPC service.

You’re not implementing an XPC service but rather a launchd daemon. This is a common source of confusion:

• An XPC service is a bundle with the .xpc extension, typically embedded within an app or a framework. See the xpcservice.plist man page.

• A launchd daemon may or may not be bundled, and is loaded by way of either a launchd property list — see the launchd.plist man page — or the SMAppService API.

• Both can published a named XPC endpoint, but they are different things.

Note Back in the day I used to use XPC Service for a .xpc bundle and XPC service (lower case S) when talking about the IPC construct. I’ve stopped doing that because it’s way too subtle. Hence the awkward-but-accurate-and-clear named XPC endpoint.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"