Unified Logging System in Go

There are two parts to this:

• Adding log points to your code, which log to the system log

• Getting information back from the system log

The first is tricky. Good integration with the system log requires compiler support. If you were up for modifying the Go compiler, I could help you with the Apple side of that (DTS tends to go the extra mile for folks building developers tools). However, it is by no means an easy task.

Fortunately it sounds like you’re primarily interested in the second part. That’s relatively straightforward. The OSLog framework is a relatively straightforward Objective-C API. You can call it like you would any other Objective-C API.

I don’t maintain expertise in third-party tools, so I can’t offer advice on how to do that. I recommend that you escalate this via your tool’s support channel.

If there’s no direct support for calling Objective-C APIs in your tools, you could create a bridge from a language that does have such support, including C (Objective-C), C++ (Objective-C++), Swift, and numerous others.

Finally, Your Friend the System Log has lots of hints and tips about the system log.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

It’s better if you reply as a reply, rather than in the comments. See Quinn’s Top Ten DevForums Tips for this and other titbits.

I believe we can modify the Golang compiler but I guess to what degree would this be needed?

I think you’d have to ask someone who knew something about the Go compiler (-:

But, again, you need to split this into two:

• Adding log points that work efficiently

• Reading the system log

The first requires careful compiler integration. The second requires the ability to import and use an Objective-C API. These are different problems and it would be a mistake to conflate them.

For example, the Swift compiler was able to access Objective-C APIs from day one, but it took a while to get good log point support (in the form of the Logger type).

can I query the unified logging system directly somehow?

I think that depends on your definition of “directly”. I know of two ways to extract information from the system log:

• The OSLog framework, which vends on Objective-C API

• The log command-line tool

IMO the first is more direct than the second.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thank you so much for your helpful links, Quinn! Your Friend the System Log was really insightful. Also I will be sure to reply and use the forums correctly! haha

I'm currently researching how to make Go compatible with Objective-C or Swift. It sounds like from what you're saying, Objective-C support might be off to a good start?

My next question would be to elaborate this point you made in your last reply: " I could help you with the Apple side of that (DTS tends to go the extra mile for folks building developers tools)".

"IMO the first is more direct than the second." I agree, if I can get this to work that would be fantastic and satisfy what I'm looking for.

By directly, I meant can I simply do an sqlite query to the unified logging system database itself on my local machine? Or does it not really work like that?

It really doesn’t work like that. The on-disk format used by the system log is customised to suit its specific requirements. It looks nothing like a normal database.

The WWDC 2016 presentation actually covers some of these implementation details. That’s no longer available from Apple, but the URL in Your Friend the System Log should help you track down this info elsewhere.

Right. There are two ways you could tackle this:

• Import the Objective-C runtime in Go. That’d let to you call any Objective-C API.

• Write an Objective-C wrapper that presents an C interface and call that from Go.

The former is very general. It also means that you do a lot more work on the Go side. That’s both good and bad. It’s good because you spend more time in your preferred environment. It’s bad because it’ll be super fiddly work with no compiler safety net.

Still, I can’t help but think that others in the Go community must’ve tackled this problem before you, and I suggest you continue your research on that side.

The connection between the system log APIs [1] and the C compiler is quite subtle, primarily because it’s central to the way that the system log works as efficiently as it does. These details are not secret — AFAIK they’re in the Clang open source — but they’re not well documented either. If you were building a compiler [2] and you wanted to use the same mechanism, that’s something I’d be happy to discuss.

DTS strives to help developers use Apple APIs in a supported fashion, so that their code works now and in the future. Given that mandate, we try to steer clear of implementation details, because those can change.

However, this isn’t a hard boundary; there are always grey areas. For example, every time you replace a C routine with some optimised assembly, you’re increasing your compatibility risk. If Apple changes CPUs, again [3], you’ll have to rewrite your code.

Building a compiler is that problem writ large. So there’s a balance to be struck between convenience for the compiler’s users and compatibility.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] I’m talking about the APIs to log stuff, not the API to read the log. The latter is just a vanilla Objective-C API.

[2] Or in this case modifying a compiler.

[3] I’m currently on my fourth CPU architecture here at Apple, or seventh, depending on how you count them (-: