Posts

Post not yet marked as solved
0 Replies
218 Views
The unified system log on Apple platforms gets a lot of stick for being ‘too verbose’. I understand that perspective: If you’re used to a traditional Unix-y system log, you might expect to learn something about an issue by manually looking through the log, and the unified system log is way too chatty for that. However, that’s a small price to pay for all its other benefits. This post is my attempt to explain those benefits, broken up into a series of short bullets. Hopefully, by the end, you’ll understand why I’m best friends with the system log, and why you should be too! If you have questions or comments about this, start a new thread and tag it with OSLog so that I see it. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Your Friend the System Log Apple’s unified system log is very powerful. If you’re writing code for any Apple platform, and especially if you’re working on low-level code, it pays to become friends with the system log! Friends with Benefits The public API for logging is fast and full-featured. And it’s particularly nice in Swift. Logging is fast enough to leave log points [1] enabled in your release build, which makes it easier to debug issues that only show up in the field. The system log is used extensively by the OS itself, allowing you to correlate your log entries with the internal state of the system. Log entries persist for a long time, allowing you to investigate an issue that originated well before you noticed it. Log entries are classified by subsystem, category, and type. Each type has a default disposition, which determines whether that log entry is enable and, if it is, whether it persists in the log store. You can customise this, based on the subsystem, category, and type, in four different ways: Install a configuration profile created by Apple (all platforms). Add an OSLogPreferences property in your app’s profile (all platforms). Run the log tool with the config command (macOS only) Create and install a custom configuration profile with the com.apple.system.logging payload (macOS only). When you log a value, you may tag it as private. These values are omitted from the log by default but you can configure the system to include them. The Console app displays the log, both on the local Mac and on attached iOS devices. It also opens log snapshots (.logarchive). It also supports surprisingly sophisticated filtering. The log command-line tool lets you do all of this and more from Terminal. There’s a public API to read back existing log entries, albeit one with significant limitations on iOS (more on that below). Every sysdiagnose log includes a snapshot of the system log, which is ideal for debugging hard-to-reproduce problems. For more information about sysdiagnose logs, see the info on Bug Reporting > Profiles and Logs. But you don’t have to use sysdiagnose logs. To create a quick snapshot of the system log, run the log tool with the collect subcommand. For more information, see: os > Logging OSLog log man page os_log man page (in section 3) os_log man page (in section 5) [1] Well, most log points. If you’re logging thousands of entries per second, the very small overhead for these disabled log points add up. Foster Your Friendship Good friendships take some work on your part, and your friendship with the system log is no exception. Follow these suggestions for getting the most out of the system log. The system log has many friends, and it tries to love them the all equally. Don’t abuse that by logging too much. One key benefit of the system log is that log entries persist for a long time, allowing you to debug issues with their roots in the distant past. But there’s a trade off here: The more you log, the shorter the log window, and the harder it is to debug such problems. Put some thought into your subsystem and category choices. One trick here is to use the same category across multiple subsystems, allowing you to track issues as they cross between subsystems in your product. Don’t use too many unique subsystem and context pairs. As a rough guide: One is fine, ten is OK, 100 is too much. Choose your log types wisely. The documentation for each OSLogType value describes the default behaviour of that value; use that information to guide your choices. Remember that disabled log points have a very low cost. It’s fine to leave chatty logging in your product if it’s disabled by default. No Friend Is Perfect The system log API is hard to wrap. The system log is so efficient because it’s deeply integrated with the compiler. If you wrap the system log API, you undermine that efficiency. For example, a wrapper like this is very inefficient: // -*-*-*-*-*- DO NOT DO THIS -*-*-*-*-*- void myLog(const char * format, ...) { va_list ap; va_start(ap, format); char * str = NULL; vasprintf(&str, format, ap); os_log_debug(sLog, "%s", str); free(str); va_end(ap); } // -*-*-*-*-*- DO NOT DO THIS -*-*-*-*-*- This is mostly an issue with the C API, because the modern Swift API is nice enough that you rarely need to wrap it. If you do wrap the C API, use a macro and have that pass the arguments through to the underlying os_log_xyz macro. iOS has very limited facilities for reading the system log. Currently, an iOS app can only read entries created by that specific process, using .currentProcessIdentifier scope. This is annoying if, say, the app crashed and you want to know what it was doing before the crash. What you need is a way to get all log entries written by your app (r. 57880434). Xcode’s console pane has no system log integration (r. 32863680). I generally work around this by ignoring the console pane and instead running Console and viewing my log entries there. Revision History 2022-06-23 Added the Foster Your Friendship section. Made other editorial changes. 2022-05-12 First posted.
Posted
by eskimo.
Last updated
.
Post not yet marked as solved
0 Replies
59 Views
Service Management framework supports installing and uninstalling services, including Service Management login items, launchd agents, and launchd daemons. General: DevForums tag: Service Management Service Management framework documentation Daemons and Services Programming Guide archived documentation Technote 2083 Daemons and Agents — It hasn’t been updated in… well… decades, but it’s still remarkably relevant. BSD Privilege Escalation on macOS DevForums post EvenBetterAuthorizationSample sample code SMJobBless sample code Sandboxing with NSXPCConnection sample code Related tags include: XPC, Apple’s preferred inter-process communication (IPC) mechanism Inter-process communication, for other IPC mechanisms Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com"
Posted
by eskimo.
Last updated
.
Post not yet marked as solved
0 Replies
59 Views
XPC is the preferred inter-process communication (IPC) mechanism on Apple platforms. XPC has two APIs: The high-level NSXPCConnection API, for Objective-C and Swift The low-level C API, which, while callable from all languages, works best with C-based languages General: DevForums tag: XPC NSXPCConnection class documentation XPC C API documentation XPC has extensive man pages — For the C API, start with the xpc man page; this is the original source for the XPC C API documentation and still contains tidbits that you can’t find elsewhere. Also read the xpcservice.plist man page, which documents the property list format used by XPC services. Daemons and Services Programming Guide archived documentation Technote 2083 Daemons and Agents — It hasn’t been updated in… well… decades, but it’s still remarkably relevant. TN3113 Testing and Debugging XPC Code With an Anonymous Listener Related tags include: Inter-process communication, for other IPC mechanisms Service Management, for installing and uninstalling Service Management login items, launchd agents, and launchd daemons Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com"
Posted
by eskimo.
Last updated
.
Post not yet marked as solved
0 Replies
78 Views
This week I’m handling a DTS incident from a developer who wants to escalate privileges in their app. This is a tricky problem. Over the years I’ve explained aspects of this both here on DevForums and in numerous DTS incidents. Rather than do that again, I figured I’d collect my thoughts into one place and share them here. If you have questions or comments, please start a new thread with an appropriate tag (Service Management or XPC are the most likely candidates here). Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" BSD Privilege Escalation on macOS macOS has multiple privilege models. Some of these were inherited from its ancestor platforms. For example, Mach messages has a capability-based privilege model. Others were introduced by Apple to address specific user scenarios. For example, macOS 10.14 and later have mandatory access control (MAC), as discussed in On File System Permissions. One of the most important privilege models is the one inherited from BSD. This is the classic users and groups model. Many subsystems within macOS, especially those with a BSD heritage, use this model. For example, a packet tracing tool must open a BPF device, /dev/bpf*, and that requires root privileges. Specifically, the process that calls open must have an effective user ID of 0, that is, the root user. That process is said to be running as root, and escalating BSD privileges is the act of getting code to run as root. IMPORTANT Escalating privileges does not bypass all privilege restrictions. For example, MAC applies to all processes, including those running as root. Indeed, running as root can make things harder because TCC will not display UI when a launchd daemon trips over a MAC restriction. Escalating privileges on macOS is not straightforward. There are many different ways to do this, each with its own pros and cons. The best approach depends on your specific circumstances. Note If you find operations where a root privilege restriction doesn’t make sense, feel free to file a bug requesting that it be lifted. This is not without precedent. For example, in macOS 10.2 (yes, back in 2002!) we made it possible to implement ICMP (ping) without root privileges. And in macOS 10.14 we removed the restriction on binding to low-number ports (r. 17427890). Nice! Decide on One-Shot vs Ongoing Privileges To start, decide whether you want one-shot or ongoing privileges. For one-shot privileges, the user authorises the operation, you perform it, and that’s that. For example, if you’re creating an un-installer for your product, one-shot privileges make sense because, once it’s done, your code is no longer present on the user’s system. In contrast, for ongoing privileges the user authorises the installation of a launchd daemon. This code always runs as root and thus can perform privileged operations at any time. Folks often ask for one-shot privileges but really need ongoing privileges. A classic example of this is a custom installer. In many cases installation isn’t a one-shot operation. Rather, the installer includes a software update mechanism that needs ongoing privileges. If that’s the case, there’s no point dealing with one-shot privileges at all. Just get ongoing privileges and treat your initial operation as a special case within that. Keep in mind that you can convert one-shot privileges to ongoing privileges by installing a launchd daemon. Just Because You Can, Doesn’t Mean You Should Ongoing privileges represent an obvious security risk. Your daemon can perform an operation, but how does it know whether it should perform that operation? There are two common ways to authorise operations: Authorise the user Authorise the client To authorise the user, use Authorization Services. For a specific example of this, look at the EvenBetterAuthorizationSample sample code. Note This sample hasn’t been updated in a while (sorry!) and it’s ironic that one of the things it demonstrates, opening a low-number port, no longer requires root privileges. However, the core concepts demonstrated by the sample are still valid. The packet trace example from above is a situation where authorising the user with Authorization Services makes perfect sense. By default you might want your privileged helper tool to allow any user to run a packet trace. However, your code might be running on a Mac in a managed environment, where the site admin wants to restrict this to just admin users, or just a specific group of users. A custom authorisation right gives the site admin the flexibility to configure authorisation exactly as they want. Authorising the client is a relatively new idea. It assumes that some process is using XPC to request that the daemon perform a privileged operation. In that case, the daemon can use XPC facilities to ensure that only certain processes can make such a request. Doing this securely is a challenge. For specific API advice, see this post. WARNING This authorisation is based on the code signature of the process’s main executable. If the process loads plug-ins [1], the daemon can’t tell the difference between a request coming from the main executable and a request coming from a plug-in. [1] I’m talking in-process plug-ins here. Plug-ins that run in their own process, such as those managed by ExtensionKit, aren’t a concern. Choose an Approach There are (at least) seven different ways to run with root privileges on macOS: A setuid-root executable The sudo command AppleScript’s do shell script command, passing true to the administrator privileges parameter The long-deprecated AuthorizationExecuteWithPrivileges function, in Security framework The SMJobBless function, in Service Management framework An installer package (.pkg) The new-in-macOS 13 (currently in beta) SMAppService facility, a much-needed enhancement to the Service Management framework To choose between them: Do not use a setuid-root executable. Ever. It’s that simple! Doing that is creating a security vulnerability looking for an attacker to exploit it. If you’re working interactively on the command line, use sudo. IMPORTANT sudo is not appropriate to use as an API. While it may be possible to make this work under some circumstances, by the time you’re done you’ll have code that’s way more complicated than the alternatives. If you’re building an ad hoc solution to distribute to a limited audience, and you need one-shot privileges, use either AuthorizationExecuteWithPrivileges or AppleScript. While AuthorizationExecuteWithPrivileges still works, it’s been deprecated for many years. Do not use it in a widely distributed product. The AppleScript approach works great from AppleScript, but you can also use it from native code using NSAppleScript. See the code snippet later in this post. If you only need escalated privileges to install your product, consider using an installer package. That’s by far the easiest solution to this problem. Keep in mind that an installer package can install a launchd daemon and thereby gain ongoing privileges. If you need ongoing privileges but don’t want to ship an installer package, use SMAppService. If you need to deploy to older systems, use SMJobBless. For instructions on using SMAppService, see Updating helper executables from earlier versions of macOS. For a comprehensive example of how to use SMJobBless, see the EvenBetterAuthorizationSample sample code. For the simplest possible example, see the SMJobBless sample code. That has a Python script to help you debug your setup. Unfortunately this hasn’t been updated in a while; see this thread for more. Hints and Tips I’m sure I’ll think of more of these as time goes by but, for the moment, let’s start with the big one… Do not run GUI code as root. In some cases you can make this work but it’s not supported. Moreover, it’s not safe. The GUI frameworks are huge, and thus have a huge attack surface. If you run GUI code as root, you are opening yourself up to security vulnerabilities. Appendix: Running an AppleScript from Native Code Below is an example of running a shell script with elevated privileges using NSAppleScript. WARNING This is not meant to be the final word in privilege escalation. Before using this, work through the steps above to see if it’s the right option for you. Hint It probably isn’t! let url: URL = … file URL for the script to execute … let script = NSAppleScript(source: """ on open (filePath) if class of filePath is not text then error "Expected a single file path argument." end if set shellScript to "exec " & quoted form of filePath do shell script shellScript with administrator privileges end open """)! // Create the Apple event. let event = NSAppleEventDescriptor( eventClass: AEEventClass(kCoreEventClass), eventID: AEEventID(kAEOpenDocuments), targetDescriptor: nil, returnID: AEReturnID(kAutoGenerateReturnID), transactionID: AETransactionID(kAnyTransactionID) ) // Set up the direct object parameter to be a single string holding the // path to our script. let parameters = NSAppleEventDescriptor(string: url.path) event.setDescriptor(parameters, forKeyword: AEKeyword(keyDirectObject)) // The `as NSAppleEventDescriptor?` is required due to a bug in the // nullability annotation on this method’s result (r. 38702068). var error: NSDictionary? = nil guard let result = script.executeAppleEvent(event, error: &error) as NSAppleEventDescriptor? else { let code = (error?[NSAppleScript.errorNumber] as? Int) ?? 1 let message = (error?[NSAppleScript.errorMessage] as? String) ?? "-" throw NSError(domain: "ShellScript", code: code, userInfo: nil) } let scriptResult = result.stringValue ?? ""
Posted
by eskimo.
Last updated
.
Post marked as solved
2 Replies
661 Views
How do you spell "test"?
Posted
by eskimo.
Last updated
.
Post not yet marked as solved
0 Replies
494 Views
I spend a lot of time here on DevForums. Over the years I’ve read many posts, both good and bad. This page is my attempt at writing down what makes a good one. Hopefully some of you will find it useful. Before you read this, read the official Apple Developer > Support > Developer Forums page. If you have questions or feedback about any of the points raised here, start a new thread with the Forums Feedback tag. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Quinn’s Top Ten DevForums Tips Here are my top ten DevForums tips: Is DevForums right for you? Search before you ask Keep your expectations realistic Tag your thread properly Craft a quality post Prefer text over images Include details with your questions Crash reports Cross posting courtesy Resize screen shots Include a Spinal Tap reference Close your threads Think about your title Post URLs in the clear Many of these boil down to one word: Empathy. Think about the person who’s reading your post. Can they read it? Will they understand it? Will it help them? 1. Is DevForums right for you? To quote Apple Developer > Support > Developer Forums, Apple Developer Forums (aka DevForums) is: a great place to post questions, exchange knowledge, and connect with fellow developers and Apple engineers on a variety of development topics. DevForums is focused on developer issues. That includes the APIs in Apple’s platform SDKs, Apple tools, developer-oriented Apple services like App Store Connect, and accessory development. If you have a user-level question, you’ll have more luck over in Apple Support Communities, run by Apple Support. DevForums is focused on Apple technologies. If you’re using a third-party tool or library, feel free to ask questions about it here, but you’re more likely to find folks with relevant expertise in that technology’s dedicated support channel. If you want to file a bug report, do that using Feedback Assistant. If you want to discuss a bug you’ve already filed, DevForums is a great place for that. Make sure to include your bug number in your post. 2. Search before you ask DevForums has a history stretching back to 2015. Many questions have been asked and answered here. Before you start a thread, search the forums for similar threads. For details about the search syntax, see Apple Developer > Support > Developer Forums. For a quick summary, hover over the help (?) button next to the search field. Remember that DevForums is world readable and thus indexed by Internet search engines. 3. Keep your expectations realistic DevForums is an informal support channel; no one is being paid to answer DevForums questions full time. Keep that in mind when you post. Apple provides a number of formal support channels. To request formal support, go to the Apple Developer > Contact Us page. One of those support channels is the code-level support provided by Apple Developer Technical Support (DTS). For more information about DTS, see Apple Developer > Support > Requesting Technical Support. Asking about Apple’s unannounced plans is unlikely to yield useful results. Apple folks can’t discuss The Future™, and non-Apple folks can only speculate. 4. Tag your thread properly DevForums uses a tag-based model. Folks willing to offer help often monitor a set of tags and only see threads with those tags. To increase your odds of getting a response, tag your thread properly. For a list of tags, go to Developer Forums Tags. That’s a lot of tags! With that many tags there are inevitably cases where a tag doesn’t mean what you think it means. For example, the ExceptionHandling tag is about the rarely used ExceptionHandling framework, not about exception handling in general. Before using a tag, read its description. That’ll tell you whether the tag is actually appropriate for your thread. 5. Craft a quality post When replying, use a reply rather than a comment. Comments are best reserved for short messages, like “Thanks!” or “See this other thread.” DevForums supports Markdown formatting, similar to that used on GitHub and by DocC. The editor UI has buttons for the most common things, like Bold and Italic, but don’t feel the need to limit yourself to that. Correct formatting is particularly important for preformatted text: Use the Inline Code button, which inserts single backquote delimiters, for inline text in code style: identifiers and so on. Use the Code Block button, which inserts triple backquote delimiters, for blocks of text in code style: code snippets, logs, and so on. After submitting your post, look it over to make sure that it reads well. If not, you have a short window where you can edit the post to fix things. 6. Prefer text over images Don’t use screen shots for data that’s essentially textual, like code snippets and logs. For example, if you want to post a log message, include that as text rather than adding a screen shot. That makes it much easier for your readers to work with the text. Use the Code Block button, which inserts triple backquote delimiters, to format blocks of text in code style. Reserve screen shots for situations where the issue is visual, for example: When discussing view layout problems When posting instructions to achieve some task in a GUI app, like Xcode 7. Include details with your questions When starting a thread, try to include all the relevant details in your post. If you skimp on these details, folks will have to reply asking for them, and that slows things down. Specifically, anticipate the following questions: What platform are you targeting? And what version of that platform? What version of Xcode are you using? What version of the OS are you testing on? What specific API are you using? What are the exact steps you took? If something failed, what are the symptoms of that failure? If an API returned an error, what was that error? If nothing failed, what results did you see? And what were you expecting? If you filed a bug, what was the bug number? What else have you tried? 8. Crash reports If you post a crash report, follow the instructions in Posting a Crash Report. 9. Cross posting courtesy Don’t start multiple threads for the same issue. That just wastes everyones time. Sometimes this happens by accident. If that’s the case, add a comment to one of the threads redirecting folks to the other one. If you post your question to another support channel, provide a link to that in your DevForums question, and vice versa. That avoids folks working on a question that’s already been answered. 10. Resize screen shots If your post includes an image, make sure it renders well. Screen shots from Retina displays are often ridiculously large. Use Preview to crop the screen shot and, if necessary, downsample it by 50% (using Tools > Adjust Size). 11. Include a Spinal Tap reference To be clear, this is a joke. While the occasional Spinal Tap reference is allowed, please don’t add one to all your posts (-: 12. Close your threads If someone replies with the answer you need, mark that as correct. That gives them some credit and helps other folks with similar questions find that answer. If you find the answer by yourself, please post a short summary of what you did. Feel free to mark that correct. 13. Think about your title When creating a thread, think carefully about your title. Your title is your thread’s ‘marketing statement’, a way to entice folks to read your post. Try to summarise your issue in 15 words or less. 14. Post URLs in the clear DevForums has a list of websites you can link to at will. For example, it places no restrictions on links to Swift Forums. To link to a site that’s not on the allowlist, skip the Markdown link syntax and post your link in the clear. So, this won’t work: Apple is based in [Cupertino](https://www.cupertino.org). but this will: Apple is based in Cupertino. https://www.cupertino.org Revision History 2022-06-17 Added tip 14. Updated the preamble to include a link to the main DevForums page. 2022-06-04 Added a discussion of unannounced plans to tip 3 (thanks to Scott). 2022-06-03 Added tip 13. 2022-05-24 Added tips suggested by Claude31 and Scott. 2022-05-23 First posted.
Posted
by eskimo.
Last updated
.
Post not yet marked as solved
0 Replies
123 Views
Trusted execution is a generic name for a Gatekeeper and other technologies that aim to protect users from malicious code. General: DevForums tag: Gatekeeper Developer > Signing Mac Software with Developer ID Apple Platform Security support document Safely open apps on your Mac support article Hardened Runtime document Testing a Notarised Product DevForums post Resolving Trusted Execution Problems DevForums post WWDC 2022 Session 10096 What’s new in privacy covers some important Gatekeeper changes in macOS 13 beta (starting at 04-32) Most trusted execution problems are caused by code signing or notarisation issues. See Code Signing Resources and Notarisation Resources. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com"
Posted
by eskimo.
Last updated
.
Post not yet marked as solved
0 Replies
1.8k Views
Modern versions of macOS use a file system permission model that’s far more complex than the traditional BSD rwx model, and this post is my attempt at explaining that new model. If you have a question about this, post it here on DevForums, tagging your thread with Files and Storage so that I see it. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" On File System Permissions Modern versions of macOS have four different file system permission mechanisms: Traditional BSD permissions Access control lists (ACLs) App Sandbox Mandatory access control (MAC) The first two were introduced a long time ago and rarely trip folks up. The second two are newer, more complex, and specific to macOS, and thus are the source of some confusion. This post is my attempt to clear that up. Error Codes App Sandbox and the mandatory access control system are both implemented using macOS’s sandboxing feature. When a file system operation fails, check the error to see whether it was blocked by this sandboxing feature. If an operation was blocked by BSD permissions or ACLs, it fails with EACCES (Permission denied, 13). If it was blocked by something else, it’ll fail with EPERM (Operation not permitted, 1). If you’re using Foundation’s FileManager, these error are reported as Foundation errors, for example, the NSFileReadNoPermissionError error. To recover the underlying error, get the NSUnderlyingErrorKey property from the info dictionary. App Sandbox File system access within the App Sandbox is controlled by two factors. The first is the entitlements on the main executable. There are three relevant groups of entitlements: The com.apple.security.app-sandbox entitlement enables the App Sandbox. This denies access to all file system locations except those on a built-in allowlist (things like /System) or within the app’s containers. The various “standard location” entitlements extend the sandbox to include their corresponding locations. The various “file access temporary exceptions” entitlements extend the sandbox to include the items listed in the entitlement. The second factor is dynamic sandbox extensions. The system issues these extensions to your sandbox based on user behaviour. For example, if the user selects a file in the open panel, the system issues a sandbox extension to your process so that it can access that file. The type of extension is determined by the main executable’s entitlements: com.apple.security.files.user-selected.read-only results in an extension that grants read-only access. com.apple.security.files.user-selected.read-write results in an extension that grants read/write access. These dynamic sandbox extensions are tied to your process; they go away when your process terminates. To maintain persistent access to an item, use a security-scoped bookmark. See Security-Scoped Bookmarks and Persistent Resource Access. If you have access to a directory — regardless of whether that’s via an entitlement or a dynamic sandbox extension — then, in general, you have access to all items in the hierarchy rooted at that directory. This does not overrule the MAC protection discussed below. For example, if the user grants you access to ~/Library, that does not give you access to ~/Library/Mail because the latter is protected by MAC. Finally, the discussion above is focused on a new sandbox, the thing you get when you launch a sandboxed app from the Finder. If a sandboxed process starts a child process, that child process inherits its sandbox from its parent. For information on what happens in that case, see the Note box in Enabling App Sandbox Inheritance. IMPORTANT The child process inherits its parent process’s sandbox regardless of whether it has the com.apple.security.inherit entitlement. That entitlement exists primarily to act as a marker for App Review. App Review requires that all main executables have the com.apple.security.app-sandbox entitlement, and that entitlements starts a new sandbox by default. Thus, any helper tool inside your app needs the com.apple.security.inherit entitlement to trigger inheritance. However, if you’re not shipping on the Mac App Store you can leave off both of these entitlement and the helper process will inherit its parent’s sandbox just fine. The same applies if you run a built-in executable, like /bin/sh, as a child process. When the App Sandbox blocks something, it typically generates a sandbox violation report. For information on how to view these reports, see Viewing Sandbox Violation Reports. To learn more about the App Sandbox, see the App Sandbox Design Guide and related documents (most notably the Entitlement Key Reference). For information about how to embed a helper tool in a sandboxed app, see Embedding a Command-Line Tool in a Sandboxed App. Mandatory Access Control Mandatory access control (MAC) has been a feature of macOS for many releases, but it’s become a lot more prominent since macOS 10.14. There are many flavours of MAC but the ones you’re most likely to encounter are: Full Disk Access (since 10.14) Files and Folders (since 10.15) Data Vaults (see below) Mandatory access control, as the name suggests, is mandatory; it’s not an opt-in like the App Sandbox. Rather, all processes on the system, including those running as root, as subject to MAC. Data Vaults are not a third-party developer opportunity. See this post if you’re curious. In the Full Disk Access and Files and Folders case users grant a program a MAC privilege using System Preferences > Security & Privacy > Privacy. Some MAC privileges are per user (Files and Folders) and some are system wide (Full Disk Access). If you’re not sure, run this simple test: On a Mac with two users, log in as user A and enable the MAC privilege for a program. Now log in as user B. Does the program have the privilege? If a process tries to access an item restricted by MAC, the system may prompt the user to grant it access there and then. For example, if an app tries to access the desktop, you’ll see an alert like this: “AAA" would like to access files in your Desktop folder. [Don’t Allow] [OK] To customise this message, set properties in your Info.plist. See the Files and Folders topic on this page. This system only displays this alert once. It remembers the user’s initial choice and returns the same result thereafter. This relies on your code having a stable code signing identity. If your code is unsigned, or signed ad hoc (“Signed to run locally” in Xcode parlance), the system can’t tell that version N+1 of your code is the same as version N, and thus you’ll encounter excessive prompts. Note For information about how that works, see TN3127 Inside Code Signing: Requirements. The Files and Folders prompts only show up if the process is running in a GUI login session. If not, the operation is allowed or denied based on existing information. If there’s no existing information, the operation is denied by default. On managed systems the site admin can use the com.apple.TCC.configuration-profile-policy payload to assign MAC privileges. For testing purposes you can reset parts of TCC using the tccutil command-line tool. For general information about that tool, see its man page. For a list of TCC service names, see the posts on this thread. Note TCC stands for transparency, consent, and control. It’s the subsystem within macOS that manages the privileges visible in System Preferences > Security & Privacy > Privacy. TCC has no API surface, but you see its name in various places, including the above-mentioned configuration profile payload and command-line tool, and the name of its accompanying daemon, tccd. While tccutil is an easy way to do basic TCC testing, the most reliable way to test TCC is in a VM, restoring to a fresh snapshot between each test. If you want to try this out, crib ideas from Testing a Notarised Product. The MAC privilege mechanism is heavily dependent on the concept of responsible code. For example, if an app contains a helper tool and the helper tool triggers a MAC prompt, we want: The app’s name and usage description to appear in the alert. The user’s decision to be recorded for the whole app, not that specific helper tool. That decision to show up in System Preferences under the app’s name. For this to work the system must be able to tell that the app is the responsible code for the helper tool. The system has various heuristics to determine this and it works reasonably well in most cases. However, it’s possible to break this link. I haven’t fully research this but my experience is that this most often breaks when the child process does something ‘odd’ to break the link, such as trying to daemonise itself. Scripting MAC presents some serious challenges for scripting because scripts are run by interpreters and the system can’t distinguish file system operations done by the interpreter from those done by the script. For example, if you have a script that needs to manipulate files on your desktop, you wouldn’t want to give the interpreter that privilege because then any script could do that. The easiest solution to this problem is to package your script as a standalone program that MAC can use for its tracking. This may be easy or hard depending on the specific scripting environment. For example, AppleScript makes it easy to export a script as a signed app, but that’s not true for shell scripts. TCC and Main Executables TCC expects its bundled clients — apps, app extensions, and so on — to use a native main executable. That is, it expects the CFBundleExecutable property to be the name of a Mach-O executable. If your product uses a script as its main executable, you are likely to encounter TCC problems. To resolve these, switch to using a Mach-O executable. Revision History 2021-04-26 Added an explanation of the TCC initialism. Added a link to Viewing Sandbox Violation Reports.  Added the TCC and Main Executables section. Made significant editorial changes. 2022-01-10 Added a discussion of the file system hierarchy. 2021-04-26 First posted.
Posted
by eskimo.
Last updated
.
Post not yet marked as solved
0 Replies
2.6k Views
Many of the trusted execution problems I see are caused by folks signing their product using the --deep option. While that can work in some circumstances, I generally recommend against it. There are two issues with --deep: It applies the same code signing options to every code item that it signs, something that’s not appropriate in general. For example, you might have an app containing a nested command-line tool, where the app and the tool need different entitlements. The --deep option will apply the same entitlements to both, which is a serious mistake. It only signs code that it can find, and it only finds code in nested code sites. If you put code in a place where the system is expecting to find data, --deep won’t sign it. The first issue is fundamental to how --deep works, and is the main reason you should not use it. Indeed, on macOS it may cause the trusted execution system to block your program from running. For the details, see the Check for Entitlements on Library Code section of Resolving Library Loading Problems. The second issue is only a problem if you don’t follow the rules for nesting code and data within a bundle, as documented in Placing Content in a Bundle. However, my experience is that the products that don’t follow those rules are exactly the same sort of products that try to use --deep. The alternative to --deep is to sign each code item separately, from the inside out. If your product has lots of nested code, automate this using a script. Note One exception to the prohibition on --deep is Automator apps. If you’re signing an Automator app, see this DevForums post. For detailed information on how to correctly sign and package Mac software, see Creating Distribution-Signed Code for Mac and Packaging Mac Software for Distribution. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Revision history: 2022-06-14 Added a link to Resolving Library Loading Problems. Replaced the link to Signing a Mac Product For Distribution with a link to Creating Distribution-Signed Code for Mac. Made other minor editorial changes. 2021-10-21 Replaced the nested code reference with one to Placing Content in a Bundle. Minor editorial changes. 2020-09-15 Adopted the correct terminology for Automator apps. 2020-03-09 First version.
Posted
by eskimo.
Last updated
.
Post not yet marked as solved
0 Replies
158 Views
This post is part of a cluster of posts related to the trusted execution system. If you found your way here directly, I recommend that you start at the top. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Resolving Library Loading Problems On macOS the dynamic linker is responsible for loading the dynamic libraries used by your process. There are two parts to this: When a process runs an executable, the dynamic linker loads all the libraries used by that executable, the libraries used by those libraries, and so on. A process may load libraries at runtime using APIs like dlopen and NSBundle. For information the dlopen and friends, see the dlopen man page. The dynamic linker works closely with the trusted execution system to ensure that it only loads appropriate libraries. A critical concern is dynamic library impersonation attacks. If your program references a library, you want the dynamic linker to load that copy of the library, and not some other copy installed by an attacker. The primary protection is library validation. If library validation is enabled on an executable, the trusted execution system only allows the process to load code signed by Apple or with the same Team ID as the executable. Library validation is enabled by the [Hardened Runtime][refHR] but you may opt out of it using the Disable Library Validation Entitlement (com.apple.security.cs.disable-library-validation) entitlement. IMPORTANT Leave library validation enabled. Only disable it if your app needs to load plug-ins from other third-party developers. Disabling library validation makes it harder to pass Gatekeeper. See Resolving Gatekeeper Problems Caused by Dangling Load Command Paths for the details. When the dynamic linker fails to load a library it includes an explanation in the crash report. For example: Termination Reason: Namespace DYLD, Code 1 Library missing Library not loaded: @rpath/libEtranger.dylib Referenced from: /Users/USER/*/LinkToEtranger.app/Contents/MacOS/LinkToEtranger Reason: … (terminated at launch; ignore backtrace) Application Specific Information: Library not loaded: @rpath/libEtranger.dylib Referenced from: … Reason: … This explanation is likely to be truncated by the crash reporting system. To see the full log, run the app from Terminal: % ./LinkToEtranger.app/Contents/MacOS/LinkToEtranger dyld[79650]: Library not loaded: @rpath/libEtranger.dylib Referenced from: …/LinkToEtranger.app/Contents/MacOS/LinkToEtranger Reason: tried: '…/LinkToEtranger.app/Contents/MacOS/../Frameworks/libEtr anger.dylib' (code signature in <E16EDD14-CE5A-33BC-9B06-554A3BC12C51> '…/LinkToEtranger.app/Contents/Frameworks/libEtranger.dylib' not valid for use in process: mapping process and mapped file (non-platform) have different Team IDs), '…/LinkToEtranger.app/Contents/MacOS/../ Frameworks/libEtranger.dylib' (code signature in <E16EDD14-CE5A-33BC- 9B06-554A3BC12C51> '…/LinkToEtranger.app/Contents/Frameworks/ libEtranger.dylib' not valid for use in process: mapping process and mapped file (non-platform) have different Team IDs), '/usr/local/lib/ libEtranger.dylib' (no such file), '/usr/lib/libEtranger.dylib' (no such file) zsh: abort ./LinkToEtranger.app/Contents/MacOS/LinkToEtranger The Reason line is super long, so break it up by attempt: '…/LinkToEtranger.app/Contents/MacOS/../Frameworks/libEtranger.dylib' (…), '/usr/local/lib/libEtranger.dylib' (no such file), '/usr/lib/libEtranger.dylib' (no such file) Each entry starts with a place that the dynamic linker attempted to find library and then has text inside parentheses, like no such file, explaining what went wrong. Note The exact format of these messages varies from release-to-release of macOS. Many of these reasons are unrelated to the trusted execution system. For example, no such file means that the library isn’t present on disk. There are, however, three common trusted execution issues: Library validation Use of an old macOS SDK Restricted entitlements on library code For more information about the dynamic linker, see the dyld man page. Specifically, the DYLD_PRINT_SEARCHING environment variable is super useful when debugging library loading problems. Library Validation In any real world situation the Reason output from the dynamic linker is super long. To understand it better, break it up by attempt: '…/LinkToEtranger.app/Contents/MacOS/../Frameworks/libEtranger.dylib' (code signature in <E16EDD14-CE5A-33BC-9B06-554A3BC12C51> '…/LinkToEtranger.app/Contents/Frameworks/libEtranger.dylib' not valid for use in process: mapping process and mapped file (non-platform) have different Team IDs), '/usr/local/lib/libEtranger.dylib' (no such file), '/usr/lib/libEtranger.dylib' (no such file) The dynamic linker looked in three different places: The app’s Frameworks directory /usr/local/lib /usr/lib The first one is the important one because its path matches the expected location of the library. And the dynamic linker has logged an excellent explanation of the problem: code signature in … '…/LinkToEtranger.app/Contents/Frameworks/ libEtranger.dylib' not valid for use in process: mapping process and mapped file (non-platform) have different Team IDs In summary, the dynamic linker didn’t load this copy of libEtranger.dylib because it’s not a system library (non-platform) and it has a different Team ID from the process’s main executable. A quick trip to codesign confirms this: % codesign -d -vvv LinkToEtranger.app … TeamIdentifier=SKMME9E2Y8 … % codesign -d -vvv LinkToEtranger.app/Contents/Frameworks/libEtranger.dylib … TeamIdentifier=VL9SQP756U … How you fix this depends on the nature of your product. If this library is installed as part of your product, re-sign the library with a signing identity associated with your Team ID. Do this even if you didn’t build the code yourself. After all, you were responsible for putting the library on the user’s machine, and its signature should reflect that. One other possibility is that you’re building a program that supports plug-ins and thus you need to load a plug-in that was signed by another third-party developer. In this case the fix is to disable library validation by signing your executable with the Disable Library Validation Entitlement entitlement (com.apple.security.cs.disable-library-validation). IMPORTANT Disabling library validation makes it harder to pass Gatekeeper. See Resolving Gatekeeper Problems Caused by Dangling Load Command Paths for the details. Use of an Old macOS SDK Another dynamic library load failure related to the trusted execution system looks like this: code signature in … '…/LinkToDodo.app/Contents/Frameworks/libDodo.dylib' not valid for use in process: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed. Note The cdhash in this message refers to a code directory hash. For more information on cdhashes, see TN3126 Inside Code Signing: Hashes This is harder to understand, not least because the library is actually signed: % codesign -d -vvv LinkToDodo.app/Contents/Frameworks/libDodo.dylib … Authority=Apple Development: … … The explanation can be found tucked away in Notarizing macOS Software Before Distribution, which says: Apple's notary service requires you to adopt the following protections: … Link against the macOS 10.9 or later SDK macOS 10.9 introduced important code signing improvements. The hardened runtime depends on those improvements. It confirms their presence by looking at the SDK that the code was built with. If the code was built with an old SDK, or has no record of the SDK it was built with, the hardened runtime refuses to load it. In this example, the LinkToDodo app was linked to a modern SDK but the libDodo.dylib has no record of the SDK it was built with: % vtool -show-build LinkToDodo.app/Contents/MacOS/LinkToDodo … cmd LC_BUILD_VERSION … sdk 12.3 … % vtool -show-build LinkToDodo.app/Contents/Frameworks/libDodo.dylib LinkToDodo.app/Contents/Frameworks/libDodo.dylib: % That explains the error: The process has the hardened runtime enabled. The hardened runtime requires that all code be built with the macOS 10.9 SDK or later. libDodo.dylib has no record of the SDK it was build with, so the trusted execution system blocks it from loading. The dynamic linker reports that in its explanation of the problem. The best fix is to rebuild the code from source with the latest tools. If you can’t do that right now, see Notarisation and the macOS 10.9 SDK for a workaround. IMPORTANT This is a short-term compatibility measure. Plan to rebuild this code from source as soon as possible. If you got the code from another third-party developer, make sure they’re aware of this issue. Finally, if you can only reproduce this problem in the field and have managed to snag a sysdiagnose log of it, look in the system log for a log entry like this: type: default time: 2022-05-20 13:12:11.185889 +0100 process: kernel category: <Missing Description> message: …/LinkToDodo.app/Contents/Frameworks/libDodo.dylib: Possible race detected. Rejecting. That’s one cryptic smoking gun! For general information about the system log, see Your Friend the System Log. Restricted Entitlements on Library Code The third dynamic library load failure related to the trusted execution system looks like this: … OS Version: macOS 11.6.5 (20G527) … Termination Reason: DYLD, [0x5] Code Signature Application Specific Information: dyld: launch, loading dependent libraries Dyld Error Message: Library not loaded: @rpath/OverlyEntitled.framework/Versions/A/OverlyEntitled Referenced from: /Users/USER/AppWithEntitlementLibrary.app/Contents/MacOS/AppWithEntitlementLibrary Reason: no suitable image found. Did find: …/AppWithEntitlementLibrary.app/Contents/MacOS/../Frameworks/OverlyEntitled.framework/Versions/A/OverlyEntitled: code signature invalid for '…/AppWithEntitlementLibrary.app/Contents/MacOS/../Frameworks/OverlyEntitled.framework/Versions/A/OverlyEntitled' … Note This crash report is from macOS 11 because macOS 12 is smart enough to completely ignore entitlements on library code. However, the code signature is valid: % codesign -v -vvv AppWithEntitlementLibrary.app/Contents/Frameworks/OverlyEntitled.framework AppWithEntitlementLibrary.app/Contents/Frameworks/OverlyEntitled.framework: valid on disk AppWithEntitlementLibrary.app/Contents/Frameworks/OverlyEntitled.framework: satisfies its Designated Requirement It also passes both of the tests outlined in the previous section: % codesign -d -vvv AppWithEntitlementLibrary.app … TeamIdentifier=SKMME9E2Y8 … % codesign -d -vvv AppWithEntitlementLibrary.app/Contents/Frameworks/OverlyEntitled.framework … TeamIdentifier=SKMME9E2Y8 … % vtool -show-build AppWithEntitlementLibrary.app/Contents/Frameworks/OverlyEntitled.framework/Versions/A/OverlyEntitled … sdk 12.3 … The issue is that the framework is signed with a restricted entitlement: % codesign -d --entitlements - AppWithEntitlementLibrary.app/Contents/Frameworks/OverlyEntitled.framework … [Dict] [Key] com.apple.developer.networking.vpn.api [Value] [Array] [String] allow-vpn Entitlements are only effective when applied to a main executable. Code that isn’t a main executable is called library code, and that includes frameworks, dynamic libraries, and bundles. Do not apply entitlements to library code. At best it’s benign. At worse, it causes a code signing crash like this. Note For details on what constitutes a main executable, see Creating Distribution-Signed Code for Mac The Entitlements on macOS section of TN3125 Inside Code Signing: Provisioning Profiles define restricted entitlement and makes it clear that, on macOS, every restricted entitlement claimed by an executable must be authorised by its provisioning profile. However, library code does not have an embedded provisioning profile: A shared library has no bundle structure, and thus can’t include a provisioning profile. Library code with a bundle structure, frameworks and bundles, could have a provisioning profile but most tools, including Xcode, do not embed one. So, the OverlyEntitled framework is claiming a restricted entitlement but that claim isn’t authorised by a profile and thus the trusted execution system tells the dynamic linker not to load it. To fix this, change your code signing setup so that only main executables claim entitlements. For detailed advice on that topic, see Creating Distribution-Signed Code for Mac. IMPORTANT The number one cause of this problem is folks signing their code with --deep. Don’t do that, for this reason and for the other reasons outlined in --deep Considered Harmful. Revision History 2022-06-13 Added the Restricted Entitlements on Library Code section. 2022-05-20 First post.
Posted
by eskimo.
Last updated
.
Post not yet marked as solved
0 Replies
194 Views
This post is part of a cluster of posts related to the trusted execution system. If you found your way here directly, I recommend that you start at the top. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Resolving Code Signing Crashes on Launch A code signing crash has the following exception information: Exception Type: EXC_CRASH (SIGKILL (Code Signature Invalid)) IMPORTANT Most developers never see a code signing crash because they use Xcode to build and sign their product. Xcode’s code signing infrastructure detects problems that could cause a code signing crash, and its automatic code signing fixes them for you! If you’re having problems with code signing crashes and you can use Xcode but aren’t, consider making the switch Xcode. The most common code signing crash is a crash on launch. To confirm that, look at the thread backtraces: Backtrace not available If you see valid thread backtraces this is not a crash on launch. Go back to Resolving Trusted Execution Problems and read through the Code Signing Crashes After Launch section. If you see no thread backtraces, your code didn’t run at all. The trusted execution system has blocked it. In most cases there is some evidence of the problem in the system log. For example: type: error time: 2022-05-19 06:29:17.640331 -0700 process: taskgated-helper subsystem: com.apple.ManagedClient category: ProvisioningProfiles message: com.example.apple-samplecode.OverClaim: Unsatisfied entitlements: com.apple.overclaim This indicates that the OverClaim app, with bundle ID com.example.apple-samplecode.OverClaim, claimed a restricted entitlement, com.apple.overclaim, that wasn’t authorised by a provisioning profile. For more information about provisioning profiles, see TN3125 Inside Code Signing: Provisioning Profiles. Specifically, the Entitlements on macOS section discusses the concept of restricted entitlements. For general information about the system log, see Your Friend the System Log. Normalise the Entitlements Property List Entitlement property list files look like text and so it’s tempting to edit them with a text editor. This can lead to all sorts of problems. If you have code whose entitlements property list contains comments, non-Unix line endings, or other weird formatting, the trusted execution system may block it. To avoid such problems, normalise your entitlements property list before passing it to codesign. For example: % plutil -convert xml1 MyApp.plist % codesign -s III --entitlements MyApp.plist MyApp.app Problems like this typically show up on older systems. Modern systems use DER-encoded entitlements, as discussed in The future is DER section of TN3125. A related gotcha is line breaks. Consider this entitlements property list file: % cat MyApp.plist … <plist version="1.0"> <dict> <key> com.apple.security.cs.disable-library-validation</key> <true/> </dict> </plist> This is a valid property list but it doesn’t do what you think it does. It looks like it claims the com.apple.security.cs.disable-library-validation entitlement but in reality it claims \ncom.apple.security.cs.disable-library-validation. The system treats the latter as a restricted entitlement and thus requires it to be authorised by a profile. Of course no such profile will authorise that entitlement, and so the app is blocked by the trusted execution system. Similarly, consider this: % cat MyApp.plist … <plist version="1.0"> <dict> <key> com.apple.security.cs.disable-library-validation</key> <true/> </dict> </plist> This claims com.apple.security.cs.disable-library-validation, note the leading space, and that’s also blocked by the trusted execution system. Check for Unauthorised Entitlements Sometimes the system log may not make it obvious what’s gone wrong. It may be easier to work this out by looking at the built program. The most common cause of problems like this is the app claiming a restricted entitlement that’s not authorised by a provisioning profile. To start your investigation, dump the entitlements to check for restricted entitlements: % codesign -d --entitlements - "OverClaim.app" …/OverClaim.app/Contents/MacOS/OverClaim [Dict] [Key] com.apple.application-identifier [Value] [String] SKMME9E2Y8.com.example.apple-samplecode.OverClaim [Key] com.apple.developer.team-identifier [Value] [String] SKMME9E2Y8 [Key] com.apple.overclaim [Value] [Bool] true [Key] com.apple.security.get-task-allow [Value] [Bool] true In this case all the entitlements except com.apple.security.get-task-allow are restricted. Note If there are no restricted entitlements, something else has gone wrong. Go back to Resolving Trusted Execution Problems and look for other potential causes. Now check that the provisioning profile was embedded correctly and extract its payload: % ls -l "OverClaim.app/Contents/embedded.provisionprofile" … OverClaim.app/Contents/embedded.provisionprofile % security cms -D -i "OverClaim.app/Contents/embedded.provisionprofile" -o "OverClaim-payload.plist" Check that the profile applies to this app by dumping the com.apple.application-identifier entitlement authorised by the profile: % /usr/libexec/PlistBuddy -c "print :Entitlements:com.apple.application-identifier" OverClaim-payload.plist SKMME9E2Y8.com.example.apple-samplecode.* This should match the com.apple.application-identifier entitlement claimed by the app. Repeat this for all the remaining restricted entitlements: % /usr/libexec/PlistBuddy -c "print :Entitlements:com.apple.developer.team-identifier" OverClaim-payload.plist SKMME9E2Y8 % /usr/libexec/PlistBuddy -c "print :Entitlements:com.apple.overclaim" OverClaim-payload.plist Print: Entry, ":Entitlements:com.apple.overclaim", Does Not Exist In this example the problem is the com.apple.overclaim entitlement, which is claimed by the app but not authorised by the profile. If that’s the case for your program, you have two choices: If you program doesn’t need this entitlement, update your code signing to not claim it. If you program relies on this entitlement, update your profile to authorise it. The entitlement allowlist in the profile is built by the Apple Developer website based on the capabilities enabled on your App ID. To change this allowlist, modify your App ID capabilities and rebuild your profile. Some capabilities are only available on some platforms and, within that platform, for some distribution channels. For these details for macOS, see Developer Account Help > Reference > Supported capabilities (macOS). Some capabilities require review and approval by Apple. For more on this, see Developer Account Help > Reference > Provisioning with managed capabilities. Check the Signing Certificate If your program’s entitlements look good, the next most likely problem is that your program was signed by a signing identity whose certificate is not authorised by the profile. To debug this, first extract the certificate chain from your program: % codesign -d --extract-certificates=signed-with- "OverClaim.app" … % for i in signed-with-* ; do mv "${i}" "${i}.cer" ; done The first certificate is the one that matters: % certtool d "signed-with-0.cer" Serial Number : 53 DB 60 CC 85 32 83 DE 72 D9 6A C9 8F 84 78 25 … Subject Name : Other name : UT376R4K29 Common Name : Apple Development: Quinn Quinn (7XFU7D52S4) OrgUnit : SKMME9E2Y8 Org : Quinn Quinn Country : US … Now check this against each of the certificates authorised by the profile. Start by extracting the first one: % plutil -extract DeveloperCertificates.0 raw -o - OverClaim-payload.plist | base64 -D > "authorised0.cer" % certtool d "authorised0.cer" Serial Number : 46 A8 EF 2C 52 54 DE FD D1 76 9D 3A 41 7C 9E 43 … Subject Name : Other name : UT376R4K29 Common Name : Mac Developer: Quinn Quinn (7XFU7D52S4) OrgUnit : SKMME9E2Y8 Org : Quinn Quinn Country : US … That’s not a match. So try the next one: % plutil -extract DeveloperCertificates.1 raw -o - OverClaim-payload.plist | base64 -D > authorised1.cer % certtool d "authorised1.cer" Serial Number : 53 DB 60 CC 85 32 83 DE 72 D9 6A C9 8F 84 78 25 … Subject Name : Other name : UT376R4K29 Common Name : Apple Development: Quinn Quinn (7XFU7D52S4) OrgUnit : SKMME9E2Y8 Org : Quinn Quinn Country : US … This matches, which means the profile applies to this code. IMPORTANT When checking for a match, look at the Serial Number field. Don’t just rely on the Common Name field. A common mistake is to have two signing identities whose certificates have identical common names but the profile only lists one of them. If you get to the end of the list of certificate list in the profile and don’t find the certificate that the program was signed with, you know what the problem is: Your program is signed with a signing identity whose certificate is not listed in its profile. To fix this, either: Reconfigure your code signing to use a signing identity whose certificate is listed. Or update the profile to include the certificate of the signing identity you’re using. Check for Expiration If your certificates aren’t the problem, check that nothing has expired. Start with the certificate from the app’s signature: % certtool d "signed-with-0.cer" Serial Number : 53 DB 60 CC 85 32 83 DE 72 D9 6A C9 8F 84 78 25 … Not Before : 10:52:56 Apr 21, 2022 Not After : 10:52:55 Apr 21, 2023 … Also check the expiry date on the profile: % plutil -extract ExpirationDate raw -o - OverClaim-payload.plist 2023-04-21T11:02:58Z If either has expired, update it and re-sign your product. IMPORTANT Developer ID-signed code and installers include a secure timestamp. When the system checks the expiry date on a Developer ID certificate, it only checks that the certificate was valid at the time that the code was signed, base on that secure timestamp. Thus, an old Developer ID-signed app will continue to run after it’s certificate has expired. Check the Supported Devices If everything else checks out, the last thing to check is that the profile authorises the code to run on this machine. There are two cases here: Developer ID profiles authorise the code on all machines. Other profiles authorise the code on a specific list of machines. If you think you have a Developer ID profile, confirm that by looking for the ProvisionsAllDevices property: % plutil -extract "ProvisionsAllDevices" xml1 -o - "OverClaim-payload.plist" … No value at that key path or invalid key path: ProvisionsAllDevices If that’s not the case, get the ProvisionedDevices property and verify that the current machine’s provisioning UDID is listed there: % plutil -extract "ProvisionedDevices" xml1 -o - "OverClaim-payload.plist" … <array> … <string>A545CA26-80D7-5B38-A98C-530A798BE342</string> … </array> </plist> % system_profiler SPHardwareDataType … Provisioning UDID: A545CA26-80D7-5B38-A98C-530A798BE342 … If you get to the end any everything looks OK, your provisioning profile is not the cause of this crash. Return to Resolving Trusted Execution Problems for more suggestions. Revision History 2022-06-08 Added the Normalise the Entitlements Property List section. 2022-05-20 First posted.
Posted
by eskimo.
Last updated
.
Post not yet marked as solved
0 Replies
124 Views
This post is part of a cluster of posts related to the trusted execution system. If you found your way here directly, I recommend that you start at the top. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Resolving Gatekeeper Problems Caused by Dangling Load Command Paths Gatekeeper strives to ensure that only trusted software runs on a user’s Mac. It’s important that your code pass Gatekeeper. If not, you’re likely to lose a lot of customers, and your users’ hard-won trust. The most common reason for Gatekeeper to block an app is a dangling load command. To understand this issue, you first need to understand library validation. Library validation is an important security feature on macOS. If library validation is enabled on an executable, the process running that executable can only load code signed by Apple or with the same Team ID as the executable. This prevents a wide range of dynamic library impersonation attacks. Library validation is enabled by the Hardened Runtime but you may opt out of it using the Disable Library Validation Entitlement (com.apple.security.cs.disable-library-validation) entitlement. IMPORTANT Leave library validation enabled. Only disable it if your app needs to load plug-ins from other third-party developers. When Gatekeeper checks an app it looks at the state of library validation. If library validation is disabled, Gatekeeper does an extensive check of the app’s Mach-O images in an attempt to prevent dynamic library impersonation attacks. If library validation is enabled, Gatekeeper skips that check because library validation prevents such attacks. The upshot of this is that, if you disable library validation, it’s harder to pass Gatekeeper. Note The Swift Package Manager creates a dangling load command when you build a command-line tool for the Mac. For more on that issue, see the Compiler adds RPATH to executable, making macOS Gatekeeper angry thread on Swift Forums. Find Dangling Load Command Paths If your app is rejected by Gatekeeper, look in the system log for entries like this: type: error time: 2022-05-11 15:00:36.296159 -0700 process: XprotectService subsystem: com.apple.xprotect category: xprotect message: File /Applications/DanglingRPath.app/Contents/MacOS/DanglingRPath failed on rPathCmd /Users/mrgumby/Work/TrustedExecutionFailures/CoreWaffleVarnishing.framework/Versions/A/CoreWaffleVarnishing (rpath resolved to: (path not found), bundleURL: /Applications/DanglingRPath.app) In this example the entry mentions rPathCmd, and that’s a good way to search for such problems. Also search for loadCmd, which indicates a similar problem. IMPORTANT If the paths in the log entry are all <private>, enable private data in the system log. For information on how to do that, see Recording Private Data in the System Log. For general information about the system log, see Your Friend the System Log. In this example Gatekeeper has rejected the DanglingRPath app because: It references a framework, CoreWaffleVarnishing, using an rpath-relative reference. The rpath includes an entry that points outside of the app’s bundle, to a directory called /Users/mrgumby/Work. This opens the app up to a dynamic library impersonation attack. If an attacker placed a malicious copy of CoreWaffleVarnishing in /Users/mrgumby/Work, the DanglingRPath app would load it. To find the offending rpath entry, run otool against each Mach-O image in the app looking for a dangling LC_RPATH load command. For example: % otool -l DanglingRPath.app/Contents/MacOS/DanglingRPath | grep -B 1 -A 2 LC_RPATH Load command 18 cmd LC_RPATH cmdsize 48 path @executable_path/../Frameworks (offset 12) Load command 19 cmd LC_RPATH cmdsize 56 path /Users/mrgumby/Work (offset 12) This app has two LC_RPATH commands. The one, for @executable_path/../Frameworks, is fine: It points to a location within the app’s bundle. In contrast, the one for /Users/mrgumby/Work is clearly dangling. In this example, the dangling rpath entry is in the main executable but that’s not always the case; you might find it lurking in some deeply nested dynamic library. Keep in mind that this is only an issue because the app has library validation disabled: % codesign -d --entitlements - DanglingRPath.app Executable=/Users/mrgumby/Desktop/DanglingRPath.app/Contents/MacOS/DanglingRPath [Dict] [Key] com.apple.security.cs.disable-library-validation [Value] [Bool] true If library validation were enabled, Gatekeeper would skip this check entirely, and thus the app would sail past Gatekeeper. IMPORTANT In this example the app’s main executable has library validation disabled, but that’s not always the case. It’s common to see this problem in app’s that have multiple executables — the app itself and, say, one or two embedded helper tools — where one of the other executables has library validation disabled. Finally, the above example is based on rpath-relative paths. If you see a log entry containing the text loadCmd, search for dangling paths in LC_LOAD_DYLIB load commands. Fix Dangling Load Command Paths The best way to fix this problem is to not disable library validation. Library validation is an important security feature. By disabling it, you introduce this problem and reduce the security of your app. Conversely, by re-enabling it, you fix this problem and you improve security overall. The only situation where it makes sense to disable library validation is if your app loads plug-ins from other third-party developers. In that case, fixing this problem requires you to find and eliminate all dangling load command paths. There are two possibilities here: The dangling load command path is in a Mach-O image that you built from source. Or it’s in a Mach-O image that you got from a vendor, for example, a library in some third-party SDK. In the first case, solve this problem by adjusting your build system to not include the dangling load command path. In the second case, work with the vendor to eliminate the dangling load command path. If the vendor is unwilling to do this, it’s possible, as a last resort, to fix this problem by modifying the load commands yourself. For an example of how you might go about doing this, see Embedding Nonstandard Code Structures in a Bundle. And once you’re done, and your product has shipped, think carefully about whether you want to continue working with this vendor. Revision History 2022-06-13 Added a link to a related Swift Forums thread. 2022-05-20 First posted.
Posted
by eskimo.
Last updated
.
Post not yet marked as solved
0 Replies
283 Views
I help a lot of developers with trusted execution problems. For example, they might have an app being blocked by Gatekeeper, or an app that crashes on launch with a code signing error. If you encounter a problem that’s not explained here, start a new thread with the details. Make sure to add relevant tags — like Gatekeeper, Code Signing, and Notarization — so that I see your post. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Resolving Trusted Execution Problems macOS supports three software distribution channels: The user downloads an app from the App Store. The user gets a Developer ID-signed program directly from its developer. The user builds programs locally using Apple or third-party developer tools. The trusted execution system aims to protect users from malicious code. It’s comprised of a number of different subsystems. For example, Gatekeeper strives to ensure that only trusted software runs on a user’s Mac, while XProtect is the platform’s built-in antivirus technology. Note To learn more about these technologies, see Apple Platform Security. If you’re developing software for macOS your goal is to avoid trusted execution entanglements. You want users to install and use your product without taking any special steps. If, for example, you ship an app that’s blocked by Gatekeeper, you’re likely to lose a lot of customers, and your users’ hard-won trust. Trusted execution problems are rare with Mac App Store apps because the Mac App Store validation process tends to catch things early. This post is primarily focused on Developer ID-signed programs. Developers who use Xcode encounter fewer trusted execution problems because Xcode takes care of many code signing and packaging chores. If you’re not using Xcode, consider making the switch. If you can’t, consult the following for information on how to structure, sign, and package your code: Placing Content in a Bundle Embedding Nonstandard Code Structures in a Bundle Embedding a Command-Line Tool in a Sandboxed App Creating Distribution-Signed Code for Mac DevForums post Packaging Mac Software for Distribution DevForums post Gatekeeper Basics User-level apps on macOS implement a quarantine system for new downloads. For example, if Safari downloads a zip archive, it quarantines that archive. This involves setting the com.apple.quarantine extended attribute on the file. Note The com.apple.quarantine extended attribute is not documented as API. If you need to add, check, or remove quarantine from a file programmatically, use the quarantinePropertiesKey API. User-level unarchiving tools preserve quarantine. To continue the above example, if you double click the quarantined zip archive in the Finder, Archive Utility will unpack the archive and quarantine the resulting files. If you launch a quarantined app, the system invokes Gatekeeper. Gatekeeper checks the app for problems. If it finds no problems, it asks the user to confirm the launch, just to be sure. If it finds a problem, it displays an alert to the user and prevents them from launching it. The exact wording of this alert varies depending on the specific problem, and from release to release of macOS, but it generally looks like the ones shown in Apple > Support > Safely open apps on your Mac. The system may run Gatekeeper at other times as well. The exact circumstances under which it runs Gatekeeper is not documented and changes over time. However, running a quarantined app always invokes Gatekeeper. Unix-y networking tools, like curl and scp, don’t quarantine the files they download. Unix-y unarchiving tools, like tar and unzip, don’t propagate quarantine to the unarchived files. Confirm the Problem Trusted execution problems can be tricky to reproduce: You may encounter false negatives, that is, you have a trusted execution problem but you don’t see it during development. You may also encounter false positives, that is, things fail on one specific Mac but otherwise work. To avoid chasing your own tail, test your product on a fresh Mac, one that’s never seen your product before. The best way to do this is using a VM, restoring to a snapshot between runs. For a concrete example of this, see Testing a Notarised Product. The most common cause of problems is a Gatekeeper alert saying that it’s blocked your product from running. However, that’s not the only possibility. Before going further, confirm that Gatekeeper is the problem by running your product without quarantine. That is, repeat the steps in Testing a Notarised Product except, in step 2, download your product in a way that doesn’t set quarantine. Then try launching your app. If that launch fails then Gatekeeper is not the problem, or it’s not the only problem! Note The easiest way to download your app to your test environment without setting quarantine is scp. Alternatively, use xattr to remove the com.apple.quarantine extended attribute from the download before you unpack it. For more information about the xattr tool, see the xattr man page. Trusted execution problems come in all shapes and sizes. The remaining sections address the most common ones. App Blocked by Gatekeeper If your product is an app and it works correctly when not quarantined but is blocked by Gatekeeper when it is, you have a Gatekeeper problem. For advice on how to investigate such issues, see Resolving Gatekeeper Problems. App Can’t Be Opened Not all failures to launch are Gatekeeper errors. In some cases the app is just broken. For example: The app’s executable might be missing the x bit set in its file permissions. The app’s executable might be subtly incompatible with the current system. The classic example of this is trying to run a third-party app that contains arm64e code. macOS requires that third-party kernel extensions use the arm64e architecture. In other circumstances, stick to arm64 for your shipping products. If you want to test arm64e code locally, see Preparing Your App to Work with Pointer Authentication. The app’s executable might claim restricted entitlements that aren’t authorised by a provisioning profile. Or the app might have some other code signing problem. Note For more information about provisioning profiles, see TN3125 Inside Code Signing: Provisioning Profiles. In such cases the system displays an alert saying: The application “NoExec” can’t be opened. [[OK]] Note In macOS 11 this alert was: You do not have permission to open the application “NoExec”. Contact your computer or network administrator for assistance. [[OK]] which was much more confusing. A good diagnostic here is to run the app’s executable from Terminal. For example, an app with a missing x bit will fail to run like so: % NoExec.app/Contents/MacOS/NoExec zsh: permission denied: NoExec.app/Contents/MacOS/NoExec And an app with unauthorised entitlements will be killed by the trusted execution system: % OverClaim.app/Contents/MacOS/OverClaim zsh: killed OverClaim.app/Contents/MacOS/OverClaim In some cases running the executable from Terminal will reveal useful diagnostics. For example, if the app references a library that’s not available, the dynamic linker will print a helpful diagnostic: % MissingLibrary.app/Contents/MacOS/MissingLibrary dyld[88394]: Library not loaded: @rpath/CoreWaffleVarnishing.framework/Versions/A/CoreWaffleVarnishing … zsh: abort MissingLibrary.app/Contents/MacOS/MissingLibrary Code Signing Crashes on Launch A code signing crash has the following exception information: Exception Type: EXC_CRASH (SIGKILL (Code Signature Invalid)) The most common such crash is a crash on launch. To confirm that, look at the thread backtraces: Backtrace not available For steps to debug this, see Resolving Code Signing Crashes on Launch. One common cause of this problem is running distribution-signed code. Don’t do that! For details on why that’s a bad idea, see Don’t Run App Store Distribution-Signed Code. Code Signing Crashes After Launch If your program crashes due to a code signing problem after launch, you might have encountered the issue discussed in Updating Mac Software. Non-Code Signing Failures After Launch The hardened runtime enables a number of security checks within a process. Some coding techniques are incompatible with the hardened runtime. If you suspect that your code is incompatible with the hardened runtime, see Resolving Hardened Runtime Incompatibilities. App Sandbox Inheritance If you’re creating a product with the App Sandbox enabled and it crashes with a trap within _libsecinit_appsandbox, it’s likely that you’re having App Sandbox inheritance problems. For the details, see Resolving App Sandbox Inheritance Problems. Library Loading Problem Most library loading problems have an obvious cause. For example, the library might not be where you expect it, or it might be built with the the wrong platform or architecture. However, some library loading problems are caused by the trusted execution system. For the details, see Resolving Library Loading Problems. Explore the System Log If none of the above resolves your issue, look in the system log for clues as to what’s gone wrong. Some good keywords to search for include: gk, for Gatekeeper xprotect syspolicy, per the syspolicyd man page cmd, for Mach-O load command oddities amfi, for Apple mobile file integrity, per the amfid man page taskgated, see its taskgated man page yara, discussed in Apple Platform Security ProvisioningProfiles For general information the system log, see Your Friend the System Log. Revision History 2022-06-09 Added the Non-Code Signing Failures After Launch section. 2022-06-03 Added a link to Don’t Run App Store Distribution-Signed Code. Fixed the link to TN3125. 2022-05-20 First posted.
Posted
by eskimo.
Last updated
.
Post not yet marked as solved
0 Replies
81 Views
This post is part of a cluster of posts related to the trusted execution system. If you found your way here directly, I recommend that you start at the top. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Resolving Hardened Runtime Incompatibilities The hardened runtime enables a number of security checks within a process. Some coding techniques are incompatible with the hardened runtime. Best practice is to enable the hardened runtime on all code so that you uncover these problems early. However, some folks only notice these problems when they go to distribute their product. Specifically, Developer ID distribution requires notarisation and the notary service requires the hardened runtime. The typical symptoms of this failure is that the program launches but then fails almost immediately afterwards. Sometimes this failure results in the program just not working, sometimes it triggers a crash, and sometimes the program terminates itself by calling exit. In some cases this failure is accompanied by diagnostic printed to stdout, so it’s worth running the program from Terminal if you can. For advice on how to do that, see Resolving Trusted Execution Problems. If you suspect that this problem is caused by a hardened runtime incompatibility, the diagnostic test is easy: Temporarily disable the hardened runtime and see if the problem goes away. Once you’ve confirmed the problem is an incompatibility with the hardened runtime, you have two choices: Fix your code to avoid the problem. Apply a hardened runtime exception entitlement to disable one specific security feature of the hardened runtime. In general, the first option is best because it leaves your product with the best security. IMPORTANT When confronted by a hardened runtime incompatibility, some folks apply all of the hardened runtime exception entitlements. This is a mistake for three reasons: Some entitlements are subsets of other entitlements. For example, there’s no point applying com.apple.security.cs.allow-unsigned-executable-memory if you’re already applying com.apple.security.cs.disable-executable-page-protection. Disabling library validation with the com.apple.security.cs.disable-library-validation entitlement makes it harder to pass Gatekeeper. For more on this, see Resolving Gatekeeper Problems Caused by Dangling Load Command Paths. The hardened runtime exists for a reason: To enhance your product’s security. Don’t apply a hardened runtime exception entitlement without first understanding what the actual problem is. Debug hardened runtime incompatibilities like you would any other problem: Step through the code, add logging, and so on. The goal is to isolate which part of your code works with the hardened runtime disabled but fails with it enabled. Once you’ve found that code, the fix is usually pretty obvious. For example, if your program needs to generate executable code on the fly, you must: Allocate that memory using mmap with the MAP_JIT flag. Write to that memory using pthread_jit_write_protect_np or one of its related APIs. See the pthread_jit_write_protect_np man page for details. Sign the problem with the com.apple.security.cs.allow-jit entitlement. If you need help with this, feel free to ask here on DevForums. If you don’t control the code that has the hardened runtime incompatibility — this happens most often when using a third-party language runtime — ask the vendor how to proceed. They might have an updated version of their code, or specific advice on what hardened runtime exception entitlements to apply. If their advice is “Apply all the hardened runtime exception entitlements!”, think carefully about your vendor choices (-:
Posted
by eskimo.
Last updated
.