User Profile

DTS regularly receives questions about how to preserve keychain items across an App ID change, and so I thought I’d post a comprehensive answer here for the benefit of all. If you have any questions or comments, or other creative solutions!, please start a new thread here on DevForums, tagging it with Security so that I see it. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" App ID Prefix Change and Keychain Access The list of keychain access groups your app can access is determined by three entitlements. For the details, see Sharing Access to Keychain Items Among a Collection of Apps. If your app changes its App ID prefix, this list changes and you’re likely to lose access to existing keychain items. This situation crops up under two circumstances: When you migrate your app from using a unique App ID prefix to using your Team ID as its App ID prefix. When you transfer your app to another team. In both cases you have to plan carefully for this change. If you only learn about the problem after you’ve made the change, consider undoing the change to give you time to come up with a plan before continuing. Note On macOS, the information in this post only applies to the data protection keychain. For more information about the subtleties of the keychain on macOS, see On Mac Keychains. For more about App ID prefix changes, see Technote 2311 Managing Multiple App ID Prefixes and QA1726 Resolving the Potential Loss of Keychain Access warning. Migrate From a Unique App ID Prefix to Your Team ID Historically each app was assigned its own App ID prefix. This is no longer the case. Best practice is for apps to use their Team ID as their App ID prefix. This enables multiple neat features, including keychain item sharing and pasteboard sharing. If you have an app that uses a unique App ID prefix, consider migrating it to use your Team ID. This is a good thing in general, as long as you manage the migration process carefully. Your app’s keychain access group list is built from three entitlements: keychain-access-groups, see Keychain Access Groups Entitlement application-identifier (com.apple.application-identifier on macOS) com.apple.security.application-groups, see App Groups Entitlement IMPORTANT A macOS app can’t use an app group as a keychain access group. The first two depend on the App ID prefix. If that changes, you lose access to any keychain items in those groups. WARNING Think carefully before using the keychain to store secrets that are the only way to access irreplaceable user data. While the keychain is very reliable, there are situations where a keychain item can be lost and it’s bad if it takes the user’s data with it. In some cases losing access to keychain items is not a big deal. For example, if your app uses the keychain to manage a single login credential, losing that is likely to be acceptable. The user can recover by logging in again. In other cases losing access to keychain items is unacceptable. For example, your app might manage access to dozens of different servers, each with unique login credentials. Your users will be grumpy if you require them to log in to all those servers again. In such situations you must carefully plan your migration. The key element here is the third item in the list above, the com.apple.security.application-groups entitlement. An app group is tied to your team, and so your app retains access to the corresponding keychain access group across an App ID change. This suggests the following approach: Release a version of your app that moves keychain items from other keychain access groups to a keychain access group corresponding to an app group. Give your users time to update to this new version, run it, and so move their keychain items. When you’re confident that the bulk of your users have done this, change your App ID prefix. Be wary of the following caveats: This approach won’t work on macOS because macOS apps can’t use an app group as a keychain access group. It’s hard to judge how long to wait at step 2. Transfer Your App to Another Team There is no supported way to maintain access to keychain items across an app transfer. This makes it critical that you plan the transfer carefully. Note The approach described in the previous section doesn’t work in this case because app groups are tied to a team. There are three potential approaches here: Do nothing Do not transfer your app Get creative Do Nothing In this case the user loses all the secrets that your app stored in the keychain. This may be acceptable for certain apps. For example, if your app uses the keychain to manage a single login credential, losing that is likely to be acceptable. The user can recover by logging in again. Do Not Transfer Another option is to not transfer your app. Instead, ship a new version of the app from the new team and have the old app recommend that the user upgrade. There are a number of advantages to this approach. The first is that there’s absolutely no risk of losing any user data. The two apps are completely independent. The second advantage is that the user can install both apps on their device at the same time. This opens up a variety of potential migration paths. For example, you might ship an update to the old app with an export feature that saves the user’s state, including their secrets, to a suitably encrypted file, and then match that with an import facility on the new app. Finally, this approach offers flexible timing. The user can complete their migration at their leisure. However, there are a bunch of clouds to go with these silver linings: Your users might never migrate to the new app. If this is a paid app, or an app with in-app purchase, the user will have to buy things again. You lose the original app’s history, ratings, reviews, and so on. Get Creative Finally, you could attempt something creative. For example, you might: Publish a new version of the app that supports exporting the user’s state, including the secrets. Tell your users to do this, with a deadline. Transfer the app and then, when the deadline expires, publish the new version with an import feature. Frankly, this isn’t very practical. The problem is with step 2: There’s no good way to get all your users to do the export, and if they don’t do it before the deadline there’s no way to do it after.

I often get questions about third-party crash reporting. These usually show up in one of two contexts: Folks are trying to implement their own crash reporter. Folks have implemented their own crash reporter and are trying to debug a problem based on the report it generated. This is a complex issue and this post is my attempt to untangle some of that complexity. If you have a follow-up question about anything I've raised here, please put it in a new thread with the Debugging tag. IMPORTANT All of the following is my own direct experience. None of it should be considered official DTS policy. If you have questions that need an official answer (perhaps you’re trying to convince your boss that implementing your own crash reporter is a very bad idea :-), open a DTS tech support incident and we can discuss things there. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Scope First, I can only speak to the technical side of this issue. There are other aspects that are beyond my remit: I don’t work for App Review, and only they can give definitive answers about what will or won’t be allowed on the store. Implementing your own crash reporter has significant privacy implications. IMPORTANT If you implement your own crash reporter, discuss the privacy impact with a lawyer. This post assumes that you are implementing your own crash reporter. A lot of folks use a crash reporter from another third party. From my perspective these are the same thing. If you use a custom crash reporter, you are responsible for its behaviour, both good and bad, regardless of where the actual code came from. Note If you use a crash reporter from another third party, run the tests outlined in Preserve the Apple Crash Report to verify that it’s working well. General Advice I strongly advise against implementing your own crash reporter. It’s very easy to create a basic crash reporter that works well enough to debug simple problems. It’s impossible to implement a good crash reporter, one that’s reliable, binary compatible, and sufficient to debug complex problems. The bulk of this post is a low-level explanation of that impossibility. Rather than attempting the impossible, I recommend that you lean in to Apple’s crash reporter. In recent years it’s acquired some really cool new features: If you’re creating an App Store app, the Xcode organiser gives you easy, interactive access to Apple crash reports. If you’re an enterprise developer, consider switching to Custom App Distribution. This yields all the benefits of App Store distribution without your app being generally available on the store. iOS 14 and macOS 12 report crashes in MetricKit. If you previously dismissed Apple crash reports as insufficient, I encourage you to reconsider that decision. Why Is This Impossible? Earlier I said “It’s impossible to implement a good crash reporter”, and I want to explain why I’m confident enough in my conclusions to use that specific word. There are two fundamental problems here: On iOS (and the other iOS-based platforms, watchOS and tvOS) your crash reporter must run inside the crashed process. That means it can never be 100% reliable. If the process is crashing then, by definition, it’s in an undefined state. Attempting to do real work in that state is just asking for problems [1]. To get good results your crash reporter must be intimately tied to system implementation details. These can change from release to release, which invalidates the assumptions made by your crash reporter. This isn’t a problem for the Apple crash reporter because it ships with the system. However, a crash reporter that’s built in to your product is always going to be brittle. I’m speaking from hard-won experience here. I worked for DTS during the PowerPC-to-Intel transition, and saw a lot of folks with custom crash reporters struggle through that process. Still, this post exists because lots of folks ignore this reality, so the subsequent sections contain advice about specific technical issues. WARNING Do not interpret any of the following as encouragement to implement your own crash reporter. I strongly advise against that. However, if you ignore my advice then you should at least try to minimise the risk, which is what the rest of this document is about. [1] On macOS it’s possible for your crash reporter to run out of process, just like the Apple crash reporter. However, possible is not the same as easy. In fact, running out of process can make things worse: It prevents you from geting critical state for the crashed process without being tightly bound to OS implementation details. It would be nice if Apple provided APIs for this sort of thing, but that’s currently not the case. Preserve the Apple Crash Report You must ensure that your crash reporter doesn’t disrupt the Apple crash reporter. This is important for three reasons: Some fraction of your crashes will not be caused by your code but by problems in framework code, and accurate Apple crash reports are critical in diagnosing such issues. When dealing with really hard-to-debug problems, you need the more obscure info that’s shown in the Apple crash report. If you’re working with someone from Apple (via a bug report, DTS incident, or whatever), they’re going to want an accurate Apple crash report. If your crash reporter is disrupting the Apple crash reporter — either preventing it from generating crash reports entirely [1], or distorting those crash reports — that limits how much they can help you. To avoid these issues I recommend that you test your crash reporter’s impact on the Apple crash reporter. The basic idea is: Create a program that generates a set of specific crashes. Run through each crash. Verify that your crash reporter produces sensible results. Verify that the Apple crash reporter produces the same results as it does without your crash reporter With regards step 1, your test suite should include: An un-handled language exception thrown by your code An un-handled language exception thrown by the OS (accessing an NSArray out of bounds is an easy way to get this) Various machine exceptions (at a minimum, memory access, illegal instruction, and breakpoint exceptions) Stack overflow Make sure to test all of these cases on both the main thread and a secondary thread. With regards step 4, check that the resulting Apple crash report includes correct values for: The exception info The crashed thread That thread’s state Any application-specific info, and especially the last exception backtrace [1] A particularly pathological behaviour here is to end your crash reporter by calling exit. This completely suppresses the Apple crash report. Some third-party language runtimes ‘helpfully’ include such a crash reporter, which makes it very hard to debug problems that occur within your process but outside of that language. Signals Many third-party crash reporters use UNIX signals to catch the crash. This is a shame because using Mach exception handling, the mechanism used by the Apple crash reporter, is generally a better option. However, there are two reasons to favour UNIX signals over Mach exception handling: On iOS-based platforms your crash reporter must run in-process, and doing in-process Mach exception handling is not feasible. Folks are a lot more familiar with UNIX signals. Mach exception handling, and Mach messaging in general, is pretty darned obscure. If you use UNIX signals for your crash reporter, be aware that this API has some gaping pitfalls. First and foremost, your signal handler can only use async signal safe functions [1]. You can find a list of these functions in sigaction man page [2] [3]. WARNING This list does not include malloc. This means that a crash reporter’s signal handler cannot use Objective-C or Swift, as there’s no way to constrain how those language runtimes allocate memory. That means you’re stuck with C or C++, but even there you have to be careful to comply with this constraint. The Operative: It’s worse than you know. Captain Malcolm Reynolds: It usually is. Many crash reports use functions like backtrace (see its man page) to get a backtrace from their signal handler. There’s two problems with this: backtrace is not an async signal safe function. backtrace uses a naïve algorithm that doesn’t deal well with cross signal handler stack frames [4]. The latter point is particularly worrying, because it hides the identity of the stack frame that triggered the signal. If you’re going to backtrace out of a signal, you must use the crashed thread’s state (accessible via the handlers uap parameter) to start your backtrace. Apropos that, if your crash reporter wants to log the state of the crashed thread, that’s the place to get it. Your signal handler must be prepared to be called by multiple threads. A typical crashing signal (like SIGSEGV) is delivered to the thread that triggered the machine exception. While your signal handler is running on that thread, other threads in your process continue to run. One of these threads could crash, causing it to call your signal handler. It’s a good idea to suspend all threads in your process early in your signal handler. However, there’s no way to completely eliminate this window. Note The need to suspend all the other threads in your process is further evidence that sticking to async signal safe functions is required. An unsafe function might depend on a thread you’ve suspended. A typical crashing signal is delivered on the thread that triggered the machine exception. If the machine exception was caused by a stack overflow, the system won’t have enough stack space to call your signal handler. You can tell the system to switch to an alternative stack (see the discussion of SA_ONSTACK in the sigaction man page) but that isn’t a complete solution (because of the thread issue discussed immediately above). Finally, there’s the question of how to exit from your signal handler. You must not call exit. There’s two problems with doing that: exit is not async signal safe. In fact, exit can run arbitrary code via handlers registered with atexit. If you want to exit the process, call _exit. Exiting the process is a bad idea anyway, because it will prevent the Apple crash reporter from running. This is very poor form. For an explanation as to why, see Preserve the Apple Crash Report (above). A better solution is to unregister your signal handler (set it to SIG_DFL) and then return. This will cause the crashed process to continue execution, crash again, and generate a crash report via the Apple crash reporter. [1] While the common signals caught by a crash reporter are not technically async signals (except SIGABRT), you still have to treat them as async signals because they can occur on any thread at any time. [2] It’s reasonable to extend this list to other routines that are implemented as thin shims on a system call. For example, I have no qualms about calling vm_read (see below) from a signal handler. [3] Be aware, however, that even this list has caveats. See my Async Signal Safe Functions vs Dyld Lazy Binding post for details. [4] Cross signal handler stack frames are pushed on to the stack by the kernel when it runs a signal handler on a thread. As there’s no API to learn about the structure of these frames, there’s no way to backtrace across one of these frames in isolation. I’m happy to go into details but it’s really not relevant to this discussion [5]. If you’re interested, start a new thread with the Debugging tag and we can chat there. [5] (Arg, my footnotes have footnotes!) The exception to this is where your trying to generate a crash report for code running in a signal handler. That’s not easy, and frankly you’re better off avoiding signal handlers in general. Where possible, handle signals via a Dispatch event source. Reading Memory A signal handler must be very careful about the memory it touches, because the contents of that memory might have been corrupted by the crash that triggered the signal. My general rule here is that the signal handler can safely access: Its code Its stack (subject to the constraints discussed earlier) Its arguments Immutable global state In the last point, I’m using immutable to mean immutable after startup. It’s reasonable to set up some global state when the process starts, before installing your signal handler, and then rely on it in your signal handler. Changing any global state after the signal handler is installed is dangerous, and if you need to do that you must be careful to ensure that your signal handler sees consistent state, even though a crash might occur halfway through your change. You can’t protect this global state with a mutex because mutexes are not async signal safe (and even if they were you’d deadlock if the mutex was held by the thread that crashed). You should be able to use atomic operations for this, but atomic operations are notoriously hard to use correctly (if I had a dollar for every time I’ve pointed out to a developer they’re using atomic operations incorrectly, I’d be very badly paid (-: but that’s still a lot of developers!). If your signal handler reads other memory, it must take care to avoid crashing while doing that read. There’s no BSD-level API for this [1], so I recommend that you use vm_read. [1] The traditional UNIX approach for doing this is to install a signal handler to catch any memory access exceptions triggered by the read, but now we’re talking signal handling within a signal handler and that’s just silly. Writing Files If your want to write a crash report from your signal handler, you must use low-level UNIX APIs (open, write, close) because only those low-level APIs are documented to be async signal safe. You must also set up the path in advance because the standard APIs for determining where to write the file (NSFileManager, for example) are not async signal safe. Offline Symbolication Do not attempt to do symbolication from your signal handler. Rather, write enough information to your crash report to support offline symbolication. Specifically: The addresses to symbolicate For each Mach-O image in the process: The image path The image UUID The image load address You can get most of the Mach-O image information using the APIs in <mach-o/dyld.h> [1]. Be aware, however, that these APIs are not async signal safe. You’ll need to get this information in advance and cache it for your signal handler to record. This is complicated by the fact that the list of Mach-O images can change as you process loads and unloads code. This requires you to share mutable state with your signal handler, which is exactly what I recommend against in Reading Memory. Note You can learn about images loading and unloading using _dyld_register_func_for_add_image and _dyld_register_func_for_remove_image respectively. [1] I believe you’ll need to parse the Mach-O load commands to get the image UUID. What to Include When deciding what to include in a crash report, there’s a three-way balance to be struck: The more information you include, the easier it is to diagnose problems. Some information is hard to obtain, either because there’s no public API to get that information, or because the API is not available to your crash reporter. Some information is so privacy-sensitive that it has no place in a crash report. Apple’s crash reporter strikes its own balance here, and I recommend that you try to include everything that it includes, subject to the limitations described in point 2. Here’s what I’d considered to be a minimal list: Information about the machine exception that triggered the crash For memory access exceptions, the address of the access that triggered the crash Backtraces of all the threads (sometimes the backtrace of a non-crashing thread can yield critical information about the crash) The crashed thread Its thread state A list of Mach-O images, as discussed in the Offline Symbolication section IMPORTANT Make sure you report the thread backtraces in a consistent order. Without that it’s hard to correlate information across crash reports. Change History 2019-02-12 — First posted. 2019-02-13 — Made minor editoral changes. Added a new footnote to the Signals section. 2019-02-14 — Clarified the complexities of an out-of-process crash reporter. Added the What to Include section. Enhanced the Signals section to cover reentrancy and stack overflow. Made minor editoral changes. 2019-02-15 — Expanded the introduction to the Preserve the Apple Crash Report section. 2019-05-13 — Added a reference to my Async Signal Safe Functions vs Dyld Lazy Binding post. 2021-02-27 — Fixed the formatting. Made minor editoral changes. 2021-09-10 — Expanded the General Advice section to include pointers to Apple crash report resources, including MetricKit. Split the second half of that section out in to a new Why Is This Impossible? section. Made minor editoral changes. 2022-05-16 — Fixed a broken link.

I regularly help developers with notarisation and trusted execution problems. The notary log is a critical tool for debugging such problems, and so I figured I should post instructions on how to fetch it. If you have comments or questions about this, start a new thread and tag it with Notarization so that I see it. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Fetching the Notary Log The notary log is a key tool for debugging notarisation and trusted execution issues. Review the log: When notarisation fails, for information as to what’s wrong When notarisation succeeds, to check for warnings If you encounter a trusted execution problem, to confirm that all your code was included in the notarised ticket The way to fetch the log depends on how you notarised your product. Reveal the Notary Log in Xcode If you notarised your app with Xcode, reveal the notary log as follows: Go to the Xcode organiser. Select the archive that you notarised. Click Show Status Log. Find the “Ready to distribute” or “Package Invalid” item and click the small arrow next to it. That reveals the notary log, a file called audit.log, in the Finder. Fetch the Notary Log with notarytool If you notarise your product from the command-line using notarytool, use the log subcommand to fetch the notary log: % xcrun notarytool log CREDENTIALS REQUEST_UUID where: CREDENTIALS is your notary service credentials, the same credentials you used to submit your request REQUEST_UUID is the UUID printed by notarytool when you submitted your request. Fetch the Notary Log with altool If you notarise your product from the command-line using altool, first run the --notarization-info subcommand to get the notary log URL: % xcrun altool CREDENTIALS --notarization-info REQUEST_UUID … LogFileURL: https://osxapps-ssl.itunes.apple.com/itunes-assets/Enigma… … where: CREDENTIALS is your notary service credentials, the same credentials you used to submit your request REQUEST_UUID is the UUID printed by notarytool when you submitted your request. In the output the URL next to LogFileURL is the notary log URL. Use your HTTP client of choice to fetch that URL. For example: % curl "https://osxapps-ssl.itunes.apple.com/itunes-assets/Enigma…" The notary log URL has a limited lifetime, so fetch the log immediately after getting the URL. If the URL is too old, more than a day or so, use --notarization-info to fetch a new URL. IMPORTANT altool has been deprecated for the purposes of notarisation. Switch to notarytool; it’s better, stronger, and faster. For the details, see WWDC 2021 Session 10261 Faster and simpler notarization for Mac apps.

I’ve discussed all of the following on DevForums before, but scattered across multiple threads. The issue of system log private data recently came up in another context and I decided to collect my thoughts in one place. If you have any questions or comments, start a new thread and tag it with OSLog so that I see it. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Recording Private Data in the System Log The unified system log on Apple platforms is a Thing of Wonder™. If you’re not convinced, go read Your Friend the System Log. One great feature of the system log is private data: If you tag a value as private, it’s omitted from the log by default. This keep personal information out the log, which is obviously a good thing. In some circumstances it makes sense to record private data. Imagine, for example, that you’re investigating a Gatekeeper problem on macOS and you see a log entry like this: type: info time: 2022-05-12 05:53:35.896830 -0700 process: XprotectService subsystem: com.apple.xprotect category: xprotect message: SecAssessment results: <private> (null) That looks interesting, but the assessment result, the thing that you really want to see, was omitted. Now look at the same log entry with the private data enabled: type: info time: 2022-05-12 05:55:21.673682 -0700 process: XprotectService subsystem: com.apple.xprotect category: xprotect message: SecAssessment results: { "assessment:authority" = { LSDownloadRiskCategoryKey = LSRiskCategoryMayContainUnsafeExecutable; "assessment:authority:flags" = 0; "assessment:authority:source" = "_XProtect"; }; "assessment:remote" = 1; "assessment:verdict" = 1; } (null) That’s much more helpful. There are two ways to enable recording of private data in the system log: Add an OSLogPreferences property in your app’s profile (all platforms). Create and install a custom configuration profile with the SystemLogging (com.apple.system.logging) payload (macOS only). Log Preferences Property Imagine an app with this code: let log = Logger(subsystem: "com.example.apple-samplecode.PrivateValueLogger", category: "app") let client: String = … log.debug("will answer the ultimate question, client: \(client)") let answer: Int = … lots of work … log.debug("did answer the ultimate question, answer: \(answer, privacy: .private)") Note This uses the Logger API in Swift but the same logic applies to all system log APIs. With this code both the client and answer values are considered private data, the first because that’s the default for strings and the second because it’s explicitly called out as private. And if you run the app outside of Xcode you’ll see that this private data is omitted: type: debug time: 16:55:23.994070+0100 process: PrivateValueLogger subsystem: com.example.apple-samplecode.PrivateValueLogger category: app message: will answer the ultimate question, client: <private> type: debug time: 16:55:23.994108+0100 process: PrivateValueLogger subsystem: com.example.apple-samplecode.PrivateValueLogger category: app message: did answer the ultimate question, answer: <private> Note When you use Xcode to run the app, it sets an environment variable, OS_ACTIVITY_DT_MODE, that changes the default handling of log entries. To record this private data, add the following to the app’s Info.plist file: <key>OSLogPreferences</key> <dict> <key>com.example.apple-samplecode.PrivateValueLogger</key> <dict> <key>app</key> <dict> <key>Enable-Private-Data</key> <true/> </dict> </dict> </dict> Now the private data is recorded in the log: … message: will answer the ultimate question, client: The Mice … message: did answer the ultimate question, answer: 42 This might not seem like a big deal — if you’re going to rebuild the app, you could just as easily change the code to log the values as public — but it has some important benefits: You only have to make this change in one place. If your app has lots of log points, this change affects them all. It works for code that you don’t control. If you need to record private data for library code running in your process, including system frameworks, you can do just that. It works on both macOS and iOS. Configuration Profile macOS has an extra trick up its sleeve, namely, the SystemLogging configuration profile payload (com.apple.system.logging). Use this to enable private data without modifying your code. For example, here’s an outline of a configuration profile that enables private data for the code shown earlier: … <dict> <key>PayloadContent</key> <array> <dict> … <key>PayloadType</key> <string>com.apple.system.logging</string> … <key>Subsystems</key> <dict> <key>com.example.apple-samplecode.PrivateValueLogger</key> <dict> <key>app</key> <dict> <key>Enable-Private-Data</key> <true/> </dict> </dict> </dict> </dict> </array> … </dict> </plist> Critically, this applies not just to your app but to all logging on the system. Continuing the Gatekeeper example from earlier, to see the assessment result, create a profile to enable private data for the com.apple.xprotect subsystem and the xprotect category. For detailed information on constructing a configuration profile, see Device Management. To get a head start with that, use Apple Configurator to create profile with a dummy payload and then use a text editor to modify that payload along the lines shown above.

The notary service requires that all Mach-O images be linked against the macOS 10.9 SDK or later. This isn’t an arbitrary limitation. The hardened runtime, another notarisation requirement, relies on code signing features that were introduced along with macOS 10.9 and it uses the SDK version to check for their presence. Specifically, it checks the SDK version using the sdk field in the LC_BUILD_VERSION Mach-O load command (or the older LC_VERSION_MIN_MACOSX command). The best way to meet this requirement is to rebuild your code with modern tools. However, in some cases that’s not possible. Imagine if your app relies on the closed source libDodo.dylib library. That library’s vendor went out of business 10 years ago, and so the library hasn’t been updated since then. Indeed, the library was linked against the macOS 10.6 SDK. What can you do? The first thing to do is come up with a medium-term plan for breaking your dependency on libDodo.dylib. Relying on an unmaintained library is not something that’s sustainable in the long term. The history of the Mac is one of architecture transitions — 68K to PowerPC to Intel, 32- to 64-bit, and so on — and this unmaintained library will make it much harder to deal with the next transition. IMPORTANT I wrote the above prior to the announcement of the latest Apple architecture transition, Apple silicon. When you update your product to a universal binary, you might as well fix this problem on the Intel side as well. Do not delay that any further: While Apple silicon Macs are currently able to run Intel code using Rosetta 2, that’s not something you want to rely on in the long term. Heed this advice from About the Rosetta Translation Environment: Rosetta is meant to ease the transition to Apple silicon, giving you time to create a universal binary for your app. It is not a substitute for creating a native version of your app. But what about the short term? Historically I wasn’t able to offer any help on that front, but this has changed recently. Xcode 11 ships with a command-line tool, vtool, that can change the LC_BUILD_VERSION and LC_VERSION_MIN_MACOSX commands in a Mach-O. You can use this to change the sdk field of these commands, and thus make your Mach-O image ‘compatible’ with notarisation and the hardened runtime. Before doing this, consider these caveats: Any given Mach-O image has only a limited amount of space for load commands. When you use vtool to set or modify the SDK version, the Mach-O could run out of load command space. The tool will fail cleanly in this case but, if it that happens, this technique simply won’t work. Changing a Mach-O image’s load commands will break the seal on its code signature. If the image is signed, remove the signature before doing that. To do this run codesign with the --remove-signature argument. You must then re-sign the library as part of your normal development and distribution process. Remember that a Mach-O image might contain multiple architectures. All of the tools discussed here have an option to work with a specific architecture (usually -arch or --architecture). Keep in mind, however, that macOS 10.7 and later do not run on 32-bit Macs, so if your deployment target is 10.7 or later then it’s safe to drop any 32-bit code. If you’re dealing with a Mach-O image that includes 32-bit Intel code, or indeed PowerPC code, make your life simpler by removing it from the image. Use lipo for this; see its man page for details. It’s possible that changing a Mach-O image’s SDK version could break something. Indeed, many system components use the main executable’s SDK version as part of their backwards compatibility story. If you change a main executable’s SDK version, you might run into hard-to-debug compatibility problems. Test such a change extensively. It’s also possible, but much less likely, that changing the SDK version of a non-main executable Mach-O image might break something. Again, this is something you should test extensively. This list of caveats should make it clear that this is a technique of last resort. I strongly recommend that you build your code with modern tools, and work with your vendors to ensure that they do the same. Only use this technique as part of a short-term compatibility measure while you implement a proper solution in the medium term. For more details on vtool, read its man page. Also familiarise yourself with otool, and specifically the -l option which dumps a Mach-O image’s load commands. Read its man page for details. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Revision history: 2020-09-11 — First posted. 2022-05-09 — Updated with a note about Apple silicon.

IMPORTANT This post is now retired in favour of TN3126 Inside Code Signing: Hashes. I’m leaving the original post here just for the record, but you should consider the official documentation authoritative. I was explaining code signing to someone today and, well, at the end of that conversation I found that I’d ‘accidentally’ written up a high-level description of how code signing works (-: Rather than let that go to waste, I thought I’d post a copy here. IMPORTANT This is all implementation details. You don’t need this info in order to sign your product for Apple platforms. If you use Xcode, it takes care of code signing for you. If you build a Mac product outside of Xcode, follow the advice in: Creating Distribution-Signed Code for Mac Packaging Mac Software for Distribution However, sometimes it’s useful to know a little bit about the man behind the curtain. It allows you to make better design choices and debug leaky abstraction layers. And hey, sometimes it’s just fun to know this stuff! If you have questions about code signing, put them in a new thread here on DevForums. Tag it with Code Signing so that I see it. Finally, if you really want to know how code signing works, most, maybe even all, of the implementation is available in Darwin. Specifically, look in the xnu project for the kernel side of it and the Security for the user space side. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" A Peek Behind the Code Signing Curtain Code signing is a fundamental technology on Apple platforms. The vast majority of developers don’t need to know how code signing works. They use Xcode, which takes care of the fiddly details. And, if they run into problems, those are usually related to high-level concepts — signing identities, entitlements, provisioning profiles — not the core code signing implementation. However, that’s not always the case. Every now and again an issue crops up where you actually need to understand how code signing works. For example: My Signing code for older versions of macOS on Apple Silicon post explains how to work around a code signing issue that cropped up during the early days of Apple silicon. It’s easier to understand issues like this if you understand some core code signing concepts. Using the Latest Code Signature Format has a diagnostic process that involves code signing hash slots. While the diagnostic itself is useful, it makes more sense if you know what those slots are for. The issue covered by Updating Mac Software makes more sense once you understand code signing’s lazy per-page signature checking. The goal of this post is to explain some of these core code signing concepts so that you can better understand, and potentially debug, weird code signing issues. I want to stress that this description is not comprehensive. I’ve elided various details for various reasons: Some stuff I haven’t yet researched in depth, some stuff is only interesting from a historical perspective, and some stuff is not relevant to my overall goal. My focus here is on macOS but these core code signing concepts apply to all Apple platfroms. Finally, a word of warning… WARNING Code signing has changed in the past and you can reasonably expect it to change again in the future. If you build a product that relies on these details, you must accept the compatibility risks that this entails. Reading this post is like reading the Darwin source: You’ll learn a lot, but you need to be careful about how you apply that knowledge. Code Signature Storage The code signature for an item is stored in one of four ways: If the item is a Mach-O image, or is a bundle wrapped around a Mach-O image, the code signature is stored within the image. % otool -l "/Applications/PCalc.app/Contents/MacOS/PCalc" | grep LC_CODE_SIGNATURE -B 1 -A 3 Load command 38 cmd LC_CODE_SIGNATURE cmdsize 16 dataoff 3170576 datasize 60656 If the item is a bundle without a Mach-O image, the code signature is stored in the bundle’s _CodeSignature directory: % codesign -d -vvv Test.bundle … Format=bundle … Authority=Apple Development: Quinn Quinn (7XFU7D52S4) … % find Test.bundle Test.bundle Test.bundle/Contents Test.bundle/Contents/_CodeSignature Test.bundle/Contents/_CodeSignature/CodeResources Test.bundle/Contents/_CodeSignature/CodeDirectory Test.bundle/Contents/_CodeSignature/CodeRequirements-1 Test.bundle/Contents/_CodeSignature/CodeSignature Test.bundle/Contents/_CodeSignature/CodeRequirements Test.bundle/Contents/Info.plist If the item exists within a bundle, it’s covered by the bundle’s code signature, as discussed below. Otherwise, the code signature is stored in extended attributes (EAs) on the item: % cat hello.txt Hello Cruel World! % codesign -d -vvv hello.txt … Format=generic … Authority=Apple Development: Quinn Quinn (7XFU7D52S4) … % ls -l@ hello.txt -rw-r--r--@ 1 quinn staff 19 14 Mar 22:09 hello.txt com.apple.cs.CodeDirectory 129 com.apple.cs.CodeRequirements 168 com.apple.cs.CodeRequirements-1 165 com.apple.cs.CodeSignature 4860 IMPORTANT Storing a code signature in EAs is brittle because many file transfer mechanisms drop these. If you follow the rules in Placing Content in a Bundle, none of your files will have code signature EAs. Code Directory The central concept in a code signature is the code directory. This is a data structure that holds all of the info about the code being signed. It’s this data structure that’s signed as part of the signing process [1], and it’s the hashes in this data structure that seal the code, resources, and metadata of your product. If you have a universal binary then each architecture is signed independently, so each has its own code directory. Hashing the code directory results in a code directory hash, or cdhash. This value uniquely identifies the code being signed. It crops up in a variety of places, most notably notarisation. A notarised ticket is actually a set of cdhash values that have been signed by the notary service. A code directory can be hashed by different algorithms, resulting in different cdhash values. Most code has a SHA-1 and a SHA-256 cdhash: % codesign -d -vvv "/Applications/PCalc.app" … CandidateCDHash sha1=eca4d50c1736d878a5c4f5f7994960735311f314 CandidateCDHashFull sha1=eca4d50c1736d878a5c4f5f7994960735311f314 CandidateCDHash sha256=a9b23d5ca054140c96fb770ee8391d96d8515923 CandidateCDHashFull sha256=a9b23d5ca054140c96fb770ee8391d96d8515923804608438e2629770d5792c6 Hash choices=sha1,sha256 … CDHash=a9b23d5ca054140c96fb770ee8391d96d8515923 … The CDHash property is the One True CDHash™. The CandidateCDHash and CandidateCDHashFull properties are alternative cdhash values, each specifying a hash algorithm. The Full variant includes the full hash, while the other variant is truncated to 20 bytes to match SHA-1. This structure was set up to enable stronger hashes (SHA-256) while still allowing the code to run on systems that only understand the old hashes (SHA-1). There’s a bunch of fine detail for this that I’ve not researched in depth. Also note that you’ll get different values for each architecture. The example above was run on Intel, and so you get the Intel architecture’s cdhash. To get the Apple silicon one, specify the --arch argument: % codesign -d -vvv --arch arm64 "/Applications/PCalc.app" … CDHash=28cf639f72ac35f8d57d45a118c101a2285f0a31 … [1] This uses CMS (Cryptographic Message Syntax) but I’m not going to go into those details here. Per-Page Hashes Within a code directory there are a set of hash slots. For a summary, look at the CodeDirectory property: % codesign -d -vvv "/Applications/PCalc.app" … CodeDirectory v=20500 size=25155 flags=0x10000(runtime) hashes=775+7 location=embedded … This means there are 775 per-page hash slots and 7 special hash slots. Add more -v options to see a dump of these hashes: % codesign -d -vvvvvv "/Applications/PCalc.app" … Page size=4096 -7=86f48b256e85ed00d054bbe3bad62b424baf1028c91cb7b193d1d3f8ebbe3f4e -6=0000000000000000000000000000000000000000000000000000000000000000 -5=bffee242018e6a39c46389fd913486fa29b3304712d32dd433f22d33b5a95da3 -4=0000000000000000000000000000000000000000000000000000000000000000 -3=8bee5419a9d27b02eee8f9897b6b94007b972cba348e4c863aa10df039b4b6c3 -2=a9ff3a03b94f75fcdea2208d8a8fadabc4cf5e4b2d4e6835a14508d1686a99b6 -1=f88548eecf39aa2e159365a9d4f0274a429e0793480abc8026d49fb2a8bf9ee9 0=07b91feb157b8238efb3d8149bacf546dc1ded500449cfd7d5b6a8bf1ebb4b34 1=f2f49595ca9daa599b4c41b69d99b6fede89eea39fa2000b452c7e26e06b8ddc 2=d62187f6573798dce6752a05aea083ae14c050e429fbc76d2092379c3533ebe5 3=a5a23c9cd825492a0ed5b5f541f2bbf12941bb8056f987f9d4e0f04d0885471f … 774=f6d1072647ec9f0e79a55c772f5e31699bbb1c3c424a2d48f64d0b896ae5c255 … The negative slots are special. More on that below. The non-negative slots are for per-page hashes, that is, 0 is the hash for the first page of code, 1 for the second, and so on. This per-page architecture means that the kernel can check each page as its loaded into memory. That is, if you take a page fault on a memory mapped file that’s code signed then, as part of satisfying that fault, the kernel may choose to verify the hash of the page’s contents. This allows the system to run a code-signed executable and check its code signature lazily [2]. macOS does not always check code as it is paged in. One key feature of the hardened runtime is that it opts the process in to this checking by default. The com.apple.security.cs.disable-executable-page-protection entitlement opts you out of this and other security features. Don’t do that! [2] I’m using the computer science definition of lazy here. Special Slots Within the code directory the negative hash slots are special. They don’t correspond to code but rather to other data structures. Each slot number corresponds to a specific type of data. I’m not going to give a comprehensive list, but here’s some highlights: Slot -1 holds a hash of the Info.plist. Slot -3 holds a hash of the resources. Slot -5 holds a hash of the entitlements. Consider this: % codesign -d -vvvvvv "/Applications/PCalc.app" … Page size=4096 … -1=f88548eecf39aa2e159365a9d4f0274a429e0793480abc8026d49fb2a8bf9ee9 … % shasum -a 256 "/Applications/PCalc.app/Contents/Info.plist" f88548eecf39aa2e159365a9d4f0274a429e0793480abc8026d49fb2a8bf9ee9 … The Info.plist hash matches the value in its hash slot. Neat-o! Now consider this advice from Using the Latest Code Signature Format: If -5 contains a value and -7 contains a zero value, or is not present, you need to re-sign your app to include the new DER entitlements. You should now have a better handle on this diagnostic. Slot -5 holds the hash for the old school property list entitlements while slot -7 holds the hash for the new-style DER entitlements. If you have an entry in -5, that is, you have entitlements, but have no entry in -7, then you’re missing your new-style DER entitlements and you must re-sign to run on iOS 15. Resources If your code exist in a bundle then the code signature protects not just your code but the resources in your bundle. Central to this is the CodeResources file. Slot -3 in the code directory holds the hash of that file: % codesign -d -vvvvvv "/Applications/PCalc.app" … Page size=4096 … -3=8bee5419a9d27b02eee8f9897b6b94007b972cba348e4c863aa10df039b4b6c3 … % shasum -a 256 "/Applications/PCalc.app/Contents/_CodeSignature/CodeResources" 8bee5419a9d27b02eee8f9897b6b94007b972cba348e4c863aa10df039b4b6c3 … So, if that file changes, the code directory hash changes and you break the seal on the code signature. Now let’s look at that file: % plcat "/Applications/PCalc.app/Contents/_CodeSignature/CodeResources" … <dict> <key>files</key> <dict> … </dict> <key>files2</key> <dict> … </dict> <key>rules</key> <dict> … </dict> <key>rules2</key> <dict> … </dict> </dict> </plist> It’s a property list with four top-level dictionaries: files, files2, rules, and rules2. Amusingly, three out of four of these items are vestigial. The one that matters is files2. Note The files dictionary contains SHA-1 hashes and is present for compatibility purposes. The rules and rules2 dictionaries are associated with resource rules, a concept that’s now obsolete. For more on the move away from resource rules, see Technote 2206 macOS Code Signing In Depth. The files2 dictionary contains two kinds of items. Firstly, there are references to resources. For example: % plutil -convert xml1 -o - "/Applications/PCalc.app/Contents/_CodeSignature/CodeResources" … <dict> … <key>files2</key> <dict> … <key>Resources/PCalc.help/Contents/Resources/PCalc.png</key> <dict> <key>hash</key> <data> wLi4SwDuno06WheWgINYKVT6LMg= </data> <key>hash2</key> <data> srK8T824L2l18d9eBixJC9nr3NUGCBh7L6qwsglQqFo= </data> </dict> … </dict> … </dict> </plist> % shasum -a 256 "/Applications/PCalc.app/Contents/Resources/PCalc.help/Contents/Resources/PCalc.png" b2b2bc4fcdb82f6975f1df5e062c490bd9ebdcd50608187b2faab0b20950a85a … % base64 -D | xxd -p srK8T824L2l18d9eBixJC9nr3NUGCBh7L6qwsglQqFo= b2b2bc4fcdb82f6975f1df5e062c490bd9ebdcd50608187b2faab0b20950 a85a As you can see, the hash2 property contains the SHA-256 checksum of the resource. The other kind of item is nested code. For example: % plutil -convert xml1 -o - "/Applications/PCalc.app/Contents/_CodeSignature/CodeResources" … <dict> … <key>files2</key> <dict> <key>Library/LoginItems/PCalc Widget.app</key> <dict> <key>cdhash</key> <data> 3pJUbWYNPPnlN8SNi1CAgdzIRWg= </data> <key>requirement</key> <string>(anchor apple generic and certificate leaf[field.1.2.840.113635.100.6.1.9] /* exists */ or anchor apple generic and certificate 1[field.1.2.840.113635.100.6.2.6] /* exists */ and certificate leaf[field.1.2.840.113635.100.6.1.13] /* exists */ and certificate leaf[subject.OU] = "9A6GM5K6XE") and identifier "com.pcalc.helper"</string> </dict> … </dict> … </dict> </plist> The cdhash property contains the code directory hash of the nested code: % codesign -d -vvv "/Applications/PCalc.app/Contents/Library/LoginItems/PCalc Widget.app" … CDHash=de92546d660d3cf9e537c48d8b508081dcc84568 … % base64 -D | xxd -p 3pJUbWYNPPnlN8SNi1CAgdzIRWg= de92546d660d3cf9e537c48d8b508081dcc84568 See, I told you cdhash values crop up all over the place! The requirement property contains the designated requirement (DR) of that nested code: % codesign -d -r - "/Applications/PCalc.app/Contents/Library/LoginItems/PCalc Widget.app" … designated => (anchor apple generic and certificate leaf[field.1.2.840.113635.100.6.1.9] /* exists */ or anchor apple generic and certificate 1[field.1.2.840.113635.100.6.2.6] /* exists */ and certificate leaf[field.1.2.840.113635.100.6.1.13] /* exists */ and certificate leaf[subject.OU] = "9A6GM5K6XE") and identifier "com.pcalc.helper" In theory this lets you update the nested code with a new version, as long as it has the same DR. No one ever does this in practice (-: For more information about code signing requirements, see Code Signing Guide, and specifically the Code Requirements section and the Code Signing Requirement Language appendix.

IMPORTANT This post is now retired in favour of TN3125 Inside Code Signing: Provisioning Profiles. I’m leaving the original post here just for the record, but you should consider the official documentation authoritative. I regularly help folks with code signing issues and have learnt that a lot of people are confused by the whole concept of a provisioning profile. This is my attempt to clear that up (-: Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" What exactly is a provisioning profile? Apple platforms will not run arbitrary third-party code [1]. All execution of third-party code must be specifically authorised by Apple. This is done using a provisioning profile [2], which describes five requirements: Who is allowed to sign code? What apps [3] are they allowed to sign? Where can those apps run? When can those apps run? How are those apps entitled? When the developer web site creates a profile [4] it cryptographically signs it. When you run an app on a device, the device checks this signature to determine if the profile is valid and, if so, checks that all of the app’s requirements are met by that profile. [1] Except for macOS, although provisioning profiles are still relevant on macOS, as we’ll see later. [2] There is an interesting edge case here. When you submit your app to the App Store, it re-signs your app as part of the distribution process. Before doing that, it checks that the app is signed and provisioned correctly. That ingestion check means that each individual iOS device does not need to perform further security checks, so the final app does not have a provisioning profile. However, this third-party code was still, technically, authorised by a profile, albeit during the App Store ingestion process. [3] In this document I’m using the term app to refer to a main executable packaged in a bundle structure. So everything I say about apps also applies to app extensions, App Clips (iOS), system extensions (macOS), and XPC Services (macOS). [4] Either directly, using the web site itself, or indirectly, using tools like Xcode. Unpack a Profile A profile is a property list wrapped within a CMS signature [1]. To view the original property list, remove the CMS wrapper using the security tool: % security cms -D -i Profile_Explainer_iOS_Dev.mobileprovision -o Profile_Explainer_iOS_Dev.plist % cat Profile_Explainer_iOS_Dev.plist … <dict> … lots of properties … </dict> </plist> IMPORTANT The exact format of a provisioning profile is not documented and could change at any time. I discuss these details here because I think it’s useful to understand how things work under the covers. I recommend against building a product based on these details; if you do build such a product be prepared to update it as the Apple development story evolves. [1] Cryptographic Message Syntax, as defined by RFC 5652. The Who Every profile has a DeveloperCertificates property holding the certificates of each developer who is allowed to sign code that’s covered by the profile. For example: % cat Profile_Explainer_iOS_Dev.plist … <dict> … <key>DeveloperCertificates</key> <array> <data>MIIFxDCC…tOLykA==</data> … more certificates … </array> … </dict> </plist> Use PlistBuddy to extract a specific certificate: % /usr/libexec/PlistBuddy -c "Print :DeveloperCertificates:0" Profile_Explainer_iOS_Dev.plist > cert0.cer % certtool d cert0.cer Serial Number : 7E FF 9C 91 BB EB D8 AB 42 81 52 35 64 F9 0F 72 Issuer Name : Common Name : Apple Worldwide Developer Relations Certification Authority … Subject Name : … Common Name : Apple Development: Quinn Quinn (7XFU7D52S4) … Not Before : 09:15:23 Apr 21, 2021 Not After : 09:15:22 Apr 21, 2022 … The What Most profiles apply to a single App ID. This is encoded in the Entitlements > application-identifier [1] property: % cat Profile_Explainer_iOS_Dev.plist … <dict> … <key>Entitlements</key> <dict> … <key>application-identifier</key> <string>SKMME9E2Y8.com.example.apple-samplecode.ProfileExplainer</string> … </dict> … </dict> </plist> This property holds an App ID, composed of an App ID prefix (SKMME9E2Y8 in this example) and a bundle ID (com.example.apple-samplecode.ProfileExplainer). It is also possible to create a profile for a wildcard App ID: % security cms -D -i Profile_Explainer_Wild_iOS_Dev.mobileprovision -o Profile_Explainer_Wild_iOS_Dev.plist % cat Profile_Explainer_Wild_iOS_Dev.plist <dict> … <key>Entitlements</key> <dict> … <key>application-identifier</key> <string>SKMME9E2Y8.com.example.apple-samplecode.*</string> … </dict> … </dict> </plist> </plist> This profile is valid for any App ID starting with SKMME9E2Y8.com.example.apple-samplecode. [1] On macOS this is Entitlements > com.apple.application-identifier. The Where Most profiles apply to a specific list of devices. This is encoded in the ProvisionedDevices property: % cat Profile_Explainer_iOS_Dev.plist … <dict> … <key>ProvisionedDevices</key> <array> <string>00008030-001544522E60802E</string> </array> … </dict> </plist> App Store distribution profiles have no ProvisionedDevices property because you can’t run a distribution-signed app locally. Developer ID and In-House (Enterprise) distribution profiles have the ProvisionsAllDevices property, indicating that they apply to all devices. The When Every profile has an ExpirationDate property which limits how long the profile remains valid. For example: % cat Profile_Explainer_iOS_Dev.plist … <dict> … <key>ExpirationDate</key> <date>2022-07-23T14:30:34Z</date> … </dict> </plist> Developer ID profiles do not expire and thus their expiration date is far in the future. The How Every profile has an Entitlements property which authorises the app to use specific entitlements. For example: % cat Profile_Explainer_iOS_Dev.plist … <key>Entitlements</key> <dict> <key>application-identifier</key> <string>SKMME9E2Y8.com.example.apple-samplecode.ProfileExplainer</string> <key>keychain-access-groups</key> <array> <string>SKMME9E2Y8.*</string> <string>com.apple.token</string> </array> <key>get-task-allow</key> <true/> <key>com.apple.developer.team-identifier</key> <string>SKMME9E2Y8</string> </dict> The entitlements in the profile act as an allowlist. This is not the same as the entitlements claimed by the app. To actually claim an entitlement you must include the entitlement in the app’s code signature. Every entitlement claimed by the app must be in the profile’s allowlist [1] but the reverse is not true. It’s fine for the allowlist to include entitlements that the app does not claim. The wildcard syntax only make sense in a profile’s allowlist. In the above example, SKMME9E2Y8.* means that the app can claim any keychain access group starting with SKMME9E2Y8. Wildcards do not make sense in the app’s code signature. To dump the entitlements claimed by the the app, use codesign with the --entitlements argument: % codesign -d --entitlements :- ProfileExplainer.app … <dict> <key>application-identifier</key> <string>SKMME9E2Y8.com.example.apple-samplecode.ProfileExplainer</string> <key>keychain-access-groups</key> <array> <string>SKMME9E2Y8.com.example.apple-samplecode.ProfileExplainer</string> </array> <key>get-task-allow</key> <true/> <key>com.apple.developer.team-identifier</key> <string>SKMME9E2Y8</string> </dict> </plist> As you can see, every entitlement claimed here is allowed by the profile, and thus the app should run. Note that the keychain-access-groups value, SKMME9E2Y8.com.example.apple-samplecode.ProfileExplainer, starts with SKMME9E2Y8.* and thus is allowed by the wildcard. [1] Except on macOS, as discussed in the next section. Entitlements on the Mac A macOS app can claim certain entitlements without them being authorised by a provisioning profile. These unrestricted entitlements include: com.apple.security.get-task-allow [1] com.apple.security.application-groups [2] Those used to enable and configure the App Sandbox Those used to configure the Hardened Runtime Other entitlements must be allowlisted by a provisioning profile, just like on iOS. This is an important security feature on macOS. For example, the fact that the keychain-access-groups entitlement must be allowlisted by a profile means that other developers can’t impersonate your app in order to steal its keychain items. [1] On iOS this is named get-task-allow and, as with all entitlements on iOS, must be allowlisted by the profile. [2] App Groups work differently on macOS and iOS; see this post for the gory details. Profile Location Historically it was common to install provisioning profiles on the device as a whole (in Settings on iOS or System Preferences > Profiles on the Mac). That’s still possible, but standard practice is to embed the profile within the app itself: iOS expects to find the profile at MyApp.app/embedded.mobileprovision. macOS expects to find the profile at MyApp.app/Contents/embedded.provisionprofile. Note The platforms use different file name extensions for these profiles (.mobileprovision on iOS, .provisionprofile on macOS) App Store apps do not contain an embedded provisioning profile because the App Store checks that the app is signed and provisioned correctly as part of its app ingestion process.

I often get questions about disabling the HTTPS default server trust evaluation done by NSURLSession. My general advice is “Don’t do that!” However, there is one case where you have to do that, namely when dealing with a hardware accessory on the local network. This post contains my advice on how to deal with that situation. IMPORTANT If you found your way here and you’re not developing a network-based accessory, I have two suggestions for you: If you’re trying to override HTTPS server trust evaluation for testing purposes, see QA1948 HTTPS and Test Servers. If not, post a question here on DevForums and we’ll try to get you heading in the right direction. Tag it with Security, and either CFNetwork or Network depending on which API you’re using. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" TLS For Accessory Developers Communicating securely with a network-based accessory is tricky because the standard network security protocol, TLS, was designed for the wider Internet, where servers have a stable DNS name. A devices on your local network doesn’t have a stable DNS name. The DNS name my-waffle-maker.local. might resolve to your waffle maker today, but tomorrow it might resolve to a different waffle maker or, worse yet, a malicious waffle varnisher. I guarantee that you won’t like them waffles! Note If you’re unfamiliar with TLS in general, I encourage you to read TLS for App Developers first. TLS puts the S into HTTPS. This post focuses on URLSession and HTTPS, because that’s where this most commonly crops up, but similar logic applies to other networking APIs. Doing the Minimum: The SSH Approach Many accessory firmware developers support TLS by generating a self-signed digital identity on the accessory and using that for their HTTPS server. IMO they should do better — the subject of later sections in this post — but sometimes you have to work with what you’ve got. If you’re working with an accessory like this, and you can’t convince the firmware developer to do better, I recommend that you adopt the SSH approach. In this approach you accept the certificate offered by the accessory the first time and then, when you connect again in the future, check the accessory still has the same certificate. Note I call this the SSH approach because that’s how many folks use SSH. To implement the SSH approach, first disable App Transport Security for local networking by setting the NSAllowsLocalNetworking property. On modern systems this isn’t strictly necessary but, as it says in the docs, this is a good “declaration of intent”. Next, override HTTPS server trust evaluation as shown below: var expectedCertificate: SecCertificate? = nil func shouldAllowHTTPSConnection(trust: SecTrust) async -> Bool { guard let chain = SecTrustCopyCertificateChain(trust) as? [SecCertificate], let actual = chain.first else { return false } guard let expected = self.expectedCertificate else { // A. First connection self.expectedCertificate = actual return true } // B. Subsequent connections return CFEqual(expected, actual) } func urlSession(_ session: URLSession, didReceive challenge: URLAuthenticationChallenge) async -> (URLSession.AuthChallengeDisposition, URLCredential?) { switch challenge.protectionSpace.authenticationMethod { case NSURLAuthenticationMethodServerTrust: let trust = challenge.protectionSpace.serverTrust! guard await self.shouldAllowHTTPSConnection(trust: trust) else { return (.cancelAuthenticationChallenge, nil) } let credential = URLCredential(trust: trust) return (.useCredential, credential) default: return (.performDefaultHandling, nil) } } There are a few things to note here: The snippet uses the new Swift async/await syntax. That simplifies the code and it also avoids a gotcha with the old completion-handler approach, where it was easy to forget to call the completion handler. Having said that, you can achieve the same result without using Swift async/await, or indeed, in Objective-C. The authentication challenge handler only deals with HTTPS server trust challenges (NSURLAuthenticationMethodServerTrust). Critically, if it finds a challenge it doesn’t recognise, it resolves that in the default way (.performDefaultHandling). The shouldAllowHTTPSConnection(trust:) method checks to see if it’s connected to the server before. If not, it defaults to allowing the connection (case A). If so, it checks that the certificate hasn’t changed (case B). In a real app you would persist expectedCertificate so that this logic continues to apply the next time your app is launched. When you do that, store the certificate using a key that uniquely identifies the accessory. Do not use the DNS name or IP address because that can change. The first time you connect (case A) you might take other steps to confirm the identity of the accessory. This is something I’ll cover below. The thing to note here is that this function is async, so you can take your time doing that confirmation. For example, you could bounce over to the main actor and display a UI. Doing Better: Add a Side Channel The above represents the absolute minimum you should do; anything less is doing the user a disservice. Aim to do better! Doing better requires some sort of side channel so that the user can confirm the identity of the accessory. For example: If the accessory has a display, you might present a UI asking the user to confirm that the code in your UI matches the one on the accessory’s display. If the accessory has a LED, it might blink that during this initial setup process. If the accessory has a serial number, you might ask the user to confirm that. Or scan that via a barcode. Note For an example of the barcode approach, see Matt’s Configuring a Wi-Fi Accessory to Join the User’s Network sample. This sample uses a different technique for TLS, namely TLS-PSK. This has some advantages — most notably, it avoids messing around with certificates — but I’m not discussing it here because it’s not supported by URLSession. If you adopt this approach make sure that you present a UI that the user can understand. If your UI contains raw certificate details, most users will just say “Sure, whatever.” and click OK, at which point you might as well have saved everyone’s time by accepting the connection on their behalf. Take care not to blindly trust any information you get from the accessory. After all, the goal here is to check the identity of the accessory, and so you mustn’t trust it before completing that check. Sometimes this can be quite subtle. Consider the accessory-with-a-display example above. In that case the accessory’s self-signed certificate might contain a common name of Acme Waffle Maker CCC, where CCC is the code showing on its display. Don’t display the entire string to the user. The presence of Acme Waffle Maker might lead the user to believe that the accessory is valid, even when it isn’t. After all, a malicious waffle varnisher can just as easily create a self-signed certificate with that common name. Rather, extract the code (CCC) and display just that. That way the user will focus on what’s important. Doing Even Better: Proper Security If the accessory’s firmware and hardware developers are on board, there are steps you can take to properly secure your communication with the accessory: Create a custom certificate authority (CA) for your accessories. At the factory, have that CA issue a certificate to each accessory as its produced, embedding identifying details, like the serial number, in that certificate. Then install that certificate and its associated private key (thus forming a digital identity) on the accessory as it comes off the line. Embed the CA’s certificate in your app. When you connect the accessory, verify that its certificate was issued by your CA. If it was, you can trust the identifying information, like the serial number, embedded in that certificate. With this setup your app will never connect to a malicious accessory. The worst that can happen is that you accidentally connect to the wrong instance of your accessory. IMPORTANT This is just an outline of one approach that I believe offers specific advantages. If you plan to deploy this in practice, hire a security profession to design your process. They can advise you on the details of creating a system that’s secure in practice. For example, they can help you: Create a mechanism that prevents your factory from leaking valid digital identities. Design your hardware to prevent an attacker from extracting an accessory’s digital identity from the accessory itself. To implement the URLSession side of this, change the shouldAllowHTTPSConnection(trust:) method like so: let waffleVarnisherCA: SecCertificate = { let u = Bundle.main.url(forResource: "WaffleVarnisherCA", withExtension: "cer")! let d = try! Data(contentsOf: u) let c = SecCertificateCreateWithData(nil, d as NSData)! return c }() func shouldAllowHTTPSConnection(trust: SecTrust) async -> Bool { var err = SecTrustSetPolicies(trust, SecPolicyCreateBasicX509()) guard err == errSecSuccess else { return false } err = SecTrustSetAnchorCertificates(trust, [self.waffleVarnisherCA] as NSArray) guard err == errSecSuccess else { return false } err = SecTrustSetAnchorCertificatesOnly(trust, true) guard err == errSecSuccess else { return false } let wasIssuedByOurCA = SecTrustEvaluateWithError(trust, nil) guard wasIssuedByOurCA else { return false } guard let chain = SecTrustCopyCertificateChain(trust) as? [SecCertificate], let trustedLeaf = chain.first else { return false } // C. Check the now-trusted leaf certificate return true } There are a few things to note about this code: It changes the trust policy (SecTrustSetPolicies) to the basic X.509 policy (SecPolicyCreateBasicX509) because the standard policy for TLS connections checks the DNS name, and that’s not appropriate for an accessory on the local network. It applies a custom anchor (SecTrustSetAnchorCertificates) and then disables all the other anchors (SecTrustSetAnchorCertificatesOnly) [1]. That prevents someone from impersonating your accessory by getting some other CA to issue a certificate that looks like the accessory’s certificate. It evaluates trust to confirm that the certificate was issued by the accessory’s CA. At point C you can trust the details in the accessory’s certificate (trustedLeaf). Here you might add code to confirm the identity of the accessory, to prevent you from accidentally connecting to the wrong accessory. [1] This is not strictly necessary because calling SecTrustSetAnchorCertificates disables other anchors. However, I’m applying both belt and braces [2] here. [2] I’m using the British idiom because belt and suspenders is so wrong (-:

The best way to view sandbox violation reports has changed over the years, so I thought I’d post some up-to-date info. I tested the following with Xcode 13.3 on macOS 12.2. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Viewing Sandbox Violation Reports After enabling the App Sandbox, you may find that your app fails in some non-obvious way. That is, something within your app doesn’t work, but your app doesn’t display a permissions error and so you have no idea where to start. If you find yourself in that situation, look for a sandbox violation report: Run the Console app. For each line below, copy the line and paste it in to the search box at the top right. type:error subsystem:com.apple.sandbox.reporting category:violation This searches for sandbox violation reports. Note The exact query terms have changed over time. The above is accurate from macOS 12.2. Click the Save button in the bar below the search box and enter a name for your saved search. I typically use “Sandbox” for this. In future, click on this saved search to skip the previous step. Click “Start streaming”. Run your app and reproduce the problem. Look for a sandbox violation report in the log. If you see one, follow the steps below to investigate. Inside a Sandbox Violation report A sandbox violation report log entry looks like this: type: error time: 11:58:26.009175+0000 process: sandboxd subsystem: com.apple.sandbox.reporting category: violation message: Sandbox: AppSandboxViolat(5807) deny(1) file-read-data /Users/quinn/.ssh/id_rsa The message includes a complete sandbox violation report. That’s too big to include here, so I’ve added it as a text attachment. sandbox-violation-report.txt Look at the Violation field first: Violation: deny(1) file-read-data /Users/quinn/.ssh/id_rsa This means that the App Sandbox blocked an attempt to read the data of the file at the path /Users/quinn/.ssh/id_rsa. Next look at the thread backtraces. It’s normally pretty easy to identify the thread responsible for the violation: It’s the one blocked in a system call that could reasonably trigger this violation. For example: Thread 0 (id: 291952): 0 libsystem_kernel.dylib … __open + 10 1 Foundation … _NSReadBytesFromFileWithExtendedAttributes + 167 2 Foundation … -[NSData(NSData) initWithContentsOfFile:options:maxLength:error:] + 119 3 libswiftFoundation.dylib … NSData.__allocating_init(contentsOf:options:) + 77 4 libswiftFoundation.dylib … Data.init(contentsOf:options:) + 76 5 AppSandboxViolator … ViewController.violateAction(_:) + 1507 (ViewController.swift:25) 6 AppSandboxViolator … @objc ViewController.violateAction(_:) + 65 (<compiler-generated>:0) 7 AppKit … -[NSApplication(NSResponder) sendAction:to:from:] + 288 … Here you see that AppKit has called the ViewController.violateAction(_:) method (frame 6) which has tried to created a Data value from the contents of a file (frame 5) which has eventually called open (frame 0), which is what triggered the violation. Using this information, investigate and fix your sandbox incompatibility. No Sandbox Violation Report It’s possible that you might not see a sandbox violation report even though the problem is caused by the App Sandbox. For example, imagine you have code like this: let url: URL = … some file URL … let data: Data if access(url.path, R_OK) == 0 { data = try Data(contentsOf: url) } else { data = Data() } The access system call never triggers a sandbox violation report. If the App Sandbox blocks access to url, this code will not fail, not generate a sandbox violation report, and set data to empty. IMPORTANT Preflighting file system calls is racy and, in the worse case, can result in TOCTTOU vulnerabilities. Avoid writing code like this. Debugging problems like this can be tricky.

On modern systems all KEXTs must be code signed with a Developer ID. Additionally, the Developer ID must be specifically enabled for KEXT development. You can learn more about that process on the Developer ID page. If your KEXT is having code signing problems, check that it’s signed with a KEXT-enabled Developer ID. Do this by looking at the certificate used to sign the KEXT. First, extract the certificates from the signed KEXT: % codesign -d --extract-certificates MyKEXT.kext Executable=/Users/quinn/Desktop/MyKEXT/build/Debug/MyKEXT.kext/Contents/MacOS/MyKEXT This creates a bunch of certificates of the form codesignNNN, where NNN is a number in the range from 0 (the leaf) to N (the root). For example: % ls -lh codesign* -rw-r--r--+ 1 quinn staff 1.4K 20 Jul 10:23 codesign0 -rw-r--r--+ 1 quinn staff 1.0K 20 Jul 10:23 codesign1 -rw-r--r--+ 1 quinn staff 1.2K 20 Jul 10:23 codesign2 Next, rename each of those certificates to include the .cer extension: % for i in codesign*; do mv $i $i.cer; done Finally, look at the leaf certificate (codesign0.cer) to see if it has an extension with the OID 1.2.840.113635.100.6.1.18. The easiest way to view the certificate is to use Quick Look in Finder. Note If you’re curious where these Apple-specific OIDs comes from, check out the documents on the Apple PKI page. In this specific case, look at section 4.11.3 Application and Kernel Extension Code Signing Certificates of the Developer ID CPS. If the certificate does have this extension, there’s some other problems with your KEXT’s code signing. In that case, feel free to create a new thread here on DevForums with your details. If the certificate does not have this extension, there are two possible causes: Xcode might be using an out-of-date signing certificate. Re-create your Developer ID signing certificate using the developer site and see if the extension shows up there. If so, you’ll have to investigate why Xcode is not using the most up-to-date signing certificate. If a freshly-created Developer ID signing certificate does not have this extension, you need to apply to get your Developer ID enabled for KEXT development per the instructions on the Developer ID page. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Change history: 20 Jul 2016 — First published. 28 Mar 2019 — Added a link to the Apple PKI site. Other, minor changes. 15 Mar 2022 — Fixed the formatting. Updated the section number in the Developer ID CPS. Made other minor editorial changes.

IMPORTANT This post has been replaced by two shiny new posts: Creating Distribution-Signed Code for Mac Packaging Mac Software for Distribution See the preamble in Creating Distribution-Signed Code for Mac for more context. I’ve left the original post here just for the record. Share and Enjoy — Quinn “The Eskimo!” Apple Developer Relations, Developer Technical Support, Core OS/Hardware let myEmail = "eskimo" + "1" + "@" + "apple.com" I spend a lot of time helping Mac developers with notarisation and Gatekeeper problems, and many of these problems are caused by incorrect code signing. The instructions for how to sign and package a Mac product for distribution are rather scattered, so I’ve written them all down in one place. And rather than keep that to myself, I’m posting it here for everyone’s benefit. If you have any corrections, feel free to get in touch with me directly (my email address is in my signature). And if have any questions about this, it’s probably best to ask them here on DevForums. I’ve locked this thread, so just start a new thread tagging it with Code Signing, Notarization, or Gatekeeper. Or, if you want one-on-one help, open a DTS tech support incident and we can pick things up in that context. IMPORTANT None of the following has been formally reviewed, so it’s not official Apple documentation. Signing a Mac Product For Distribution The best way to sign and package an app is via Xcode: Build a version of your app to distribute using Xcode’s Product > Archive command, and then package that archive for your distribution channel via the Organizer. See Xcode Help > Distribute your app for the details. However, not all Mac products can be distributed this way. For example: An app that’s distributed outside of the Mac App Store on a disk image A product that has to be installed via an installer package An app that uses a third-party development environment In these cases you must manually sign and package your product. Note If you find this post a little abstract, and would prefer to follow a concrete example, see Manual Code Signing Example. Consult Resources for Third-Party Development Environments Many third-party development environments have their own strategies for signing and packaging the products they build. If you’re using a third-party development environment, consult its support resources for advice before continuing. Decide on a Container Format To get started, decide on your container format. Mac products support two distribution channels: An app can be distributed via the Mac App Store Apps and non-apps can be distributed outside of the Mac App Store using Developer ID signing A Mac App Store app must be submitted as an installer package. In contrast, products distributed outside of the Mac App Store can use a variety of different container formats, the most common being: Zip archive (.zip) Disk image (.dmg) Installer package (.pkg) It’s also possible to nest these. For example, you might have an app inside an installer package on a disk image. Each container format has its own pros and cons, so pick an approach based on the requirements of your product. However, this choice affects how you package your product, something discussed in more detail below. Structure Your Code Correctly All code that you distribute must be signed. There’s two parts to this: Structuring your code to support signing Actually signing it You must structure your code correctly. If you don’t, it may be hard (or in some cases impossible) to sign it. First things first, identify all the code in your product. There are many types of code, including apps, app extensions, frameworks, other bundled code (like XPC Services), shared libraries, and command-line tools. Each type of code has two key attributes Is it bundled code? (apps, app extensions, frameworks, other bundled code) Is it a main executable? (apps, app extensions, command-line tools) Both of these attributes affect how you sign the code. In addition, whether the code is bundled is critical to how you structure it. Specifically, bundled code supports the notion of nested code. For example, you might have an app extension nested within your app’s bundle. When dealing with nested code, follow these rules: Place any nested code in the appropriate nested code location. For more on that, see Placing Content in a Bundle. Do not place non-code items in a nested code location. Rather, place these in the bundle’s resources directory (typically Contents/Resources). IMPORTANT Scripts are not considered code. If you have scripts — shell, Python, AppleScript, or whatever — place them in the resources directory. These will still be signed, but as a resource rather than as code. Provisioning Profile If you have a main executable that uses a restricted entitlement, one that must be allowlisted by a provisioning profile, place the profile in your bundle at the path Contents/embedded.provisionprofile. The profile is sealed by the code signature, so do this before signing the code. If your product contains multiple executables that need a profile — for example, you have an app with an embedded Network Extension app extension, both of which need the Network Extensions entitlement — repeat this process for each of these code executables. If your product includes a non-bundled executable that uses a restricted entitlement, you must package that executable in an app-like structure. For the details, see Signing a Daemon with a Restricted Entitlement. Handling Alien Code Structures If you’re using a complex third-party library, you may find that the structure required by the library does not match up with the structure required by macOS. For an in-depth discussion of the techniques you can use to resolve this, see Embedding Nonstandard Code Structures in a Bundle. Sign Your Code Sign code using the codesign tool. Read the following sections to learn about the specific arguments to use, but also keep these general rules in mind: Do not use the --deep argument. This feature is helpful in some specific circumstances but it will cause problems when signing a complex program. For a detailed explanation as to why, see --deep Considered Harmful. Rather, sign each code item separately. For a complex app, you should create a script to do this. Sign from the inside out. That is, if A depends on B, sign B before you sign A. When you sign A, the code signature encodes information about B, and changing B after the fact can break the seal on that code signature. Basic Signing No matter what sort of code you’re signing, the basic codesign command looks like this: % codesign -s III /path/to/your/code` where III is the name of the code signing identity to use. The specific identity varies depending on your target platform. See the following sections for details. When signing bundled code (as defined in Structure Your Code Correctly) pass in the path to the bundle, not the path to the code. If you’re re-signing code — that is, the code you’re signing is already signed — pass the -f option. If you’re signing a main executable (as defined in Structure Your Code Correctly) that needs entitlements, add --entitlements EEE.entitlements, where EEE.entitlements is a path to a property list file that contains your entitlements. IMPORTANT The entitlements property list file must be in the standard XML format with LF line endings, no comments, and no BOM. If you’re not sure of the file’s provenance, use plutil to convert it to the standard format. See Ensure Properly Formatted Entitlements in Resolving Common Notarization Issues. If you’re signing non-bundled code, set the code signing identifier by adding -i BBB, where BBB is the bundle ID the code would have if it had a bundle ID. For example, if you have an app whose bundle ID is com.example.flying-animals that has a nested command-line tool called pig-jato, the bundle ID for that tool would logically be com.example.flying-animals.pig-jato, and that’s a perfectly fine value to use for BBB. Note For bundled code, you don’t need to supply a code signing identifier because codesign defaults to using the bundle ID. Mac App Store Signing If you’re distributing via the Mac App Store, use your Mac App Distribution signing identity in place of III in the example above. This will typically be named 3rd Party Mac Developer Application: TTT, where TTT identifies your team. You can also use an Apple Distribution signing identity, with the name Apple Distribution: TTT. Developer ID Signing If you’re distributing outside of the Mac App Store, use your Developer ID Application signing identity in place of III in the example above. This will typically be named Developer ID Application: TTT, where TTT identifies your team. All Developer ID signed code needs a secure timestamp; enable this by adding the --timestamp option. If you’re signing a main executable (as defined in Structure Your Code Correctly), enable the hardened runtime by adding -o runtime option. The hardened runtime enables additional security checks within your process. You may need to make minor code changes to be compatible with those additional security checks. For some specific examples, watch WWDC 2019 Session 703 All About Notarization. Failing that, you can opt out of these additional security checks using entitlements. See Hardened Runtime Entitlements Build Your Container Once you’ve signed the code in your product, it’s time to wrap it in a container for distribution. Follow the advice appropriate for your chosen container format in the following sections. If you’re using a nested container format — for example, an app inside an installer package on a disk image — work from the inside out, following the advice for each level of nesting. Build a Zip Archive Use the ditto tool to create a zip archive for your product: Create a directory that holds everything you want to distribute. Run the ditto as shown below, where DDD is the path to the directory from step 1 and ZZZ is the path where ditto creates the zip archive. % ditto -c -k --keepParent DDD ZZZ Zip archives cannot be signed (although their contents can be). Build an Installer Package Use the productbuild tool to create a simple installer package for a single app: % productbuild --sign III --component AAA /Applications PPP In this example: III is either your Mac Installer Distribution or Developer ID Installer signing identity, depending on your distribution channel. This will typically be named 3rd Party Mac Developer Installer: TTT or Developer ID Installer: TTT, where TTT identifies your team. AAA is the path to your app. PPP is the path where productbuild creates the installer package. IMPORTANT The above is the simplest possible example. There are many different ways to create installer packages. See the man pages for productbuild, productsign, pkgbuild, and pkgutil for more details. Build a Disk Image Use the hdiutil tool to create a disk image for distribution: Create a directory to act as the source for the root directory of your disk image’s volume. Populate that directory with the items you want to distribute. Use hdiutil command shown below to create the disk image, where SSS is the directory from step 1 and DDD is the path where hdiutil creates the disk image. Use codesign command shown below to sign the disk image, where III is your Developer ID Application signing identity (typically named Developer ID Application: TTT, where TTT identifies your team), BBB is a pseudo bundle ID as discussed in Basic Signing, and DDD is the path to the disk image from step 3. % hdiutil create -srcFolder SSS -o DDD % codesign -s III --timestamp -i BBB DDD IMPORTANT There are various third-party tools that can help you create a disk image in exactly the right way. For example, the tool might arrange the icons nicely, set a background image, and add a symlink to /Applications. If you use such a tool, or create your own tool for this, make sure that the resulting disk image: Is signed with your Developer ID Application signing identity Is a UDIF-format read-only zip-compressed disk image (type UDZO) Notarisation If you’re distributing outside of the Mac App Store, you must notarise the file you intend to distribute to your users. For instructions on doing this, see Customizing the Notarization Workflow. Skip the Export a Package for Notarization section because you already have the file that you want to submit. If you’re using a nested container format, only notarise the outermost container. For example, if you have an app inside an installer package on a disk image, sign the app, sign the installer package, and sign the disk image, but only notarise the disk image. The exception to this rule is if you have a custom third-party installer. In that case, see the discussion in Customizing the Notarization Workflow. Stapler Once you have notarised your product, you should staple the resulting ticket to the file you intend to distribute. Customizing the Notarization Workflow discusses how to do this for a zip archive. The other common container formats (installer package and disk image) support stapling directly. For example: % xcrun stapler staple FlyingAnimals.dmg Note Stapling is recommended but not mandatory. If you don’t staple, a user may have problems if they try to install or run your app for the first time when the Mac is offline. Change history: 20 Jan 2020 — First version. 27 Jan 2020 — Minor editorial changes. 9 Mar 2020 — Moved the details of --deep into a separate post, --deep Considered Harmful. 10 Mar 2020 — Fixed a typo. 30 Mar 2020 — Added a link to Manual Code Signing Example. 26 Feb 2021 — Fixed the formatting. Add a discussion of the entitlements file format. Minor editorial changes. 1 Mar 2021 — Added the Provisioning Profile section. 21 Oct 2021 — Updated the Structure Your Code Correctly section to reference Placing Content in a Bundle. 22 Dec 2021 — Replaced links to two DevForums posts with links to the official documentation, namely those for Signing a Daemon with a Restricted Entitlement and Embedding Nonstandard Code Structures in a Bundle. Made some other editorial changes.

This post is one of a pair of posts, the other one being Creating Distribution-Signed Code for Mac, that replaces my earlier Signing a Mac Product For Distribution post. For more background on this, see the notes at the top of Creating Distribution-Signed Code for Mac. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Packaging Mac Software for Distribution Build a zip archive, disk image, or installer package for distributing your Mac software. Overview Xcode is a great tool for creating and distributing Mac apps. Once you’ve written your code you can upload it to the App Store with just a few clicks. However, Xcode cannot do everything. For example: Some Mac software products are not apps. You might, for example, be creating a product that includes a daemon. Some Mac products include multiple components. Your daemon might include an app to configure it. Some Mac products ship outside of the App Store, and so need to be packaged for distribution. For example, you might choose to distribute your daemon and its configuration app in an installer package. Some Mac products are built with third-party developer tools. If your product cannot be built and distributed using Xcode alone, follow these instructions to package it for distribution. Note If you use a third-party developer tool to build your app, consult its documentation for advice specific to that tool. To start this process you need distribution-signed code. For detailed advice on how to create distribution-signed code, see Creating Distribution-Signed Code for Mac. If you ship your product frequently, create a script to automate the distribution process. Decide on a Container Format To get started, decide on your container format. Mac products support two distribution channels: The Mac App Store, for apps Independent distribution, for apps and non-apps, using Developer ID signing A Mac App Store app must be submitted as an installer package. In contrast, products distributed outside of the Mac App Store use a variety of different container formats, the most common being: Zip archive (.zip) Disk image (.dmg) Installer package (.pkg) You may choose to nest these containers. For example, you might ship an app inside an installer package on a disk image. Nesting containers is straightforward: Just work from the inside out, following the instructions for each container at each step. IMPORTANT Sign your code and each nested container (if the container supports signing). For example, if you ship an app inside an installer package on a disk image, sign the app, then create the installer package, then sign that package, then create the disk image, then sign the disk image. Each container format has its own pros and cons, so choose an approach based on the requirements of your product. Build a Zip Archive If you choose to distribute your product in a zip archive, use the ditto tool to create that archive: Create a directory that holds everything you want to distribute. Run the ditto tool as shown below, where DDD is the path to the directory from step 1 and ZZZ is the path where ditto creates the zip archive. % ditto -c -k --keepParent DDD ZZZ Zip archives cannot be signed, although their contents can be. Build an Installer Package If you choose to distribute your product in an installer package, start by determining your installer signing identity. Choose the right identity for your distribution channel: If you’re distributing an app on the Mac App Store, use a Mac Installer Distribution signing identity. This is named 3rd Party Mac Developer Installer: TTT, where TTT identifies your team. If you’re distributing a product independently, use a Developer ID Installer signing identity. This is named Developer ID Installer: TTT, where TTT identifies your team. For information on how to set up these installer signing identities, see Developer Account Help. Run the following command to confirm that your installer signing identity is present and correct: % security find-identity -v 1) 6210ECCC616B6A72F238DE6FDDFDA1A06DEFF9FB "3rd Party Mac Developer Installer: …" 2) C32E0E68CE92936D5532E21BAAD8CFF4A6D9BAA1 "Developer ID Installer: …" 2 valid identities found The -v argument filters for valid identities only. If the installer signing identity you need is not listed, see Developer Account Help. IMPORTANT Do not use the -p codesigning option to filter for code signing identities. Installer signing identities are different from code signing identities and the -p codesigning option filters them out. If your product consists of a single app, use the productbuild tool to create a simple installer package for it: % productbuild --sign III --component AAA /Applications PPP In this command: III is your installer signing identity. AAA is the path to your app. PPP is the path where productbuild creates the installer package. The above is the simplest possible use of productbuild. If you’re submitting an app to the Mac App Store, that’s all you need. If you have a more complex product, you’ll need a more complex installer package. For more details on how to work with installer packages, see the man pages for productbuild, productsign, pkgbuild, and pkgutil. For instructions on how to read a man page, see Reading UNIX Manual Pages. Build a Disk Image If you choose to distribute your product in a disk image: Create a directory to act as the source for the root directory of your disk image’s volume. Populate that directory with the items you want to distribute. If you’re automating this, use ditto rather than cp because ditto preserves symlinks. Use hdiutil command shown below to create the disk image, where SSS is the directory from step 1 and DDD is the path where hdiutil creates the disk image. Decide on a code signing identifier for this disk image. If you were signing bundled code, you’d use the bundle ID as the code signing identifier. However, disk images have no bundle ID and thus you must choose a code signing identifier for your image. For advice on how to do this, see the Sign Each Code section in Creating Distribution-Signed Code for Mac. Use the codesign command shown below to sign the disk image, where III is your Developer ID Application code signing identity (named Developer ID Application: TTT, where TTT identifies your team), BBB is the code signing identifier you chose in the previous step, and DDD is the path to the disk image from step 3. % hdiutil create -srcFolder SSS -o DDD % codesign -s III --timestamp -i BBB DDD For more information on code signing identities, see the Confirm Your Code Signing section in Creating Distribution-Signed Code for Mac. IMPORTANT Sign your disk image with a code signing identity, not an installer signing identity. There are various third-party tools that configure a disk image for distribution. For example, the tool might arrange the icons nicely, set a background image, and add a symlink to the Applications folder. If you use such a tool, or create your own tool for this, make sure that the resulting disk image: Is signed with your Developer ID Application code signing identity Is a UDIF-format read-only zip-compressed disk image (type UDZO) Submit Your App to the Mac App Store If you’re creating an app for the Mac App Store, submit your signed installer package using either the altool command-line tool or the Transporter app. For detailed instructions, see App Store Connect Help &gt; Reference &gt; Upload tools. Notarize Your Product If you’re distributing outside of the Mac App Store, notarize the file you intend to distribute to your users. For detailed instructions, see Customizing the Notarization Workflow. Skip the Export a Package for Notarization section because you already have the file that you want to submit. If you’re using nested containers, only notarize the outermost container. For example, if you have an app inside an installer package on a disk image, sign the app, sign the installer package, and sign the disk image, but only notarize the disk image. The exception to this rule is if you have a custom third-party installer. In that case, see the discussion in Customizing the Notarization Workflow. Staple Your Product Once you’ve notarized your product, staple the resulting ticket to the file you intend to distribute. Staple the Ticket to Your Distribution discusses how to do this for an app within a zip archive. The other common container formats, installer packages and disk images, support stapling directly. For example, to staple a tick to a disk image: % xcrun stapler staple FlyingAnimals.dmg Stapling is recommended but not mandatory. However, if you don’t staple a user might find that your product is blocked by Gatekeeper if they try to install or use it while the Mac is offline.

This post is one of a pair of posts, the other one being Packaging Mac Software for Distribution, that replaces my earlier Signing a Mac Product For Distribution post. Over the past year I’ve been trying to convert my most useful code signing posts here on DevForums to official documentation, namely: Placing Content in a Bundle Updating Mac Software Signing a Daemon with a Restricted Entitlement Embedding a Command-Line Tool in a Sandboxed App Embedding Nonstandard Code Structures in a Bundle Unfortunately in the past month or so my Day Job™, answering developer questions for DTS, has become super busy, and so I’ve not had chance to complete this work by publish a replacement for Signing a Mac Product For Distribution. This post, and Packaging Mac Software for Distribution, represent the current state of that effort. I think these are sufficiently better than Packaging Mac Software for Distribution to warrant posting them here on DevForums while I wait for the quiet time needed to finish the official work. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Creating Distribution-Signed Code for Mac Sign Mac code for distribution using either Xcode or command-line tools. Overview Before shipping a software product for the Mac, you must first create distribution-signed code, that is, code that you can package up and then submit to either the Mac App Store or the notary service. The way you do this depends on the nature of your product and how it was built: If your product is a standalone app, possibly with nested code such as an app extension, that you build using Xcode, use Xcode to export a distribution-signed app. If your product isn't a standalone app, but you build it using Xcode, create an Xcode archive, and then manually export distribution-signed code from that archive. If you build your product using an external build system, such as make, add a manual signing step to your build system. Once you have distribution-signed code, package it for distribution. For more information, see Packaging Mac Software for Distribution. Note If you use a third-party developer tool to build your app, consult its documentation for advice specific to that tool. Export an App from Xcode If your product is a standalone app that you build with Xcode, follow these steps to export a distribution-signed app: Build an Xcode archive from your project. Export a distribution-signed app from that Xcode archive. You can complete each step from the Xcode app or automate the steps using xcodebuild. To build an Xcode archive using the Xcode app, select your app’s scheme and choose Product &gt; Archive. This creates the Xcode archive and selects it in the organizer. To create a distribution-sign app from that archive, select the archive in the organizer, click Distribute App, and follow the workflow from there. Note If the button says Distribute Content rather than Distribute App, your archive has multiple items in its Products directory. Make sure that every target whose output is embedded in your app has the Skip Install (SKIP_INSTALL) build setting set; this prevents the output from also being copied into the Xcode archive’s Products directory. For more information about the Xcode archives and the organizer, see Distributing Your App for Beta Testing and Releases. To build an Xcode archive from the command line, run xcodebuild with the archive action. Once you have an Xcode archive, export a distribution-signed app by running xcodebuild with the -exportArchive option. For more information about xcodebuild, see its man page. For instructions on how to read a man page, see Reading UNIX Manual Pages. For information about the keys supported by the export options property list, run xcodebuild with the -help argument. Export a Non-App Product Built with Xcode If you build your product with Xcode but it’s not a standalone app, you can build an Xcode archive using the techniques described in the previous section but you cannot export distribution-signed code from that archive. The Xcode organizer and the -exportArchive option only work for standalone apps. To export a distribution-signed product from the Xcode archive: Copy the relevant components from the archive. Sign those components manually. The exact commands for doing this vary depending on how your product is structured, so let’s consider a specific example. Imagine your product is a daemon but it also has an associated configuration app. Moreover, the configuration app has a share extension, and an embedded framework to share code between the app and the extension. When you build an Xcode archive from this project it has this structure: DaemonWithApp.xcarchive/ Info.plist Products/ usr/ local/ bin/ Daemon Applications/ ConfigApp.app/ Contents/ embedded.provisionprofile Frameworks/ Core.framework/ … PlugIns/ Share.appex/ Contents/ embedded.provisionprofile … … … The Products directory contains two items: the daemon itself (Daemon) and the configuration app (ConfigApp.app). To sign this product, first copy these items out of the archive: % mkdir "to-be-signed" % ditto "DaemonWithApp.xcarchive/Products/usr/local/bin/Daemon" "to-be-signed/Daemon" % ditto "DaemonWithApp.xcarchive/Products/Applications/ConfigApp.app" "to-be-signed/ConfigApp.app" IMPORTANT When you copy code, use ditto rather than cp. ditto preserves symlinks, which are critical to the structure of Mac frameworks. For more information on this structure, see Placing Content in a Bundle. Symlinks are also useful when dealing with nonstandard code structures. For more details, see Embedding Nonstandard Code Structures in a Bundle. The code you copy from the Xcode archive is typically development-signed: % codesign -d -vv to-be-signed/Daemon … Authority=Apple Development: … … To ship this code, you need to re-sign it for distribution. Confirm Your Code Signing Identity To sign code for distribution you need a code signing identity. Choose the right identity for your distribution channel: If you’re distributing an app on the Mac App Store, use a Mac App Distribution code signing identity. This is named 3rd Party Mac Developer Application: TTT, where TTT identifies your team. If you’re distributing a product independently, use a Developer ID Application code signing identity. This is named Developer ID Application: TTT, where TTT identifies your team. For information on how to set up these code signing identities, see Developer Account Help. To confirm that your code-signing identity is present and correct, run the following command: % security find-identity -p codesigning -v 1) A06E7F3F8237330EE15CB91BE1A511C00B853358 "Apple Distribution: …" 2) ADC03B244F4C1018384DCAFFC920F26136F6B59B "Developer ID Application: …" 2 valid identities found The -p codesigning argument filters for code-signing identities. The -v argument filters for valid identities only. If the code-signing identity that you need isn't listed, see Developer Account Help. Each output line includes a SHA-1 hash that uniquely identifies the identity. If you have multiple identities with the same name, sign your code using this hash rather than the identity name. Identify the Code to Sign To sign your product, first identify each code item that you need to sign. For example, in the DaemonWithApp product, there are four code items: ConfigApp.app, Core.framework, Share.appex, and Daemon. For each code item, determine the following: Is it bundled code? Is it a main executable? IMPORTANT For a code item to be considered bundled code it must be the main code within a bundle. If, for example, you have an app with a nested helper tool, there are two code items: the app and the helper tool. The app is considered bundle code but the helper tool is not. In some cases, it might not be obvious whether the code item is a main executable. To confirm, run the file command. A main executable says Mach-O … executable. For example: % file "to-be-signed/ConfigApp.app/Contents/Frameworks/Core.framework/Versions/A/Core" … … Mach-O 64-bit dynamically linked shared library x86_64 … % file "to-be-signed/ConfigApp.app/Contents/PlugIns/Share.appex/Contents/MacOS/Share" … … Mach-O 64-bit executable x86_64 … The Core.framework is not a main executable but Share.appex is. To continue the DaemonWithApp example, here’s a summary of this info for each of its code items: | Code Item | Bundled Code? | Main Executable | | --------- | ------------- | --------------- | | ConfigApp.app | yes | yes | | Core.framework | yes | no | | Share.appex | yes | yes | | Daemon | no | yes | Determine the Signing Order Sign code from the inside out. That is, if A depends on B, sign B before you sign A. For the DaemonWithApp example, the signing order for the app is: Core.framework Share.appex ConfigApp.app The app and daemon are independent, so you can sign them in either order. Configure Your Entitlements A code signature may include entitlements. These key-value pairs grant an executable permission to use a service or technology. For more information about this, see Entitlements. Entitlements only make sense on a main executable. When a process runs an executable, the system grants the process the entitlements claimed by its code signature. Do not apply entitlements to library code. It doesn’t do anything useful and can prevent your code from running. When signing a main executable, decide whether it needs entitlements. If so, create an entitlements file to use when signing that executable. This entitlements file is a property list containing the key-value pairs for the entitlements that the executable claims. If you build your product with Xcode, you might be able to use the .entitlements file that Xcode manages in your source code. If not, create the .entitlements file yourself. IMPORTANT The entitlements file must be a property list in the standard XML format with LF line endings, no comments, and no BOM. If you’re not sure of the file’s provenance, use plutil to convert it to the standard format. For specific instructions, see Ensure Properly Formatted Entitlements. If you have a development-signed version of your program you can get a head start on this by dumping its entitlements. For example: % codesign -d --entitlements - --xml "to-be-signed/ConfigApp.app" | plutil -convert xml1 -o - - … &lt;dict&gt; &lt;key&gt;com.apple.application-identifier&lt;/key&gt; &lt;string&gt;SKMME9E2Y8.com.example.apple-samplecode.DaemonWithApp.App&lt;/string&gt; &lt;key&gt;com.apple.developer.team-identifier&lt;/key&gt; &lt;string&gt;SKMME9E2Y8&lt;/string&gt; &lt;key&gt;com.apple.security.app-sandbox&lt;/key&gt; &lt;true/&gt; &lt;key&gt;keychain-access-groups&lt;/key&gt; &lt;array&gt; &lt;string&gt;SKMME9E2Y8.com.example.apple-samplecode.DaemonWithApp.SharedKeychain&lt;/string&gt; &lt;/array&gt; &lt;/dict&gt; &lt;/plist&gt; Keep in mind that some entitlements vary between development and distribution builds. For example: The value of the APS Environment (macOS) Entitlement changes from development to production. The com.apple.security.get-task-allow entitlement allows the debugger to attach to your program, so you rarely apply it to a distribution-signed program. To check whether an entitlement varies in distribution builds, see the documentation for that specific entitlement in Entitlements. For information about when it makes sense to distribute a program signed with the get-task-allow entitlement, see Avoid the Get-Task-Allow Entitlement section in Resolving Common Notarization Issues). Embed Distribution Provisioning Profiles In general, all entitlement claims must be authorized by a provisioning profile. This is an important security feature. For example, the fact that the keychain-access-groups entitlement must be authorized by a profile prevents other developers from shipping an app that impersonates your app in order to steal its keychain items. However, macOS allows programs to claim some entitlements without such authorization. These unrestricted entitlements include: com.apple.security.get-task-allow com.apple.security.application-groups Those used to enable and configure the App Sandbox Those used to configure the Hardened Runtime If your program claims a restricted entitlement, include a distribution provisioning profile to authorize that claim: Create the profile on the developer web site. Copy that profile into your program’s bundle. Note If your product includes a non-bundled executable that uses a restricted entitlement, package that executable in an app-like structure. For details on this technique, see Signing a Daemon with a Restricted Entitlement. To create a distribution provisioning profile, follow the instructions in Developer Account Help. Make sure to choose a profile type that matches your distribution channel (Mac App Store or Developer ID). Once you have a distribution provisioning profile, copy it into your program’s bundle. For information about where to copy it, see Placing Content in a Bundle. To continue the DaemonWithApp example, the configuration app and its share extension use a keychain access group to share secrets. The system grants the programs access to that group based on their keychain-access-groups entitlement claim, and such claims must be authorized by a provisioning profile. The app and the share extension each have their own profile. To distribute the app, update the app and share extension bundles with the corresponding distribution provisioning profile: % cp "ConfigApp-Dist.provisionprofile" "to-be-signed/ConfigApp.app/Contents/embedded.provisionprofile" % cp "Share-Dist.provisionprofile" "to-be-signed/ConfigApp.app/Contents/PlugIns/Share.appex/Contents/embedded.provisionprofile" Modifying the app in this way will break the seal on its code signature. This is fine because you are going to re-sign the app before distributing it. IMPORTANT If you’re building your product with Xcode then you might find that Xcode has embedded a provisioning profile within your bundle. This is a development provisioning profile. You must replace it with a distribution provisioning profile. Sign Each Code Item For all code types, the basic codesign command looks like this: % codesign -s III PPP Here III is the name of the code signing identity to use and PPP is the path to the code to sign. The specific identity you use for III varies depending on your distribution channel, as discussed in Confirm Your Code Signing, above. Note If you have multiple identities with the same name, supply the identity’s SHA-1 hash to specify it unambiguously. For information on how to get this hash, see Confirm Your Code Signing, above. When signing bundled code, as defined in Identify the Code to Sign, above, use the path to the bundle for PPP, not the path to the bundle’s main code. If you’re re-signing code — that is, the code you’re signing is already signed — add the -f option. If you’re signing a main executable that needs entitlements, add the --entitlements EEE option, where EEE is the path to the entitlements file for that executable. For information on how to create this file, see Configure Your Entitlements, above. If you’re signing for Developer ID distribution, add the --timestamp option to include a secure timestamp. If you’re signing a main executable for Developer ID distribution, add the -o runtime option to enable the Hardened Runtime. For more information about the Hardened Runtime, see Hardened Runtime. If you’re signing non-bundled code, add the -i BBB option to set the code signing identifier. Here BBB is the bundle ID the code would have if it had a bundle ID. For example, if you have an app whose bundle ID is com.example.flying-animals that has a nested command-line tool called pig-jato, the bundle ID for that tool would logically be com.example.flying-animals.pig-jato, and that’s a perfectly fine value to use for BBB. Note For bundled code, you don’t need to supply a code signing identifier because codesign defaults to using the bundle ID. Repeat this signing step for every code item in your product, in the order you established in Determine the Signing Order, above. If you have a complex product with many code items to sign, create a script to automate this process. Here's the complete sequence of commands to sign the DaemonWithApp example for Developer ID distribution: % codesign -s "Developer ID Application" -f --timestamp "to-be-signed/ConfigApp.app/Contents/Frameworks/Core.framework" to-be-signed/ConfigApp.app/Contents/Frameworks/Core.framework: replacing existing signature % codesign -s "Developer ID Application" -f --timestamp -o runtime --entitlements "Share.entitlements" "to-be-signed/ConfigApp.app/Contents/PlugIns/Share.appex" to-be-signed/ConfigApp.app/Contents/PlugIns/Share.appex: replacing existing signature % codesign -s "Developer ID Application" -f --timestamp -o runtime --entitlements "ConfigApp.entitlements" "to-be-signed/ConfigApp.app" to-be-signed/ConfigApp.app: replacing existing signature % codesign -s "Developer ID Application" -f --timestamp -o runtime -i "com.example.apple-samplecode.DaemonWithApp.Daemon" "to-be-signed/Daemon" to-be-signed/Daemon: replacing existing signature Consider Deep Harmful When signing code, do not pass the --deep option to codesign. This option is helpful in some specific circumstances but it will cause problems when signing a complex product. Specifically: It applies the same code signing options to every code item that it signs, something that’s not appropriate. For example, you might have an app with an embedded command-line tool, where the app and the tool need different entitlements. The --deep option will apply the same entitlements to both, which is a serious mistake. It only signs code that it can find, and it only finds code in nested code sites. If you put code in a place where the system is expecting to find data, --deep won’t sign it. The first issue is fundamental to how --deep works, and is the main reason you should avoid it. The second issue is only a problem if you don’t follow the rules for nesting code and data within a bundle, as documented in Placing Content in a Bundle.

IMPORTANT This post is now retired in favour of official documentation, namely TN3113 Testing and Debugging XPC Code With an Anonymous Listener. I’m leaving the original post here just for the record. Testing and debugging XPC code can be tricky because there are two processes involved. Imagine you might have an app with an embedded XPC Service. To debug this you have to run two instances of the debugger, one connected to the app and another to the service, and then bounce between them. There is, however, an easier way, a way that allows you to test and debug all of your XPC code in a single process. The trick is to set up an anonymous XPC listener. To start, tweak your XPC listener abstraction to accept an NSXPCListener instance. For example, say you have a MyListener class like this: class MyListener { init() { self.listener = NSXPCListener.service() } let listener: NSXPCListener … more code … } Change the initialiser to look like this: init(listener: NSXPCListener = .service()) { self.listener = listener } This uses the XPC Service’s listener by default, but allows you to override that by passing a value to the listener parameter. Now, in your test environment, call the anonymousListener method to create a anonymous listener and pass that to your listener abstraction: let myListener = MyListener(listener: .anonymous()) On the client side, tweak your XPC connection abstraction to accept an NSXPCConnection instance. For example, say you have a MyConnection class like this: class MyConnection { init() { self.connection = NSXPCConnection(serviceName: "com.example.MyService") } let connection: NSXPCConnection } Change the initialiser to look like this: init(connection: NSXPCConnection = .init(serviceName: "com.example.MyService")) { self.connection = connection } This sets up a connection to the XPC Service’s listener by default, but allows you to override that by passing a value to the connection parameter. Finally, in your test environment, use the init(listenerEndpoint:) initialiser to create a connection to your anonymous listener: let connection = NSXPCConnection(listenerEndpoint: myListener.listener.endpoint) let myConnection = MyConnection(connection: connection) You now have a connection connected to your listener, both running in the same process. This makes it much easier to debug your XPC code. It’s also perfect for unit tests. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com"