What’s new in assessment on macOS

Hi, I'm Chris, an engineer on the Education Technologies team. And I'm excited to take you through the enhancements available in the Automatic Assessment Configuration framework in macOS 27. This framework helps to create a secure, locked down environment for testing organizations delivering education assessments or certifications on Apple devices. To use this framework, your app needs the restricted Automatic Assessment Configuration entitlement. If you haven't already requested it, you can do so through the Apple Developer portal.

In this session, I'll cover five areas. First, system preconditions: checks your app can require before allowing an assessment to continue.

Then, accessibility restrictions that ensure these features are only available for the students who are approved to use them.

After that, system experience customization for tailoring how users interact with the Mac during an assessment.

From there, application launch restrictions your app can set to ensure that only the processes you deem trustworthy are running during an assessment.

And lastly, best practices that will help you make the most of what assessment mode has to offer.

I'll be using this sample assessment app to demonstrate the behavior of the API capabilities covered in this video.

First up, I'll review the precondition checks that can be done before an assessment starts. Performing system precondition checks before allowing a student to begin an exam is critical for ensuring the Mac is in a hardened state, and helps ensure the security requirements of the assessment from the very start.

Your app can require that System Integrity Protection is enabled, that the Mac is MDM enrolled, that only a single account is signed in, and that the account is a specific type, such as a standard account; four checks that together help ensure the device is in a hardened, tamper-resistant state.

Additionally, your app can require that Lockdown Mode and iCloud Private Relay are disabled; two checks that help ensure that Apple privacy and security features don't interfere with your assessment infrastructure requirements.

I'll now add the precondition checks to my sample app, by setting properties on my AEAssessmentConfiguration object, which is the object that defines an assessment session's parameters. This code snippet requires the signed-in user account to be a standard account. With these precondition checks configured, if a student's device does not meet one or more of these requirements then an alert is displayed informing the student of the issues that must be addressed before they may continue.

Next up: managing the availability of accessibility features. MacOS includes a comprehensive suite of built-in accessibility features. These features are essential for providing equitable access to exams, enabling individuals with visual, auditory, motor, or cognitive needs to fully participate without requiring third-party assistive software.

By default, the Menu Bar and the Dock are hidden, but any currently enabled accessibility features will continue to work during the assessment session.

As demonstrated here, Switch Control continues to run after the assessment session begins.

However, it's important to note that certain accessibility features can be customized with user-generated content. Therefore, disabling access to those features for students who do not require them as part of an approved accommodation is an important security measure. In this configuration, every accessibility feature is allowed except one. I'm restricting the use of Switch Control. Note that setting the value to true for a configuration property does not enable the corresponding accessibility feature. It simply allows it to be used during an assessment when enabled by the user.

When the assessment session starts, Switch Control is automatically quit and cannot be relaunched during the assessment session.

One of the most powerful areas of the framework is system experience customization, tailoring how students interact with macOS during an assessment.

The macOS system experience is designed to provide a seamless, intuitive environment through familiar elements like the Menu Bar, Menu Bar items such as Wi-Fi and Volume, the Dock, various input technologies like Dictation and AutoFill, and filesystem interaction using the Finder and the Open and Save file dialogs. Your app can make the Menu Bar available to allow students to access essential application functions during an assessment. Additionally, your app can customize which Menu Bar items are available, like volume or Wi-Fi, while removing items that could serve as a vector for information or content leakage.

I've already demonstrated how the assessment session looks without a Menu Bar. In this example, I'm setting additional properties on my AEAssessmentConfiguration object to enable the Menu Bar and to define a set of allowlisted menu extras.

Note that these menu extras are not forced on but rather continue to be available during an assessment session if they were already present in the Menu Bar.

I've also customized the Apple menu by setting a property on my configuration object to show only the sleep menu item. Optionally, I could set an empty array to hide all items except for "About This Mac".

As the assessment session begins, the app's menu items are displayed and the set of Menu Bar items is filtered down to the allowlisted set I just specified. Additionally, selecting the Apple menu reveals that its contents have been filtered.

Now I'll review some input methods you may wish to disable that can inadvertently provide students with hints or correct answers.

Dictation can produce correct spelling automatically. The emoji picker exposes a searchable symbol library. And structural input reveals character composition clues, any of which could bypass assessments designed to test unaided recall.

AutoFill can provide pre-loaded answers, notes, or reference material into response fields from sources such as Contacts. To restrict these input methods, I set each of these properties to false in my AEAssessmentConfiguration object. This hides them in menus and prevents their use within UI controls that normally support them.

With the input restrictions in place, the AutoFill, Dictation, and Emoji & Symbols menu items are no longer available in the Edit menu.

The Dock is another valuable system experience you may wish to enable during an assessment session because it provides students with a clear, focused workspace where they can easily find and switch between applications. In this code snippet, I'm enabling a filtered Dock experience by setting allowsDock to true.

During the assessment, the Dock displays only the allowed apps, giving students a focused workspace to find and switch between them.

Upon entering the assessment session, note that in addition to the always present anchor elements of the Finder and the Trash, only the allowed apps for the session are present. However, even though the Finder is present, it is not accessible unless it is explicitly added as a participant.

If your assessment app requires interaction with designated files, then you may wish to allow access to the Finder and make use of the standard Open and Save dialogs. They provide a consistent, intuitive way to browse, organize, and access files across macOS. Next, I'm adding the Finder as a participant in the assessment session. With the allowedDirectoriesAndFiles property on the AEAssessmentConfiguration object, I can allowlist the Documents directory into which a student can save their scratch paper work in the Sample Assessment app. This same setting also filters the directories and files available in the standard Open and Save panels.

When the student chooses to save their scratch paper work, they are presented with a standard Save panel, but only the allowlisted directory is available for them to save their work into.

Bringing up a Finder window shows the same filtered access and the allowlisted directory contains the scratch paper file the student just saved.

There's one more layer of control to cover: restricting which processes are allowed to run during an assessment. Processes that are not essential to the execution of your assessment app can pose a threat to the integrity of the assessment session. They can be used to capture screen content, log keystrokes, transmit data to external parties, or otherwise interact with the system in ways that undermine the secure testing environment. Consequently, you may wish to restrict the allowed processes to just your assessment app and participant apps you allowlist. Additionally, shortcuts and automator actions can also be shut down and their execution blocked. Setting allowOnlyParticipantsToRun to true tells the system to shut down non-essential processes when the assessment begins. With this enabled, only the main assessment app, its explicitly allowlisted participants, and essential system processes are permitted to run. I'm also setting allowsUserScriptExecution to false to prevent Shortcut and Automator scripts from executing.

Before starting the assessment, both Safari and Notes are running as well as a long running Shortcut. However, after starting the assessment, only the sample assessment app and its allowlisted participant, Finder, are still running. Any user-initiated background processes, including the Shortcut, have also been stopped and cannot be accessed during a secure exam.

Whether adopting Assessment Mode for the first time or hardening an existing integration, the following best practices are ones that every developer should consider.

These practices will help with how you adopt the framework, how you shape the test-taker's experience, and how you keep your app functioning well across macOS releases.

With its deep system integration, let the Assessment framework do the work to secure the system environment for you. Resist the urge to roll your own equivalents by adopting the framework's APIs directly and deleting the redundant code you've been maintaining.

Only restrict the minimum that your assessment actually requires. Restrictions you add can detract from the test-taker's experience, so start permissive and tighten deliberately.

Accessibility should be treated as a requirement in an assessment context. Every student deserves a fair testing experience, so design your app from day one to accommodate assistive technologies rather than treating them as exceptions to be carved out later.

If you're new to AAC, take note that session transitions don't occur the moment you call the begin and end session API. Register for the framework's transition callbacks and drive your app's state off of those events so you always know when a session truly started, ended, or was terminated unexpectedly.

It's critical to re-validate your app on every macOS beta the day it drops. Run your full assessment test matrix against it, and file Feedback reports immediately. Waiting until the release to discover a regression means your customers discover it too.

Now that you've seen what's possible, consider these next steps to take your assessment app to the next level. Use system pre-checks to validate device integrity before starting an assessment.

Enable accessibility features to provide an equitable assessment experience for all users.

Provide a familiar and intuitive experience by customizing access to Menu Bar items, the Dock, and the file system.

Secure your assessment's runtime environment by blocking non-essential processes.

And lastly, test your assessment solution with real exam workflows on macOS.

Whether you're building a classroom quiz app or a nationwide standardized testing platform, the AutomaticAssessmentConfiguration framework on macOS gives you the tools to verify system security prerequisites, tailor accessibility accommodations, customize the user experience, and lock down the runtime environment, all through a single, unified API, tailorable to your needs. I can't wait to see how you use these new capabilities to deliver safer, more inclusive, and more polished assessment experiences for students everywhere. Thank you for joining me.