메시지와 맞춤형 협업 앱 통합

♪ Mellow instrumental hip-hop music ♪ ♪ Devin Clary: Hi. I'm Devin, an engineer on the Messages team.

Lance Parker: And I'm Lance, also a Messages engineer.

Devin: Welcome to "Integrate your custom collaboration app with Messages." Collaboration starts with a conversation, and in iOS 16 and macOS Ventura, you can bring your app's custom collaboration experience right into the fabric of the conversation.

In this video, we'll go over the life cycle of a collaboration.

Then, we'll show you how to prepare your app's collaborative content to be shared through Messages.

Next, we'll give you everything you need to instantly verify recipient access, and respond to participant changes, all without compromising privacy.

Finally, we'll show you how your app can post notices about the content right to the Messages conversation.

This video assumes your app has existing collaboration infrastructure, and has already adopted universal links.

We'll also build on some concepts introduced in "Add Shared with You to your app" and "Enhance collaboration experiences with Messages." First up, I'll go over the life cycle of a custom collaboration message to illustrate how this API allows your users to start collaborating faster than ever.

When a user decides to share a collaboration from your app through Messages, you first create metadata to represent the content.

The metadata includes share options the user can configure prior to sending the message, and a number of other properties you can customize.

Next, you provide that metadata to the share sheet, or to drag and drop.

This allows a draft of the content to be staged in the Messages compose field.

The collaboration needs to be represented by a universal link.

That can be created immediately, but it's best deferred until right before the message is sent.

This is useful if your app's link creation depends on the selected share options, or recipients, as configured in the Messages compose field.

The user chooses the recipients and share options and taps the send button.

Before the message is sent, Messages asks your app for the universal link and a device independent identifier for the content.

Using that identifier, Messages provides a set of cryptographic identities representing the recipients of that specific collaboration message.

Your app will use these identities later to allow the recipients to immediately open the link on any of their devices.

Your app stores those identities on its servers and associates them with the shared content.

Once your app finishes this step, the message is sent to the recipients.

Now, here's what happens on the receiving device.

The goal is to instantly verify access, pairing a recipient identity with an account on your server.

When the recipient opens the link, your app receives a call to open the URL, just like it does for any other link.

When your app detects that a user account doesn't yet have access to the document, it queries the system for a proof of user identity cryptographically signed by the recipient device.

Your app sends that signed identity proof to your server for validation.

If the signature is valid, the server compares the proof against the identities previously provided by the sending device.

If there's a match, your server grants access to the user's account.

And with that, the recipient has gained instant and secure access to the content, all without exchanging account information! And that's the life cycle of a collaboration message! Next, let's look more closely at the API for starting a collaboration.

The system needs some metadata about the collaboration.

And for that, you use a new class in the Shared with You framework called SWCollaborationMetadata.

This class has a few properties for you to configure: the content's title, a local identifier to reference the content before its been shared; the initiator name and account handle, to provide transparency to the user about the account they're sharing from; and the default share options, for the user to configure.

Here's how to create a metadata object and configure its properties.

Create a local identifier using SWLocalCollaborationIdentifier initialized with a string.

The string only needs to be sufficient for your app to identify the content locally, not across devices.

Initialize a new metadata instance using the local identifier.

Set the content title, the initiator's account handle, and their name using PersonNameComponents from the foundation framework.

The handle and name are only displayed locally so the user can confirm the account they're sharing from.

Next, set the defaultShareOptions.

Before I show you how to do that, I'll first describe how options work.

Share options are the settings a user configures on the collaboration in Messages or the share sheet.

The options selected by the user are provided to you before the message is sent.

Share options might include settings like who can make edits to a collaboration or who should have access to the content.

You use a few classes to define options, starting with SWCollaborationOption.

Depending on how they're grouped, options represent individual switches, or mutually exclusive values for a setting.

Options have a title and an identifier, and they are either selected or unselected.

There are two classes to represent a group of options: SWCollaborationOptionsGroup and SWCollaborationOptions PickerGroup.

You use SWCollaborationOptionsGroup to represent a collection of switches, while SWCollaborationOptions PickerGroup represents mutually exclusive values for a setting.

Finally, SWCollaborationShareOptions defines the full set of option groups, to be set on the metadata's defaultShareOptions property.

You can also provide a summary string to describe the options.

Now that I've described the option classes, here's an example showing how to use them.

This code defines two option groups.

The first group is initialized with an identifier and two possible options.

The identifier is an arbitrary string you later use to identify which option was selected by the user.

Since this is a picker group, the options are mutually exclusive.

This group represents the permission settings for the content: readwrite or readonly.

Then, the first option in that group is selected by default.

And the title is set to a string describing this group.

The second option group is initialized the same way, and also contains two options.

But since this is a generic option group, the user will be able to configure whether to allow mentions and comments independently.

Finally, the two option groups are used to initialize an instance of SWCollaborationShareOptions, which is then set on the metadata.

Next, the metadata is provided to the share sheet or drag and drop, depending on how the user decides to share the content.

If your app uses SwiftUI, SWCollaborationMetadata is compatible with the new ShareLink API.

Watch "Meet Transferable" and "What's new in SwiftUI" to learn more about Transferable and ShareLink.

Here's how easy it is to support collaboration on a proxy representation in SwiftUI! From within a Transferable model object, set up a ProxyRepresentation to return a collaboration metadata instance.

Then, from a view, initialize ShareLink with that model object.

For UIKit and AppKit apps, you use NSItemProvider to support sharing.

And SWCollaborationMetadata conforms to NSItemProviderReading and writing.

So you simply register a metadata instance with an item provider, to support collaboration.

It's also good practice to register multiple representations of the content to support sharing through as many channels as possible.

For example, Messages automatically offers an option to send the content as a copy if you provide a file representation.

You'll use the NSItemProvider API with UIActivityViewController and UIDragItem on iOS and iPadOS and NSSharingServicePicker on macOS.

Here's how to set that up with the share sheet on iOS.

Create an NSItemProvider instance.

Register the collaboration metadata created in the previous example, with visibility set to all processes on the system.

Initialize UIActivityItemsConfiguration with the item provider, then initialize UIActivityViewController with that configuration.

And finally, present the view controller.

It's just as easy to support drag and drop.

Initialize NSItemProvider and register the metadata the same way, then create a UIDragItem with the item provider to use with the drag and drop APIs.

The API is similar on macOS for the sharing popover.

Again, set up the item provider.

And this time, use it to initialize NSSharingServicePicker.

And then show the picker relative to a target view.

Drag and drop on macOS utilizes NSPasteboardItem rather than NSItemProvider.

To support this, SharedWithYou exports an NSPasteboardItem extension.

Using that extension, set the collaboration metadata directly on a new NSPasteboardItem instance in order to support drag and drop.

And that's all you need for a draft of your collaborative content to be staged in Messages! Next, when the user taps the send button, the system coordinates with your app to set up the share.

It does this through a new class called SWCollaborationCoordinator.

SWCollaborationCoordinator is a singleton, meaning there is a global shared instance.

That shared instance coordinates the collaboration through a delegate you define called an actionHandler.

To ensure your app is always available to coordinate collaborations, it will be launched in the background when needed.

So you should register the delegate soon after launch and handle actions immediately to avoid timeouts.

Here's how to set up the collaboration coordinator after your app finishes launching.

Access the singleton coordinator instance through the shared property.

Then, in the app delegate's didFinishLaunchingWithOptions method, set the actionHandler property to an object that conforms to the SWCollaborationActionHandler protocol.

The action handler protocol uses a new class called SWAction.

SWActions represent work your app is expected to perform.

You fulfill actions to mark them as complete, and fail them otherwise.

The first action your app needs to handle is the start collaboration action.

SWStartCollaborationAction contains the collaboration metadata you set up earlier, updated with the user's selected share options.

Once you've performed the necessary setup, you fulfill the start action with the universal link and a device-independent identifier for the collaboration.

If you explicitly fail the start action, the message is canceled.

Here's an implementation to handle the start action using an example server request.

First, retrieve the local identifier, and user-selected share options from the action's metadata property.

Set up a server request to prepare the collaboration using the identifier and options.

Then, send the request to the server.

This example uses async await.

Finally, fulfill the action with the universal link and the device independent identifier from the response.

Or, if there was an error, fail the action to cancel the message.

If the start action was successful, the system sends your app a second action to update the collaboration participants.

The SWUpdateCollaboration ParticipantsAction contains the cryptographic identities for the participants.

The identities are derived from the collaboration identifier fulfilled by the start action in the previous step.

Store the identities on your server associated with the content.

You'll use this data for verifying access on the recipient devices.

Finally, fulfilling this action will send the universal link in Messages.

This example shows how to handle the update participants action.

Retrieve the collaboration identifier from the action's metadata.

This is the identifier you fulfilled while handling the start action.

Next, retrieve the participant data to store on your servers using the action's addedIdentities property.

Each identity has a Data property called a root hash.

This is the data you should store on your server for later use.

Lance will go over more of the details about this property in the Verifying Access section.

Set up another server request, this time to add the participants to the collaboration with the target identifier.

And just like before, send the request to your server, and fulfill or fail the action.

This time, the fulfill method does not take any parameters.

Now that you've set up the collaboration, your app has everything it needs to grant immediate access to the recipients of the message.

I'll hand it over to Lance to show you how to do that! Lance: Thanks, Devin. In this section, I'll show how to provide immediate access to the recipients using the identity data you stored on your server in the previous step.

The rootHash property on SWPersonIdentity is used to do this verification.

A rootHash is a secure value used to uniquely identify a participant on their devices.

In order to perform verification, you'll need to understand how to compute a root hash.

I'll take you through that now.

When a collaboration message is sent, it's actually sent individually to each of a person's devices.

Messages identifies each device using a cryptographic public key.

Since the goal is to allow access only on this set of devices, the root hash is derived from the set of public keys registered to each recipient.

The root hash is the root node of a data structure called a Merkle tree.

A Merkle tree is a binary tree that is built by performing a sequence of hashing operations.

In order to derive an identity for the user based on their public keys, the keys are used as the leaves of this tree.

The hashing algorithm used in the Merkle tree ensures that the root node can only be computed from that set of keys.

In this example, this user has three devices and three public keys.

The keys will be unique for each collaboration identifier provided by your app, using a process called key diversification.

To prevent tracking the number of devices registered to a user, the set is padded with random keys up to a fixed size.

The leaf nodes of the tree are created by hashing the padded set of diversified keys.

The SHA256 algorithm is used for the hashing operations in this tree.

Then, each pair of leaf nodes are concatenated and then hashed to derive their parent nodes.

This process is repeated with the parent nodes and repeated again until a single root node remains.

This is the root hash used to uniquely represent this recipient's identity across their devices.

Notice that it's possible to generate a root hash using a subset of the nodes from a complete Merkle tree.

The root hash in this tree can be reproduced using just the hashes H4, 7, and 11, along with the diversified public key P3.

First, hash the public key to get the missing leaf node H3.

Use H3 and H4 to generate H8.

Use the given H7 node with H8 to generate H10.

And finally, H10 and H11 produce the root hash.

It's important to note that you can prove the public key P3 was used to generate a given root hash, without needing to reconstruct the entire tree.

The subset of nodes needed to do this is called a proof of inclusion.

Verification begins when a universal link is opened in your app.

To do this, you first need to check that the link is collaborative.

SWCollaborationHighlight represents a collaborative link and is retrieved from SWHighlightCenter.

Use that collaboration highlight to generate the proof of inclusion.

To represent a proof of inclusion, use a class called SWPersonIdentityProof.

To perform verification, you'll first generate this object along with a cryptographic signature to send to your server.

Retrieve the proof using the getSignedIdentityProof method on SWHighlightCenter.

It takes an SWCollaborationHighlight and some arbitrary data to be signed by the device.

Use the signature to ensure the request cannot be replayed by a bad actor to gain access to your collaboration.

The data could be a challenge you request from your server, or a nonce generated on the device.

This example uses the challenge approach.

The URL is passed to this method on your app's UIApplicationDelegate.

This URL is the universal link associated with the collaboration.

The URL is used to fetch the associated SWCollaborationHighlight from the SWHighlightCenter.

Next, I'll request the challenge from my server, and pass the data I get back to the getSignedIdentityProof method on SWHighlightCenter, along with the highlight.

This method returns a signed identity proof.

I'll discuss what your server should do to validate this data later on.

Now I can send the signed proof to my server for verification.

Finally, I update my user interface with the result.

The app sends the proof to the server, along with the public key and the signed data.

The data is signed using the elliptic curve digital signature algorithm over the P-256 elliptic curve, using SHA256 as a hash function.

Verify the signature on the data using the public key in the identity proof.

You can do this with most commonly used encryption libraries.

Once you have verified the signature, you can trust that the identity proof was sent from the device associated with that public key.

Next, you use the identity proof to recompute the root hash.

Here is an example of what an SWPersonIdentityProof would contain using the example tree we looked at before.

Use it to reconstruct the root hash of a Merkle tree.

The public key is P3.

The inclusion hashes are H4, 7, and 11.

A local key index of 2 indicates the position of the public key in the tree.

Here is an example implementation that reconstructs a root hash from the properties on the proof.

A recursive algorithm works nicely when working with tree data structures, so that's what I've done here.

On the initial invocation, pass in the hash of the public key, the set of inclusion hashes, and the public key index.

Next, the first inclusion hash is pulled out.

The public key index is checked to see if the key is on the left or the right of its sibling.

The selected hashes are concatenated in the correct order, and then hashed.

Next, the consumed node in the inclusionHashes array is removed, and the rest are passed to a recursive call to this same function.

The public key index is also updated so that it's ready for the next node in the tree.

With this simple function, you can quickly compute a root hash given an identity proof.

The server can now check that this generated root hash is in the list of root hashes the owner of the document uploaded during sending.

The hash is present in the list of known hashes, so the server can grant access to the document.

Now you can grant access to the document with confidence! To recap the steps you'll follow to verify an identity: first, look up the collaboration highlight for your content while handling its universal link.

Next, sign some data and retrieve the proof of inclusion.

Send the signed data and proof to your server.

Verify the signature on the data.

Using the proof of inclusion, generate the root hash.

Finally, compare the root hash to the list of known identities associated with that content.

Now that you know all about verifying access to your collaboration links, I'll talk about how to coordinate participant changes with Messages.

When the participants in a Messages group change, and that group is collaborating together, a user can choose to propagate those changes to your app, right from a banner in the Messages thread.

In this scenario, your app receives another SWUpdateCollaboration ParticipantsAction containing the added and removed identities.

You'll use the same code you wrote to handle this action when setting up a collaboration, but you'll also need to handle removed participants.

For removal, simply look up any account associated with a removed identity and revoke their access.

If no account is yet associated, simply delete the root hash from your database.

Here's the implementation for the update participants action that Devin went over earlier.

This example uses the removed identities property on the action and passes them to a similar removal API request.

Note that this code only shows handling removed identities, but a complete implementation should handle both added and removed identities.

And that's all you need to handle participant changes! Lastly, when changes are made to a collaboration, your app posts notices about those changes to be shown directly in Messages.

There are a few types of supported notices I'll go over in this section.

Notices are displayed as a banner right in the conversation where the link was shared.

The banner includes a description of what changed, as well as who made the change.

In this conversation, Charlie made edits to the Baking Recipes document.

Tapping the show button connects them right back to the content.

To represent a notice, the SharedWithYou framework has a protocol named SWHighlightEvent.

Highlight events are initialized with SWHighlights retrieved from the SWHighlightCenter API.

Messages supports several categories of events.

A change event for content updates or comments, a membership event when a participant joins or leaves, a mention event when a user is mentioned in a collaboration, and a persistence event when content is moved or deleted.

Here's an example showing how to post a change event for an edit to a collaboration.

Using the highlight center API, retrieve a collaboration highlight for the target identifier.

Remember, this identifier is one you defined during the collaboration initiation, so your app should have this available for use when a content change is made.

Next, create a highlight change event instance.

The initializer takes a highlight, and a trigger enum value; in this case, set it to the edit type.

Finally, again using the highlight center, post the notice for that event.

Similarly, for membership changes, post a membership event, this time passing the addedCollaborator or removedCollaborator trigger type.

Next, if your app supports user mentions, you can post a mention event.

Initialize a person identity with the root hash of the mentioned user.

Recall that you associated a person identity with a user account in your app while verifying access.

Then, post the mention event in the same way, this time passing the mentioned identity as a parameter.

This notice will only be shown in Messages to the mentioned user.

Finally, use the persistence event type when content is moved, renamed, or deleted.

Here, the renamed trigger type is used, to signify that the user changed the name of the content.

And that is how your app can notify collaborators, and they will get those updates right in Messages.

Devin: And with that, you're ready to integrate your app's collaboration experience with messages by following a few steps.

Set up your content to be shared collaboratively, cryptographically verify participant access, keep track of participant changes, and post notices in Messages to connect your users right back to the content.

Be sure to check out the "Enhance collaboration experiences with Messages" video to learn more about the new UI elements you can display for collaborations.

Lance: We can't wait to get collaborating with your apps! Devin and Lance, cryptographically signing off.

Devin: Thanks for watching! ♪