Binary Frameworks in Swift

Good afternoon, everyone.

My name is Harlan. And I am super excited to talk to you about how Xcode 11 allows you to create and distribute Binary Frameworks in Swift.

Now, before I talk about binary frameworks, I actually want to take a moment to talk about Swift Packages.

With the new support for Swift Packages in Xcode 11, it's easy to create and use them in your projects and distribute them to others. And Swift Packages are a great way to distribute your code, because Xcode knows how to manage their dependencies, and it will figure out which version of your Packages to use automatically. And because they are distributed in Source form, there is no requirement to maintain binary compatibility with your clients. If you have the ability to ship the source code of your project, then Swift Packages are really, really great.

But not everyone has the ability to ship the source of their libraries, and if you don't, then Xcode 11 supports distributing binary libraries using the new XCFrameworks format.

So, in this talk, I'm going to introduce you to XCFrameworks, the new supported way to distribute binary frameworks. And I will also talk about some things that clients should consider when they're choosing to use third-party code. Next, I'll talk about what's inside an XCFramework, and how you can go about creating one for your project.

And then finally, my colleague, Jordan, will come up and talk to you about some things that framework authors should consider to make using their framework as smooth as possible.

So XCFrameworks are a new way to bundle up multiple variants of your framework in a way that will work across Xcode versions going forward. A single XCFramework can contain a variant for the simulator, and for the device. Not done yet. Because a single XCFramework can also contain a variant for any of the platforms that Xcode supports. You can also have a variant for Mac apps that use AppKit, and a variant for Mac apps that use UIKit. So, no matter which API your clients want to use, they will be able to use your framework effectively. And not only can you bundle up frameworks, but you can also use XCFrameworks to bundle up static libraries, and their corresponding headers. And Xcode will set up your client's search pads automatically.

And of course-- XCFrameworks support binary distribution of Swift and C-based code. So, now I'd actually like to just show you how easy it is to get started using an XCFramework.

So here, I've got a pretty simple iOS app. And I will go ahead and click Run to run it on the iPad Simulator. You can see, it has got a big, blue Launch button, and when you click it, it does nothing. Well, that is because it's hooked up to this launch method right here, and its body is totally empty.

Well, I've actually got this awesome XCFramework I'd like to use, called FlightKit.

And FlightKit gives me some UI that I'd to present in my application.

So, to actually be able to use the FlightKit XCFramework, all I need to do is click on the Project in the Project Navigator, select my target, and make sure the General tab is selected.

Then, I'll scroll down to Frameworks, Libraries, and Embedded Content.

Then, I'll just drag in my XCFramework, and that has been wired up automatically as a dependency of my target.

So let's go back to the code and start using it.

Well, just like any framework you're used to using already, the first thing you'll do is Import it.

And now, I'd like to go ahead and get started using some of the APIs from FlightKit. So, either I could go look at the documentation, or I can Command click the name FlightKit, and click Jump to Definition.

What this will do is it will take me to the generated interface for FlightKit.

And this shows all the public APIs. Every public type, public method, everything that I can use when I import FlightKit. So I can see that there is this LaunchViewController, which is a subclass of UIViewController.

That seems to be some little piece of UI that I might want to show.

Great. So, now I need to know actually how to create one of these, and I can see in the interface that there is an Initializer that takes a Spaceship. And that Spaceship is also part of FlightKit.

So, if I jump to its definition, it will take me further down in this generated interface and show me everything that is in Spaceship that I can make use of. I can see that there is a public stored property called Name, and there is also a public initializer that takes a name.

Okay, so I can create a Spaceship, and I can create a LaunchViewController, and then I can present it.

So, let's go back to the code and do exactly that.

So, first I'll create a ship, and I can see that auto completion is already suggesting to me all the things that were in FlightKit, again, just like any framework that you are already used to using.

I can accept this auto completion, then I'll pick a name for my ship. Well, I've got this array of ship names already, and any one of them would be a great name for this ship.

So I will actually go ahead and pick a random element out of the ship names array. Now, I will create a LaunchViewController, and I'll pass it the ship that I just created. And finally, I'll show the controller passing myself as the sender. So, created a ship, created some UI, and now I'll go to show it.

I'll build and run my program in the Simulator. And when I click the Launch button, it picks a random name, and launches the UI. If I click it again, it will pick another name, and another. You get it.

So, that's how to use an XCFramework for just one platform, but one of the great things about XCFrameworks is that you can put multiple variants in the same bundle. So just by dragging the one XCFramework in, not only am I able to build and run for the simulator, but if I select Generic iOS Device, then I can go to Product Archive, and build an archive for the App Store as well. So that is how easy it is to use an XCFramework from your code.

So when you are making the choice to use a framework, it's really important to be aware of what you're making available to third party code.

Importantly, you want to make sure that you trust the source of the framework.

You trust that they're not going to introduce bugs or instability to your app, and you trust that they'll respect your user's privacy. For example, if you've been granted Entitlements for your application, and you use a framework, that framework is also granted those entitlements. And those permissions, if your users have granted them permissions.

Additionally, if you're adopting a framework that expects a certain entitlement to be available, it's your responsibility to add that entitlement to your app.

Another thing to consider about using frameworks is that sometimes you will use a framework that brings its own dependencies along, and those dependencies might have their own dependencies, and it's your responsibility not only to add all of these to your project, but also to extend the same trust to each of them as well. Now, it is worth noting that this trust extends to using Swift Packages as well.

One advantage of packages over binary frameworks is that you can inspect the code, and you can step into it while you're debugging. If you want more information about using Swift Packages in Xcode, I'd recommend these talks from earlier this week. But whether you're using a package or a binary framework, Xcode 11 makes it easy to use third party code that you trust.

So now, I'd like to talk about how to create an XCFramework.

Well, the first thing you'll want to do is have some source code that you'd like to distribute. So let's actually take a look at some of the source code from FlightKit, from earlier.

This is just a subset of the objects in FlightKit, just for an example. But you can see that Spaceship type that we looked at earlier.

You can also see an enum called Speed, that describes how fast something can move in space.

You also see a struct called Location, that describes the location of an object in space.

Great. So we have this code. Now, how do we make sure that we build this library for distribution? Well, in Xcode 11, there's a new Build setting called Build Libraries for Distribution.

And it does exactly that. It turns on all the features that are necessary to build your library in such a way that it can be distributed.

Now, let's talk about one of those features right now.

If you've tried sending somebody a built Swift Framework before, you may have seen a variant of this error.

Compiled module was created by a newer version of the compiler. What does this error actually mean? Well, when the Swift compiler goes to import a module, it looks for a file called the Compiled Module for that library.

If it finds one of these files, it reads off the manifest of public APIs that you can call into, and lets you use them. Now, this Compiled Module Format is a binary format that basically contains internal compiler data structures.

And since they're just internal data structures, they're subject to change with every version of the Swift Compiler.

So what this means is that if one person tries to import a module using one version of Swift, and that module was created by another version of Swift, their compiler can't understand it, and they won't be able to use it.

Well, in order to solve this version lock, Xcode 11 introduces a new format for Swift Modules, called Swift Module Interfaces.

And just like the Compiled Module Format, they list out all the public APIs of a module, but in a textual form that behaves more like source code.

And since they behave like source code, then future versions of the Swift Compiler will be able to import module interfaces created with older versions. And when you enable Build Libraries for Distribution, you're telling the compiler to generate one of these stable interfaces whenever it builds your framework. So, what does one of these interfaces actually look like? Let's take a look again at the source of FlightKit.

So that is the source from FlightKit. And on the right, you'll see the Module Interface for FlightKit. Now, this is a lot, and it goes off the screen, so we are going to look at it piece by piece. So first you'll see this section of meta data.

So this includes the version of the compiler that produced this interface, but it also contains the subset of command line flags that the Swift Compiler needs to import this as a module.

Next, you'll see all the modules that this framework imports, and then we'll start seeing the types that are part of the interface.

So, here's the public API of the Spaceship class. Now, I want you to notice three things here. Number one is that the public name property is included in the interface, but the private current location property is not.

It's not part of the public API. Next, notice that the public initializer and the fly method are included in the interface.

But their bodies are not included, again, because they're not part of the public API.

And finally, notice that the class has a de-initializer in the interface, but there wasn't one written in the original source code. Now, when you write a class in Swift, and you don't provide an explicit de-initializer, the compiler generates one for you. And this sort of highlights one of the underlying principles of Module Interfaces. If this format is supposed to be stable across versions of the compiler, then the compiler should not make any assumptions about the underlying source code.

So we include it in the Module Interface. Next, let's look at that Speed enum.

Well, the first thing to see is that both cases of the enum are included. Those are part of the public API. However, in the interface, there's an explicit conformance to Hashable. And we list off the methods that are required to conform to both Hashable and Equitable.

Well, this is because in Swift, if you make an enum without any associated values, then the compiler implicitly makes that conform to an Equitable and Hashable, and automatically derives the methods that are required. So, in the spirit of being explicit, and not making assumptions, it's included in the Module Interface.

And finally, the Location struct is included as is, because it only has public stored properties, and does not declare any conformances. So that's a quick look at the Module Interface for FlightKit.

Now that you've taken a look at what's inside a framework, let's talk about how to build a distributable binary XCFramework yourself. Well, the first step to building your framework is by building an archive.

Archiving Your Framework will build it in Release Mode, and it will package it up for distribution and you can see that in the Organizer window.

And as an added benefit, this archive will also contain the debug information that corresponds to that build of your framework, which means if your clients have any crashes or any instability that originate in your framework, they'll be able to send it to you, and you will be able to look at the symbols and debug it. To Archive your framework, you can use the xcodebuild archive command.

You'll pass in the scheme of your framework in your project, and list out the destinations that you'd like to compile it for. So if you're building for iOS, this can be one for the simulator, one for the device, and one for the Mac that's running UIKit. You will also need to pass the Skip Install build setting, and set it to No. This tells xcodebuild archive to install your framework in the resulting archive. So, by doing this, you will be building archives of each variant of your framework, and those will be available in the Archives directory in the Xcode Locations tab, in the Preferences window. Once you've built these archives, you can extract the frameworks, and bundle them up together in one XCFramework. And to do this, you'll run the xcodebuild -create-xcframework command.

You'll pass in the path to each framework on disk, and then provide a path that you'd like the output XCFramework to be output to. So, that's how to build an XCFramework. And in summary, remember, you'll want to enable Build Libraries for Distribution, to make sure that your library is built to be distributed. You'll run xcodebuild archive, to build archives of your framework, and finally you'll run xcodebuild -create-xcframework, to package it up for distribution. And then you can start sending it to your clients, and they can start adopting it. So that was XCFrameworks.

Now, my teammate, Jordan, will come up to talk to you about what you should consider as a framework author to make using your framework as smooth as possible.

Thanks Harlan. All right. So we saw how easy it was to bring one of these XCFrameworks into an app that is a client of the framework, and we saw the steps required to produce an XCFramework. But that's just the first step, because you're framework authors, and you're developing new capabilities every year, and making things better and better for your clients. So, in this section, I'm going to talk about three major things.

Evolving your framework from release to release. Trading some flexibility that Swift gives you for optimizability of your clients, and helping your clients have the smoothest experience possible. So, start with evolving your framework. And what do I mean when I say evolving your framework? Well, like I said, every time you release a new version of your framework, it will have new capabilities, new APIs, maybe some bug fixes, and we want to be able to do that without breaking source or binary compatibility.

Now, why is binary compatibility important here? It's because you don't necessarily know who your clients are going to be. A lot of times it will just be an app target. They'll take your framework, bundle it up, and send it off to the store.

But other times, you'll have clients that are themselves binary frameworks, either from your company or another company entirely. And in that case, the two of you probably have separate release schedules. They might get all the way up to version 2.1, while you're still working on your newest version. And when you finally do release that version 1.1, well, they shouldn't have to do any extra effort to adopt it. You don't want to get in a situation where two binary frameworks are version locked with one another, because then the application who is using them might decide not to update. So, I'm saying here that the version of your framework is important, and not only do you want to put that on your website, and your documentation, but you should also put it in the framework itself, and the place to do that is the Bundle version string setting in the framework's Info.plist.

This is the place for a human readable version number to communicate to your clients what changes you've made since the last release. And the way that we recommend to do that is with Semantic Versioning. Now, if you weren't in the Packages talk, I'll do a quick review of what Semantic Versioning is. The smallest component is the Patch Version, and represents when you make bug fixes, or implementation changes to your framework that shouldn't affect your clients. The middle component is for backwards compatible editions, new APIs, or new capabilities. And the Major component is for any breaking changes that you have to make, whether that's source breaking, binary breaking, or semantics breaking in a way where clients will have to rebuild, and possibly redo some of their client code, in order to adopt the new version of the framework. Let's see what this looks like in practice, with the FlightKit model objects.

So here is the same thing on the left that we had from before.

And now, on the right, I've made a bunch of changes to this framework.

Let's go through them piece by piece and see how each change can affect the framework's version number.

We'll start at the top.

I've added a new private property to the Spaceship class. And I'm using it in the Spaceship's initializer.

Now, neither of these things are going to appear in the module interface. They're not part of your framework's public API.

So this sort of change only requires updating the minor, or the Patch Version component.

Keep in mind though that I did change the behavior of the initializer, and so if this was documented behavior before, then this would be a semantics breaking change, and clients would have to consider whether to update, and therefore, I should change the major version number instead. Now, the next change I've made here is to add a new method to the Spaceship class. It's a new public method, which means clients will start using it and depending on it.

So, the right thing to do is to increment the Minor Version number. And you'll notice, I also reset the Patch Version to zero. Finally, I've also added a new parameter to the fly method.

I've given it a default, so that most of the use sites won't have to change. But in Swift, a function is uniquely identified by its name, and its parameters. Both the argument labels, and the types. So, here I've broken both source and binary compatibility, so this requires updating the Major Version number, and asking any clients to re-compile. Maybe I should have made a new overload instead? Now, these are all changes to the Spaceship class, but I've also changed some of the value types in FlightKit as well. I've added a new case to the Speed edum.

I've made locations Hashable, so that clients can have sets of them, and this is my favorite change, I've added a new stored property to the Location struct, without breaking source or binary compatibility. Now, in Swift, all of these changes are backwards compatible, so I only need to bump the Minor Version number. Now, this flexibility here has some implications for how you design the API of your frameworks. The most important one is to start small. It's easy to add new capabilities if you find out that you need them, or if your clients file feedback saying that more capabilities are needed. But it's really hard to remove something, because it will most likely break source or binary compatibility for at least one of your clients.

For the things that you won't be able to change after the fact, like the names of your types, make sure you consider them carefully up front, that those names are not just going to make sense in this release, but also in all future releases. And finally, don't add extensibility too early. You don't need to make your classes open, or to provide arbitrary callbacks in the first version of your framework.

Why is this important? Because it makes reasoning about your framework's behavior much harder when you have to consider what your clients might be doing at the same time.

So, you can always make your classes open in the future. You can always add properties that represent additional callbacks, but you can't remove the flexibility that you put in by default. So, how does this all work? Indirection.

That's just a word, so let's step through an example. On the left, here again, I have the Spaceship class, stripped down to its module interface this time, and on the right, I have a use of the fly method.

This is from a client code that's outside of the FlightKit framework.

And what's going to happen here at runtime is that the client is going to have to ask which method is the fly method? And the framework will respond, it's the second one. This is how Swift ensures binary compatibility even when you add new methods to a class.

And it's basically the same way that Objective-C does message dispatch, doing it in a call from one Library to another, but Swift only does it when you're crossing this client framework boundary.

There's another form of indirection as well. And that's when clients are using the structs or enums defined in the framework.

So in this case, one of the arguments to the fly method is that fast case from the Speed edum. And I said before that enums could have new cases added without breaking binary compatibility.

That means that the client can't assume that it knows how big the enum is going to be in memory.

So this use of the enum requires the client to ask the framework how big is it? And the framework responds, it's just one byte.

The other possibility here is that one of the new cases added in the future might have associated values.

And those associated values might require some kind of cleanup.

And so the client will also ask the framework to cleanup the enum value when it's done with it, and the framework will do so. Now, a couple of you in the audience at this point are probably getting a little antsy because we're talking about all this extra communication between the client and the framework, and that's because you have performance sensitive frameworks, and that's why the next section is about trading the flexibility that Swift is giving you for the optimizability of your clients.

Now, this really is a tradeoff. As framework authors, we want the flexibility to change things, to add things, to improve things, without breaking source or binary compatibility.

But in order for the compiler to make the client code as fast as possible, it needs to make assumptions about what's in the framework. And so Swift needs to be able to handle both sides of the spectrum. And the way that this works is through the Build Libraries for Distribution build setting. Harlan said before that this has multiple effects in addition to generating the Module Interface file, and one of those effects is to set the default to the Flexibility side.

But again, Swift needs to be able to handle all of these use cases, and so in this section, I'm going to talk about what you can do once you've profiled the behavior of your framework from the outside, and seen that you need additional performance. And there's three ways to do that: inlinable functions, frozen enums, and frozen structs. So, we'll start with inlinable functions, a feature introduced last year in Swift 4.2.

In this case, I have a CargoShip subclass of the Spaceship class we saw earlier, and it has a method canCarry that just determines whether the CargoShip is able to carry some piece of cargo.

And I've made this inlinable, because I think that this is important for the performance of my clients.

What this will do is make this method part of my public interface, not just its declaration, but also its body.

And the effect of that is to copy that body into the Module Interface file. If you're reading very quickly, you'll also see that this method references an internal property of the CargoShip class.

And that's possible, because I've marked that property as usable from inline.

This lets you get the best of both worlds. The property is available as part of your framework's public interface, but it's only available to the inlinable code. It's still protected from being arbitrarily read or written by outside clients. So it's still internal, but usable from inline.

And it's important to note that this is a per declaration decision. The current cargo property here that's also internal is not included in the Module Interface.

So, okay, we have the body of the canCarry method in the Module Interface. And when a client is compiling against that interface, they'll be able to copy that body directly into their own code, and possibly optimize it even further if they know something about the cargo that's being checked. But what happens if the framework owner changes the body of the method, and the clients aren't recompiled? For example, what if there's a new rule that says that CargoShips are not allowed to carry Radioactive cargo? Well, in this case, we're going to run into trouble. Because now two different parts of the program have different ideas about what this method is supposed to do.

And on some inputs, they're still going to agree, for some regular cargo, both the client and the framework will say that it's okay. But if we try to test Radioactive cargo, then the client code will say that it's okay, because that's what it saw in the Module Interface when it was compiled.

While the framework has the new version of the method, and will disallow it. This could indicate a serious logic error in the program.

So, as a rule of thumb, if you're a framework author who has made a function inlinable, make sure not to change the output or observable behavior.

It's okay to add a better algorithm, or some additional fast pads, but if you change the observable behavior of the function, then you could end up with these really subtle problems that are only visible at runtime, and possibly only under certain inputs. If you need to do this, all your clients need to recompile. So next I want to talk about enums.

Swift enums are great. I love them. And one thing that we talked about here is that you can add new cases to an enum without breaking source or binary compatibility.

What this means for clients is that they always have to have a default case when they switch over the enum. And in this client, they've decided to use the unknown default syntax that was also introduced in Swift 4.2. What this means is that they've handled all the known cases in the enum but will still handle any future cases that are added, and this is necessary when switching over C enums, and also enums built in binary frameworks. The other effect that this has is what I talked about earlier, this sort of handshake between the client and the framework, about how big the enum is, and whether it needs any cleanup. But the example I've picked here is a Flight Plan. You can really only have one-way flights, or round-trip flights.

So by marking this enum with the frozen attribute, then I as the framework author can promise that there are no new cases added in future releases of the framework.

The first effect of this is that clients no longer have to write that default case. It can just go away.

And next, the compiler is able to compile it more efficiently. The clients are able to assume that this enum won't have any additional cases and won't require any cleanup.

So that's great.

Except I forgot something. There is another kind of Flight Plan. A multiHop flight.

And now we're in trouble, because that client code no longer has a default case, so adding a new case to a frozen enum is both source and binary breaking and requires incrementing the Major Version and asking all clients to recompile. Now, after frozen enums, frozen structs are much the same. By default, a Struct in a binary framework can have new stored properties added, or the existing ones reordered without any trouble, but that does result in that same sort of handshake and extra communication between the client and the framework. So, in order to avoid this, for structs that are known to have a frozen layout, the frozen attribute can be used to promise that the stored properties will not change. They will not be added, or reordered, or removed. And the other thing that this does is require that the stored properties all have types that are public, or usable from inline. Because remember what the goal is here. We want the compiler when it's working with client code, to be able to manipulate the stored properties of this struct directly, so that it can generate more efficient code on the client side.

This also has a semantic effect, which is that the framework author can now write inlinable initializers. An initializer is already required to set up all of the stored properties in the struct, but now the compiler can be sure that it will also do so in future versions of the framework. Now, I want to close up this section by reminding you that flexibility is the default for reasons. And the main one of these is that breaking changes are really inconvenient for their clients.

A client is going to have-- make a second guess over whether or not to take the new version of your framework, because it might break them in some way. And you could also get into trouble when you have one binary framework depending on another.

It's also worth the reminder that these attributes only affect client code. Within your framework, you still get the full power of the compiler optimizations.

So, before reaching for frozen or inlinable, make sure that you have profiled the behavior of your framework form the outside, and demonstrated that you have a need for additional performance. Otherwise, keep that flexibility because you may need it.

Now, the last section I want to talk about is making sure that your client's experience is the best it can be. And this is going to mirror Harlan's section a lot from the first half of the talk. And so we'll start off with Entitlements.

If your framework has certain Entitlements that it needs to get its job done, well, let's start with the basics. Make sure that you document them, so that any potential client knows what it needs to do to successfully adopt your framework. And furthermore, try to minimize the entitlement requests of your particular framework, because that means that it will be applicable in more contexts. And you can get more clients using your framework. And finally, keep in mind that while both the framework and the application can request permissions from the user, it's ultimately the user's choice whether or not to grant them.

So if you get denied a particular permission, make sure your framework handles that denial gracefully. It should not crash the app, it should not stop working. Make sure it still does something useful so that your clients can use the framework without having to give up.

Now, Dependencies have a lot of the same concerns as Entitlements. Because like Entitlements, your framework's Dependencies become the application's Dependencies.

And so again, you should start off by documenting them, so that a potential client knows what they are signing up for. And you should minimize your Dependencies, so that you're asking less of your clients. Less in extending trust, and even practical matters like the code size taken up by your Dependencies.

And finally, all of your Dependencies do have to be built with the Build Libraries for Distribution build setting in order to get that binary compatibility guarantee that we talked about. This does have a particular implication that binary frameworks cannot depend on Packages.

Let's look at a Dependency graph.

I said just a few minutes ago that the Dependencies of the framework become the Dependencies of the app. But when an app builds a package, it has to pick a particular tag to do so.

And that may not match the version that your framework was built against. It may not be compatible at all.

And beyond that, not all frameworks can necessarily be built in a mode that is compatible with Build Libraries for Distribution.

So this configuration is not supported. Now, the last thing I want to talk about is your Objective-C Interface. Yes, you, Swift framework authors, you have an Objective-C Interface, most likely, because Xcode's default template is set up for a mixed source framework that has both an Objective-C Umbrella Header, and a generated header containing the Objective-C parts of the Swift in your framework. But if your Swift code doesn't have any Objective-C API that it's trying to publish, well, you don't need to install that second header at all. There's an Install Objective-C Compatibility Header build setting that you can just turn off. And if your framework doesn't vend any Objective-C API, well then there's no reason to support this Objective-C Import Syntax, and you can turn that off as well, with the Defines Module Build setting. Set it to No, and that will no longer be valid Objective-C code.

Once you've done that, you can delete the Umbrella Header that Xcode generated for you.

So, let's wrap things up. We talked about a lot of things today, but the most important is XCFrameworks. They're the new Bundle format for distributing multiple framework variants in a way that is super easy for your users to use.

In order to build an XCFramework, you'll need to turn on the Build Libraries for Distribution build setting, which activates everything that you need to get a proper binary compatible framework. And as framework owners, make sure that you know the responsibilities that you have to your clients, so that you can serve them the best. Harlan and I will be down in the Lab immediately after this session, but for everyone who came here, thank you very much, and let's see some great frameworks. [ Applause and Cheering ]