Meet Safari Web Extensions on iOS

♪ ♪ Hi. Welcome to Meet Safari Web Extensions on iOS. I'm David Quesada, an engineer on the Safari team. And today, I'll be walking through an exciting new feature in iOS 15, Safari Web Extensions. Web Extensions are a powerful type of browser extension that allows users to extend and customize their browsing in a huge number of ways. They're built with the standard web technologies: HTML, CSS, and JavaScript. And the WebExtension API has been available for all the major desktop browsers for a while, which means you can write a browser extension once and deploy it across all those browsers, reaching so many users. But now, I'm thrilled to tell you that Web Extensions are coming to iOS 15, letting you build a browser extension for more users than ever. I'd love to show you a demo now of a Safari Web Extension running on iOS. I'll be showing off an extension called Sea Creator. This extension was originally written for browsers on the Mac, but I've been working on bringing it to iOS. This extension shows me fun facts about sea creatures and makes reading more about sea life fun by replacing the names of sea creatures with emoji. Let's take a look at this on an iPad. Now, I've just installed Sea Creator, and I've opened up Safari, where I'm reading an article about fish. I'd really love to start using Sea Creator here so I can liven up this page with some emoji. So I'll open up the action menu and drill into the new Extensions list, which shows me all the extensions that I have available, including Sea Creator. So I'll turn it on. Now, another feature of Sea Creator is that I can put it front and center in Safari with a custom Start Page that I use every time I open a new tab. I'll check that out in a bit. And now that I've turned on Sea Creator, it's right there in the Action menu ready for me to use. And when I do that, Safari asks me if I'm OK with giving Sea Creator full access to this site. We brought the same Web Extensions permission model from Safari on the Mac to iOS, which means I, as the user, have full control over how much of my browsing extensions can access. I'll talk in more detail about this later. But right now, I'll allow Sea Creator to run. And just like that, it swapped in all these fish emoji on the page, which is just how I like to read it. It's also opened this popup showing me a running total of just how many words I've replaced so far. But it's not just this one page. I gave Sea Creator permission to work across this website as a whole. So as I navigate to other pages, Sea Creator is still working for me and gives me emoji here as well. I don't need to go out of my way to open the extension again once I've started using it. Of course, Sea Creator has one more feature that I mentioned earlier. When I open up a new tab to do some more browsing, I'm greeted by this bold New Tab page from Sea Creator with a fact of the day today about starfish-- and once again, that counter showing me just how many emoji Sea Creator has given me so far. This is pretty cool. So that was a quick look at Sea Creator working great on iOS. In this session, I want to show how you too can create a Safari Web Extension for iOS. And I'll be using Sea Creator as an example of how I took an existing extension and brought it to iOS with just a few changes. But don't worry if you've never written a browser extension before. I'll give you an overview of how Web Extensions work and how you can get started creating a new one. Along the way, I'll cover some debugging techniques and best practices to help your extension be the best it can be. And I'll wrap it up with some thoughts on user privacy and how it relates to the permissions model of Safari Web Extensions. So, our journey begins with creating a web extension that can run on iOS in the first place. Let's look behind the scenes to understand how this works. It's important to understand that for Safari, Web Extensions are parts of apps. So when you want to install a web extension, you install its app. And like any other type of iOS app, apps with Safari Web Extensions can be found on the App Store. And Xcode has everything you need to build an app to ship your extension. So when you think about building a Safari Web Extension for iOS, you may be starting from one of three places. Maybe you're looking at creating a new extension from scratch, or maybe you already wrote one for another browser, and you want to bring it to Safari, or perhaps you already have an extension for Safari on the Mac, and you want to make it available on iOS too. I want to go over what you can do in each of these cases. So first, if you want to create a brand-new Web Extension, you can use Xcode's included templates, which make it super easy to get started. You can create a new Xcode project and use the Safari Extension App template to start building a web extension as well as its containing app. And when you use this template, you get an extension that comes complete with all the resources a typical web extension might have. You can use this as a starting point by customizing what's already there or adding or removing pieces according to what your extension needs. Now, if you've already built a Web Extension for another browser, you can take advantage of a tool called the Safari Web Extension Converter to automatically create an Xcode project from that existing extension. You simply run the converter on the command line, provide it a path to your extension's resources, and it will create a new Xcode project that packages your extension in a native app. And new in Xcode 13, the Safari Web Extension Converter creates a project that supports Safari on both iOS and macOS by default. And the newly created project, by default, references your existing extension's resources at the original path rather than copying them. Finally, if you've already set up an Xcode project in the past to build a Safari Web Extension for macOS, you can also use the converter to upgrade that project so it supports iOS too. You can run the converter and provide a --rebuild-project option along with a path to your Xcode project. Then the converter will add an iOS-compatible version of the extension and a containing app to your project. Now I want to show the Safari Web Extension Converter in action by retracing my steps of bringing Sea Creator to iOS for the very first time.

So here is my Sea Creator Xcode project, which I used to develop Sea Creator for Safari on the Mac. But it only supports the Mac right now, and I want to fix that. So I'll open up a terminal and type xcrun safari-web-extension-converter, and I'll use the --rebuild-project option, and I'll provide a path to my Xcode project and run the converter. And it confirms some details about my existing project and stops to ask me if I'd like to overwrite it with a cross-platform version. Also, any time you run the converter, it'll warn you about potential compatibility issues with your extension. In this case, it's warning me about the use of something called a persistent background page, which isn't supported on iOS. I'll need to fix this later. But for now, I'll let the converter keep going.

And when it's done, my project automatically opens. Notice there are folders here for both the iOS and macOS versions of my app and the extension. Now, I want to take a look at the Resources folder within the Shared extension group. Here is where I'll find the core pieces of the Sea Creator Web Extension. For those of you who are new to the Web Extension API, I want to explain what some of these pieces are, since I'll be referring to them later. I'll start with the manifest. This is a JSON file that describes the structure of the extension. It includes important information such as the extension's name, which websites it wants to access, and what features it supports, such as a pop-up page or a New Tab page like Sea Creator has. Next, there's a JavaScript source called the background script. The browser can run this script in the background when your extension is enabled, and it allows your extension to listen for various events coming from the browser or other parts of your extension. Sea Creator's background script keeps track of that counter of just how many emoji it's added to web pages. Next, there's another type of script called the content script. The browser automatically runs this script on web pages that the user visits. An extension can have any number of content scripts, and the manifest specifies which content scripts should run on which websites. This script gives your extension the power to extend and customize pages by directly manipulating them. For example, Sea Creator's content page is what actually does the replacement of animal names with emoji. And after it does that, it sends a message to the background script so it can update that central counter. Of course, the extension also has some HTML, CSS, and other JavaScript files for the New Tab page, and that pop-up page as well. And finally, there are some other resources here such as localized strings, as well as the extension's icons and other graphics. So all of these pieces come together to make the Sea Creator Web Extension. And thanks to the Safari Web Extension Converter, I now have a project that packages all of this up into a native app. And I can run my project now, so I can start using Sea Creator in Safari right away. I'll do that now by going to the toolbar at the top of the screen, selecting Sea Creator for iOS, and choosing the destination where I want to run the app. I don't have any iOS devices connected, so in this case, I'll use an iPhone simulator.

I'll then click the Run button. And Xcode will then build my app project, load the simulator, and run my app.

So this is the Sea Creator app generated by the Web Extension Converter, and it contains the Sea Creator Web Extension. At the beginning of this session, I showed that I can turn on extensions directly within Safari, but as the app suggests, I can also use Settings to manage my extensions. And I want to show that off here, so I'll switch to Settings.

I'll jump into Safari, Extensions, and then Sea Creator. And here's the details page for Sea Creator with some info about the extension and a switch to turn it on. But that switch is grayed out, and I can't turn on the extension yet. This means there's a problem with the extension that keeps it from loading. I definitely need to look into this next. And this is where I want to talk about debugging. It's a great first step to just be able to build your project and install your extension, but of course, it's essential to know how to dig deeper to find out why things go wrong if they do. It's easy to identify errors in web extensions and use tools such as Web Inspector to debug parts of your extension, and that's what I'll show you now. Looking back at Settings, there's an error here at the bottom telling me that Extensions on iOS must have a non-persistent background page. This sounds familiar. It's the same issue the converter warned me about earlier. And now I want to actually go fix it. So I'll go back to Xcode, open up the manifest, and add a "persistent": false key in the background section. I'll talk about this change in more detail later, but for now, this should get me past the error. Now that I've made this change, I'll run the app again to install the updated extension on the simulator.

And here's the Sea Creator app once again. It looked like I only had that one error to fix, so now I'll go straight to Safari instead of going back to Settings.

I'll open up Safari, open the Extensions list, and there's Sea Creator, and it looks like I can turn it on now. Safari offers me the choice to use Sea Creator's New Tab experience. And I do want to check out what that looks like on iPhone since I haven't seen it before. So I'll set that as my New Tab page, close out the Extensions list, and open up a new tab. And just as I'd expect, there's that New Tab page from Sea Creator. But the text is really small, and the page just doesn't look good. This is the first time I've ever run this extension on iOS, so it's not surprising that the content hasn't yet been adjusted to work well with iPhone. I want to go fix this now, and to do that, I'll use Web Inspector to take a closer look at this page and try out a few changes I have in mind. So to get to Web Inspector, I'll open up Safari on my Mac, and then I'll double-check in Safari's Advanced Preferences to make sure I have the Develop menu enabled.

And now that I have that Develop menu, I can open it up, select the iPhone Simulator I've been using, and choose a page to inspect. Now, there are two options here for Sea Creator. I could select the Extension Background Page if I'd like to debug the background script, but in this case, I want to inspect the New Tab page. So I'll choose the other option.

And now I'm in Web Inspector. The first thing I want to address here is the overall size of the page. By default, Safari on iPhone renders the web page as if it were the size of a desktop browser, and then it scales that larger content down so it all fits on screen. But I don't want this behavior here since it makes the text too small. So I'll use a very common practice when it comes to web design for iPhone. I'll add a viewport meta-tag to tell Safari not to scale the content this way. I'll select the Head element, add a Child element, and add the contents of that viewport meta-tag. I'll add the tag, and the page updates live in the simulator in response to that change. Now all the text is at a readable size, but it doesn't all fit on screen, which is my next problem. I want to inspect the elements on this page to understand why it lays out like this.

I'll look at the body tag, and I notice there's this div here that contains all the content on the page. If I focus it and look in the stylesheet sidebar to the right, I notice that the element has a fixed width of 850. That would make sense on a desktop, where windows can be really wide and you'd want the text to wrap at some point. But that doesn't work on iPhone because the phone just isn't that wide. I think it would make more sense here if that rule set a maximum width for the content rather than demanding an exact width. And I can simply edit the rule here. I'll click width and change it to max-width, and the page in the simulator updates to reflect that change. Now the content's width is much more appropriate for iPhone, and all of the text fits on screen. This looks much better now, but these changes that I've made are only temporary to this Inspector session. I want to actually save these changes back into my project so I don't lose them. And it's really easy to save an updated copy of the stylesheet while I work in the stylesheet Inspector. I can simply type Command+S to save my stylesheet that has that updated rule. And I'll overwrite the new_tab_page.css in my project.

But for the viewport change, I added that tag to a live web page rather than a static resource, so I'll have to make the same change to the original source. I'll select that tag, copy it, go back to Xcode, open up that source for the New Tab page, and paste the contents there.

So the next time I run Sea Creator, it'll use that updated page source and stylesheet, and it'll look just as good again. And that's been a few examples of using Web Inspector to look under the hood of a web extension, understand exactly what's happening, and experiment with potential changes while iterating on the extension. For now, I'm only going to look at the New Tab page, but later on in development, I should obviously go test all the other parts of the extension on iOS, and I'd use the same tools and techniques for that. You can use Safari's extension settings on iOS to view any errors in the configuration of your Web Extension. Specifically, Sea Creator had the fatal error of using a persistent background page, but there may also be non-fatal warnings here that you should check out. These details will only appear in Settings for debug builds of your app from Xcode and not for copies from the App Store or TestFlight.

As you make changes to your web extension, you simply run your app again to update the extension on the device or simulator. And of course, Web Inspector allows you to investigate issues with your extension's web content. Remember that in order to access Web Inspector, I had to enable the Develop menu in Safari's Advanced Preferences on the Mac. In my demo, I used the iOS simulator for my extension, but if you wanted to use a physical iOS device, you would also need to enable Web Inspector support on that device in Safari's Advanced Settings. To take a deeper dive into Web Inspector and learn about some of its newest features, check out Discover Web Inspector improvements. Now that I've created a Safari Web Extension for iOS and I've debugged a few basic issues, I'd like to walk through some best practices to keep in mind, including some things you might need to watch out for as you build an extension for Safari on iOS. I'll focus on five topics that may be relevant to your extension, starting with non-persistent background pages. The background page is a web page that the browser loads to run your extension's background script. And this page allows your extension to handle events sent by the browser or other parts of your extension. But keeping this page loaded has a performance cost. It can use memory and power as if you were keeping one more tab open and running for every enabled extension. Keeping all these pages loaded all the time can be pretty wasteful. But you can make a background page non-persistent, which means the browser will only load it when your extension actually needs to do work, and the browser can later unload that page when it's been idle for some time. That way, the performance cost is only paid while your extension is doing something useful. This is important because background pages must be non-persistent on iOS, where system memory and battery life are especially at a premium. The web extension templates in Xcode already come with a non-persistent background page, so they're ready to run on iOS. But if you have an existing extension that uses a persistent background page like Sea Creator did, you'll need to change it to be non-persistent. And you can do that by adding that "persistent:" False key in the background section of your manifest. To learn more about some best practices and potential issues you may face when updating an extension to use a non-persistent background page, check out Explore Safari Web Extension Improvements. Up next, let's talk about responsive design. As we learned with Sea Creator, bringing an extension to iOS means its web content may be rendered in new environments that you haven't considered before. Just like I did with Sea Creator's New Tab page, be sure to test the layout of your extension's web content on iPhone and iPad and use a responsive design that can accommodate various screen sizes. But the difference in screen size isn't the only consideration when it comes to making your web content look great on iOS. Let me tell you about a few things you should be aware of. If your extension has full-page web content that lays out important elements near the bottom of the screen, you may find it being covered by Safari's Tab Bar or the device's home indicator. In CSS terminology, this area near the edge of the screen where content may be partially hidden is called the unsafe area, while the usable area of the viewport is called the safe area. By using CSS environment variables, you can calculate the safe area inset to make sure important elements are positioned within the safe area. On iPhone, this is also worth considering in Landscape, where devices may have safe area insets on the left and right side of the display as well. By using similar CSS and by specifying the viewport-fit parameter in your viewport, you can give your web content an edge-to-edge design that still keeps important content within the safe area.

Check out Design for Safari 15 to learn more about these APIs that can make your web content feel more at home on iOS. On iPad and desktop browsers, if your extension has a pop-up page, you may be used to it being shown as a popover that's comfortably sized to fit its content. But on iPhone, however, Safari will display this web page as a sheet, which may be a surprise to your content. The sheet spans the full width of the device, and it may be laid out taller than the content expects. In this screenshot of an early version of Sea Creator, the content didn't have much padding from the edge of the screen, and we set a background color on an individual element in the page rather than on the body. So the text looks a bit cramped and the background doesn't fill the whole page. But we've since updated the alignment and padding of the content to give it a little more space to breathe, and we now specify a background color on the body so it covers the whole sheet. If your extension has a pop-up page, think about whether you should make similar changes for your extension. And note that Safari will use a similar presentation in Landscape as well. Be sure to test your extension's interface in these configurations to make sure its layout makes sense when given this extra space. And finally among the design considerations, I want to mention Dynamic Type. Dynamic Type is a feature that allows users to adjust the size of text and other visual elements. You can make content smaller to fit more or larger so it's more visible. Be sure to test your extension's interface under smaller or larger text sizes to make sure it looks all right with any size the user chooses. To help your web content make the most of Dynamic Type, WebKit has a variety of system fonts that respect the user's text size preference and resize to match. You should adopt these fonts in your extension so that its text remains comfortably readable to the user, just like the rest of Safari's interface. So the main takeaway here is to design your extension's web content with Safari's UI in mind. Test any full page web content on iPhone to make sure that it adapts well to the screen size and doesn't clash with Safari's UI at the bottom of the screen. You should test your pop-up web page on iPhone to make sure its layout makes sense with the sheet-style presentation, and you should test your interface across a wide range of Dynamic Type sizes to make sure it adjusts for the user's preference. For more tips on designing web content to look great in Safari's new interface, check out Design for Safari 15. Up next, pointer events.

If your extension currently depends on handling mouse events for arbitrary clicks and drags, be aware that the same events won't be sent when the user taps on iOS. You should instead adopt the Pointer Events API. It's similar to the Mouse Events API, and it works just the same with mouse input, but the Pointer Events API also reports touches and Apple Pencil input. And now let's talk about the Web Extension Windows API. On desktop browsers, users may have multiple windows open and your Web Extension can use the browser.windows API to work with these windows. And the same is true on iPad, where you can also open multiple windows of Safari. Each Safari window might be full screen, or it might be in split view, side-by-side with another app, maybe another Safari. Under the hood though, each of these windows is actually called a scene on iOS. If your extension uses the Windows API, you should know that each scene of Safari actually has two windows: one for regular browsing and one for Private browsing. This is also true on iPhone, even though there's only one scene of Safari. If I call the browser.windows.getAll API to query what windows are open, then the API returns these two window objects. In the first window, the incognito property is false, and focused is true. This represents the window I'm looking at, which is in regular browsing and not Private Browsing. The second window holds the Private Browsing tabs in the Safari scene. And of course, its properties are different from the first window. Incognito is true, and focused is false. Now when I switch Safari to Private Browsing and call windows.getAll again, the API returns different window objects. Now the focused property has changed in both windows, and the second window now has focus.

This works exactly the same on iPad where one Safari scene is represented by two windows. But if I open a second scene of Safari in split view, the API would now report four windows. And if my extension listens for the windows.onCreated event, it would observe that event fire twice when opening split view, once for each of the two new windows in the new Safari scene. So if you use this API, keep this model in mind as you encounter extra windows you might not expect at first. In addition to this model of what a window actually is, there are a few limitations of the windows API that may affect your extension. The methods to create, remove, and update the state of windows aren't available. On iOS, window placement is fully controlled by the user, not any app or extension. The windows.onRemoved event won't be fired as you'd maybe expect when the user closes Safari from the app switcher. That doesn't truly close or remove a window. It just leaves it for the user to pick up where they left off later. And note that these restrictions only apply to the windows themselves and not the tabs within those windows. With the browser.tabs API, Web Extensions still have full control to add, remove, and update individual tabs as they'd like. Now, last up in the best practices is feature detection. When bringing an existing extension to iOS, you may find some APIs are unavailable-- for example, the windows APIs that I mentioned a moment ago. But there are a few others, like context menus and WebRequest. If non-essential parts of your extension use such APIs, be sure to use feature detection to handle the case that they're not available. So, instead of unconditionally calling these APIs, conditionalize that code based on the existence of those APIs, so you can cleanly exclude parts of your extension and make it more flexible when it comes to which browsers it supports. You should also use this pattern when adopting new APIs that are added in the future, as they might not immediately be available across all browsers or in previous versions of browsers that some users may still be using. And that wraps up the best practices. I've now covered many parts of Safari Web Extensions, but I've saved arguably the most important topic for last, and that's user privacy. User privacy is a huge part of everything we do, so we believe users deserve transparency and control when it comes to how their personal data is handled. And of course that includes everything they do in Safari. Web Extensions can gain a tremendous amount of access to the user's browsing, so the decision of whether that should be allowed and on which websites should be up to the user. Historically, other browsers would give an extension full access to every website it requests right away when you turn it on, which is potentially asking you to sacrifice privacy up front just to get a taste of the extension. But Safari aims to do better by providing transparency and control to Web Extension users with an opt-in model where extensions are only given access to websites when the user consents. You caught a glimpse of this earlier with Sea Creator. It didn't immediately start modifying my pages when I turned it on, but it was only able to do that when I explicitly told Safari, "Yes, I want to use this extension here, and I'm okay with it having access." And once I give an extension that access, Safari tells me in the Tab Bar that I have an extension running on this page, so I know that the extension might be seeing my web browsing here. But if I navigate to any website that I haven't yet allowed the extension to access, that indicator disappears. And I know that that extension isn't running on this page and can't see what I'm doing here. Let's look at this permissions model in more detail. Again, users opt in to your extension when they choose to use it on specific websites. Safari will ask the user for consent by presenting a dialog which makes it clear which websites the extension is trying to access. This happens when the user invokes your extension by selecting it in the Action menu or by using any of its keyboard shortcuts. And this consent is required for any privacy sensitive API to work. Specifically, any API that reveals the URL or title of a tab will only include this info if your extension has permission for that URL. The Cookies API will only let your extension read and write cookies for websites your extension has permission for. And injecting JavaScript and stylesheets will only be allowed on websites where, you guessed it, your extension has permission.

If your extension's scripts call any of these APIs when your extension doesn't have the required permissions but hasn't asked the user yet, Safari will wait to call your completion handlers and will show a non-disruptive banner at the top of the screen. This lets the user know that your extension wants some more access, and they can then review the set of websites your extension is asking for, and they can make a decision to allow or reject access for those websites. You should avoid requesting more permission this way than your extension really needs. For some types of extensions, this way of thinking about permission might be a bit more than you need. For example, extensions that share or annotate individual web pages don't need full access to the site any time the user visits in any tab, but rather they just want permission once to do one thing at a time for the user.

And there's a great solution for those extensions called the activeTab permission. When your extension requests the activeTab permission, Safari will automatically grant your extension permission on tabs where the user explicitly uses your extension. And this permission will be limited to just the current website in just the current tab. So it'll be revoked if the user navigates that tab to a different website. Safari won't display a prompt when granting this permission, since there's no long-term commitment needed from the user.

To adopt this feature, simply add activeTab to the Permissions section of your manifest.

So all of this is to say that in Safari, users are always in control of which websites their extensions can work on. So be aware that your extension won't automatically be able to go to work on every page the user visits. For many extensions, the activeTab permission is a great way to get access to just the pages where the user is using your extension without bugging them about any access beyond that. All of these concepts are the same across all our platforms that support Safari Web Extensions. To see this in action on macOS and get a closer look at updating an extension to use activeTab, check out Meet Safari Web Extensions from WWDC 2020. And that's the privacy preserving permissions model of Safari Web Extensions. Along with everything else I've covered today, I hope this gives you a more complete view of Safari Web Extensions on iOS and the tools available to you for building them. We're so excited to see the Web Extension experiences that will be brought to Safari on iOS 15, including the many extensions we already love on the Mac today but also the great new extensions that are yet to come. So if you're new to extension development, I encourage you to check out the linked resources associated with this session. There, you'll find sample code that you can download and try out for yourself. And you can read more about the Web Extension APIs available to you in Apple's developer documentation and the MDN Web Docs. If you already develop a Web Extension for another browser or for Safari on the Mac, I encourage you to try out the Safari Web Extension Converter in Xcode 13 to easily bring your extension to iOS. If you have feedback like bug reports, suggestions, or compatibility issues, I encourage you to send us that feedback at feedbackassistant.apple.com. You can also reach out to us at the Apple Developer Forums if you'd like to get in touch or have questions. And finally, I'd like to again recommend two other sessions you may be interested in. Explore Safari Web Extension Improvements will teach you about some recent API additions to Safari Web Extensions. And Design for Safari 15 will show you how to make your web content look great with Safari's new design in iOS 15. Thanks for watching, and have a great WWDC. [upbeat music]