command-line appstore app and sandbox: accessing user files

Are you using Security Scoped Bookmarks? That is how you handle this in an XPC context. It should be the same for a helper app.

Thanks for your reply. The command-line app is seperate, not a helper app. As an example, think of the app 'handbrake' to encode videos. There is a graphical part, but also a independent command-line version for batch processing. So you basically just want to run from the terminal:

mycommandlineapp [options] [file ...]

(for files that have not been granted access before in the GUI) but access to files is prevented by the sandboxing required by the App-store. As far as I know, Security Scoped Bookmarks are used for persistent access to files once you already have acces granted via the NSopenPanel. But for commandline batch processing, displaying a dialog for file-access is not an option.

The fact that the user has granted permission should ideally be directly following from the fact that the user explicitely typed in the file-names, but I guess no such mechanism or API exists.

One ugly option is to have an entitlement for access to the 'Downloads'-folder (use 'com.apple.security.files.downloads.read-write') and require the user to first copy the input files over there. But the Downloads folders is not a good match for doing that. Other option is to create a symbol link to the sandbox:

ln -s ~/Library/Containers/Commandline.app/Data ~/Documents/CommandlineData

and required the user to work from that documents directory. Another option, using /tmp for unrestricted access is not allowed with sandboxing.

If you app includes a command-line tool, it is a helper. It may not be a formal XPC task or anything, but it effectively serves the same role. What you do in this case is use the UI to select a file and create the security scoped bookmark. Then you pass that bookmark to the helper - in any way that is convenient. But you have to pass the bookmark, not the file. Then, even without a UI, you can resolve the bookmark. You may need to use application groups or something so that both executables can share files. I believe using a true XPC might help with that. That's why I have the "or something" qualifier. I had apps that did these things but I haven't looked at them in a couple of years.

The sandbox is designed to protect the user from a malicious app. If your UI could just run your helper with an absolute path, the sandbox may as well not exist.

If you wanted to avoid sandbox hassles, an easy way would be to not use the sandbox. Provide a developer ID-signed command-line tool that only works if the App Store app is installed and properly licensed. There are few apps that do that.

You can use a temporary directory for sharing files. You may need to use application groups (or something) so that both processes have access. You shouldn't ever use direct paths. Use NSTemporaryDirectory() instead. Don't use the download folder unless you want to download files. Otherwise that is a hack and those don't end well. To access the download folder, use URLsForDirectory:inDomains: or URLForDirectory:inDomain:appropriateForURL:create:error: NSFileManager methods with NSDownloadsDirectory.

Thanks for the suggestions! I understand the cocoa-main app can be used to grant permission and that these can be store in Security Scoped Bookmarks. But that is not what a normal command-line app does, it just runs by itself and get a list of string arguments, often corresponding to files that need to be read (and initiated/picked by the user). I also understand the reason for sandboxing. But... it overshoots its goals here. For example, the command-line app does not need network-server or client permissons. In that case: what is the harm that a commandline-app, reviewed and checked by apple, reads a file?

The work-around is that the input-files first need to be copied (by hand using the bsd cp-command) to a location that is accessible by the sandbox before the command-line app is run. So NSTemporaryDirectory() is problematic, because that is a directory available in code while you first need to know that user-specific tmp-directory beforehand to put the files there in the first place.

It boils down to how a user can grant permission for file access in a sandboxed app. For Cocoa it is done via NSOpenPanel/NSSavePanel.

For a command-line app one possible implementation would be:

commandline-app [options] [file:///path/to/user/ ...]

for example, suppose the executable is called 'convert' which converts a jpg into tiff-file using the commandline:

convert file:///path/to/user/folder/file.jpg file:///path/to/user/folder/file.tiff

By specifying explicit file-urls the user explicitly selects files (similar to NSOpenPanel and NSSavePanel), the system can check these command-line input-strings for file-url format and grant permission via powerbox. This would then be in line with the entitlement 'com.apple.security.files.user-selected.read-write'.

I was expecting Apple to allow the entitlement 'com.apple.security.temporary-exception.files.home-relative-path.read-only' until such an implementation becomes available for command-line apps.

Perhaps your suggestion to not distribute the command-line tool via the appstore, but via an developer ID-signed command-line tool, is then best.

You can't have a normal command-line app in the Mac App Store. You can only have sandboxed apps. You have to follow the rules of the sandbox. That's just the way it is.

There is a great deal of harm that could be done by reading files. You have to think like Apple. You have a billion users, each with at least $10,000 in credit. That is a lot to steal. You've got about at least a million fraudulent developers hammering at it 24x7 trying to do just that. They would love to be able to silently read user documents and upload them.

Apple doesn't do all that much reviewing and checking. They have some automated checks to look for the relatively honest developer who are merely trying to sell game tokens and avoid Apple's 30% cut. A human reviewer might launch the app and run it for 30 seconds or so just to make sure it runs on the latest OS. They are now also looking for some kind of substantial difference between the app under review and the developer's other 47, virtually identical apps. But that's pretty much it. It is easy to find apps that blatanly ignore every guideline in the book - all the way up to issuing sudo commands. That kind of stuff will make it past reviewers. But using a temporary entitlement will not. That is something they can check with automated tools.

If you use application groups, you can define a shared directory. See developer.apple.com/documentation/foundation/nsfilemanager/1412643-containerurlforsecurityapplicati

I'm don't recommend this method. I'm just pointing it out. Security scoped bookmarks are the way to go.

I suggest you also review why you even have this tool and what you are doing with it. It really isn't worth your time just to give users a command-line option. You are talking about maybe one user out of 1,000, if that. A tool outside of the Mac App Store will solve that problem, but introduce new ones. If this is a tool for your UI to use, then just use security-scoped bookmarks or maybe even do the helper as a true XPC. You can also do something really crude, but effective, like asking the user to open the root of the disk and saving that as a security-scoped bookmark. Presto! Problem solved. I think there is a risk that Apple will disallow that practice at some point, but what do I know about Apple? Anything command-line is going to be for power users, so just don't make the root access a requirement. Tuck it away under "advanced" or something. It is allowed, but frowned upon.

Thanks you so much for your suggestions, I really appreciate it!

I did not realize you could grant permission to "/" using security-scoped bookmarks. Putting that under an advanced option in my preference panel is a very nice and workable solution (it is only a one time action)! The bookmarks can easily be stored in the bundle using groups (as you mentioned). Thanks!!

PS. I understand that malicious apps like to read files to upload them to a server. But I see no harm in a sandbox app that does not have the "client" and "sever" network entitlements. I use e.g. 'homebrew' a lot which provide me with a very rich set of unix utilities, handbrake-cli, etc. Would be even beter if these utils would be available via the appstore.

I assume that Apple does check with automatic tools which Cocoa and library functions are called and that these are consistent with the entitlements you specify. Apple does catch the use of private functions I hear.

Actually, I am a big fan that Apple checks apps, and that can not be thorough enough. But power users should also be allowed to tune their system, and the security-scoped bookmark using "/" is a conscious user decision in my opinion. I hope this access system becomes more refined over time. For example, my command-line app only needs access to specific types of files.

I can virtually guarantee that the access system will become more refined over time. It will inevitably become more restrictive like iOS. One of the restrictions that Apple (supposedly) imposes on the root access technique is that your app must provide functionality even if the user denies that access. Apple really doesn't enforce this restriction, but it is always possible they might change their minds about that one day.

There are many, many different ways to get data into and out of a process than anything that would be covered by client or server network entitlements. Those are entitlements. Developers are entitled to enable if they have a legitimate need. As such, they are not effective in terms of security. When thinking about things from a security perspective, it is only logical to assume that malicious developers will not be honest about their intentions.

Homebrew doesn't even bother to sign their apps. I can't see them doing anything to get into the Mac App Store. Homebrew is just a package manager for open-source software. Most of it, with the notable exception of any GPL software like handbrake, would work fine in the Mac App Store. It would not, however, facilite the import into the sandbox. A fair amount of Homebrew is already installed on your Mac.

Thanks for pointing me to a working solution. I think I got it working. Since it is perhaps slightly non-trivial I will write here the steps that work.

0) setup "group app" and appropriate entitlements, think of:

com.apple.security.application-groups

com.apple.security.files.bookmarks.app-scope

com.apple.security.files.user-selected.read-write

1) In the UI-app, use the NSOpenPanel to obtain a directory-url.

2) Use url.bookmarkData with the option .minimalBookmark and save this data to the user defaults of the group (using UserDefaults(suiteName: "groupname") for example). This will store a regular bookmark in the userdefaults of the group-container.

3) Do NOT close the UI-app yet, but first run the command-line app, read the data from the group-defaults and using the data resolve the url BookmarkData using the option .withoutUI

4) Then, in order to persist the use of the url in the Helper command-line app, create a security scoped url using url.bookmarkData with the option .withSecurityScope

5) Write the data to the local user defaults of the command-line app, and next time, read the data from the user defaults of the command-line app and resolve the url from data with the option .withSecurityScope

6) use: let succeeded: Bool = permissionURL.startAccessingSecurityScopedResource()

The boolean will be true.

So, note an important thing: you can NOT create a security scoped bookmark in the main app, write it to the group, read that from a helper command-line utility and resolve it. You will get: "error: The file couldn’t be opened because it isn’t in the correct format."

But a normal Bookmark works while both programs are running, which gives you a chance to read the normal Bookmark from the group-defaults and write the security scoped Bookmark to the userdefaults of the Helper-app (which needs to be written by the Helper-app, NOT by the main UI-app).

I hope this helps other people.