Sandbox activated macOS application crashes immediately after execution

Question

I want to submit my python based application to apple store, but whenever I activate sandboxing for my application(no matter the content of code, even just “hello world” printing crashes),

It crashes in a couple of sec. with following console error after execution, Although it works fine without sandbox activation :

“Sandbox: test(8409) deny(1) forbidden-sandbox-reinit”

1-Requirments:

macOS Catalina 10.15.4
Python 3.6
PyInstaller 4.0
Certification:Developer ID Application
Manual(without Xcode): build, signing, notarizing and sandboxing

2.Implementing Simple Code Ex(test.py):(No matter which code content is used, all crash)

test.py

from AppKit import NSOpenPanel
from objc import YES, NO
panel = NSOpenPanel.openPanel()
panel.setTitle("open file")
panel.setAllowsMultipleSelection(NO)
panel.setCanChooseDirectories_(YES)
panel.runModal()

3.Creating test.spec:

test.spec

blockcipher = None
a = Analysis(['test.py'],
   pathex=['/Users/Emre/Documents/Work/models/research/objectdetection'],
   binaries=[],
   datas=[],
   hiddenimports=[],
   hookspath=[],
   runtimehooks=[],
   excludes=[],
   winnopreferredirects=False,
   winprivateassemblies=False,
   cipher=blockcipher)
pyz = PYZ(a.pure, a.zippeddata,
   cipher=blockcipher)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
name='test',
debug=False,
strip=False,
upx=True,
runtimetmpdir=None,
console=False )
app = BUNDLE(exe,
   name='test.app',
   icon=None,
   bundle_identifier=None)

4.Creating .App File:

python3.6 -m PyInstaller -F test.spec

5.Creating Entitlements(Entitlements.plist):

Entitlements.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-url(not allowed to be added in developer forum)">
<plist version="1.0">
<dict>
<key>com.apple.security.app-sandbox</key>
<true/>
</dict>
</plist>

6.Codesiging Manually:

codesign --force --options=runtime --sign "Developer ID Application: Emre (xxxx)” test.app --entitlements Entitlements.plist

7.Generate dmg file via disk utility for Notarizing from test.py:

-

8.Notarizing the application:

xcrun altool --notarize-app --primary-bundle-id "com.emre" --username "emre@gmail.com" --password "@keychain:AppSpecPass" --file /Users/Emre/Documents/Work/models/research/objectdetection/dist/test.dmg

9.Verification Of Certificate and sandboxing:
spctl -a -t exec -vv test.app
test.app: accepted
source=Notarized Developer ID
origin=Developer ID Application: Emre Guenaydin (93DS7PC26P)

codesign --verify --deep --strict --verbose=2 test.app
test.app: valid on disk
test.app: satisfies its Designated Requirement

codesign -dvvv --entitlements :- test.app
Executable=/Users/Emre/Documents/Work/models/research/objectdetection/dist/untitled folder/test.app/Contents/MacOS/test
Identifier=test
Format=app bundle with Mach-O thin (x86_64)
CodeDirectory v=20500 size=73712 flags=0x10000(runtime) hashes=2295+5 location=embedded
Hash type=sha256 size=32
CandidateCDHash sha1=e3716fc5262107748b2658e4cbbcee97670268c1
CandidateCDHashFull sha1=e3716fc5262107748b2658e4cbbcee97670268c1
CandidateCDHash sha256=6fe01092e170231a9fdc855871ce3161b71b8af5
CandidateCDHashFull sha256=6fe01092e170231a9fdc855871ce3161b71b8af597cd78410dcea3b80092e5c0
Hash choices=sha1,sha256
CMSDigest=cf0a13051e77eba16609e67ca580b24d2698becad591a45fac911d44ddbef0b6
CMSDigestType=2
CDHash=6fe01092e170231a9fdc855871ce3161b71b8af5
Signature size=9058
Authority=Developer ID Application: Emre (***)
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=20. Sep 2020 at 23:40:39
Info.plist entries=8
TeamIdentifier=***
Runtime Version=10.11.0
Sealed Resources version=2 rules=13 files=1
Internal requirements count=1 size=164
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-url(not allowed to be commented in developer forum)">
<plist version="1.0">
<dict>
<key>com.apple.security.app-sandbox</key>
<true/>
</dict>
</plist>

10.Execute .app file:

Crashes with following console report follow:
default 23:45:08.671813+0200 secinitd test[8055]: AppSandbox request successful
default 23:45:09.275405+0200 secinitd test[8056]: AppSandbox request successful
error 23:45:09.504553+0200 sandboxd Failed to produce a full report for: test[8056].
error 23:45:09.504643+0200 sandboxd Sandbox: test(8056) deny(1) forbidden-sandbox-reinit

3.1k

Posted by

EmreGun

Reply

Add a Comment

Answer 1

Does this generate a crash report? If so, please post it here. Use the text attachment feature to avoid clogging up the log.

ps Does the problem reproduce if you skip steps 7 through 9? I think it will and, if so, you can skip those steps while testing this. Those steps are important when you finally deploy your app, but you can improve you turnaround time by skipping them whilst debugging.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"

Posted by

eskimo

Add a Comment

Answer 2

I want to submit my python based application to apple store

OK. Wait a second. Let's focus on that. Do you really want to do that? Apparently this is something that people have been trying for years and haven't gotten it working. See github.com/pyinstaller/pyinstaller/issues/2198

Also, you are trying to Notarize with a Developer ID too? Are you aware that has nothing to do with the Mac App Store? I don't know if trying to build a Notarized Developer ID version would be any easier. It may just give you a completely different set of problems. But you definitely don't want to try both. For one, it simply isn't possible. For another, you may just confuse yourself. Mac App Store and Developer ID are two separate tasks. Apple does try to get people to sandbox their Developer ID apps, but that is a different question and not something you should probably be attempting at this point.

That GitHub link contains a reference to this person getting something close working: medium.com/python-pandemonium/embedding-a-python-application-in-macos-d866adfcaf94

However, in this case, the developer has generic, Objective-C Mac app running and is then using Python as the back end. That is a much more manageable problem.

Posted by

Etresoft

Add a Comment

Answer 3

@Etresoft:Thank you very much for your hints. Theys were really helpful to understand the problem in details.
Answers are below for your questions:

1-Do you really want to do that?
Unfortunately, yes! Before I wasted several days to create my .app file via Pyinstaller, I was not aware of this sandboxing problem.
Therefore I want to research a couple of days more to solve the problem.

2-you are trying to Notarize with a Developer ID too?
Actually It works fine somehow :) you can see in 9.Verification Of Certificate and sandboxing. But you can just ignore step9.
When I was debugging my sandboxing problem, I saw that “spctl -a -t exec -vv test.app” command was throwing “rejected” error, therefore I thought that my problem could be relevant with this rejection. But apparently it is not :)

3-That GitHub link contains a reference to this person getting something close working
I guess my problem is a similar one which you mentioned in your second link.
During my researches, I found following description in Pyinstaller tutorial:

PyInstaller Tutorial Bootloader Section

So I had a look into my activity monitor(test.app without sandboxing, since sandboxed application crashes immediately), and I saw that they are 2 processes indeed:

activity monitor view

Process id 8049 is parent of 8048. Could it be that I have to (somehow) inherit from my application itself for code signing?(<key>com.apple.security.inherit</key>) Maybe a similar situation as it is mentioned in your second link? Now "“Sandbox: test(8409) deny(1) forbidden-sandbox-reinit” error makes also sense.

I tried to codesign my test.py with following inheritance entitlements:

entitlements

But unfortunately I got following crash report:

crash report

Do you have any idea?
Thank you very much in advance

Posted by

EmreGun

Add a Comment

Answer 4

@ eskimo:Thank you very much for your response.
Your answers are below:

1-Does this generate a crash report?
you can see my crash reports below:

Crash Report1

Crash Report2

2-Does the problem reproduce if you skip steps 7 through 9?
Etresoft has also mentioned it. I guess it was a mistake to add these steps to my question :) you can just ignore step9.
When I was debugging my sandboxing problem, I saw that “spctl -a -t exec -vv test.app” command was throwing “rejected” error, therefore I thought that my problem could be relevant with this rejection. But apparently it is not.

Thanks a lot in advance.

Posted by

EmreGun

Add a Comment

Answer 5

you can see my crash reports below:

That’s what I was looking for.

Etresoft has pointed you in the right direction here but I want to explain some of the backstory. All of the following is relative to Crash Report1.crash.

To start, consider:

Code Block  Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)

This indicates that your process died because it tried to execute an illegal instruction. In some cases this is caused by your code running off into the weeds but in this case it’s clear that this is a deliberate crash. Code running within your process has detected a problem and run an illegal instruction to cause the process to terminate.

You can see who did this by looking at the crashing thread backtrace:

Code Block  Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_secinit.dylib … _libsecinit_appsandbox.cold.2 + 95
1   libsystem_secinit.dylib … _libsecinit_appsandbox + 1511
2   libsystem_secinit.dylib … _libsecinit_initializer + 35
3   libSystem.B.dylib       … libSystem_initializer + 268
4   dyld                    … ImageLoaderMachO::doModInitFunctions(ImageLoader::LinkContext const&) + 535
5   dyld                    … ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) + 40
6   dyld                    … ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned in…
…
16  dyld                    … ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned in…
17  dyld                    … ImageLoader::processInitializers(ImageLoader::LinkContext const&, unsigned int, I…
18  dyld                    … ImageLoader::runInitializers(ImageLoader::LinkContext const&, ImageLoader::Initia…
19  dyld                    … dyld::initializeMainExecutable() + 199
20  dyld                    … dyld::_main(macho_header const*, unsigned long, int, char const, char const, …
21  dyld                    … dyldbootstrap::start(dyld3::MachOLoaded const*, int, char const**, dyld3::MachOLo…
22  dyld                    … _dyld_start + 37

This is very early in the startup sequence for your process. The dynamic linker (frames 22 through 4) is in the middle of linking together your code. As part of this it runs the module initialiser for the System framework (frame 3). That calls the initialiser for the security subsystem (frame 2). That detects that your process is sandbox and attempts to set up the App Sandbox (frames 1 through 0). That setup operation has failed, at which point the code has no choice but to crash your process (it can’t let it continue without a sandbox).

Both dyld and the security subsystem have left breadcrumbs in the Application Specific Information section to explain the problem:

Code Block  dyld: launch, running initializers
/usr/lib/libSystem.B.dylib
Could not set sandbox profile data: Operation not permitted (1)
Application Specific Signatures:
SYSCALL_SET_PROFILE

You can see that the system call to set up the sandbox has failed with EPERM. As to why that’s happened, this is again something that Etresoft pointed to. Consider this:

Code Block  Process:               test [8519]

and this:

Code Block  Parent Process:        ??? [8518]

Normally an app’s parent process is launchd. In this case the process IDs are sequential, suggesting that the app has relaunched itself. Doing this is incompatible with the App Sandbox.

A process can set up the App Sandbox in one of two ways:

From scratch, by setting the com.apple.security.app-sandbox — In this case the process is expected to be launched by a non-sandboxed process, like launchd. This is how apps and app extensions work.
By inheriting it from its parent, by setting com.apple.security.app-sandbox and com.apple.security.inherit — If you embed a helper tool within your app, this is how you set it up to inherit the app’s sandbox [1].

Your process seems to be trying to combine these, that is, it’s signed like a normal app but it’s trying to inherit it sandbox from its parent. This won’t work because the system tries to re-apply the sandbox and that is not allowed (hence the EPERM).

It seems that you’ve inherited (so to speak) this approach from PyInstaller. If so, you’ll need to discuss this issue with them. The current approach they’re using is fundamentally incompatible with the App Sandbox.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"

[1] A child process will inherit its parent sandbox even if it has no entitlements. These entitlements make that inheritance explicit, primarily for the benefit of the App Store ingestion process.

Posted by

eskimo

Add a Comment

Answer 6

@eskimo

Thank you very much for your fast and detailed response. Now it is clear to me, what is happening.
I will try a couple of hours to find a way to prevent that pyinstaller app calls (inherits) itself,
If I cannot find a solution then I will try further with py2app tool instead of pyinstaller to bundle my application.

Thank you very much for your help.

Posted by

EmreGun

Add a Comment

Sandbox activated macOS application crashes immediately after execution

Replies