com.apple.developer.endpoint-security.client in python executable

Question

Created Sep ’20

Replies 7

Boosts 0

Views 1.5k

Participants 2

I have a .dylib that is creating an endpoint security client using esnewclient() and subscribes to few events.
I linked my console app with that .dylib and given com.apple.developer.endpoint-security.client.
The console app loads fine with SIP disabled and esnewclient() is created and works successfully.

I have another console app that is an executable created with pyinstaller (compiled python code).
If I try to load/link the endpoint security client .dylib into that python-compiled app, it is killed by the system with "Invalid signature" in the crash log. I verified endpoint-security, app, group etc entitlements and they are correctly set. But the app can not launch.
I wonder if there are special requirements that python executable can not satisfy to be entitled as endpoint-security.client?

Boost

Answer 1

DTS Engineer OP

Apple

Sep ’20

Does your Python executable have the hardened runtime enabled? That’s required for ES clients.

Does your Python executable use any hardened runtime exception entitlements? Those are not allowed in ES clients.

Beyond that, it’s hard to say. Normally when the system refuse to run a process it will leave a clue as to what the problem is in the system log, but that can be hard to find. If you can’t track it down, open a DTS tech support incident and I can take a look.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"

1

Answer 2

vachooho OP

Oct ’20

Thanks, Quinn!

Yes, the executable is correctly entitled, has hardened runtime and no exceptions.

Here is what I see in the console when i try to execute it from the terminal:

Code Block amfid 	/Library/Application Support/Path/to/executable signature not valid: -67050
kernel	proc 24873: load code signature error 4 for file “executable"
kernel	ASP: Security policy would not allow process: 24873, /Library/Application Support/Path/to/executable

And the rest is a crash report generation which contains only this:

Code Block Process:               executable [24873]
Path:                  /Library/Application Support/Path/*/executable
Identifier:            executable
Version:               ???
Code Type:             X86-64 (Native)
Parent Process:        ??? [24872]
Responsible:           Terminal [20782]
User ID:               0
Date/Time:             2020-10-09 19:38:06.356 -0700
OS Version:            Mac OS X 10.16 (20A5384c)
Report Version:        12
Anonymous UUID:        BE0CE7B3-DC87-D815-8080-E239397792B1
Sleep/Wake UUID:       4FC733F9-E1AF-4E83-A763-D37E3DB3757C
Time Awake Since Boot: 97000 seconds
System Integrity Protection: enabled
Crashed Thread:        Unknown
Exception Type:        EXC_CRASH (Code Signature Invalid)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY
Termination Reason:    Namespace CODESIGNING, Code 0x1
kernel messages:
Backtrace not available
Unknown thread crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x0000000000000000  rcx: 0x0000000000000000  rdx: 0x0000000000000000
  rdi: 0x0000000000000000  rsi: 0x0000000000000000  rbp: 0x0000000000000000  rsp: 0x00007ffedfe50c30
   r8: 0x0000000000000000   r9: 0x0000000000000000  r10: 0x0000000000000000  r11: 0x0000000000000000
  r12: 0x0000000000000000  r13: 0x0000000000000000  r14: 0x0000000000000000  r15: 0x0000000000000000
  rip: 0x000000011699e000  rfl: 0x0000000000000200  cr2: 0x0000000000000000
  
Logical CPU:     0
Error Code:      0x00000000
Trap Number:     0
Unknown thread instruction stream not available.
Unknown thread last branch register state not available.
Binary images description not available
External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 6832793
    thread_create: 0
    thread_set_state: 0

I believe the following has some relevance to the issue.
The executable is built and signed using macOS 10.11 SDK with minimum supported version macOS 10.7, so the signature contains both SHA-1 and SHA-256 entries.
There are 2 processes launched from the binary, not sure how python implemented that, but the parent process might be fork()-ing itself in order to run interpreter or something like that (
Process: executable [24873]
Parent Process: ??? [24872]
Responsible: Terminal [20782]). and that's the child process is being killed.
Could that be an issue ? And if so, is there any way to make OS accept it for launch?

Any input greatly appreciated!

0

Answer 3

vachooho OP

Oct ’20

I have tried this on macOS 11.0 Beta 9 but the same behaviour is observed on 10.15.6 Catalina as well.

0

Answer 4

DTS Engineer OP

Apple

Oct ’20

Are you trying to run this executable directly from Terminal? That won’t work, even for native code, because the process inherits a whole bunch of context from the GUI login session that Terminal is running in. An ES client must be started either as a launchd daemon or be packaged as a system extension.

In the former case, make sure to read Packaging a Daemon with a Provisioning Profile.

Frankly, I think it’d behoove you to get this working from native code first, then try adding Python into the mix.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"

0

Answer 5

vachooho OP

Oct ’20

Yes I am running it from Terminal with SIP disabled.
And if the container is written in C/C++/Obj-C, everything works.
However if the container is a compiled python tool then it doesn't even get to run anything.
It is killed on start-up!
When i removed "endpoint-security" entitlement, the python tool is running but (apparently because of the lack of the entitlement) is failing to create es_client.

Any ideas?

0

Answer 6

DTS Engineer OP

Apple

Oct ’20

Yes I am running it from Terminal with SIP disabled.

This isn’t a supported approach. Your ES client must be started by the system, either as an ES sysex or as a launchd daemon. In the latter case, you need to put it in app-like packaging, per my previous post.

If your code continues to crash when started in a supported fashion, please post the resulting crash report (use the paperclip icon to attach it as a text attachment) and I’ll take a deeper look.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"

0

Answer 7

vachooho OP

Nov ’20

Thanks Quinn,

Here is the crash log for python compiled and packaged as a daemon according to your other post https://developer.apple.com/forums/thread/129596 ES client.

HelloWorld_2020-11-23-113814_my-Mac.crash

Process:               HelloWorld [37170]
Path:                  /Applications/HelloWorld.app/Contents/MacOS/HelloWorld
Identifier:            HelloWorld
Version:               ???
Code Type:             X86-64 (Native)
Parent Process:        ??? [37168]
User ID:               0

Date/Time:             2020-11-23 11:38:13.166 -0800
OS Version:            macOS 11.0.1 (20B28)
Report Version:        12
Anonymous UUID:        BE0CE7B3-DC86-D815-8000-E239494692B1

Sleep/Wake UUID:       CE95E52E-3F0B-4D7B-851D-A685362DB063

Time Awake Since Boot: 850000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (Code Signature Invalid)
Exception Codes:       0x0000000000000032, 0x000000011019b000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Reason:    Namespace CODESIGNING, Code 0x2

kernel messages:

VM Regions Near 0x11019b000:
    MALLOC metadata             11019a000-11019b000    [    4K] r--/rwx SM=PRV  
--> mapped file                 11019b000-11019c000    [    4K] r--/r-- SM=PRV  Object_id=c1b35dd9
    __TEXT                      114ec8000-114f64000    [  624K] r-x/r-x SM=COW  /usr/lib/dyld

Application Specific Information:
dyld: in dlopen()
/tmp/_MEIahGl8B/.Python

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   dyld                          	0x0000000114f20090 bcmp + 208
1   dyld                          	0x0000000114ee3294 ImageLoaderMachO::validateFirstPages(linkedit_data_command const*, int, unsigned char const*, unsigned long, long long, ImageLoader::LinkContext const&) + 104
2   dyld                          	0x0000000114ee9826 ImageLoaderMachOCompressed::instantiateFromFile(char const*, int, unsigned char const*, unsigned long, unsigned long long, unsigned long long, stat const&, unsigned int, unsigned int, linkedit_data_command const*, encryption_info_command const*, ImageLoader::LinkContext const&) + 310
3   dyld                          	0x0000000114ee2320 ImageLoaderMachO::instantiateFromFile(char const*, int, unsigned char const*, unsigned long, unsigned long long, unsigned long long, stat const&, ImageLoader::LinkContext const&) + 158
4   dyld                          	0x0000000114ecbeeb dyld::loadPhase6(int, stat const&, char const*, dyld::LoadContext const&) + 781
5   dyld                          	0x0000000114ed4a64 dyld::loadPhase5(char const*, char const*, dyld::LoadContext const&, unsigned int&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) + 1419
6   dyld                          	0x0000000114ed4477 dyld::loadPhase4(char const*, char const*, dyld::LoadContext const&, unsigned int&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) + 185
7   dyld                          	0x0000000114ed41f2 dyld::loadPhase3(char const*, char const*, dyld::LoadContext const&, unsigned int&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) + 1434
8   dyld                          	0x0000000114ed3845 dyld::loadPhase1(char const*, char const*, dyld::LoadContext const&, unsigned int&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) + 186
9   dyld                          	0x0000000114ecba4e dyld::loadPhase0(char const*, char const*, dyld::LoadContext const&, unsigned int&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) + 487
10  dyld                          	0x0000000114ecb5d2 dyld::load(char const*, dyld::LoadContext const&, unsigned int&) + 185
11  dyld                          	0x0000000114ed94ab dlopen_internal + 419
12  libdyld.dylib                 	0x00007fff20331fd0 dlopen_internal(char const*, int, void*) + 177
13  libdyld.dylib                 	0x00007fff2032087e dlopen + 28
14  HelloWorld                    	0x000000010ff4d879 0x10ff4a000 + 14457
15  HelloWorld                    	0x000000010ff4c6d3 0x10ff4a000 + 9939
16  HelloWorld                    	0x000000010ff4ca6a 0x10ff4a000 + 10858
17  HelloWorld                    	0x000000010ff4b2d4 0x10ff4a000 + 4820

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x000000011019b000  rcx: 0x0000000000000000  rdx: 0x0000000000000780
  rdi: 0x000000011019b000  rsi: 0x00007ffedfcabea0  rbp: 0x00007ffedfcab870  rsp: 0x00007ffedfcab870
   r8: 0x0000000000000003   r9: 0x0000000000000000  r10: 0x0000000000040001  r11: 0x0000000000000202
  r12: 0x0000000000000000  r13: 0x0000000000000003  r14: 0x0000000000000780  r15: 0x00007fdca54056e0
  rip: 0x0000000114f20090  rfl: 0x0000000000010246  cr2: 0x000000011019b000
  
Logical CPU:     1
Error Code:      0x00000004 (no mapping for user data read)
Trap Number:     14

Thread 0 instruction stream:
  0c 0e 66 0f 74 c1 66 0f-d7 c0 3d ff ff 00 00 75  ..f.t.f...=....u
  7a 48 83 c1 10 4c 39 c9-75 de 4c 89 d2 4c 01 cf  zH...L9.u.L..L..
  4c 01 ce 48 31 c9 48 0f-b6 07 4c 0f b6 06 4c 29  L..H1.H...L...L)
  c0 0f 85 91 00 00 00 48-ff c7 48 ff c6 48 83 ea  .......H..H..H..
  01 0f 84 81 00 00 00 48-f7 c7 0f 00 00 00 75 d6  .......H......u.
  49 c7 c1 f0 0f 00 00 eb-97 0f 1f 80 00 00 00 00  I...............
 [66]0f 6f 04 0f f3 0f 6f-0c 0e 66 0f 74 c1 66 0f  f.o....o..f.t.f.	<==
  d7 c0 3d ff ff 00 00 75-12 48 83 c1 10 48 39 d1  ..=....u.H...H9.
  72 de 48 31 c0 66 0f ef-c9 5d c3 f7 d0 0f bc c0  r.H1.f...]......
  48 01 c1 48 0f b6 04 0f-4c 0f b6 04 0e 4c 29 c0  H..H....L....L).
  4d 31 c0 48 39 d1 49 0f-43 c0 66 0f ef c9 5d c3  M1.H9.I.C.f...].
  48 0f b6 04 0f 4c 0f b6-04 0e 4c 29 c0 75 09 48  H....L....L).u.H
  
Thread 0 last branch register state not available.


Binary Images:
       0x10ff4a000 -        0x10ff50fff +HelloWorld (???) <E8DCD841-9F46-32C5-A82F-1A12CA97E1BD> /Applications/HelloWorld.app/Contents/MacOS/HelloWorld
       0x114ec8000 -        0x114f63fff  dyld (832.7.1) <2705F0D8-C104-3DE9-BEB5-B1EF6E28656D> /usr/lib/dyld
    0x7fff2004e000 -     0x7fff2004ffff  libsystem_blocks.dylib (78) <83E2F888-F4C1-3181-B21B-136A0D7293DD> /usr/lib/system/libsystem_blocks.dylib
    0x7fff20050000 -     0x7fff20085fff  libxpc.dylib (2038.40.38) <DC50F7DD-1C4B-3BD2-A54D-C5D7D97A3718> /usr/lib/system/libxpc.dylib
    0x7fff20086000 -     0x7fff2009dfff  libsystem_trace.dylib (1277.50.1) <94196441-5456-342D-84D1-C60A9249D273> /usr/lib/system/libsystem_trace.dylib
    0x7fff2009e000 -     0x7fff20139fff  libcorecrypto.dylib (1000.40.21) <2FA5C1BD-F41C-3FCF-B1FA-7BED6E19ABEE> /usr/lib/system/libcorecrypto.dylib
    0x7fff2013a000 -     0x7fff20166fff  libsystem_malloc.dylib (317.40.8) <3688DBCE-2FFD-3F98-B12B-69ABE231908D> /usr/lib/system/libsystem_malloc.dylib
    0x7fff20167000 -     0x7fff201abfff  libdispatch.dylib (1271.40.12) <C55547DC-A05B-34A1-BD41-D54948D8F57F> /usr/lib/system/libdispatch.dylib
    0x7fff201ac000 -     0x7fff201e4fff  libobjc.A.dylib (818.2) <DD09F187-80F1-308F-9395-912F27E9A390> /usr/lib/libobjc.A.dylib
    0x7fff201e5000 -     0x7fff201e7fff  libsystem_featureflags.dylib (28.40.2) <3E1E8F98-E791-301D-909F-4CA205C94D1A> /usr/lib/system/libsystem_featureflags.dylib
    0x7fff201e8000 -     0x7fff20270fff  libsystem_c.dylib (1439.40.11) <8D791F3A-4848-302F-88CD-46D91DE73127> /usr/lib/system/libsystem_c.dylib
    0x7fff20271000 -     0x7fff202c6fff  libc++.1.dylib (904.4) <E7F465AC-C8FF-31DB-8EC4-7EBBA7468EB5> /usr/lib/libc++.1.dylib
    0x7fff202c7000 -     0x7fff202dffff  libc++abi.dylib (904.4) <253848B5-ED35-3202-8593-5DCC763A8192> /usr/lib/libc++abi.dylib
    0x7fff202e0000 -     0x7fff2030efff  libsystem_kernel.dylib (7195.50.7) <41068F5C-74E3-3C98-9256-6A18364FB5BA> /usr/lib/system/libsystem_kernel.dylib
    0x7fff2030f000 -     0x7fff2031afff  libsystem_pthread.dylib (454.40.3) <78072EC6-2257-361A-AAF5-4A3C1832B5EB> /usr/lib/system/libsystem_pthread.dylib
    0x7fff2031b000 -     0x7fff20355fff  libdyld.dylib (832.7.1) <0C66AB9F-E22C-3286-B76B-DA4008698CD2> /usr/lib/system/libdyld.dylib
    0x7fff20356000 -     0x7fff2035ffff  libsystem_platform.dylib (254.40.4) <6C05D209-129A-373A-A90D-CC27AB18C091> /usr/lib/system/libsystem_platform.dylib
    0x7fff20360000 -     0x7fff2038bfff  libsystem_info.dylib (542.40.3) <90DDEF9D-9DBA-3599-A9DD-A7F730AE945F> /usr/lib/system/libsystem_info.dylib
    0x7fff22774000 -     0x7fff2277dfff  libsystem_darwin.dylib (1439.40.11) <3BD9046F-FAED-3FD9-8191-49A4618555A3> /usr/lib/system/libsystem_darwin.dylib
    0x7fff22b8e000 -     0x7fff22b99fff  libsystem_notify.dylib (279.40.4) <73D358B2-6C2D-33DF-975C-BE9EE5F45A0F> /usr/lib/system/libsystem_notify.dylib
    0x7fff24ae3000 -     0x7fff24af1fff  libsystem_networkextension.dylib (1295.40.29) <84A5011E-1511-30C7-86D8-514CFA59EF42> /usr/lib/system/libsystem_networkextension.dylib
    0x7fff24b4f000 -     0x7fff24b65fff  libsystem_asl.dylib (385) <EB8BA94D-1B45-322E-9D26-272872423807> /usr/lib/system/libsystem_asl.dylib
    0x7fff26245000 -     0x7fff2624cfff  libsystem_symptoms.dylib (1431.40.36) <81345ED1-342F-347F-B979-39846E607C8C> /usr/lib/system/libsystem_symptoms.dylib
    0x7fff2828c000 -     0x7fff2829cfff  libsystem_containermanager.dylib (318.41.1) <BF178091-2151-3462-AB4F-A2E111650BA2> /usr/lib/system/libsystem_containermanager.dylib
    0x7fff28f9b000 -     0x7fff28f9efff  libsystem_configuration.dylib (1109.40.9) <EA408233-7EA5-33B5-B099-E9609DD3A0FF> /usr/lib/system/libsystem_configuration.dylib
    0x7fff28f9f000 -     0x7fff28fa3fff  libsystem_sandbox.dylib (1441.40.38) <349F71A1-4C7C-3AE0-8A1B-B95CCBB2327F> /usr/lib/system/libsystem_sandbox.dylib
    0x7fff29ca6000 -     0x7fff29ca8fff  libquarantine.dylib (119.40.2) <3DD7F9EE-9B9A-3896-A8CC-4D5BE49385AE> /usr/lib/system/libquarantine.dylib
    0x7fff2a227000 -     0x7fff2a22bfff  libsystem_coreservices.dylib (127) <670DDFEB-8100-3C51-BC96-76BD46A21DF6> /usr/lib/system/libsystem_coreservices.dylib
    0x7fff2a42f000 -     0x7fff2a441fff  libz.1.dylib (76) <0AF4F195-1F20-3F59-B09F-31E7249890D5> /usr/lib/libz.1.dylib
    0x7fff2a442000 -     0x7fff2a489fff  libsystem_m.dylib (3186.40.2) <0276200D-C6F2-3E79-AFDC-E4D7F60A7837> /usr/lib/system/libsystem_m.dylib
    0x7fff2a48b000 -     0x7fff2a490fff  libmacho.dylib (973.4) <F2E3F7A6-B15A-30ED-A6AD-4EFEC37CF3E7> /usr/lib/system/libmacho.dylib
    0x7fff2a4ad000 -     0x7fff2a4b8fff  libcommonCrypto.dylib (60178.40.2) <66E10FDF-896D-35F4-B9A8-8EE9400AC23A> /usr/lib/system/libcommonCrypto.dylib
    0x7fff2a4b9000 -     0x7fff2a4c3fff  libunwind.dylib (200.10) <BD13DE19-15F8-3982-ACCB-CD8C036D5D2B> /usr/lib/system/libunwind.dylib
    0x7fff2a4c4000 -     0x7fff2a4cbfff  liboah.dylib (203.1.3) <5EC28751-3BE8-3BB7-9DBE-305225CBD41B> /usr/lib/liboah.dylib
    0x7fff2a4cc000 -     0x7fff2a4d6fff  libcopyfile.dylib (173.40.2) <17A9CD2E-0527-3FF7-9CFA-57B30133DFB5> /usr/lib/system/libcopyfile.dylib
    0x7fff2a4d7000 -     0x7fff2a4defff  libcompiler_rt.dylib (102.2) <FE958653-7FEC-3ADD-9AC0-A386E47208CF> /usr/lib/system/libcompiler_rt.dylib
    0x7fff2a4df000 -     0x7fff2a4e1fff  libsystem_collections.dylib (1439.40.11) <31C10DA5-F9B8-36E4-B859-8F6B845FF65D> /usr/lib/system/libsystem_collections.dylib
    0x7fff2a4e2000 -     0x7fff2a4e4fff  libsystem_secinit.dylib (87.40.10) <BB2A70F5-BECC-34BD-90FE-14D31C5FD80D> /usr/lib/system/libsystem_secinit.dylib
    0x7fff2a4e5000 -     0x7fff2a4e7fff  libremovefile.dylib (49.40.3) <24CE2291-AEB3-30C9-94E8-535AA6E9C6F2> /usr/lib/system/libremovefile.dylib
    0x7fff2a4e8000 -     0x7fff2a4e8fff  libkeymgr.dylib (31) <B47F66C0-27F4-3B77-BAA0-85E895A7421D> /usr/lib/system/libkeymgr.dylib
    0x7fff2a4e9000 -     0x7fff2a4f0fff  libsystem_dnssd.dylib (1310.40.42) <F54EC6FB-DEC3-3DA8-A749-4892F50D1F08> /usr/lib/system/libsystem_dnssd.dylib
    0x7fff2a4f1000 -     0x7fff2a4f6fff  libcache.dylib (83) <C83C4423-D12A-32A0-875D-81450DF40F42> /usr/lib/system/libcache.dylib
    0x7fff2a4f7000 -     0x7fff2a4f8fff  libSystem.B.dylib (1292.50.1) <14E3BB7B-B95F-3700-8D12-B9B2012C0AF8> /usr/lib/libSystem.B.dylib
    0x7fff2d922000 -     0x7fff2d922fff  liblaunch.dylib (2038.40.38) <D71781B5-4034-3349-BBE5-4B796CD3629A> /usr/lib/system/liblaunch.dylib
    0x7fff2fdc0000 -     0x7fff2fdc0fff  libsystem_product_info_filter.dylib (8.40.1) <FF59B842-1290-3C4A-9015-AC972563406E> /usr/lib/system/libsystem_product_info_filter.dylib

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 67898733
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=491.0M resident=0K(0%) swapped_out_or_unallocated=491.0M(100%)
Writable regions: Total=17.3M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=17.3M(100%)
 
                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Kernel Alloc Once                    8K        1 
MALLOC                            9256K        7 
MALLOC guard page                   16K        4 
STACK GUARD                       56.0M        1 
Stack                             8192K        1 
__DATA                             409K       45 
__DATA_CONST                       242K       34 
__DATA_DIRTY                        58K       21 
__LINKEDIT                       486.0M        4 
__OBJC_RO                         61.0M        1 
__OBJC_RW                         2468K        2 
__TEXT                            5080K       45 
mapped file                          4K        1 
shared memory                        4K        1 
===========                     =======  ======= 
TOTAL                            628.2M      168

If I remove the endpoint-security entitlement then it loads without crash but apparently is not able to create ES client.

0