Develop kernel-resident device drivers and kernel extensions using Kernel.

Posts under Kernel tag

200 Posts

Post

Replies

Boosts

Views

Activity

Using libproc to get ports used by processes on macos
I am trying to get ports used by processes. It can be done via lsof on macOS, i am trying to do it via libproc. #include <iostream> #include <libproc.h> int main(int argc, const char * argv[]) { pid_t pids[3072]; int count = proc_listpids(PROC_ALL_PIDS, 0, pids, sizeof(pids)); for (int i = 0; i < count; i++) { char buffer[1024]; for (int j = 1; j < 50000; j++) { //port range int ret = proc_pidfileportinfo(pids[i], j, PROC_PIDFILEPORTVNODEPATHINFO, buffer, sizeof(buffer)); if(ret != 0) { printf("proc_pidfileportinfo returned %d bytes of data\n", ret); printf("%s\n", name); } } } return 0; } proc_pidfileportinfo function is not working for any port, i tried iterating till 50K. What i am doing wrong with proc_pidfileportinfo? how to properly use proc_pidfileportinfo?
9
0
2.9k
Oct ’23
XCode 15: simple execv program exits with code 5 in the console.
Hello, found the following curious behaviour, If I try to run from within xcode (pressing Run) the following code: #include <unistd.h> int main(int argc, const char * argv[]) { char *args[] = {"/bin/ls", "-r", "-t", "-l", (char *) 0 }; execv(args[0], args); return 0; } the program does not print the expected list of files and folders but instead exits with: Message from debugger: Terminated due to signal 5 Program ended with exit code: 5 But if I try to run the exact same compiled program from the terminal, it works as expected. I lost so many hours wondering what I was doing wrong, but apparently it was the xcode console that does not play nice with execing? Could it be that changing the process image throws a wrench into xcode? Anybody has any idea why this could be? Thanks.
1
0
1k
Oct ’23
Disable peripherals on macOS programmatically
I know with Disk Arbitration framework, I can use DARegisterDiskMountApprovalCallback to prevent external disks from mounting. The disks includes thumb drive, external hard disk, etc., but there are many types of peripherals out there, like a usb wireless receiver or a USB ethernet adapter. Is there any other framework for us to use to enable/disable peripherals based on their I/O Registry properties? Thanks!
4
0
2.1k
Sep ’23
DriverKit crash at 0x1d4d60010 OSMetaClassBase::Invoke(IORPC) (.cold.1) + 44
I created a driver using DriverKit on Intel macOS 12.6.1 and Xcode 13.3. I enabled auto-manage signing, and set the signing certificate to 'Sign to Run Locally'. Then, I created a provision profile for the driver and selected my M1 test device. After installing the profile, I ran the app on the M1 device and successfully activated the driver. When I plugin the USB device, I can see the following log: DK: epusbfilter-0x100009dce::start(IOUSBHostInterface-0x10000946d) ok epusbfilter - init com.injection.epusbfilter.dext[57573] Corpse failure, too many 6 I also found a crash log ------------------------------------- Translated Report (Full Report Below) ------------------------------------- Process: com.injection.epusbfilter.dext [53185] Path: /Library/SystemExtensions/*/com.injection.epusbfilter.dext Identifier: com.injection.epusbfilter.dext Version: 1.0 (1) Code Type: ARM-64 (Native) Parent Process: launchd [1] User ID: 270 Date/Time: 2023-09-19 15:01:01.8502 +0800 OS Version: macOS 13.2 (22D49) Report Version: 12 Anonymous UUID: 5EB7EBD9-A435-FC45-73E6-C2C5844A8082 Time Awake Since Boot: 79000 seconds System Integrity Protection: disabled Crashed Thread: 1 Dispatch queue: Root Exception Type: EXC_CRASH (SIGABRT) Exception Codes: 0x0000000000000000, 0x0000000000000000 Application Specific Information: abort() called Thread 0: 0 libsystem_kernel.dylib 0x1d5043b78 __semwait_signal_nocancel + 8 1 libsystem_c.dylib 0x1d4fcfec8 nanosleep$NOCANCEL + 212 2 libsystem_c.dylib 0x1d4fee204 sleep$NOCANCEL + 48 3 libdispatch.dylib 0x1d4f807b4 _dispatch_queue_cleanup2 + 200 4 libsystem_pthread.dylib 0x1d50fbc50 _pthread_tsd_cleanup + 132 5 libsystem_pthread.dylib 0x1d50f3220 _pthread_exit + 88 6 libsystem_pthread.dylib 0x1d50f4180 pthread_exit + 88 7 libdispatch.dylib 0x1d4f7bbcc dispatch_main + 128 8 DriverKit 0x1d4d33178 DriverExecutableMain + 84 9 dyld 0x104e95e50 start + 2544 Thread 1 Crashed:: Dispatch queue: Root 0 libsystem_kernel.dylib 0x1d5043720 __pthread_kill + 8 1 libsystem_pthread.dylib 0x1d50f40ec pthread_kill + 268 2 libsystem_c.dylib 0x1d5033cac abort + 180 3 DriverKit 0x1d4d5f890 panic + 256 4 DriverKit 0x1d4d5fa60 __assert_rtn + 88 5 DriverKit 0x1d4d60010 OSMetaClassBase::Invoke(IORPC) (.cold.1) + 44 6 DriverKit 0x1d4d32064 OSMetaClassBase::Invoke(IORPC) + 1396 7 DriverKit 0x1d4d32c5c Server(void*, mach_msg_header_t*, mach_msg_header_t*) + 520 8 DriverKit 0x1d4d3b420 uiomessage(void*) + 180 9 DriverKit 0x1d4d34694 uiomachchannel(void*, dispatch_mach_reason_t, dispatch_mach_msg_s*, int) + 380 10 libdispatch.dylib 0x1d4f8868c _dispatch_mach_msg_invoke + 472 11 libdispatch.dylib 0x1d4f74484 _dispatch_lane_serial_drain + 380 12 libdispatch.dylib 0x1d4f89620 _dispatch_mach_invoke + 852 13 libdispatch.dylib 0x1d4f74484 _dispatch_lane_serial_drain + 380 14 libdispatch.dylib 0x1d4f75130 _dispatch_lane_invoke + 436 15 libdispatch.dylib 0x1d4f7640c _dispatch_workloop_invoke + 1784 16 libdispatch.dylib 0x1d4f7ff5c _dispatch_workloop_worker_thread + 652 17 libsystem_pthread.dylib 0x1d50f5024 _pthread_wqthread + 404 18 libsystem_pthread.dylib 0x1d50fc678 start_wqthread + 8 Thread 2: 0 libsystem_pthread.dylib 0x1d50fc670 start_wqthread + 0 Thread 3: 0 libsystem_kernel.dylib 0x1d504401c __sigsuspend_nocancel + 8 1 libdispatch.dylib 0x1d4f808b4 _dispatch_sigsuspend + 48 2 libdispatch.dylib 0x1d4f80884 _dispatch_sig_thread + 56 Thread 1 crashed with ARM Thread State (64-bit): x0: 0x0000000000000000 x1: 0x0000000000000000 x2: 0x0000000000000000 x3: 0x0000000000000000 x4: 0xffffa0016a011948 x5: 0x0000000000000010 x6: 0x00006000010481b0 x7: 0x0000000000000000 x8: 0x725b4b6e56620c88 x9: 0x725b4b6f3d67bc88 x10: 0x00000000000001b0 x11: 0x0000600001048000 x12: 0x0000000000000090 x13: 0x00000000ffffff92 x14: 0x00000000000007fb x15: 0x0000000080636ffb x16: 0x0000000000000148 x17: 0x00000001d7176c60 x18: 0x0000000000000000 x19: 0x0000000000000006 x20: 0x0000000000004003 x21: 0x000000016b05b0e0 x22: 0x0000000000000000 x23: 0x00006000010480e8 x24: 0x0000600001048058 x25: 0xd200fde7d57ecca6 x26: 0x0000000000000085 x27: 0x000060000374c328 x28: 0x0000600001d4c000 fp: 0x000000016b059a90 lr: 0x00000001d50f40ec sp: 0x000000016b059a70 pc: 0x00000001d5043720 cpsr: 0x40001000 far: 0x0000600002c48000 esr: 0x56000080 Address size fault Binary Images: 0x1d503a000 - 0x1d5075fe3 libsystem_kernel.dylib (*) <60df52bd-fc1a-3888-b05b-24b44be3af15> /System/DriverKit/usr/lib/system/libsystem_kernel.dylib 0x1d4fc6000 - 0x1d5039fff libsystem_c.dylib (*) <eee04d9a-7574-3a74-8f4e-cfb05f89f7da> /System/DriverKit/usr/lib/system/libsystem_c.dylib 0x1d4f62000 - 0x1d4fadfff libdispatch.dylib (*) <4e310a5c-9629-305e-a1dd-6632bddd3362> /System/DriverKit/usr/lib/system/libdispatch.dylib 0x1d50ee000 - 0x1d50fdff3 libsystem_pthread.dylib (*) <c1ed564d-b480-3058-937e-b40c3d3df09d> /System/DriverKit/usr/lib/system/libsystem_pthread.dylib 0x1d4d27000 - 0x1d4d6b00d DriverKit (*) <839dc0a2-1e69-38e8-8bf5-ff0ecc531539> /System/DriverKit/System/Library/Frameworks/DriverKit.framework/DriverKit 0x104e90000 - 0x104f1bfff dyld (*) <fe8a9d9e-f65d-34ca-942c-175b99c0601b> /usr/lib/dyld Could anyone please help me with resolving this problem?
1
0
1.1k
Sep ’23
Can't load KEXT in VMs on M1
Trying to get some minimum development working again, I've been waiting to be able to macOS in VMs on M1. Currently both VirtualBuddy, and UTM, can install macOS, I can go to Recovery Boot to disable SIP and enable 3rd party extensions. My M1 runs: ProductVersion: 13.0 BuildVersion: 22A5331f I've tested VM macOS versions of Monterey and Ventura. Here is my old kext (known to be working) loaded on M1 (Ventura) bare-metal 250 0 0xfffffe0006b70000 0x862ac 0x862ac org.openzfsonosx.zfs (2.1.0) BE4DF1D3-FF77-3E58-BC9A-C0B8E175DD97 &lt;21 7 5 4 3 1&gt; The same pkg, using the same steps in the VM, will after clicking Allow, ask to reboot (suspiciously fast), then come up with: System Extension Error: An error occurred with your system extensions during startup and they need to be rebuilt before they can be used. Of course clicking Allow just does the same, reboot, fail, ask to approve again, reboot..fail... Directly on the hardware, the dialog "rebuilding cache" pops up for a few seconds, but with the VMs I do not see it. I'm unfamiliar with the new system, so I'm not sure which log files to look at, but here is the output from kmtuil log, both at Allow and after reboot: https://www.lundman.net/kmutil-log.txt If I was going to make an uneducated guess and pull out some lines by random, maybe: 2022-08-29 20:01:13.169897+0900 0x251 Error 0x0 100 0 kernelmanagerd: Kcgen roundtrip failed with: Boot policy error: Error creating linked manifest: code BOOTPOLICY_ERROR_ACM 2022-08-29 20:01:13.170200+0900 0x251 Error 0x0 100 0 kernelmanagerd: Kcgen roundtrip failed checkpoint saveAuxkc: status:error fatalError:Optional("Boot policy error: Error creating linked manifest: code BOOTPOLICY_ERROR_ACM") 2022-08-29 20:01:13.170201+0900 0x251 Error 0x0 100 0 kernelmanagerd: Kcgen roundtrip failed: missing last checkpoint or errors found 2022-08-29 20:01:13.170242+0900 0x251 Default 0x0 100 0 kernelmanagerd: Deleting Preboot content Any work arounds? Loading kexts on my only M1 is a hard way to develop.
3
2
2.4k
Sep ’23
Restrict access to user space applications accessing a kernel extension
I have developed a kernel extension (KEXT) for driving SCSI devices and I am able to successfully use it to send commands to the underlying device. The driver class overrides the newUserClient method which gets called whenever IOServiceOpen is called from the user space so that apps can make use of the driver. Is there any way to restrict access to this kernel extension such that only my app would be able to open a user client to access the driver and communicate with it using IOConnectCallMethod?
3
1
1.2k
Aug ’23
Prioritizing program/renice
Hi, How would I go about keeping a program at full capacity even as I leave the application? I am on Mac M1 (Ventura 13.5) and working in Unreal Engine. Specifically, I want to be able to stream my scene as I am working in it/playing the scene but this requires me to leave to application as I am working with 3 applications at the same time when doing so. Whenever I leave UE it becomes a background application and frame rate drops from 60 FPS to 3. I tried to renice but although UE is prioritized it nevertheless becomes treated as a background application and FPS drops. I suspect this has something to do with energy saving but I need to find a way around it. Anyone have an idea of how to sort this?
0
0
712
Aug ’23
Can we use clock_gettime_nsec_np(CLOCK_MONOTONIC_RAW) in IOS?
Hello! We develop IOS application and it’s necessary for us to check if user changed the device time in Device settings or not. We use function "clock_gettime_nsec_np(CLOCK_MONOTONIC_RAW)" to check time and it works perfect. But I saw this function only here in Documentation(in discussion section): https://developer.apple.com/documentation/kernel/1646199-mach_continuous_time and this is for Mac OS so I want to know if I can use this function in IOS application that we are going to submit to App Store or we can face with some problems? For example this function can not work on some iPhones or on some IOS or we can have some problems during the review the application in App Store? Or maybe there are some alternatives function "clock_gettime_nsec_np(CLOCK_MONOTONIC_RAW)" for IOS? We tried to use ProcessInfo.processInfo.systemUptime but it counts only when device is awaken. So we can't use it to check if user changed the device time in Device settings or not.( Thank you for any help in advance!
1
0
2.3k
Aug ’23
Availability of Low-Level APIs
I regularly see folks confused by this point so I decided to write it up in a single place. If you have questions or comments about this, start a new thread here on DevForums. Tag your thread with something appropriate for the API you’re trying to use. If nothing leaps out, Kernel is a good option [1]. IMPORTANT I don’t work for App Review and can’t make definitive statements on their behalf. All of the following is about whether an API is available for you to use, not whether App Review will approve your specific usage. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" [1] Because of the KPI aspect of this, discussed below. Availability of Low-Level APIs Every now and again I see questions like this: The developer documentation has no entry for getpid. How can that not be API? Or this: I want to call mach_absolute_time on iOS. Its documentation says that it’s only available on macOS. But it’s in the iOS SDK and it works just fine. Is it OK for me to use it in my iOS app? These questions arise because: Apple Developer Documentation focuses on Apple’s frameworks. Most low-level APIs, specifically those with a BSD heritage, are documented in man pages. These man pages aren’t available in Apple Developer Documentation (r. 16512537). For information about how to access them, see Reading UNIX Manual Pages. Some low-level APIs are documented in both man pages and Apple Developer Documentation. A classic example of this is Dispatch, which has comprehensive man pages (start at dispatch 3) and good documentation in Apple Developer Documentation. Some low-level APIs, like Compression, are documented in Apple Developer Documentation but have no man pages. Some low-level APIs aren’t documented in either Apple Developer Documentation or the man pages. A classic example of this is SQLite. In such cases the best documentation may be the doc comments in the APIs headers, or on a third-party website. For Mach APIs, you often find that the best documentation is in the Darwin open source! Some low-level APIs have equivalent KPIs (Kernel Programming Interfaces). For example, mach_absolute_time is both an API and a KPI. Many KPIs are documented in Apple Developer Documentation, in a special area known as the Kernel framework [1]. Kernel development is only supported on macOS, so those KPIs are flagged as being macOS-only. However, their equivalent APIs are typically available on Apple’s other platforms. These questions most often crop up in the context of obscure low-level APIs, but many obviously valid low-level APIs have the same issue and no one worries about those. For example, Apple Developer Documentation has no documentation for the printf API [2], but no one asks whether it’s OK to use printf! Given the above, it’s clear that you can’t infer anything about the availability of an API based on its state in Apple Developer Documentation. Rather, the acid test for availability is: Is it declared in a platform SDK header? Is it not marked as unavailable on that platform? [3] Does the platform SDK have a stub library [4] that makes its symbol available to the linker? If the answer to all four questions is “yes”, the API is available on that platform. [1] The kernel does not support frameworks but bundling these KPIs into Kernel.framework within the macOS SDK makes some degree of sense from a tooling perspective, and that logic flows through to the documentation. [2] There’s a documentation page for the KPI, but that’s not the same thing. [3] Sorry for the double negative but it’s the only way to capture an important subtlety. If the header contains a declaration with no availability markings, that API is available on all platforms. A classic example of this is printf. [4] If you’re not familiar with the term stub library, see An Apple Library Primer. Hints and Tips In general, prefer high-level APIs over low-level ones. For example, prefer Date, or even gettimeofday or clock_gettime, over mach_absolute_time. However, that’s only a general guideline. In some cases the low-level API really is the right one to use. Just because something is an API doesn’t mean that there aren’t any restrictions on it. mach_absolute_time is a perfect example of this. Using it for highly accurate performance analysis of your code is fine, but using it for fingerprinting is not. See Describing use of required reason API. If you can’t find adequate documentation for an API you’re using, always look in the headers for doc comments. In some cases that’s the only source of documentation. However, even if the API is reasonably well documented, the headers might contain critical tidbits that slipped through the cracks.
0
0
2.4k
Aug ’23
kmutil fails when booted into OS on external drive
"kmutil load" will fail to load a 3rd party kernel extension when booted into an OS on an external drive. The same kernel extension will load fine when booting from the internal "Macintosh HD" . "kmutil inspect" also fails with the same error message when booted using an external drive, but works fine when booting with the internal "Macintosh HD". Both errors indicate the "kernelcache" file cannot be found: sh-3.2/usr/bin/kmutil load -p /Library/Extensions/XXXX.kext Error Domain=KMErrorDomain Code=71 "Could not find: Unable to get contents of boot kernel collection collection at /System/Volumes/Preboot/3B670FAA-F124-41AB-98A8-7C3940B5ECAC/boot/16FE8A65862647F7F8752DA7C4EF320E4CADEB250FCF438FB84A55F822BEB4A98108829E21658BB08B1E314BBC85169A/System/Library/Caches/com.apple.kernelcaches/kernelcache" UserInfo={NSLocalizedDescription=Could not find: Unable to get contents of boot kernel collection collection at /System/Volumes/Preboot/3B670FAA-F124-41AB-98A8-7C3940B5ECAC/boot/16FE8A65862647F7F8752DA7C4EF320E4CADEB250FCF438FB84A55F822BEB4A98108829E21658BB08B1E314BBC85169A/System/Library/Caches/com.apple.kernelcaches/kernelcache} sh-3.2/usr/bin/kmutil inspect No variant specified, falling back to release Error Domain=KMErrorDomain Code=71 "Invalid argument: Unable to read contents of file at /System/Volumes/Preboot/3B670FAA-F124-41AB-98A8-7C3940B5ECAC/boot/904DF99E6EB1281F8E510B3FFB953383F530F313CCFB30A9C1F98231A81B02BDA5E6E0D2BF26FC5D4EB3D2E226B8BC1C/System/Library/Caches/com.apple.kernelcaches/kernelcache" UserInfo={NSLocalizedDescription=Invalid argument: Unable to read contents of file at /System/Volumes/Preboot/3B670FAA-F124-41AB-98A8-7C3940B5ECAC/boot/904DF99E6EB1281F8E510B3FFB953383F530F313CCFB30A9C1F98231A81B02BDA5E6E0D2BF26FC5D4EB3D2E226B8BC1C/System/Library/Caches/com.apple.kernelcaches/kernelcache} The "kernelcache" file path does not exist. In fact, there is no "904DF..." directory under "boot". There is no "kernelcache" file in the entire "/System/Volumes/Preboot" mount. sh-3.2find /System/Volumes/Preboot -name kernelcache find: /System/Volumes/Preboot/.Trashes: Operation not permitted This is on an M1 MacBook Air running 11.3. Is there a special way to load kernel extensions when booting from an external drive? Thanks.
16
1
6.3k
Jul ’23
VFS deadlock (WebDAV on Apple Silicon only)
FB9108925 FB10408005 Since Apple Silicon we've seen a lot of WebDAV instability in macOS 11.x, 12.x and now 13.x that isn't found on x86 Macs. Some were fixed in earlier minor OS upgrades (e.g. webdavfs-387.100.1 that added a missing mutex init), but it's still highly unreliable. The purpose of this post is to put more focus on the bug, see if there is something else we can do to help solve this, as well as hear about potential workarounds from other people experiencing the same problems. I've got a reproducible case described below that triggers a deadlock in VFS every time, requiring a hard reboot to fully recover. Before reboot I've captured this stack trace showing the WebDAV/VFS/UBC/VM layers getting tangled up (macOS 13.2 Build 22D49 running on Macmini9,1): Thread 0x16358 1001 samples (1-1001) priority 46 (base 31) 1001 thread_start + 8 (libsystem_pthread.dylib + 7724) [0x18d2e0e2c] 1001 _pthread_start + 148 (libsystem_pthread.dylib + 28780) [0x18d2e606c] 1001 ??? (diskarbitrationd + 99400) [0x100d5c448] 1001 unmount + 8 (libsystem_kernel.dylib + 55056) [0x18d2b2710] *1001 ??? (kernel.release.t8103 + 30712) [0xfffffe00083437f8] *1001 ??? (kernel.release.t8103 + 1775524) [0xfffffe00084ed7a4] *1001 ??? (kernel.release.t8103 + 7081508) [0xfffffe00089fce24] *1001 ??? (kernel.release.t8103 + 2522264) [0xfffffe00085a3c98] *1001 ??? (kernel.release.t8103 + 2523168) [0xfffffe00085a4020] *1001 vnode_iterate + 728 (kernel.release.t8103 + 2410988) [0xfffffe00085889ec] *1001 ??? (kernel.release.t8103 + 6095404) [0xfffffe000890c22c] *1001 ??? (kernel.release.t8103 + 1097172) [0xfffffe0008447dd4] *1001 ??? (kernel.release.t8103 + 1100852) [0xfffffe0008448c34] *1001 ??? (kernel.release.t8103 + 1024804) [0xfffffe0008436324] *1001 ??? (kernel.release.t8103 + 1025092) [0xfffffe0008436444] *1001 ??? (kernel.release.t8103 + 6497736) [0xfffffe000896e5c8] *1001 ??? (kernel.release.t8103 + 2705840) [0xfffffe00085d09b0] *1001 webdav_vnop_pageout + 432 (com.apple.filesystems.webdav + 16920) [0xfffffe000b2db7b8] *1001 webdav_vnop_close + 64 (com.apple.filesystems.webdav + 9492) [0xfffffe000b2d9ab4] *1001 webdav_vnop_close_locked + 96 (com.apple.filesystems.webdav + 19708) [0xfffffe000b2dc29c] *1001 webdav_close_mnomap + 264 (com.apple.filesystems.webdav + 20004) [0xfffffe000b2dc3c4] *1001 webdav_fsync + 404 (com.apple.filesystems.webdav + 20484) [0xfffffe000b2dc5a4] *1001 ubc_msync + 184 (kernel.release.t8103 + 6096856) [0xfffffe000890c7d8] *1001 ??? (kernel.release.t8103 + 1097172) [0xfffffe0008447dd4] *1001 ??? (kernel.release.t8103 + 1100728) [0xfffffe0008448bb8] *1001 lck_rw_sleep + 136 (kernel.release.t8103 + 505804) [0xfffffe00083b77cc] *1001 ??? (kernel.release.t8103 + 607656) [0xfffffe00083d05a8] *1001 ??? (kernel.release.t8103 + 613952) [0xfffffe00083d1e40] I've spent countless hours reading the xnu-8792.81.2 and webdavfs-392 sources trying to understand what happens. Symbols mapped back to the source code tell me it's trying to flush a dirty mmap'ed file back to the WebDAV host when the volume is about to get unmounted, but I suspect the pageout request is triggered recursively, perhaps because the mmap'ed file has shrunk and pages need to be released? The test case: Use Finder to connect to a WebDAV volume which holds a fairly large image (200 MB Photoshop file in my case). Navigate to this file in column mode so Finder renders a preview (using a QuickLook process). I believe this mmap's the file, but that alone isn't sufficient, so I think the Finder tries to write an updated thumbnail back to the volume as well. Click the Eject icon in the Finder to unmount the volume, which now deadlocks that file system. In the end something remains unreleased in the filesystem since the unmount request never completes, so whether that's a VNode lock or just open file refcount or something else I don't know. Now, why this deadlock is only seen on Apple Silicon is a mystery. Is Finder/QuickLook executing different code paths for generating or storing the thumbnail? Or is there yet more cases of uninitialized mutexes/locks that happen to be accidentally functional on x86 but expose a problem on AS? I've been through a lot of kernel source code trying to find any but have come up short. But since the above is easily reproduced I'm hoping someone with filesystem/kernel debug capability can succeed in pinpointing the bug. It's at least positive that the overall architecture works on x86 so I'm hoping it is a simple fix in the end. The reason I'm debugging this is we've got a lot of customers running WebDAV on M1/M2 and they find Finder file copying highly unreliable (i.e. writing many files to the WebDAV server, possibly overwriting existing files; some users have reported a need to reboot 20 times a day). I'm really looking for a bug that's common to all of these tasks, not just the mmap + unmount problem which is a minimal test case that I've cooked up in the lab. The few spindumps I've seen from end users have also included the combination of webdav_vnop_pageout + webdav_fsync + ubc_msync + lck_rw_sleep even if unmounting wasn't the initial op that forced the deadlock. This problem has been reproduced with different WebDAV server vendors, and there is a test account on a server running Apache provided in FB10408005 (though please select the PSD file, not just the tiny JPG). Thanks in advance!
1
0
1.3k
Jul ’23
iOS system panic, panic string Kernel data abort
Our user reported lately that their devices were rebooted automatically after using our app for more than 2 hours , at the beginnig I cannot , I never seen an app will make the iOS system get rebooted, until our user send me a video prove it what they said is true.And after that I asked them for the panic-full logs in their devices which indicate the device was rebooted unusually, the log shows that it's exactly our app's process leads to the reboot, OMG, I got shocked...Our app is a live broadcasting app, most of our users encountered the reboot problem are when they finished broadcasting a live for a long time (maybe more than 2 hours) and close the room button, another infomation is that we add WebRTC.framework to our app lately, Our user's devices and system version vary, so maybe there is no relation with these factors.I reproduce this problem once on my iPhone 7P (I boradcasted for 2 hours, my phone rebooted exactly after I clicked the close button of my live room), and the panic log in my phone is very similar with logs from our users. Here is part of my log, I put the full on pastebin: https://pastebin.com/Hh9TqZgh{ "build":"iPhone OS 12.3.1 (16F203)", "product":"iPhone9,2", "kernel":"Darwin Kernel Version 18.6.0: Thu Apr 25 22:14:06 PDT 2019; root:xnu-4903.262.2~2/RELEASE_ARM64_T8010", "incident":"82BE9C5C-B6C3-40B4-98C7-B467ACEEF879", "crashReporterKey":"7800f8ead4380bbeae183732c167c3a212d0b0fe", "date":"2019-10-09 14:03:49.04 +0800", "panicString":"panic(cpu 0 caller 0xfffffff021e53bd0): Kernel data abort. (saved state: 0xffffffe050fbaf80) x0: 0x0000000000000000 x1: 0x000f076700041104 x2: 0xfffffffffffffffb x3: 0xffffffe04a4be81b x4: 0x0000000000000000 x5: 0x0000000000000020 x6: 0x0000000000000000 x7: 0xfffffff0221fc940 x8: 0x139a6525663e0012 x9: 0x139a6525663e0012 x10: 0x0000000000000000 x11: 0x0200000010000860 x12: 0x0000000000000000 x13: 0x3a3a79726f6d654d x14: 0x656e72654b746567 x15: 0x002928617461446c x16: 0xfffffff021cb2df7 x17: 0x0000000000000020 x18: 0xfffffff021d31000 x19: 0xffffffe006609000 x20: 0xffffffe007192880 x21: 0x0000000281f9e760 x22: 0x0000000008600000 x23: 0x0000000000000006 x24: 0x000000000000000c x25: 0xffffffe0070b4600 x26: 0x0000000085020080 x27: 0x0000000000000018 x28: 0x0000000000000000 fp: 0xffffffe050fbb350 lr: 0xfffffff020f9ed74 sp: 0xffffffe050fbb2d0 pc: 0xfffffff020f9ed74 cpsr: 0x60400304 esr: 0x96000006 far: 0x0000000000000000 Debugger message: panic Memory ID: 0x1 OS version: 16F203 Kernel version: Darwin Kernel Version 18.6.0: Thu Apr 25 22:14:06 PDT 2019; root:xnu-4903.262.2~2/RELEASE_ARM64_T8010 KernelCache UUID: 2AEBFD081F36C4C81402573F19F4DEE7 Kernel UUID: 897227C4-A9D9-3E80-AA04-C1C2029A8430 iBoot version: iBoot-4513.260.81 secure boot?: YES Paniclog version: 12 Kernel slide: 0x000000001ac54000 Kernel text base: 0xfffffff021c58000 Epoch Time: sec usec Boot : 0x5d9c64c8 0x000468bc Sleep : 0x5d9d3a6c 0x000d0471 Wake : 0x5d9d3aa5 0x00014f47 Calendar: 0x5d9d7819 0x0007c4af Panicked task 0xffffffe00634f2a0: 43873 pages, 21 threads: pid 339: EXXXLXXX Panicked thread: 0xffffffe001324000, backtrace: 0xffffffe050fba710, tid: 121839 lr: 0xfffffff021d661c4 fp: 0xffffffe050fba7a0 lr: 0xfffffff021e54538 fp: 0xffffffe050fba8e0 lr: 0xfffffff021d31610 fp: 0xffffffe050fba8f0 lr: 0xfffffff021d65768 fp: 0xffffffe050fbac60 lr: 0xfffffff021d65ae0 fp: 0xffffffe050fbaca0 lr: 0xfffffff021d65934 fp: 0xffffffe050fbacc0 lr: 0xfffffff021e53bd0 fp: 0xffffffe050fbae20 lr: 0xfffffff021e54c5c fp: 0xffffffe050fbaf60 lr: 0xfffffff021d31610 fp: 0xffffffe050fbaf70 lr: 0xfffffff020f9ed74 fp: 0xffffffe050fbb350 lr: 0xfffffff020f9d590 fp: 0xffffffe050fbb3a0 lr: 0xfffffff020f9d39c fp: 0xffffffe050fbb410 lr: 0xfffffff020f9c5f0 fp: 0xffffffe050fbb470 lr: 0xfffffff022217ee0 fp: 0xffffffe050fbb580 lr: 0xfffffff02221f8d4 fp: 0xffffffe050fbb710 lr: 0xfffffff021e2b128 fp: 0xffffffe050fbb820 lr: 0xfffffff021d49c90 fp: 0xffffffe050fbb9b0 lr: 0xfffffff021d5c17c fp: 0xffffffe050fbbb40 lr: 0xfffffff021e54d28 fp: 0xffffffe050fbbc80 lr: 0xfffffff021d31610 fp: 0xffffffe050fbbc90 lr: 0x00000001a343b0f4 fp: 0x0000000000000000 ", "panicFlags":"0x2", "otherString":" ** Stackshot Succeeded ** Bytes Traced 357072 ** ", "memoryStatus":{ "compressorSize":8413, "compressions":50291, "decompressions":21892, "busyBufferCount":0, "pageSize":16384, "memoryPressure":false, "memoryPages":{ "active":68032, "throttled":0, "fileBacked":61496, "wired":24379, "purgeable":2883, "inactive":40858, "free":11191, "speculative":27437 } } ........ }I have no idea about what to do next to resolve this problem, any suggestions will help, thanks!
6
0
14k
Jun ’23
THREAD_TIME_CONSTRAINT_POLICY for Apple Silicon
Hello, I use https://juce.com to develop an audio app for macOS. With previous macOSes on Intel, I used THREAD_TIME_CONSTRAINT_POLICY as described in https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/KernelProgramming/scheduler/scheduler.html to set real time priority for the audio processing thread, according to CPU usage values. Juce has an API to get the CPU frequency so I was able to set usage values. With macOS Monterey on Apple Silicon, the Juce API is unable to give the CPU frequency (the getCpuSpeedInMegahertz API just returns 0). Is THREAD_TIME_CONSTRAINT_POLICY still valid on latest macOS on Apple Silicon? If so, what CPU frequency value should be used - ie how can we determine it? If not, how should the real time priority be set? Thanks! -Mathieu Fyi Juce code to get CPU (SystemStats::getCpuSpeedInMegahertz) can be found at https://github.com/juce-framework/JUCE/blob/master/modules/juce_core/native/juce_mac_SystemStats.mm
0
0
968
Jun ’23
System Settings Asking Me to go to recoveryOS and change system security to allow kexts
After installing Xcode Command Line Tools, the System Settings App now asks me, to reboot my machine into recoveryOS and change the systems security to allow kernel extensions/ system extensions. It's not prominently mentioned, but just there and I never, ever had this. Must be a new dialog. Well, if you click on it... My Question is: Why? Kernel Extensions have been deprecated long ago (I thought) and the dialog mentions nothing specific.
0
0
723
Jun ’23
12 core M2 mini only using 4 cores, Intel Mac mini used 6
Developing a program that needs maximum cores. My previous Intel Mac mini used all 6 cores, according to Activity Monitor. It showed 600% cpu usage. I assumed the M2 would use twelve, but only shows 400% cpu usage in Activity Monitor, ergo, two less! I don't recall setting a core usage parameter in Xcode. Any ideas? Ventura 13.3 and Xcode 14.3 up to date as of this post.
2
0
1k
May ’23
`kmutil showloaded` shows address of 0
We have a KEXT that lists an address of 0 when we run kmutil showloaded on Apple Silicon (M1 & M2). The KEXT appears to work fine, but we are concerned that we have something misconfigured. It shows a proper address on x86_64. Any thoughts would be appreciated. macOS 13.4. Cheers, Dave ... 252 0 0xfffffe00078461f0 0x998 0x998 com.apple.driver.driverkit.serial (6.0.0) 447C62ED-B7E8-3B91-A3B1-19D2B50A20EC <46 5 4 3 1> 254 0 0xfffffe0007918e30 0x1710 0x1710 com.apple.filesystems.autofs (3.0) 7FF20081-54D5-329E-AA76-6BD3018C3167 <210 7 6 5 4 3 2 1> 256 0 0 0x1976 0x1976 com.google...
0
1
1.1k
Apr ’23
Using libproc to get ports used by processes on macos
I am trying to get ports used by processes. It can be done via lsof on macOS, i am trying to do it via libproc. #include &lt;iostream&gt; #include &lt;libproc.h&gt; int main(int argc, const char * argv[]) { pid_t pids[3072]; int count = proc_listpids(PROC_ALL_PIDS, 0, pids, sizeof(pids)); for (int i = 0; i &lt; count; i++) { char buffer[1024]; for (int j = 1; j &lt; 50000; j++) { //port range int ret = proc_pidfileportinfo(pids[i], j, PROC_PIDFILEPORTVNODEPATHINFO, buffer, sizeof(buffer)); if(ret != 0) { printf("proc_pidfileportinfo returned %d bytes of data\n", ret); printf("%s\n", name); } } } return 0; } proc_pidfileportinfo function is not working for any port, i tried iterating till 50K. What i am doing wrong with proc_pidfileportinfo? how to properly use proc_pidfileportinfo?
Replies
9
Boosts
0
Views
2.9k
Activity
Oct ’23
XCode 15: simple execv program exits with code 5 in the console.
Hello, found the following curious behaviour, If I try to run from within xcode (pressing Run) the following code: #include &lt;unistd.h&gt; int main(int argc, const char * argv[]) { char *args[] = {"/bin/ls", "-r", "-t", "-l", (char *) 0 }; execv(args[0], args); return 0; } the program does not print the expected list of files and folders but instead exits with: Message from debugger: Terminated due to signal 5 Program ended with exit code: 5 But if I try to run the exact same compiled program from the terminal, it works as expected. I lost so many hours wondering what I was doing wrong, but apparently it was the xcode console that does not play nice with execing? Could it be that changing the process image throws a wrench into xcode? Anybody has any idea why this could be? Thanks.
Replies
1
Boosts
0
Views
1k
Activity
Oct ’23
task_info and TASK_POWER_INFO_V2
On iOS we are trying to log energy use by our network extension to help us isolate troubled areas. task_info with TASK_POWER_INFO_V2 does return task_energy value and it's growing over time, but it's not clear what exactly it means and how it supposed to correlate to battery usage screen from iOS settings. Does anybody know anything about this?
Replies
2
Boosts
0
Views
958
Activity
Oct ’23
Kernel Debug Kit for 23A344 ETA?
Hi! Any ETA for a KDK for 23A344? The most recent available KDK I see is 22G91. Thanks!
Replies
0
Boosts
0
Views
1.3k
Activity
Sep ’23
Disable peripherals on macOS programmatically
I know with Disk Arbitration framework, I can use DARegisterDiskMountApprovalCallback to prevent external disks from mounting. The disks includes thumb drive, external hard disk, etc., but there are many types of peripherals out there, like a usb wireless receiver or a USB ethernet adapter. Is there any other framework for us to use to enable/disable peripherals based on their I/O Registry properties? Thanks!
Replies
4
Boosts
0
Views
2.1k
Activity
Sep ’23
DriverKit crash at 0x1d4d60010 OSMetaClassBase::Invoke(IORPC) (.cold.1) + 44
I created a driver using DriverKit on Intel macOS 12.6.1 and Xcode 13.3. I enabled auto-manage signing, and set the signing certificate to 'Sign to Run Locally'. Then, I created a provision profile for the driver and selected my M1 test device. After installing the profile, I ran the app on the M1 device and successfully activated the driver. When I plugin the USB device, I can see the following log: DK: epusbfilter-0x100009dce::start(IOUSBHostInterface-0x10000946d) ok epusbfilter - init com.injection.epusbfilter.dext[57573] Corpse failure, too many 6 I also found a crash log ------------------------------------- Translated Report (Full Report Below) ------------------------------------- Process: com.injection.epusbfilter.dext [53185] Path: /Library/SystemExtensions/*/com.injection.epusbfilter.dext Identifier: com.injection.epusbfilter.dext Version: 1.0 (1) Code Type: ARM-64 (Native) Parent Process: launchd [1] User ID: 270 Date/Time: 2023-09-19 15:01:01.8502 +0800 OS Version: macOS 13.2 (22D49) Report Version: 12 Anonymous UUID: 5EB7EBD9-A435-FC45-73E6-C2C5844A8082 Time Awake Since Boot: 79000 seconds System Integrity Protection: disabled Crashed Thread: 1 Dispatch queue: Root Exception Type: EXC_CRASH (SIGABRT) Exception Codes: 0x0000000000000000, 0x0000000000000000 Application Specific Information: abort() called Thread 0: 0 libsystem_kernel.dylib 0x1d5043b78 __semwait_signal_nocancel + 8 1 libsystem_c.dylib 0x1d4fcfec8 nanosleep$NOCANCEL + 212 2 libsystem_c.dylib 0x1d4fee204 sleep$NOCANCEL + 48 3 libdispatch.dylib 0x1d4f807b4 _dispatch_queue_cleanup2 + 200 4 libsystem_pthread.dylib 0x1d50fbc50 _pthread_tsd_cleanup + 132 5 libsystem_pthread.dylib 0x1d50f3220 _pthread_exit + 88 6 libsystem_pthread.dylib 0x1d50f4180 pthread_exit + 88 7 libdispatch.dylib 0x1d4f7bbcc dispatch_main + 128 8 DriverKit 0x1d4d33178 DriverExecutableMain + 84 9 dyld 0x104e95e50 start + 2544 Thread 1 Crashed:: Dispatch queue: Root 0 libsystem_kernel.dylib 0x1d5043720 __pthread_kill + 8 1 libsystem_pthread.dylib 0x1d50f40ec pthread_kill + 268 2 libsystem_c.dylib 0x1d5033cac abort + 180 3 DriverKit 0x1d4d5f890 panic + 256 4 DriverKit 0x1d4d5fa60 __assert_rtn + 88 5 DriverKit 0x1d4d60010 OSMetaClassBase::Invoke(IORPC) (.cold.1) + 44 6 DriverKit 0x1d4d32064 OSMetaClassBase::Invoke(IORPC) + 1396 7 DriverKit 0x1d4d32c5c Server(void*, mach_msg_header_t*, mach_msg_header_t*) + 520 8 DriverKit 0x1d4d3b420 uiomessage(void*) + 180 9 DriverKit 0x1d4d34694 uiomachchannel(void*, dispatch_mach_reason_t, dispatch_mach_msg_s*, int) + 380 10 libdispatch.dylib 0x1d4f8868c _dispatch_mach_msg_invoke + 472 11 libdispatch.dylib 0x1d4f74484 _dispatch_lane_serial_drain + 380 12 libdispatch.dylib 0x1d4f89620 _dispatch_mach_invoke + 852 13 libdispatch.dylib 0x1d4f74484 _dispatch_lane_serial_drain + 380 14 libdispatch.dylib 0x1d4f75130 _dispatch_lane_invoke + 436 15 libdispatch.dylib 0x1d4f7640c _dispatch_workloop_invoke + 1784 16 libdispatch.dylib 0x1d4f7ff5c _dispatch_workloop_worker_thread + 652 17 libsystem_pthread.dylib 0x1d50f5024 _pthread_wqthread + 404 18 libsystem_pthread.dylib 0x1d50fc678 start_wqthread + 8 Thread 2: 0 libsystem_pthread.dylib 0x1d50fc670 start_wqthread + 0 Thread 3: 0 libsystem_kernel.dylib 0x1d504401c __sigsuspend_nocancel + 8 1 libdispatch.dylib 0x1d4f808b4 _dispatch_sigsuspend + 48 2 libdispatch.dylib 0x1d4f80884 _dispatch_sig_thread + 56 Thread 1 crashed with ARM Thread State (64-bit): x0: 0x0000000000000000 x1: 0x0000000000000000 x2: 0x0000000000000000 x3: 0x0000000000000000 x4: 0xffffa0016a011948 x5: 0x0000000000000010 x6: 0x00006000010481b0 x7: 0x0000000000000000 x8: 0x725b4b6e56620c88 x9: 0x725b4b6f3d67bc88 x10: 0x00000000000001b0 x11: 0x0000600001048000 x12: 0x0000000000000090 x13: 0x00000000ffffff92 x14: 0x00000000000007fb x15: 0x0000000080636ffb x16: 0x0000000000000148 x17: 0x00000001d7176c60 x18: 0x0000000000000000 x19: 0x0000000000000006 x20: 0x0000000000004003 x21: 0x000000016b05b0e0 x22: 0x0000000000000000 x23: 0x00006000010480e8 x24: 0x0000600001048058 x25: 0xd200fde7d57ecca6 x26: 0x0000000000000085 x27: 0x000060000374c328 x28: 0x0000600001d4c000 fp: 0x000000016b059a90 lr: 0x00000001d50f40ec sp: 0x000000016b059a70 pc: 0x00000001d5043720 cpsr: 0x40001000 far: 0x0000600002c48000 esr: 0x56000080 Address size fault Binary Images: 0x1d503a000 - 0x1d5075fe3 libsystem_kernel.dylib (*) <60df52bd-fc1a-3888-b05b-24b44be3af15> /System/DriverKit/usr/lib/system/libsystem_kernel.dylib 0x1d4fc6000 - 0x1d5039fff libsystem_c.dylib (*) <eee04d9a-7574-3a74-8f4e-cfb05f89f7da> /System/DriverKit/usr/lib/system/libsystem_c.dylib 0x1d4f62000 - 0x1d4fadfff libdispatch.dylib (*) <4e310a5c-9629-305e-a1dd-6632bddd3362> /System/DriverKit/usr/lib/system/libdispatch.dylib 0x1d50ee000 - 0x1d50fdff3 libsystem_pthread.dylib (*) <c1ed564d-b480-3058-937e-b40c3d3df09d> /System/DriverKit/usr/lib/system/libsystem_pthread.dylib 0x1d4d27000 - 0x1d4d6b00d DriverKit (*) <839dc0a2-1e69-38e8-8bf5-ff0ecc531539> /System/DriverKit/System/Library/Frameworks/DriverKit.framework/DriverKit 0x104e90000 - 0x104f1bfff dyld (*) <fe8a9d9e-f65d-34ca-942c-175b99c0601b> /usr/lib/dyld Could anyone please help me with resolving this problem?
Replies
1
Boosts
0
Views
1.1k
Activity
Sep ’23
Can't load KEXT in VMs on M1
Trying to get some minimum development working again, I've been waiting to be able to macOS in VMs on M1. Currently both VirtualBuddy, and UTM, can install macOS, I can go to Recovery Boot to disable SIP and enable 3rd party extensions. My M1 runs: ProductVersion: 13.0 BuildVersion: 22A5331f I've tested VM macOS versions of Monterey and Ventura. Here is my old kext (known to be working) loaded on M1 (Ventura) bare-metal 250 0 0xfffffe0006b70000 0x862ac 0x862ac org.openzfsonosx.zfs (2.1.0) BE4DF1D3-FF77-3E58-BC9A-C0B8E175DD97 &lt;21 7 5 4 3 1&gt; The same pkg, using the same steps in the VM, will after clicking Allow, ask to reboot (suspiciously fast), then come up with: System Extension Error: An error occurred with your system extensions during startup and they need to be rebuilt before they can be used. Of course clicking Allow just does the same, reboot, fail, ask to approve again, reboot..fail... Directly on the hardware, the dialog "rebuilding cache" pops up for a few seconds, but with the VMs I do not see it. I'm unfamiliar with the new system, so I'm not sure which log files to look at, but here is the output from kmtuil log, both at Allow and after reboot: https://www.lundman.net/kmutil-log.txt If I was going to make an uneducated guess and pull out some lines by random, maybe: 2022-08-29 20:01:13.169897+0900 0x251 Error 0x0 100 0 kernelmanagerd: Kcgen roundtrip failed with: Boot policy error: Error creating linked manifest: code BOOTPOLICY_ERROR_ACM 2022-08-29 20:01:13.170200+0900 0x251 Error 0x0 100 0 kernelmanagerd: Kcgen roundtrip failed checkpoint saveAuxkc: status:error fatalError:Optional("Boot policy error: Error creating linked manifest: code BOOTPOLICY_ERROR_ACM") 2022-08-29 20:01:13.170201+0900 0x251 Error 0x0 100 0 kernelmanagerd: Kcgen roundtrip failed: missing last checkpoint or errors found 2022-08-29 20:01:13.170242+0900 0x251 Default 0x0 100 0 kernelmanagerd: Deleting Preboot content Any work arounds? Loading kexts on my only M1 is a hard way to develop.
Replies
3
Boosts
2
Views
2.4k
Activity
Sep ’23
Restrict access to user space applications accessing a kernel extension
I have developed a kernel extension (KEXT) for driving SCSI devices and I am able to successfully use it to send commands to the underlying device. The driver class overrides the newUserClient method which gets called whenever IOServiceOpen is called from the user space so that apps can make use of the driver. Is there any way to restrict access to this kernel extension such that only my app would be able to open a user client to access the driver and communicate with it using IOConnectCallMethod?
Replies
3
Boosts
1
Views
1.2k
Activity
Aug ’23
Prioritizing program/renice
Hi, How would I go about keeping a program at full capacity even as I leave the application? I am on Mac M1 (Ventura 13.5) and working in Unreal Engine. Specifically, I want to be able to stream my scene as I am working in it/playing the scene but this requires me to leave to application as I am working with 3 applications at the same time when doing so. Whenever I leave UE it becomes a background application and frame rate drops from 60 FPS to 3. I tried to renice but although UE is prioritized it nevertheless becomes treated as a background application and FPS drops. I suspect this has something to do with energy saving but I need to find a way around it. Anyone have an idea of how to sort this?
Replies
0
Boosts
0
Views
712
Activity
Aug ’23
Can we use clock_gettime_nsec_np(CLOCK_MONOTONIC_RAW) in IOS?
Hello! We develop IOS application and it’s necessary for us to check if user changed the device time in Device settings or not. We use function "clock_gettime_nsec_np(CLOCK_MONOTONIC_RAW)" to check time and it works perfect. But I saw this function only here in Documentation(in discussion section): https://developer.apple.com/documentation/kernel/1646199-mach_continuous_time and this is for Mac OS so I want to know if I can use this function in IOS application that we are going to submit to App Store or we can face with some problems? For example this function can not work on some iPhones or on some IOS or we can have some problems during the review the application in App Store? Or maybe there are some alternatives function "clock_gettime_nsec_np(CLOCK_MONOTONIC_RAW)" for IOS? We tried to use ProcessInfo.processInfo.systemUptime but it counts only when device is awaken. So we can't use it to check if user changed the device time in Device settings or not.( Thank you for any help in advance!
Replies
1
Boosts
0
Views
2.3k
Activity
Aug ’23
Availability of Low-Level APIs
I regularly see folks confused by this point so I decided to write it up in a single place. If you have questions or comments about this, start a new thread here on DevForums. Tag your thread with something appropriate for the API you’re trying to use. If nothing leaps out, Kernel is a good option [1]. IMPORTANT I don’t work for App Review and can’t make definitive statements on their behalf. All of the following is about whether an API is available for you to use, not whether App Review will approve your specific usage. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" [1] Because of the KPI aspect of this, discussed below. Availability of Low-Level APIs Every now and again I see questions like this: The developer documentation has no entry for getpid. How can that not be API? Or this: I want to call mach_absolute_time on iOS. Its documentation says that it’s only available on macOS. But it’s in the iOS SDK and it works just fine. Is it OK for me to use it in my iOS app? These questions arise because: Apple Developer Documentation focuses on Apple’s frameworks. Most low-level APIs, specifically those with a BSD heritage, are documented in man pages. These man pages aren’t available in Apple Developer Documentation (r. 16512537). For information about how to access them, see Reading UNIX Manual Pages. Some low-level APIs are documented in both man pages and Apple Developer Documentation. A classic example of this is Dispatch, which has comprehensive man pages (start at dispatch 3) and good documentation in Apple Developer Documentation. Some low-level APIs, like Compression, are documented in Apple Developer Documentation but have no man pages. Some low-level APIs aren’t documented in either Apple Developer Documentation or the man pages. A classic example of this is SQLite. In such cases the best documentation may be the doc comments in the APIs headers, or on a third-party website. For Mach APIs, you often find that the best documentation is in the Darwin open source! Some low-level APIs have equivalent KPIs (Kernel Programming Interfaces). For example, mach_absolute_time is both an API and a KPI. Many KPIs are documented in Apple Developer Documentation, in a special area known as the Kernel framework [1]. Kernel development is only supported on macOS, so those KPIs are flagged as being macOS-only. However, their equivalent APIs are typically available on Apple’s other platforms. These questions most often crop up in the context of obscure low-level APIs, but many obviously valid low-level APIs have the same issue and no one worries about those. For example, Apple Developer Documentation has no documentation for the printf API [2], but no one asks whether it’s OK to use printf! Given the above, it’s clear that you can’t infer anything about the availability of an API based on its state in Apple Developer Documentation. Rather, the acid test for availability is: Is it declared in a platform SDK header? Is it not marked as unavailable on that platform? [3] Does the platform SDK have a stub library [4] that makes its symbol available to the linker? If the answer to all four questions is “yes”, the API is available on that platform. [1] The kernel does not support frameworks but bundling these KPIs into Kernel.framework within the macOS SDK makes some degree of sense from a tooling perspective, and that logic flows through to the documentation. [2] There’s a documentation page for the KPI, but that’s not the same thing. [3] Sorry for the double negative but it’s the only way to capture an important subtlety. If the header contains a declaration with no availability markings, that API is available on all platforms. A classic example of this is printf. [4] If you’re not familiar with the term stub library, see An Apple Library Primer. Hints and Tips In general, prefer high-level APIs over low-level ones. For example, prefer Date, or even gettimeofday or clock_gettime, over mach_absolute_time. However, that’s only a general guideline. In some cases the low-level API really is the right one to use. Just because something is an API doesn’t mean that there aren’t any restrictions on it. mach_absolute_time is a perfect example of this. Using it for highly accurate performance analysis of your code is fine, but using it for fingerprinting is not. See Describing use of required reason API. If you can’t find adequate documentation for an API you’re using, always look in the headers for doc comments. In some cases that’s the only source of documentation. However, even if the API is reasonably well documented, the headers might contain critical tidbits that slipped through the cracks.
Replies
0
Boosts
0
Views
2.4k
Activity
Aug ’23
kmutil fails when booted into OS on external drive
"kmutil load" will fail to load a 3rd party kernel extension when booted into an OS on an external drive. The same kernel extension will load fine when booting from the internal "Macintosh HD" . "kmutil inspect" also fails with the same error message when booted using an external drive, but works fine when booting with the internal "Macintosh HD". Both errors indicate the "kernelcache" file cannot be found: sh-3.2/usr/bin/kmutil load -p /Library/Extensions/XXXX.kext Error Domain=KMErrorDomain Code=71 "Could not find: Unable to get contents of boot kernel collection collection at /System/Volumes/Preboot/3B670FAA-F124-41AB-98A8-7C3940B5ECAC/boot/16FE8A65862647F7F8752DA7C4EF320E4CADEB250FCF438FB84A55F822BEB4A98108829E21658BB08B1E314BBC85169A/System/Library/Caches/com.apple.kernelcaches/kernelcache" UserInfo={NSLocalizedDescription=Could not find: Unable to get contents of boot kernel collection collection at /System/Volumes/Preboot/3B670FAA-F124-41AB-98A8-7C3940B5ECAC/boot/16FE8A65862647F7F8752DA7C4EF320E4CADEB250FCF438FB84A55F822BEB4A98108829E21658BB08B1E314BBC85169A/System/Library/Caches/com.apple.kernelcaches/kernelcache} sh-3.2/usr/bin/kmutil inspect No variant specified, falling back to release Error Domain=KMErrorDomain Code=71 "Invalid argument: Unable to read contents of file at /System/Volumes/Preboot/3B670FAA-F124-41AB-98A8-7C3940B5ECAC/boot/904DF99E6EB1281F8E510B3FFB953383F530F313CCFB30A9C1F98231A81B02BDA5E6E0D2BF26FC5D4EB3D2E226B8BC1C/System/Library/Caches/com.apple.kernelcaches/kernelcache" UserInfo={NSLocalizedDescription=Invalid argument: Unable to read contents of file at /System/Volumes/Preboot/3B670FAA-F124-41AB-98A8-7C3940B5ECAC/boot/904DF99E6EB1281F8E510B3FFB953383F530F313CCFB30A9C1F98231A81B02BDA5E6E0D2BF26FC5D4EB3D2E226B8BC1C/System/Library/Caches/com.apple.kernelcaches/kernelcache} The "kernelcache" file path does not exist. In fact, there is no "904DF..." directory under "boot". There is no "kernelcache" file in the entire "/System/Volumes/Preboot" mount. sh-3.2find /System/Volumes/Preboot -name kernelcache find: /System/Volumes/Preboot/.Trashes: Operation not permitted This is on an M1 MacBook Air running 11.3. Is there a special way to load kernel extensions when booting from an external drive? Thanks.
Replies
16
Boosts
1
Views
6.3k
Activity
Jul ’23
VFS deadlock (WebDAV on Apple Silicon only)
FB9108925 FB10408005 Since Apple Silicon we've seen a lot of WebDAV instability in macOS 11.x, 12.x and now 13.x that isn't found on x86 Macs. Some were fixed in earlier minor OS upgrades (e.g. webdavfs-387.100.1 that added a missing mutex init), but it's still highly unreliable. The purpose of this post is to put more focus on the bug, see if there is something else we can do to help solve this, as well as hear about potential workarounds from other people experiencing the same problems. I've got a reproducible case described below that triggers a deadlock in VFS every time, requiring a hard reboot to fully recover. Before reboot I've captured this stack trace showing the WebDAV/VFS/UBC/VM layers getting tangled up (macOS 13.2 Build 22D49 running on Macmini9,1): Thread 0x16358 1001 samples (1-1001) priority 46 (base 31) 1001 thread_start + 8 (libsystem_pthread.dylib + 7724) [0x18d2e0e2c] 1001 _pthread_start + 148 (libsystem_pthread.dylib + 28780) [0x18d2e606c] 1001 ??? (diskarbitrationd + 99400) [0x100d5c448] 1001 unmount + 8 (libsystem_kernel.dylib + 55056) [0x18d2b2710] *1001 ??? (kernel.release.t8103 + 30712) [0xfffffe00083437f8] *1001 ??? (kernel.release.t8103 + 1775524) [0xfffffe00084ed7a4] *1001 ??? (kernel.release.t8103 + 7081508) [0xfffffe00089fce24] *1001 ??? (kernel.release.t8103 + 2522264) [0xfffffe00085a3c98] *1001 ??? (kernel.release.t8103 + 2523168) [0xfffffe00085a4020] *1001 vnode_iterate + 728 (kernel.release.t8103 + 2410988) [0xfffffe00085889ec] *1001 ??? (kernel.release.t8103 + 6095404) [0xfffffe000890c22c] *1001 ??? (kernel.release.t8103 + 1097172) [0xfffffe0008447dd4] *1001 ??? (kernel.release.t8103 + 1100852) [0xfffffe0008448c34] *1001 ??? (kernel.release.t8103 + 1024804) [0xfffffe0008436324] *1001 ??? (kernel.release.t8103 + 1025092) [0xfffffe0008436444] *1001 ??? (kernel.release.t8103 + 6497736) [0xfffffe000896e5c8] *1001 ??? (kernel.release.t8103 + 2705840) [0xfffffe00085d09b0] *1001 webdav_vnop_pageout + 432 (com.apple.filesystems.webdav + 16920) [0xfffffe000b2db7b8] *1001 webdav_vnop_close + 64 (com.apple.filesystems.webdav + 9492) [0xfffffe000b2d9ab4] *1001 webdav_vnop_close_locked + 96 (com.apple.filesystems.webdav + 19708) [0xfffffe000b2dc29c] *1001 webdav_close_mnomap + 264 (com.apple.filesystems.webdav + 20004) [0xfffffe000b2dc3c4] *1001 webdav_fsync + 404 (com.apple.filesystems.webdav + 20484) [0xfffffe000b2dc5a4] *1001 ubc_msync + 184 (kernel.release.t8103 + 6096856) [0xfffffe000890c7d8] *1001 ??? (kernel.release.t8103 + 1097172) [0xfffffe0008447dd4] *1001 ??? (kernel.release.t8103 + 1100728) [0xfffffe0008448bb8] *1001 lck_rw_sleep + 136 (kernel.release.t8103 + 505804) [0xfffffe00083b77cc] *1001 ??? (kernel.release.t8103 + 607656) [0xfffffe00083d05a8] *1001 ??? (kernel.release.t8103 + 613952) [0xfffffe00083d1e40] I've spent countless hours reading the xnu-8792.81.2 and webdavfs-392 sources trying to understand what happens. Symbols mapped back to the source code tell me it's trying to flush a dirty mmap'ed file back to the WebDAV host when the volume is about to get unmounted, but I suspect the pageout request is triggered recursively, perhaps because the mmap'ed file has shrunk and pages need to be released? The test case: Use Finder to connect to a WebDAV volume which holds a fairly large image (200 MB Photoshop file in my case). Navigate to this file in column mode so Finder renders a preview (using a QuickLook process). I believe this mmap's the file, but that alone isn't sufficient, so I think the Finder tries to write an updated thumbnail back to the volume as well. Click the Eject icon in the Finder to unmount the volume, which now deadlocks that file system. In the end something remains unreleased in the filesystem since the unmount request never completes, so whether that's a VNode lock or just open file refcount or something else I don't know. Now, why this deadlock is only seen on Apple Silicon is a mystery. Is Finder/QuickLook executing different code paths for generating or storing the thumbnail? Or is there yet more cases of uninitialized mutexes/locks that happen to be accidentally functional on x86 but expose a problem on AS? I've been through a lot of kernel source code trying to find any but have come up short. But since the above is easily reproduced I'm hoping someone with filesystem/kernel debug capability can succeed in pinpointing the bug. It's at least positive that the overall architecture works on x86 so I'm hoping it is a simple fix in the end. The reason I'm debugging this is we've got a lot of customers running WebDAV on M1/M2 and they find Finder file copying highly unreliable (i.e. writing many files to the WebDAV server, possibly overwriting existing files; some users have reported a need to reboot 20 times a day). I'm really looking for a bug that's common to all of these tasks, not just the mmap + unmount problem which is a minimal test case that I've cooked up in the lab. The few spindumps I've seen from end users have also included the combination of webdav_vnop_pageout + webdav_fsync + ubc_msync + lck_rw_sleep even if unmounting wasn't the initial op that forced the deadlock. This problem has been reproduced with different WebDAV server vendors, and there is a test account on a server running Apache provided in FB10408005 (though please select the PSD file, not just the tiny JPG). Thanks in advance!
Replies
1
Boosts
0
Views
1.3k
Activity
Jul ’23
iOS system panic, panic string Kernel data abort
Our user reported lately that their devices were rebooted automatically after using our app for more than 2 hours , at the beginnig I cannot , I never seen an app will make the iOS system get rebooted, until our user send me a video prove it what they said is true.And after that I asked them for the panic-full logs in their devices which indicate the device was rebooted unusually, the log shows that it's exactly our app's process leads to the reboot, OMG, I got shocked...Our app is a live broadcasting app, most of our users encountered the reboot problem are when they finished broadcasting a live for a long time (maybe more than 2 hours) and close the room button, another infomation is that we add WebRTC.framework to our app lately, Our user's devices and system version vary, so maybe there is no relation with these factors.I reproduce this problem once on my iPhone 7P (I boradcasted for 2 hours, my phone rebooted exactly after I clicked the close button of my live room), and the panic log in my phone is very similar with logs from our users. Here is part of my log, I put the full on pastebin: https://pastebin.com/Hh9TqZgh{ "build":"iPhone OS 12.3.1 (16F203)", "product":"iPhone9,2", "kernel":"Darwin Kernel Version 18.6.0: Thu Apr 25 22:14:06 PDT 2019; root:xnu-4903.262.2~2/RELEASE_ARM64_T8010", "incident":"82BE9C5C-B6C3-40B4-98C7-B467ACEEF879", "crashReporterKey":"7800f8ead4380bbeae183732c167c3a212d0b0fe", "date":"2019-10-09 14:03:49.04 +0800", "panicString":"panic(cpu 0 caller 0xfffffff021e53bd0): Kernel data abort. (saved state: 0xffffffe050fbaf80) x0: 0x0000000000000000 x1: 0x000f076700041104 x2: 0xfffffffffffffffb x3: 0xffffffe04a4be81b x4: 0x0000000000000000 x5: 0x0000000000000020 x6: 0x0000000000000000 x7: 0xfffffff0221fc940 x8: 0x139a6525663e0012 x9: 0x139a6525663e0012 x10: 0x0000000000000000 x11: 0x0200000010000860 x12: 0x0000000000000000 x13: 0x3a3a79726f6d654d x14: 0x656e72654b746567 x15: 0x002928617461446c x16: 0xfffffff021cb2df7 x17: 0x0000000000000020 x18: 0xfffffff021d31000 x19: 0xffffffe006609000 x20: 0xffffffe007192880 x21: 0x0000000281f9e760 x22: 0x0000000008600000 x23: 0x0000000000000006 x24: 0x000000000000000c x25: 0xffffffe0070b4600 x26: 0x0000000085020080 x27: 0x0000000000000018 x28: 0x0000000000000000 fp: 0xffffffe050fbb350 lr: 0xfffffff020f9ed74 sp: 0xffffffe050fbb2d0 pc: 0xfffffff020f9ed74 cpsr: 0x60400304 esr: 0x96000006 far: 0x0000000000000000 Debugger message: panic Memory ID: 0x1 OS version: 16F203 Kernel version: Darwin Kernel Version 18.6.0: Thu Apr 25 22:14:06 PDT 2019; root:xnu-4903.262.2~2/RELEASE_ARM64_T8010 KernelCache UUID: 2AEBFD081F36C4C81402573F19F4DEE7 Kernel UUID: 897227C4-A9D9-3E80-AA04-C1C2029A8430 iBoot version: iBoot-4513.260.81 secure boot?: YES Paniclog version: 12 Kernel slide: 0x000000001ac54000 Kernel text base: 0xfffffff021c58000 Epoch Time: sec usec Boot : 0x5d9c64c8 0x000468bc Sleep : 0x5d9d3a6c 0x000d0471 Wake : 0x5d9d3aa5 0x00014f47 Calendar: 0x5d9d7819 0x0007c4af Panicked task 0xffffffe00634f2a0: 43873 pages, 21 threads: pid 339: EXXXLXXX Panicked thread: 0xffffffe001324000, backtrace: 0xffffffe050fba710, tid: 121839 lr: 0xfffffff021d661c4 fp: 0xffffffe050fba7a0 lr: 0xfffffff021e54538 fp: 0xffffffe050fba8e0 lr: 0xfffffff021d31610 fp: 0xffffffe050fba8f0 lr: 0xfffffff021d65768 fp: 0xffffffe050fbac60 lr: 0xfffffff021d65ae0 fp: 0xffffffe050fbaca0 lr: 0xfffffff021d65934 fp: 0xffffffe050fbacc0 lr: 0xfffffff021e53bd0 fp: 0xffffffe050fbae20 lr: 0xfffffff021e54c5c fp: 0xffffffe050fbaf60 lr: 0xfffffff021d31610 fp: 0xffffffe050fbaf70 lr: 0xfffffff020f9ed74 fp: 0xffffffe050fbb350 lr: 0xfffffff020f9d590 fp: 0xffffffe050fbb3a0 lr: 0xfffffff020f9d39c fp: 0xffffffe050fbb410 lr: 0xfffffff020f9c5f0 fp: 0xffffffe050fbb470 lr: 0xfffffff022217ee0 fp: 0xffffffe050fbb580 lr: 0xfffffff02221f8d4 fp: 0xffffffe050fbb710 lr: 0xfffffff021e2b128 fp: 0xffffffe050fbb820 lr: 0xfffffff021d49c90 fp: 0xffffffe050fbb9b0 lr: 0xfffffff021d5c17c fp: 0xffffffe050fbbb40 lr: 0xfffffff021e54d28 fp: 0xffffffe050fbbc80 lr: 0xfffffff021d31610 fp: 0xffffffe050fbbc90 lr: 0x00000001a343b0f4 fp: 0x0000000000000000 ", "panicFlags":"0x2", "otherString":" ** Stackshot Succeeded ** Bytes Traced 357072 ** ", "memoryStatus":{ "compressorSize":8413, "compressions":50291, "decompressions":21892, "busyBufferCount":0, "pageSize":16384, "memoryPressure":false, "memoryPages":{ "active":68032, "throttled":0, "fileBacked":61496, "wired":24379, "purgeable":2883, "inactive":40858, "free":11191, "speculative":27437 } } ........ }I have no idea about what to do next to resolve this problem, any suggestions will help, thanks!
Replies
6
Boosts
0
Views
14k
Activity
Jun ’23
THREAD_TIME_CONSTRAINT_POLICY for Apple Silicon
Hello, I use https://juce.com to develop an audio app for macOS. With previous macOSes on Intel, I used THREAD_TIME_CONSTRAINT_POLICY as described in https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/KernelProgramming/scheduler/scheduler.html to set real time priority for the audio processing thread, according to CPU usage values. Juce has an API to get the CPU frequency so I was able to set usage values. With macOS Monterey on Apple Silicon, the Juce API is unable to give the CPU frequency (the getCpuSpeedInMegahertz API just returns 0). Is THREAD_TIME_CONSTRAINT_POLICY still valid on latest macOS on Apple Silicon? If so, what CPU frequency value should be used - ie how can we determine it? If not, how should the real time priority be set? Thanks! -Mathieu Fyi Juce code to get CPU (SystemStats::getCpuSpeedInMegahertz) can be found at https://github.com/juce-framework/JUCE/blob/master/modules/juce_core/native/juce_mac_SystemStats.mm
Replies
0
Boosts
0
Views
968
Activity
Jun ’23
System Settings Asking Me to go to recoveryOS and change system security to allow kexts
After installing Xcode Command Line Tools, the System Settings App now asks me, to reboot my machine into recoveryOS and change the systems security to allow kernel extensions/ system extensions. It's not prominently mentioned, but just there and I never, ever had this. Must be a new dialog. Well, if you click on it... My Question is: Why? Kernel Extensions have been deprecated long ago (I thought) and the dialog mentions nothing specific.
Replies
0
Boosts
0
Views
723
Activity
Jun ’23
how to trigger sysdiagnose on iPhoneX
how to trigger sysdiagnose on iPhoneX
Replies
11
Boosts
0
Views
27k
Activity
May ’23
12 core M2 mini only using 4 cores, Intel Mac mini used 6
Developing a program that needs maximum cores. My previous Intel Mac mini used all 6 cores, according to Activity Monitor. It showed 600% cpu usage. I assumed the M2 would use twelve, but only shows 400% cpu usage in Activity Monitor, ergo, two less! I don't recall setting a core usage parameter in Xcode. Any ideas? Ventura 13.3 and Xcode 14.3 up to date as of this post.
Replies
2
Boosts
0
Views
1k
Activity
May ’23
`kmutil showloaded` shows address of 0
We have a KEXT that lists an address of 0 when we run kmutil showloaded on Apple Silicon (M1 & M2). The KEXT appears to work fine, but we are concerned that we have something misconfigured. It shows a proper address on x86_64. Any thoughts would be appreciated. macOS 13.4. Cheers, Dave ... 252 0 0xfffffe00078461f0 0x998 0x998 com.apple.driver.driverkit.serial (6.0.0) 447C62ED-B7E8-3B91-A3B1-19D2B50A20EC <46 5 4 3 1> 254 0 0xfffffe0007918e30 0x1710 0x1710 com.apple.filesystems.autofs (3.0) 7FF20081-54D5-329E-AA76-6BD3018C3167 <210 7 6 5 4 3 2 1> 256 0 0 0x1976 0x1976 com.google...
Replies
0
Boosts
1
Views
1.1k
Activity
Apr ’23
What Device Drivers are in macOS? How are they different?
I am researching the drivers in macOS. Apple Developer lists that the Default Drivers are ApplePlatformExpert, IODTPlatformExpert, IOPlatformExpert, and the Apple Drivers are IODTNVRAM, AppleMacIO, AppleMacIODevice, AppleNMI. What are these drivers and how are they different?
Replies
5
Boosts
0
Views
2.4k
Activity
Apr ’23