Develop kernel-resident device drivers and kernel extensions using Kernel.

Posts under Kernel tag

48 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

KEXT Code Signing Problems
On modern systems all KEXTs must be code signed with a Developer ID. Additionally, the Developer ID must be specifically enabled for KEXT development. You can learn more about that process on the Developer ID page. If your KEXT is having code signing problems, check that it’s signed with a KEXT-enabled Developer ID. Do this by looking at the certificate used to sign the KEXT. First, extract the certificates from the signed KEXT: % codesign -d --extract-certificates MyKEXT.kext Executable=/Users/quinn/Desktop/MyKEXT/build/Debug/MyKEXT.kext/Contents/MacOS/MyKEXT This creates a bunch of certificates of the form codesignNNN, where NNN is a number in the range from 0 (the leaf) to N (the root). For example: % ls -lh codesign* -rw-r--r--+ 1 quinn staff 1.4K 20 Jul 10:23 codesign0 -rw-r--r--+ 1 quinn staff 1.0K 20 Jul 10:23 codesign1 -rw-r--r--+ 1 quinn staff 1.2K 20 Jul 10:23 codesign2 Next, rename each of those certificates to include the .cer extension: % for i in codesign*; do mv $i $i.cer; done Finally, look at the leaf certificate (codesign0.cer) to see if it has an extension with the OID 1.2.840.113635.100.6.1.18. The easiest way to view the certificate is to use Quick Look in Finder. Note If you’re curious where these Apple-specific OIDs comes from, check out the documents on the Apple PKI page. In this specific case, look at section 4.11.3 Application and Kernel Extension Code Signing Certificates of the Developer ID CPS. If the certificate does have this extension, there’s some other problems with your KEXT’s code signing. In that case, feel free to create a new thread here on DevForums with your details. If the certificate does not have this extension, there are two possible causes: Xcode might be using an out-of-date signing certificate. Re-create your Developer ID signing certificate using the developer site and see if the extension shows up there. If so, you’ll have to investigate why Xcode is not using the most up-to-date signing certificate. If a freshly-created Developer ID signing certificate does not have this extension, you need to apply to get your Developer ID enabled for KEXT development per the instructions on the Developer ID page. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Change history: 20 Jul 2016 — First published. 28 Mar 2019 — Added a link to the Apple PKI site. Other, minor changes. 15 Mar 2022 — Fixed the formatting. Updated the section number in the Developer ID CPS. Made other minor editorial changes.
0
0
6.0k
Mar ’22
macOS mmap / dlopen problem
We are having a problem in our C++ app with dlopen returning memory addresses which were previous reserved using mmap() with the MAP_ANON | MAP_PRIVATE | MAP_JIT flags. The mmap is memory is 4Kb page-aligned and returns normally, however sometime later dlopen() is returning an address within the mmap range when no munmap() has been performed. This looks like a bug in the macOS kernal memory manager. Back in July, I opened support ticket FB14442215 where one of our Engineers was able to create a similar and reproducible problem using Preview to load a large bitmap. This ticket has not yet been acted upon, still showing a status of "Open" . Any help or suggestions would be most welcome. Norm Green norm(dot)green(at)gemtalksystems(dot)com
1
0
114
1w
iOS writeback behavior for mmap(MAP_SHARED) dirty pages
I'm evaluating a technique to implement a sort of an event logger that uses MAP_SHARED mapping of a file in the app sandbox as an event ring buffer. The reason to use mapping instead of traditionally allocated memory is to achieve log persistence across app termination of any kind (crashes, sigkill, etc.) and keep logs fast by avoiding syscalls. By definition MAP_SHARED area must be coherent with any other RW operations in the system on that file slice which practically means that kernel has to use page cache that is used to serve RW requests. This in turn means that after app process terminates by any reason - content of that memory will not be discarded but rather will be available on next app start via open()/read() or mmap() for that file. msync() can be used to tell kernel to initiate "writeback" - to flush modified mapping pages to the corresponding locations in the non-volatile storage but I haven't found any description of what is the writeback policy if user opts to NOT use msync() at all. And similarly no means to control this. In my case it appears to be important as if kernel does some automatic writebacks on its own - intensive logger traffic would put unneeded IO load to a disk device. After some experiments I was able to figure out that e.g. Linux is able to issue periodical writebacks w/o explicit msync(). For OS X according to "fs_usage -f diskio" no writeback occurs until app terminates (better to say until last reference in the system to that MAP_SHARED area is dropped). I'm now interested to learn about iOS behavior. Is it the same as OS X (no automatic writebacks)? Alternatively I'd happy to hear if there are other techniques available for iOS app to "pin" some memory so its content could survive app termination. Shared memory with an associated "retainer" process would work on other platforms but here we are limited to a single process. Thanks.
5
0
220
5d
How to Symbolicate an Apple Silicon Panic?
Investigating a kernel panic, I discovered that Apple Silicon Panic traces are not working with how I know to symbolicate the panic information. I have not found proper documentation that corrects this situation. Attached file is an indentity-removed panic, received from causing an intentional panic (dereferencing nullptr), so that I know what functions to expect in the call stack. This is cut-and-pasted from the "Report To Apple" dialog that appears after the reboot: panic_1_4_21_b.txt To start, I download and install the matching KDK (in this case KDK_14.6.1_23G93.kdk), identified from this line: OS version: 23G93 Kernel version: Darwin Kernel Version 23.6.0: Mon Jul 29 21:14:04 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T8122 Then start lldb from Terminal, using this command: bash_prompt % lldb -arch arm64e /Library/Developer/KDKs/KDK_14.6.1_23G93.kdk/System/Library/Kernels/kernel.release.t8122 Next I load the remaining scripts per the instructions from lldb: (lldb) settings set target.load-script-from-symbol-file true I need to know what address to load my kext symbols to, which I read from this line of the panic log, after the @ symbol: com.company.product(1.4.21d119)[92BABD94-80A4-3F6D-857A-3240E4DA8009]@0xfffffe001203bfd0->0xfffffe00120533ab I am using a debug build of my kext, so the DWARF symbols are part of the binary. I use this line to load the symbols into the lldb session: (lldb) addkext -F /Library/Extensions/KextName.kext/Contents/MacOS/KextName 0xfffffe001203bfd0 And now I should be able to use lldb image lookup to identify pointers on the stack that land within my kext. For example, the current PC at the moment of the crash lands within the kext (expected, because it was intentional): (lldb) image lookup -a 0xfffffe001203fe10 Which gives the following incorrect result: Address: KextName[0x0000000000003e40] (KextName.__TEXT.__cstring + 14456) Summary: "ffer has %d retains\n" That's not even a program instruction - that's within a cstring. No, that cstring isn't involved in anything pertaining to the intentional panic I am expecting to see. Can someone please explain what I'm doing wrong and provide instructions that will give symbol information from a panic trace on an Apple Silicon Mac? Disclaimers: Yes I know IOPCIFamily is deprecated, I am in process of transitioning to DriverKit Dext from IOKit kext. Until then I must maintain the kext. Terminal command "atos" provides similar incorrect results, and seems to not work with debug-built-binaries (only dSYM files) Yes this is an intentional panic so that I can verify the symbolicate process before I move on to investigating an unexpected panic I have set nvram boot-args to include keepsyms=1 I have tried (lldb) command script import lldb.macosx but get a result of error: no images in crash log (after the nvram settings)
3
0
242
2w
vnop_advlock not being called for my filesystem
When running AJA System Test for my custom filesystem, the write and read tests get stuck intermittently. I didn't observe any error codes being returned by my vnop_read/write or sock_receive/send functions. Dtrace(1)'ing the vnops being called by AJA System Test for smbfs revealed that amongst other things vnop_advlock is being called: 0 -> smbfs_vnop_advlock ajasystemtest -> smbfs_vnop_advlock(ajatest.dat, op: 0x2, fl->l_start: 0, fl->l_len: 0, fl->l_pid: 0, fl->l_type: 2, fl->l_whence: 0, flags: 0x40, timeout: 0) 0 <- smbfs_vnop_advlock ajasystemtest -> smbfs_vnop_advlock(ajatest.dat) -> -1934627947504 op: 0x2 #define F_SETFD 2 /* set file descriptor flags */ fl->l_len: 0 ;len = 0 means until end of file fl->l_type: 2 ;#define F_UNLCK 2 /* unlock */ fl->l_whence: 0 ;#define SEEK_SET 0 /* set file offset to offset */ flags: 0x40 ;#define F_POSIX 0x040 /* Use POSIX semantics for lock */ As my filesystem didn't implement vnop_advlock, I thought I'd explore that avenue. My vnop_advlock simply returns KERN_SUCCESS. Both f_capabilities.valid and f_capabilities.capabilities of struct vfs_attr have VOL_CAP_INT_ADVLOCK and VOL_CAP_INT_FLOCK set. Yet, vnop_advlock doesn't get called for my filesystem when running AJA System Test. Any tips on what could be amiss there would be much appreciated.
4
0
255
2w
Kext run well on Xcode but kextload failed after archive to others
I was developing kext for use on Apple Silicon Mac, the kext ran well when I put into the app project. However, after I archived the app and installed it on other computers, I got some error messages while doing kextload command. I have no idea what to do, the error messages are as following: "Error domain=KMErrorDomain Code=71 Incompatible architecture: Cannot find arm64e in fat binary. Unsupported Error: one or more extensions are unsupported to load." I run Xcode projects on M2 Pro MAX and I'm sure the other Mac have been closed SIP. If I use the other Mac to run xcode project, the kext works fine too.
1
0
178
2w
vnop_strategy unexpectedly zero-extends files
On implementing vnop_mmap, vnop_strategy and other related VNOPs as suggested in https://developer.apple.com/forums/thread/756358 my vnop_strategy routine ends up zero-extending files. I don't understand why my filesystem behaves as described above. Perusing the source code of both the relevant parts of Darwin/XNU and SMBClient did not clarify things for me. A nudge in the right direction would be greatly appreciated. The technical details of the issue are given in the plain text file attached, as some text was found to be sensitive. Unsure what exactly it was. apple-dts-issue-desc.txt
3
0
314
Jul ’24
APPUL OS X LION 10.7 kernel debug kit problems (10.7.{0,4}) + 11G63 unavailable
i'm trying to get firefox running in 10.7 but the kernel is crashing. i have it working/running/looking great on 10.8 and higher. something is happening in 10.7 that is causing the panic. i need the right kit. someone asked a similar question for 10.11: https://forums.developer.apple.com/forums/thread/108732 but feedback assistant doesn't seem like the right answer. i need this kit to move forward. right now using either the 10.7.0 or 10.7.4 kernel debug kit causes a panic on reboot; something to do with AVX and the fpu. i am hoping APPUL had enough foresight to see this would be an issue (even one year later) for people on newer architectures debugging for older, supported (until 2014) targets: this use-case definitely falls within the parameters.
0
0
243
Jul ’24
Low-level event-posting help needed.
Hi there, I am working on a little application which processes cursor and graphics tablet data and adds some extra control to the output. So far it makes use of... if let eventTap = CGEvent.tapCreate(tap: .cgSessionEventTap, //.cghidEventTap place: .headInsertEventTap, options: .defaultTap, eventsOfInterest: eventMask, callback: handleTapEvent, userInfo: userInfo) ... to modify existing events. The issue that in some cases arises (it's a globally working app) - that some other applications pull and process pointer-data aside the event stream and therefor create conflicting values. Would creating and posting events to a 'virtual pointing device' on a lower system level (kext) help? Let's discuss. BR, E
0
0
222
Jul ’24
Missing Developer Kit Error(Code=34)
sudo kmutil create -n boot -B /usr/share/file/magic/kernelcache -k /Library/Extensions Error Domain=KMErrorDomain Code=34 "Missing Developer Kit: As of macOS 13.0, you will need to install a KDK matching your build 23F79 to rebuild kernel collections." UserInfo={NSLocalizedDescription=Missing Developer Kit: As of macOS 13.0, you will need to install a KDK matching your build 23F79 to rebuild kernel collections.} I'm using KDK 14.5(23F5074a) on macOS 14.5(23F79), and trying to build kernel collection. It seems the kernel debug kit for macOS 14.5(23F79) is not supported. Is there any replacement for macOS 14.5(23F79)?
0
0
313
Jun ’24
Spotlight / Finder Search / Finder Tags not working on virtual file system Monterey/Ventura
I'm writing a virtual file system as my educational project (generic kernel extension). Currently, mostly everything is implemented, however, I'm having trouble using Finder search and tags. The results simply don't show up - despite I am having vnop_... calls to those files. The extended attributes are supported. Inodes are stable. Mmap is implemented. Vnop_ioctl returns KERN_SUCCESS (but no implementation). An important moment: Previously, the search didn't work at all. Researching the web has shown me, that Spotlight indexation and Finder search are tightly glued. So basically I was trying to enable support for spotlight, thinking that would be the source of the problem. I was receiving "Unknown indexing state". All those tricks with mdutil, launchd, manual and reindexation either were doing nothing or returning error. The problem was resolved FOR SONOMA by making by VFS appear as local one (adding flags for MNT_LOLCAL and MNT_DOVOLFS). This has changed the state from Unknown indexing state for spotlight to Indexing is disabled. No need to turn it on for me - I am interested only in search and tags, not the spotlight itself. Basically, whether spotlight recognises my driver as no-error, the Finder works correctly, even with indexation disabled. Whether on Monterey*, or Ventura, I get the same problem. However, neither system logs nor my driver show any kinds of errors. The spotlight simply returns error. Reindexation attempt via Security&Privacy returns "Unknown error occured". The metadata for Ventura and Monterey read attempt (mdls) returns "Unable to locate file", however returns a huge list for Sonoma. *Monterey and Ventura never have .Spotlight-V100 folder. No disable indexing files or other spotlight restrictions are present. No user space solutions seem to help. The kext is unsigned and running in an environment with SIP disabled and Security Mode reduced to Permissive. Maybe there some abstract rules for what is required on VFS side to be recognised okay'ish by Spotlight ? Or maybe something specific right for my case ? Any pointers and/or assistance would be greatly appreciated.
9
0
525
Jun ’24
Maximum stack space
According https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/Multithreading/CreatingThreads/CreatingThreads.html maximum stack size is 512 KB for secondary threads, 8 MB for OS X main thread and 1 MB for iOS main thread. Can someone tell actual information about maximum stack size on different OS?
2
0
306
Jun ’24
How can I get Apple to fix a major bug that happens under specific circumstances?
This is the bug: https://forums.developer.apple.com/forums/thread/729348?answerId=780151022#780151022 [quote='780151022, LostButFound, /thread/729348?answerId=780151022#780151022, /profile/LostButFound'] This issue still happens for me. With and without VPN, with both OpenVPN and Wireguard. With and without filtering (mostly with filters though, both LuLu and Little Snitch). On two different machines, M1 and M3. It's random. If I have to bet, there's a deadlock caused by running x86 code on Arm hardware. As a software engineer I saw cases where Intel tolerates bad multI-threading more than Arm. Are the Apple devs working on this? This is a very serious issue that makes these very expensive laptop unusable! At least provide a way to reset the network stack! 0  comments [/quote] It seems that the Network Extension Framework has a bug where combining VPN and Network filters causes deadlocks and timeouts in networking on irregular basis, regardless of the network interface used. This issue happens at a lower level than network interfaces. It can be either a Network Extension Framework bug or a kernel module bug. Given that the network extension frame isn't open source, I can't even debug it, which I would've happily done. Yes, even though I'm not an apple developer, but this bug is so bad that I'm happy to build MacOS components in debug mode and attach a debugger on them when the issue happens... but it's not open source. So, we have a bug, and we need help from apple devs to fix it... what am I supposed to do? Is there a chance I can get a dev to contact me to debug this together? What are my options?
0
0
388
May ’24
SA_RESETHAND flag in sigaction not working for swift exceptions
I have an exception handling frame for an Xcode application in macOS, which contains Cpp and Swift code. I am using the Unix signals frame for handling exceptions using sigaction. My sigaction signal handler get invoked when there is a swift or Cpp exception. However for some exceptions like SIGSEGV, the signal handler gets called repeatedly. To handle this I am using the SA_RESETHAND flag so that the handler gets invoked only once, and then the default action for the signal take over to terminate the process. This approach works well when an exception occurs due to Cpp code, however when it occurs due to Swift code, the signal handler still gets invoked repeatedly. Can someone explain why is this happening and What is the solution to this?
1
0
394
Apr ’24
How to get the new created vnode since kauth_listen_scope is invalid now
I'm finding a way to hook vnode operations, following is a snippet of the code: IOReturn FltIOKitKAuthVnodeGate::RegisterVnodeScopeCallback(void) { // // register our listener // this-&gt;VnodeListener = kauth_listen_scope( KAUTH_SCOPE_VNODE, // for the vnode scope FltIOKitKAuthVnodeGate::VnodeAuthorizeCallback, // using this callback this ); // give a cookie to callback if( NULL == this-&gt;VnodeListener ){ DBG_PRINT_ERROR( ( "kauth_listen_scope failed\n" ) ); return kIOReturnInternalError; } return kIOReturnSuccess; } Here use kauth_listen_scope to get the newly created vnode object, then will hook on it. But now kauth_listen_scope is deprecated, and there is no way to get the vnode by using EndpointSecurity. So is there any other way to get the newly created vnode object?
8
0
898
Apr ’24
How to use mach Exception handling
I wanted to perform handling for the exception in my mac and ios application, I am following this link, where it is suggested to follow either the mach exception handling or use Unix signals. I did not find many resources that could be followed to implement mach exception as suggested. Below are the few resources I could find. Can someone point to the some documentation that apple provides for this or some other helpful documentation. https://gist.github.com/rodionovd/01fff61927a665d78ecf
3
0
622
Apr ’24
How to make sysctl and gettimeofday thread-safe in Swift?
I am trying to sync the ntp time from the server using Kronos library. However, I believe the code is not fully protected from multithreading access since it is using low level system code. So, does anyone know how can I ensure sysctl and gettimeofday are thread-safe when calling them? Or, is there any thread-safe alternative to get the same result? func currentTime() -&gt; TimeInterval { var current = timeval() let systemTimeError = gettimeofday(&amp;current, nil) != 0 assert(!systemTimeError, "system clock error: system time unavailable") return Double(current.tv_sec) + Double(current.tv_usec) / 1_000_000 } static func systemUptime() -&gt; TimeInterval { var mib = [CTL_KERN, KERN_BOOTTIME] var size = MemoryLayout&lt;timeval&gt;.stride var bootTime = timeval() let bootTimeError = sysctl(&amp;mib, u_int(mib.count), &amp;bootTime, &amp;size, nil, 0) != 0 assert(!bootTimeError, "system clock error: kernel boot time unavailable") let now = currentTime() let uptime = Double(bootTime.tv_sec) + Double(bootTime.tv_usec) / 1_000_000 assert(now &gt;= uptime, "inconsistent clock state: system time precedes boot time") return now - uptime } I have thought of using NSLock but I can only protect from the getter (caller) not the setter (system)
14
0
768
May ’24