Hey everyone,
I'm currently working on developing a kernel extension (kext) for the custom file system on macOS.
I opted for a kernel extension due to its potential for higher performance compared to using FileProvider. However, during development, I've noticed a significant performance bottleneck related to synchronous I/O operations within the VFS subsystem.
It appears that all I/O operations in the macOS kernel, such as vnop_read/vnop_write (sock_receive/sock_send), are executed synchronously. (https://forums.swift.org/t/task-safe-way-to-write-a-file-asynchronously/54639/7)
For example, the Linux kernel supports asynchronous I/O operations, which utilize struct file_operations.read_iter/write_iter.
This discrepancy in implementation leads to a considerable performance gap, with macOS performing approximately 8-15 times slower than Linux implementation.
Given this performance difference, I'm reaching out to seek advice and insights from the community.
Are there any known strategies or best practices for improving the performance of kernel extensions related to file systems on macOS?
Any guidance or suggestions on how to optimize the performance of file system operations on macOS kext would be greatly appreciated. Thank you in advance for your assistance!
Files and Storage
RSS for tagAsk questions about file systems and block storage.
Posts under Files and Storage tag
200 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
I'm developing a encrypte&decrypt filesystem on Mac. I use MacFuse to realize this filesystem and mount it under a folder.
By doing this I can hook the open file method when user are trying to open the encrypted file(such as A.rtf) under the mounted folder. Then I will decrypt A.rtf and generate a new decrypted file, let's call it as B.rtf. In the hooked open file method, I will return B.rtf file descriptor so that the user can open the decrypted file.
All works fine, until I opend the encrypted file once and copy it. It seems Mac system directly using the decrypted cache data when doing copy, so the decrypted content will be copied.
I tried add fcntl(fd, F_NOCACHE, 1); fcntl(fd, F_NODIRECT, 1) after int fd = open([p UTF8String], mode);, but it not work.
So is there a way to clearly tell the Mac system, do not cache my data when open files?
Hi, so I have this case where I would like the user to pick a folder where they want to create a file/folder using UIDocumentPicker/Browser and I make the file using open() in cpp and use its fd to read/write to the file. Now, the first thing is I have to call startAccessingSecurityScopedResource() on the directory url, then I make the file, get its fd(file descriptor) and I leave this makefile() function.
Every startAccessingSecurityScopedResource() needs to be matched with a stopstartAccessingSecurityScopedResource().
So my question is do I 'have' to call stopAccessingSecurityScopedResource() 'just before' calling close() on the fd.
Or is it fine to call it after I have made the fd i.e., at the end of the makefile() function?
In the tests I did it seems that once the fd is opened, even if stopAccessingSecurityScopedResource() is called on it(the directory), I can continue to read/write from the fd until I close() the fd?
ios app from app store
some day,unable to read, write any file, included SecItemAdd
What is the possible cause?
thank you
Hi~recently I have storage issue as follows:
Go to iOS Settings -> iPhone Storage , it shows available storage is enough
(ex.180.38 GB)
But app will get disk full error when trying to write large file (ex. 26 GB)
Error Domain=NSCocoaErrorDomain Code=640 "The file couldn’t be saved because there isn’t enough space." UserInfo={NSUnderlyingError=0x282db8ae0 {Error Domain=NSPOSIXErrorDomain Code=28 "No space left on device"}}
Then I write the following code to log available space and found it is only 26.95 GB available not 180.38 GB as UI displayed
NSError *error = nil;
NSDictionary *dictionary = [[NSFileManager defaultManager] attributesOfFileSystemForPath:path error: &error];
NSNumber *sizeValue = [dictionary objectForKey:NSFileSystemFreeSize];
uint64_t totalFreeSpace = [sizeValue unsignedLongLongValue];
print(@“ %@ Free space available.", [NSByteCountFormatter stringFromByteCount:totalFreeSpace countStyle:NSByteCountFormatterCountStyleFile]);
Does any one know what reason may cause this strange situation and how can I do to make available space displayed in UI could be reliable?
Any suggestion will be very appreciated!
good day everyone, I'm new to Xcode and I would like to start the first steps with a DB. I have two problems: 1) I have no idea which free practice DB you can use. 2) consequently I am not yet familiar with any use of the DB.
I would be useful for an indication of a free DB that can be used with Xcode and possibly an example of code to be able to write and read it. I thank anyone who wants to help me
suddenly
iOS app release
sandbox file
fail to open [/var/mobile/Containers/Data/Application/52B6A7C2-2F2C-400F-8D6B-4B500434F918/Library/com.zhibo8.client81136870/UserDefault/mmkv/cache], 1(Operation not permitted)
some device restart recover
Hello everybody,
I am struggling with accessing files from the Location OneDrive through UIDocumentViewController.
The error says:
Error Domain=NSCocoaErrorDomain Code=260 "Die Datei „Testfile.txt“ konnte nicht geöffnet werden, da sie nicht existiert." UserInfo={NSFilePath=/private/var/mobile/Containers/Shared/AppGroup/11E04153-649E-416F-9860-2EA9C0913A18/File Provider Storage/item|1|18a17c69%2D5d6d%2D4b16%2Db388%2D4a9834e9440b/Testfile.txt, NSUnderlyingError=0x281202310 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}
The Controller is initialised the following way:
let ctrl = UIDocumentPickerViewController(forOpeningContentTypes: [.image, .audio, .video, .item, .content])
And in the delegate method I do the following:
func documentPicker(_ controller: UIDocumentPickerViewController, didPickDocumentsAt urls: [URL]) {
guard let documentUrl = urls.first else { return }
guard documentUrl.startAccessingSecurityScopedResource() else {
parent.errorText = "Developer Error: Can't access security scoped resource."
return
}
defer {
documentUrl.stopAccessingSecurityScopedResource()
}
do {
let data = try Data(contentsOf: documentUrl)
} catch {
parent.errorText = error.localizedDescription
}
}
Any help is appreciated!
Thanks
Recently I've had a project which always had behaved normally start placing new files in my home folder. I'm not sure what started this, but I've found out that the issue occurs if the project folder or one of its parents have a Finder tag. I don't know if the same happens with the 'old' Finder labels. I'm surprised this hasn't been discussed yet, one would think at least one developer would be using tags in their filesystem.
Xcode 15.1 seems to default to the first parent it can find that doesn't hava a tag. I don't know about earlier versions.
Apparently, the solution is to remove the tag AND clear the build data. An Xcode restart seems not to be needed.
Here's how my app used to work:
On one device, generate a text file with a custom extension.
Send it via AirDrop or in Messages to another device.
Open the file in another copy of my app.
The app processes the data correctly the way I want.
Now, when I try to do this, I get this:
Error Domain=NSCocoaErrorDomain Code=257 "The file “Shopping List.sld” couldn’t be opened because you don’t have permission to view it." UserInfo={NSFilePath=/private/var/mobile/Library/Mobile Documents/com~apple~CloudDocs/Downloads/Shopping List.sld, NSUnderlyingError=0x282280a50 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}}
file:///private/var/mobile/Library/Mobile%20Documents/com~apple~CloudDocs/Downloads/Shopping%20List.sld
I think this broke with the iOS 17 upgrade.
What permissions or capabilities do I need to add to my app to do that, and how do I go about adding them? I am quite new to xCode and iOS developement, but I had this working before.
I want to store sensitive data, which has to be removed when the application is moved to bin, but not when the application is updated (install newer version of application). When I move application to bin the data still remain on the MacBook so when I install the application again, the sensitive data is loaded. This problem is only on macOS, because iOS and iPadOS clear all data when I uninstall them.
I tried many options of storing. SQLite file stored in applicationSupportDirectory, temporaryDirectory, but nothing worked. I also tried to store this data using UserDefaults, but also unsuccessful.
I expect that I install application and it will store this data somewhere, when I update the application, the data remain, but when I uninstall this application by moving it to BIN, it will remove this data.
So, I'm looking into startaccessingsecurityscopedresource() function and from my current understanding this is to get temporary access to files/folders you don't implicitly have access to i.e., that don't belong to your sandbox. I can understand what it means wrt macOS, iOS, iPadOS, but what does it mean in watchOS and tvOS where there isn't any file sharing between different apps? And what is it's relevance wrt using iCloud (if there is any?)
Hi
So Apple Support sent me here ... (Case: 102237195769 if anyone has access)
My entire file system seems to be tagged with "com.apple.provenance" tags ... they're everywhere
drwxr-xr-x@ 3 jon staff 96 20 Jan 2023 .vs-kubernetes
com.apple.provenance 11
drwxr-xr-x@ 5 jon staff 160 21 Jun 2023 .vscode
com.apple.provenance 11
drwxr-xr-x 12 jon staff 384 28 Feb 00:17 .vscode-server
drwxr-xr-x@ 3 jon staff 96 11 Dec 2022 .warp
com.apple.provenance 11
-rw-r--r--@ 1 jon staff 691 29 Nov 19:14 .wget-hsts
com.apple.provenance 11
-rw-r--r--@ 1 jon staff 361 11 Dec 15:55 .zprofile
com.apple.provenance 11
-rw-r--r--@ 1 jon staff 300 23 Jul 2023 .zprofile.bak
com.apple.provenance 11
-rw-r--r--@ 1 jon staff 136 4 Dec 2022 .zprofile.pysave
com.apple.provenance 11
-rw------- 1 jon staff 16802 27 Feb 22:20 .zsh_history
drwx------@ 7 jon staff 224 21 Feb 14:28 .zsh_sessions
com.apple.provenance 11
-rw-r--r--@ 1 jon staff 1681 24 Feb 18:24 .zshrc
Now I don't know if this is related to the CrossOver (aka Wine) problem we're experiencing, or if it's a coincidence .. however I'd like to understand why every file that gets created ends up with this tag.
When installing a Bottle in CO none of the shortcuts appear within the CO App .. which is where we started all of this.
jon@ziggy ~ % cd Library/Application\ Support
jon@ziggy Application Support % ls -lad@ CrossOver
drwxr-xr-x 7 jon staff 224 28 Feb 02:02 CrossOver
jon@ziggy Application Support % cd CrossOver
jon@ziggy CrossOver % ls -la@
total 784
drwxr-xr-x 7 jon staff 224 28 Feb 02:02 .
drwx------+ 184 jon staff 5888 28 Feb 08:56 ..
drwxr-xr-x 3 jon staff 96 28 Feb 01:59 Bottles
-rw-------@ 1 jon staff 239584 28 Feb 01:59 cxfixes.xml
com.apple.provenance 11
-rw-------@ 1 jon staff 256 28 Feb 01:59 cxfixes.xml.sha256
com.apple.provenance 11
-rw-------@ 1 jon staff 155031 28 Feb 02:02 gstreamer-1.0-registry.x86_64.bin
com.apple.provenance 11
drwxr-xr-x 5 jon staff 160 28 Feb 01:59 tie
Problem Summary
The Desktop shortcut keeps randomly vanishing from the Finder Favourites section
Every file is tagged as com.apple.provenance and despite trying to clear the tags (which does work) they re-appear moments later.
At this stage .. I've run out of ideas, Codeweavers support has gone silent and Apple Support said "try developer.apple.com" ... sigh
Last night I flattened my M1 MBP and reinstalled. All looked good until Migration Assistant got involved and then it migrated across whatever is doing this as the problem persists.
Hi,
I've got swiftUI based application. It seems that on some occasions, when the app starts, I get the following popup window but I don't know which restricted items it attempts to access (passwords,network, etc..) . How can I tell what trigger this elevation message ?
Thanks !
Anyone have any experience, started to get permission and failed to stat error for apple script which has shell do rysnc to backup files after moving files to new drive.
I have app that is using container for small settings data and iCloud for larger storage, but I also want to be able to save to local documents folder. When I do that the url that is created is not local docs, but:
path: file:///var/mobile/Containers/Data/Application/85A8B8C9-C0C3-4843-A74C-5A951F593790/Documents/Dialog08:45,%2022%20Feb%202024
Here is code using to save to local docs folder.
func saveDataToFile(data: String) {
//file will be datetimedialog
let formatter = DateFormatter()
formatter.dateFormat = "HH:mm, d MMM y"
var dateTime = Date.getCurrentDate()
dateTime = formatter.string(from: Date.now)
var saveFilename = "Dialog"
saveFilename.append(dateTime)
let path = getDocumentsDirectory().appendingPathComponent(saveFilename)
print("path: \(path)")
// let fileURL = URL(fileURLWithPath: data, relativeTo: path)
do {
try data.write(to: path, atomically: true, encoding: String.Encoding.utf8)
} catch {
print(error.localizedDescription)
}
}
func getDocumentsDirectory() -> URL {
let paths = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)
return paths[0]
}
Hi, so I want to have a set of config files and data files which need to be accessible for say 3 of the apps I am building. And these files should be accessible to all 3 regardless of who makes them. The config file can have details like theme etc or sign in details etc.
How do I go about doing this? Some of the things I thought of were,
Using the app group storage folder, where if I understand correctly, if my apps belong to the same app group and have the required entitlement I can create files, they can be accessed by all 3 apps and they will exist until the last of the applications is uninstalled. Is this right?
Letting the user pick a location/folder in "On my iPhone" in each of the apps and store the folder in bookmarks for each of the app. In this way the data will remain even after it is uninstalled.
Is there any other way this can be done. Is there a recommended way of doing this?
Hi all
In pursuit of adopting widgets in my application, I have transitioned from AppSupport to AppGroup as storage location for Core Data. I have done a migration process/flow that goes as follows and which have been tested multiple times although I have yet to publish the update:
Check if migration has taken place or not
1a. if yes continue to app
1b. If no continue flow
Begin migration process
2a. Backup original store in AppSupport
2b. Migrate store to AppGroup
2c. Migrate userdefaults to AppGroup
2d. Update userdefaults with true for both hasMigratedToAppGroup and hasMigratedUserDefaultsToAppGroup
Is there any tips or stuff to look for that hasn’t been taken in to account? How have you done it previously, and what would be recommended? Also, is there some specific tests to run/over many times get a baseline of how many is succeeding or failing?
Thanks in advance.
I want to check if a given string matches a file pattern.
Does Swift have builtin support for filename globbing? Or is there any 3rd party packages for this?
Hi there,
i have an macOS app, sandboxed, compatibility 10.13 up to Sonoma, objective-C.
I have a dropzone (or alternatively selection with NSOpenPanel) where users can drop files which results in an array of NSURLs. I create bookmarks to securely access them. This worked for years.
Now i want to add iCloud support. Everything works good so far. I have methods to check the file status and download the file from icloud if NSURLUbiquitousItemDownloadingStatusKey == NSURLUbiquitousItemDownloadingStatusNotDownloaded
Then i listen for the file, once the status key changes to NSURLUbiquitousItemDownloadingStatusCurrent i continue with processing the NSURL and i want to create bookmarkData:
[filePathUrl bookmarkDataWithOptions:NSURLBookmarkCreationWithSecurityScope
includingResourceValuesForKeys:nil
relativeToURL:nil
error:&error]];
But this returns the error "Could not open() the item: [1: Operation not permitted]"
So i was wondering if downloading the file from iCloud now changed the NSURL itself so the given permissions by dropping do not match the downloaded file?
Adding [filePathUrl startAccessingSecurityScopedResource]; didn't change anything.
Any help appreciated