Files and Storage

RSS for tag

Ask questions about file systems and block storage.

Posts under Files and Storage tag

200 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

Optimizing I/O operations in kernel (VFS)
Hey everyone, I'm currently working on developing a kernel extension (kext) for the custom file system on macOS. I opted for a kernel extension due to its potential for higher performance compared to using FileProvider. However, during development, I've noticed a significant performance bottleneck related to synchronous I/O operations within the VFS subsystem. It appears that all I/O operations in the macOS kernel, such as vnop_read/vnop_write (sock_receive/sock_send), are executed synchronously. (https://forums.swift.org/t/task-safe-way-to-write-a-file-asynchronously/54639/7) For example, the Linux kernel supports asynchronous I/O operations, which utilize struct file_operations.read_iter/write_iter. This discrepancy in implementation leads to a considerable performance gap, with macOS performing approximately 8-15 times slower than Linux implementation. Given this performance difference, I'm reaching out to seek advice and insights from the community. Are there any known strategies or best practices for improving the performance of kernel extensions related to file systems on macOS? Any guidance or suggestions on how to optimize the performance of file system operations on macOS kext would be greatly appreciated. Thank you in advance for your assistance!
0
0
355
Mar ’24
Deny Mac system using cached data when copy
I'm developing a encrypte&decrypt filesystem on Mac. I use MacFuse to realize this filesystem and mount it under a folder. By doing this I can hook the open file method when user are trying to open the encrypted file(such as A.rtf) under the mounted folder. Then I will decrypt A.rtf and generate a new decrypted file, let's call it as B.rtf. In the hooked open file method, I will return B.rtf file descriptor so that the user can open the decrypted file. All works fine, until I opend the encrypted file once and copy it. It seems Mac system directly using the decrypted cache data when doing copy, so the decrypted content will be copied. I tried add fcntl(fd, F_NOCACHE, 1); fcntl(fd, F_NODIRECT, 1) after int fd = open([p UTF8String], mode);, but it not work. So is there a way to clearly tell the Mac system, do not cache my data when open files?
1
0
350
Mar ’24
The right way to use start/stop AccessingSecurityScopedResource in swift-cpp interop
Hi, so I have this case where I would like the user to pick a folder where they want to create a file/folder using UIDocumentPicker/Browser and I make the file using open() in cpp and use its fd to read/write to the file. Now, the first thing is I have to call startAccessingSecurityScopedResource() on the directory url, then I make the file, get its fd(file descriptor) and I leave this makefile() function. Every startAccessingSecurityScopedResource() needs to be matched with a stopstartAccessingSecurityScopedResource(). So my question is do I 'have' to call stopAccessingSecurityScopedResource() 'just before' calling close() on the fd. Or is it fine to call it after I have made the fd i.e., at the end of the makefile() function? In the tests I did it seems that once the fd is opened, even if stopAccessingSecurityScopedResource() is called on it(the directory), I can continue to read/write from the fd until I close() the fd?
1
0
433
Mar ’24
Available Storage displayed in iOS setting is not correct
Hi~recently I have storage issue as follows: Go to iOS Settings -> iPhone Storage , it shows available  storage  is enough (ex.180.38 GB) But app will get disk full error when trying to write large file (ex. 26 GB) Error Domain=NSCocoaErrorDomain Code=640 "The file couldn’t be saved because there isn’t enough space." UserInfo={NSUnderlyingError=0x282db8ae0 {Error Domain=NSPOSIXErrorDomain Code=28 "No space left on device"}} Then I write the following  code to log available space and found it is only 26.95 GB available not 180.38 GB as UI displayed NSError *error = nil; NSDictionary *dictionary = [[NSFileManager defaultManager] attributesOfFileSystemForPath:path error: &error]; NSNumber *sizeValue = [dictionary objectForKey:NSFileSystemFreeSize]; uint64_t totalFreeSpace = [sizeValue unsignedLongLongValue]; print(@“ %@ Free space available.", [NSByteCountFormatter stringFromByteCount:totalFreeSpace countStyle:NSByteCountFormatterCountStyleFile]); Does any one know what reason  may cause this strange situation and how can I do to make available space displayed in UI could be reliable? Any suggestion will be very appreciated!
2
0
385
Mar ’24
help for DB
good day everyone, I'm new to Xcode and I would like to start the first steps with a DB. I have two problems: 1) I have no idea which free practice DB you can use. 2) consequently I am not yet familiar with any use of the DB. I would be useful for an indication of a free DB that can be used with Xcode and possibly an example of code to be able to write and read it. I thank anyone who wants to help me
1
0
278
Mar ’24
UIDocumentPickerViewController: Cannot access file from OneDrive
Hello everybody, I am struggling with accessing files from the Location OneDrive through UIDocumentViewController. The error says: Error Domain=NSCocoaErrorDomain Code=260 "Die Datei „Testfile.txt“ konnte nicht geöffnet werden, da sie nicht existiert." UserInfo={NSFilePath=/private/var/mobile/Containers/Shared/AppGroup/11E04153-649E-416F-9860-2EA9C0913A18/File Provider Storage/item|1|18a17c69%2D5d6d%2D4b16%2Db388%2D4a9834e9440b/Testfile.txt, NSUnderlyingError=0x281202310 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}} The Controller is initialised the following way: let ctrl = UIDocumentPickerViewController(forOpeningContentTypes: [.image, .audio, .video, .item, .content]) And in the delegate method I do the following: func documentPicker(_ controller: UIDocumentPickerViewController, didPickDocumentsAt urls: [URL]) { guard let documentUrl = urls.first else { return } guard documentUrl.startAccessingSecurityScopedResource() else { parent.errorText = "Developer Error: Can't access security scoped resource." return } defer { documentUrl.stopAccessingSecurityScopedResource() } do { let data = try Data(contentsOf: documentUrl) } catch { parent.errorText = error.localizedDescription } } Any help is appreciated! Thanks
4
0
433
Mar ’24
Xcode 'New File' default location issue when the path contains a Finder tag
Recently I've had a project which always had behaved normally start placing new files in my home folder. I'm not sure what started this, but I've found out that the issue occurs if the project folder or one of its parents have a Finder tag. I don't know if the same happens with the 'old' Finder labels. I'm surprised this hasn't been discussed yet, one would think at least one developer would be using tags in their filesystem. Xcode 15.1 seems to default to the first parent it can find that doesn't hava a tag. I don't know about earlier versions. Apparently, the solution is to remove the tag AND clear the build data. An Xcode restart seems not to be needed.
2
0
406
Mar ’24
App doesn't have permission to open files
Here's how my app used to work: On one device, generate a text file with a custom extension. Send it via AirDrop or in Messages to another device. Open the file in another copy of my app. The app processes the data correctly the way I want. Now, when I try to do this, I get this: Error Domain=NSCocoaErrorDomain Code=257 "The file “Shopping List.sld” couldn’t be opened because you don’t have permission to view it." UserInfo={NSFilePath=/private/var/mobile/Library/Mobile Documents/com~apple~CloudDocs/Downloads/Shopping List.sld, NSUnderlyingError=0x282280a50 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}} file:///private/var/mobile/Library/Mobile%20Documents/com~apple~CloudDocs/Downloads/Shopping%20List.sld I think this broke with the iOS 17 upgrade. What permissions or capabilities do I need to add to my app to do that, and how do I go about adding them? I am quite new to xCode and iOS developement, but I had this working before.
1
0
565
Mar ’24
Remove data when application is moved to bin - Swift multiplatform project
I want to store sensitive data, which has to be removed when the application is moved to bin, but not when the application is updated (install newer version of application). When I move application to bin the data still remain on the MacBook so when I install the application again, the sensitive data is loaded. This problem is only on macOS, because iOS and iPadOS clear all data when I uninstall them. I tried many options of storing. SQLite file stored in applicationSupportDirectory, temporaryDirectory, but nothing worked. I also tried to store this data using UserDefaults, but also unsuccessful. I expect that I install application and it will store this data somewhere, when I update the application, the data remain, but when I uninstall this application by moving it to BIN, it will remove this data.
1
0
323
Mar ’24
What does startaccessingsecurityscopedresource() mean for different OS?
So, I'm looking into startaccessingsecurityscopedresource() function and from my current understanding this is to get temporary access to files/folders you don't implicitly have access to i.e., that don't belong to your sandbox. I can understand what it means wrt macOS, iOS, iPadOS, but what does it mean in watchOS and tvOS where there isn't any file sharing between different apps? And what is it's relevance wrt using iCloud (if there is any?)
1
0
478
Mar ’24
Codeweaver Crossover problems, com.apple.provenance and SIP
Hi So Apple Support sent me here ... (Case: 102237195769 if anyone has access) My entire file system seems to be tagged with "com.apple.provenance" tags ... they're everywhere drwxr-xr-x@ 3 jon staff 96 20 Jan 2023 .vs-kubernetes com.apple.provenance 11 drwxr-xr-x@ 5 jon staff 160 21 Jun 2023 .vscode com.apple.provenance 11 drwxr-xr-x 12 jon staff 384 28 Feb 00:17 .vscode-server drwxr-xr-x@ 3 jon staff 96 11 Dec 2022 .warp com.apple.provenance 11 -rw-r--r--@ 1 jon staff 691 29 Nov 19:14 .wget-hsts com.apple.provenance 11 -rw-r--r--@ 1 jon staff 361 11 Dec 15:55 .zprofile com.apple.provenance 11 -rw-r--r--@ 1 jon staff 300 23 Jul 2023 .zprofile.bak com.apple.provenance 11 -rw-r--r--@ 1 jon staff 136 4 Dec 2022 .zprofile.pysave com.apple.provenance 11 -rw------- 1 jon staff 16802 27 Feb 22:20 .zsh_history drwx------@ 7 jon staff 224 21 Feb 14:28 .zsh_sessions com.apple.provenance 11 -rw-r--r--@ 1 jon staff 1681 24 Feb 18:24 .zshrc Now I don't know if this is related to the CrossOver (aka Wine) problem we're experiencing, or if it's a coincidence .. however I'd like to understand why every file that gets created ends up with this tag. When installing a Bottle in CO none of the shortcuts appear within the CO App .. which is where we started all of this. jon@ziggy ~ % cd Library/Application\ Support jon@ziggy Application Support % ls -lad@ CrossOver drwxr-xr-x 7 jon staff 224 28 Feb 02:02 CrossOver jon@ziggy Application Support % cd CrossOver jon@ziggy CrossOver % ls -la@ total 784 drwxr-xr-x 7 jon staff 224 28 Feb 02:02 . drwx------+ 184 jon staff 5888 28 Feb 08:56 .. drwxr-xr-x 3 jon staff 96 28 Feb 01:59 Bottles -rw-------@ 1 jon staff 239584 28 Feb 01:59 cxfixes.xml com.apple.provenance 11 -rw-------@ 1 jon staff 256 28 Feb 01:59 cxfixes.xml.sha256 com.apple.provenance 11 -rw-------@ 1 jon staff 155031 28 Feb 02:02 gstreamer-1.0-registry.x86_64.bin com.apple.provenance 11 drwxr-xr-x 5 jon staff 160 28 Feb 01:59 tie Problem Summary The Desktop shortcut keeps randomly vanishing from the Finder Favourites section Every file is tagged as com.apple.provenance and despite trying to clear the tags (which does work) they re-appear moments later. At this stage .. I've run out of ideas, Codeweavers support has gone silent and Apple Support said "try developer.apple.com" ... sigh Last night I flattened my M1 MBP and reinstalled. All looked good until Migration Assistant got involved and then it migrated across whatever is doing this as the problem persists.
0
0
454
Feb ’24
local doc save and iCloud data in same app
I have app that is using container for small settings data and iCloud for larger storage, but I also want to be able to save to local documents folder. When I do that the url that is created is not local docs, but: path: file:///var/mobile/Containers/Data/Application/85A8B8C9-C0C3-4843-A74C-5A951F593790/Documents/Dialog08:45,%2022%20Feb%202024 Here is code using to save to local docs folder. func saveDataToFile(data: String) { //file will be datetimedialog let formatter = DateFormatter() formatter.dateFormat = "HH:mm, d MMM y" var dateTime = Date.getCurrentDate() dateTime = formatter.string(from: Date.now) var saveFilename = "Dialog" saveFilename.append(dateTime) let path = getDocumentsDirectory().appendingPathComponent(saveFilename) print("path: \(path)") // let fileURL = URL(fileURLWithPath: data, relativeTo: path) do { try data.write(to: path, atomically: true, encoding: String.Encoding.utf8) } catch { print(error.localizedDescription) } } func getDocumentsDirectory() -> URL { let paths = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask) return paths[0] }
1
0
416
Feb ’24
Options to keep config files or data files for a suite of apps
Hi, so I want to have a set of config files and data files which need to be accessible for say 3 of the apps I am building. And these files should be accessible to all 3 regardless of who makes them. The config file can have details like theme etc or sign in details etc. How do I go about doing this? Some of the things I thought of were, Using the app group storage folder, where if I understand correctly, if my apps belong to the same app group and have the required entitlement I can create files, they can be accessed by all 3 apps and they will exist until the last of the applications is uninstalled. Is this right? Letting the user pick a location/folder in "On my iPhone" in each of the apps and store the folder in bookmarks for each of the app. In this way the data will remain even after it is uninstalled. Is there any other way this can be done. Is there a recommended way of doing this?
1
0
362
Feb ’24
Advice for AppSupport to AppGroup migration before publishing
Hi all In pursuit of adopting widgets in my application, I have transitioned from AppSupport to AppGroup as storage location for Core Data. I have done a migration process/flow that goes as follows and which have been tested multiple times although I have yet to publish the update: Check if migration has taken place or not 1a. if yes continue to app 1b. If no continue flow Begin migration process 2a. Backup original store in AppSupport 2b. Migrate store to AppGroup 2c. Migrate userdefaults to AppGroup 2d. Update userdefaults with true for both hasMigratedToAppGroup and hasMigratedUserDefaultsToAppGroup Is there any tips or stuff to look for that hasn’t been taken in to account? How have you done it previously, and what would be recommended? Also, is there some specific tests to run/over many times get a baseline of how many is succeeding or failing? Thanks in advance.
4
0
559
Apr ’24
File (NSURL) programmatically downloaded from iCloud -> Could not open() the item: [1: Operation not permitted]
Hi there, i have an macOS app, sandboxed, compatibility 10.13 up to Sonoma, objective-C. I have a dropzone (or alternatively selection with NSOpenPanel) where users can drop files which results in an array of NSURLs. I create bookmarks to securely access them. This worked for years. Now i want to add iCloud support. Everything works good so far. I have methods to check the file status and download the file from icloud if NSURLUbiquitousItemDownloadingStatusKey == NSURLUbiquitousItemDownloadingStatusNotDownloaded Then i listen for the file, once the status key changes to NSURLUbiquitousItemDownloadingStatusCurrent i continue with processing the NSURL and i want to create bookmarkData: [filePathUrl bookmarkDataWithOptions:NSURLBookmarkCreationWithSecurityScope includingResourceValuesForKeys:nil relativeToURL:nil error:&error]]; But this returns the error "Could not open() the item: [1: Operation not permitted]" So i was wondering if downloading the file from iCloud now changed the NSURL itself so the given permissions by dropping do not match the downloaded file? Adding [filePathUrl startAccessingSecurityScopedResource]; didn't change anything. Any help appreciated
0
0
325
Feb ’24