Files and Storage

RSS for tag

Ask questions about file systems and block storage.

Posts under Files and Storage tag

200 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

Loading CoreML model increases app size?
Hi, i have been noticing some strange issues with using CoreML models in my app. I am using the Whisper.cpp implementation which has a coreML option. This speeds up the transcribing vs Metal. However every time i use it, the app size inside iphone settings -> General -> Storage increases - specifically the "documents and data" part, the bundle size stays consistent. The Size of the app seems to increase by the same size of the coreml model, and after a few reloads it can increase to over 3-4gb! I thought that maybe the coreml model (which is in the bundle) is being saved to file - but i can't see where, i have tried to use instruments and xcode plus lots of printing out of cache and temp directory etc, deleting the caches etc.. but no effect. I have downloaded the container of the iphone from xcode and inspected it, there are some files stored inthe cache but only a few kbs, and even though the value in the settings-> storage shows a few gb, the container is only a few mb. Please can someone help or give me some guidance on what to do to figure out why the documents and data is increasing? where could this folder be pointing to that is not in the xcode downloaded container?? This is the repo i am using https://github.com/ggerganov/whisper.cpp the swiftui app and objective-C app both do the same thing i am witnessing when using coreml. Thanks in advance for any help, i am totally baffled by this behaviour
2
1
438
2w
Xcode 15.3+ bug: framework tests executed in fresh simulator fail to write data to disk
I have filed this as FB13722352. I am sharing it here because I haven't seen it mentioned anywhere online yet and am curious if anyone else has run into it. In Xcode 15.3+, writing data to disk fails when running the tests for a Framework project in a fresh simulator. Specifically, if the selected simulator has never been launched before (i.e. is newly-created), any test execution that attempts to write data into the NSDocumentDirectory directory will fail for a period of time after the simulator is first launched (I've observed between 10s and 20s). After that period of time, the same data write action will succeed. It appears that Xcode 15.3+ is starting test execution too soon, without waiting a sufficient amount of time for the Simulator to fully boot. This issue does not occur in Xcode 15.2 or prior versions. Since the issue only appears in a fresh (never-before booted) simulator, it is likely to pop up consistently in CI test runs (where simulators are not re-used). This can cause confusion because the same test would not fail locally when re-using an existing simulator. When the issue appears, the file write API returns the following error: Domain=NSCocoaErrorDomain Code=4 "The folder “testFile” doesn’t exist." UserInfo={NSFilePath=[...]/data/Documents/testFile, NSUserStringVariant=Folder, NSUnderlyingError= {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"} } Reproduction Steps: Open Xcode 15.3 or 15.4. Make sure Simulator.app is closed. Using the "Devices and Simulators" window, create a new iPhone 15 Pro simulator with iOS 17.4 (other devices and OS versions work as well). Do not launch this new simulator. Create a new Framework project and add a test that performs and then checks the output of a data write to the Document directory (see example test code below). Select the new simulator (created in step 2) as the test run target and run the test. Here's an example test that fails in the scenario outlined above: - (void)testBasicRepro { NSString *testString = @"Hello, World!"; NSData *data = [testString dataUsingEncoding:NSUnicodeStringEncoding]; // Get documents directory NSURL *url = [[[NSFileManager defaultManager] URLsForDirectory:NSDocumentDirectory inDomains:NSUserDomainMask] lastObject]; NSURL *testFileURL = [url URLByAppendingPathComponent:@"testFile"]; // Write the data NSError *error; bool result = [data writeToURL:testFileURL options:NSDataWritingAtomic error:&error]; // Check if it was successful XCTAssertTrue(result); XCTAssertNil(error); XCTAssertTrue([[NSFileManager defaultManager] fileExistsAtPath:testFileURL.path]); } Workaround The workaround that I have come up with is to create a test that runs first (by disabling parallelization and randomization, and making sure the test class filename is alphabetically first). Alternatively, it could be called from the setUp method in any test files that are affected. This test performs a data write and checks the result in a loop in order to block until the data write succeeds (i.e. the Simulator is sufficiently booted for data write operations to complete). - (void)testWorkaroundBug { NSString *testString = @"Hello, World!"; NSData *data = [testString dataUsingEncoding:NSUnicodeStringEncoding]; NSError *error; // Get documents directory NSURL *documentsURL = [[[NSFileManager defaultManager] URLsForDirectory:NSDocumentDirectory inDomains:NSUserDomainMask] lastObject]; NSURL *testFileURL; NSDate *startTime = [NSDate date]; NSLog(@"Starting test at %@", startTime); for (int i = 0; i < 120; i++) { // Create unique URL testFileURL = [documentsURL URLByAppendingPathComponent:[NSString stringWithFormat:@"testFile-%@", @(i)]]; // Write the data BOOL success = [data writeToURL:testFileURL options:NSDataWritingAtomic error:&error]; // Check if it exists if (success && [[NSFileManager defaultManager] fileExistsAtPath:testFileURL.path]) { NSLog(@"Test file %@ was created successfully! Elapsed time %@s", @(i), @(fabs([startTime timeIntervalSinceNow]))); return; } else { NSLog(@"Test file %@ was not created. Error: %@. Sleeping for 0.5s and trying again.", @(i), error); [NSThread sleepForTimeInterval:0.5]; } } }
1
0
348
May ’24
Are Open OutputStreams Buffered or Unbuffered?
I am working on an app that is streaming data from a bluetooth device to an iPhone. As data constantly arrives to the phone, I am repeatedly decoding data segments and sending it to a file through a OutputStream (from Foundation). Trying to streamline this background work, I want to make sure the data I/O is using a buffered write. Looking over documentation for the OutputStream class, I cannot find any mention of how to flush the buffer if needed, and am unsure if the Stream object is using a buffer across my repeated calls. Are OutputStreams sending binary data to a file buffered or unbuffered? Thank you in advance for clarifying this mechanic! I am using XCode Version 15.3, and Swift 5.10
1
0
207
May ’24
MacOS Finder Preview/Thumbnail Generation Limited to 20-21 Alpha Channels?
I am using the following shell script to return an image preview for use in FileMaker: qlmanage -t [sourcePath] -s 512 -o [outputPath] This usually works well, but it hangs if the RGB image (.tif, .psb, or .psd) has too many Alpha Channels ( >20 if on transparent background; >21 if flattened). This issue can be also be seen when looking at the image thumbnail or preview in the Finder. It appears MacOS won't create a thumbnail when the image has over 21 Alpha Channels... it just shows the default tif/psb/psd thumbnail, even if the image is very small. Environment MacOS Sonoma 14.4.1 Adobe Photoshop 2024 (25.6.0) Maximize PSD and PSB File Compatibility is enabled when saved from Photoshop Since I'm only able to upload a screenshot to this post, the original test files can be found in the Adobe Forum with the Title: "MacOS Finder Preview Limited to 20-21 Alpha Channels?"
1
0
262
May ’24
UTI Conflicts, iOS
My iOS app wants to associate itself with a certain file extension, let's say .aaa. To do so I declare the following exported type: &lt;dict&gt; &lt;key&gt;UTTypeConformsTo&lt;/key&gt; &lt;array&gt; &lt;string&gt;public.data&lt;/string&gt; &lt;/array&gt; &lt;key&gt;UTTypeDescription&lt;/key&gt; &lt;string&gt;AAA File&lt;/string&gt; &lt;key&gt;UTTypeIdentifier&lt;/key&gt; &lt;string&gt;com.myapp.aaa&lt;/string&gt; &lt;key&gt;UTTypeTagSpecification&lt;/key&gt; &lt;dict&gt; &lt;key&gt;public.filename-extension&lt;/key&gt; &lt;array&gt; &lt;string&gt;aaa&lt;/string&gt; &lt;/array&gt; &lt;/dict&gt; &lt;/dict&gt; As well as this under the supported document types: &lt;dict&gt; &lt;key&gt;CFBundleTypeExtensions&lt;/key&gt; &lt;array&gt; &lt;string&gt;aaa&lt;/string&gt; &lt;/array&gt; &lt;key&gt;CFBundleTypeName&lt;/key&gt; &lt;string&gt;AAA File&lt;/string&gt; &lt;key&gt;LSItemContentTypes&lt;/key&gt; &lt;array&gt; &lt;string&gt;com.myapp.aaa&lt;/string&gt; &lt;/array&gt; &lt;key&gt;LSHandlerRank&lt;/key&gt; &lt;string&gt;Owner&lt;/string&gt; &lt;/dict&gt; Turns out, several other apps on the App Store also register the .aaa file extension, each under a different UTI, making it impossible to open .aaa in my app if they installed it after already having installed another app claiming this extension. Moreover, some of these app declare all of their supported file extensions under a single UTI (e.g. both .aaa and .bbb are associated with the com.theirapp.generic UTI), so I can't even add their UTI to LSItemContentTypes without associating myself with files I don't support. How do I force iOS to allow opening every file with the .aaa extension with my app, regardless of any potential third-party app registering the same extension? For clarification – the .aaa file extension in this example is always associate with a single type and format, but it doesn't have an agreed-on UTI identifier or MimeType, nor is there a single app that should be the sole "exporter" of the UTI type.
2
0
330
May ’24
Trigger permission dialog for file access from kind of user supplied path.
I have the following situation: My SwiftUI App for macOS is using App Sandbox and is currently configured for read/write access for all the locations selectable in XCode I have added a file selector using a button and NSOpenPanel() to let the user select a folder containing a database file, to which I successfully get permissions using URL.bookmarkData() and URL.startAccessingSecurityScopedResource() I then try to read file paths from the database file and open those but I instantly get a permission error without a permission dialog/prompt appearing In my test I am using paths to files in my iCloud Drive folder I added all file/folder related usage string entries to the Info.plist for testing I think this is weird, since I can paste one of those file:// URLs from the database into a (non-Safari) browser and it shows the native permission dialog/prompt before downloading the file as expected. Is there any usage string that's not shown in the Info.plist Dropdown in XCode that I need to add to my app in order for this to work?
3
0
471
May ’24
Does macOS clean /tmp dir automatically in app containers?
I know that system /tmp and $TMPDIR are cleaned periodically and on reboot, but what about /tmp directory inside app containers? Because it looks like on macOS Sonoma it is not cleaned automatically and I was wondering if it is by design? And what should I do about it? Should I delete these files manually for existing users or is it possible to somehow nudge macOS into doing it?
2
0
344
May ’24
Why macOS Sonoma doesn't delete /tmp files in app containers?
It looks like macOS Sonoma doesn't delete /tmp files in containers periodically or on reboot, like it does for system /tmp or $TMPDIR. Is it a bug or is it in on purpose? Our app didn't delete its temp files and some users have quite a few of them. We will make the fix so that we clear after ourselves in the future but should we delete temp files manually on start for existing users?
2
0
253
May ’24
Skip FileProvider folders without metadata
I want to traverse my local Google Drive folder to calculate the size of all the files on my drive. I'm not interested in files or directories that are not present locally. I use getattrlistbulk for traversing and it takes way too much time. I think it is because FileProvider tries to download metadata for the directories that are not yet materialised. Is there a way to skip non-materialised directories?
2
0
334
May ’24
FileDescriptor writing to an unexpected file
I'm using a file descriptor to write into a file. I've encountered a problem where if the underlying file is removed or recreated, the file descriptor becomes unstable. I have no reliable way to confirm if it's writing on the expected file. let url = URL(fileURLWithPath: "/path/") try FileManager.default.removeItem(at: url) FileManager.default.createFile(atPath: url.path, contents: .empty) let filePath = FilePath(url.path) var fileDescriptor = try FileDescriptor.open(filePath, .readWrite) // The file is recreated - may be done from a different process. try FileManager.default.removeItem(at: url) // L9 FileManager.default.createFile(atPath: url.path, contents: .empty) // L10 let dataToWrite = Data([1,1,1,1]) try fileDescriptor.writeAll(dataToWrite) // L13 let dataWritten = try Data(contentsOf: url) print(dataToWrite == dataWritten) // false I would expect L13 to result in an error. Given it doesn't: Is there a way to determine where fileDescriptor is writing? Is there a way to ensure that fileDescriptor is writing the content in the expected filePath?
8
0
520
Jun ’24
Files App Share Context with Security scoped resource fails
I'm creating an App that can accepted PDFs from a shared context. I am using iOS, Swift, and UIKit with IOS 17.1+ The logic is: get the context see who is sending in (this is always unknown) see if I can open in place (in case I want to save later) send the URL off to open the (PDF) document and load it into PDFKit's pdfView.document I have no trouble loading PDF docs with the file picker. And everything works as expected for shares from apps like Messages, email, etc... (in which case URLContexts.first.options.openInPlace == False) The problem is with opening (sharing) a PDF that is sent from the Files App. (openInPlace == True) If the PDF is in the App's Document Folder, I need the Security scoped resource, to access the URL from the File's App so that I can copy the PDF's data to the PDFViewer.document. I get Security scoped resource access granted each time I get the File App's context URL. But, when I call fileCoordinator.coordinate and try to access a file outside of the App's document folder using the newUrl, I get an error. FYI - The newUrl (byAccessor) and context url (readingItemAt) paths are always same for the Files App URL share context. I can, however, copy the file to a new location in my apps directory and then open it from there and load in the data. But I really do not want to do that. . . . . . Questions: Am I missing something in my pList or are there other parameters specific to sharing a file from the Files App? I'd appreciate if someone shed some light on this? . . . . . Here are the parts of my code related to this with some print statements... . . . . . SceneDelegate func scene(_ scene: UIScene, openURLContexts URLContexts: Set<UIOpenURLContext>) { // nothing to see here, move along guard let urlContext = URLContexts.first else { print("No URLContext found") return } // let's get the URL (it will be a PDF) let url = urlContext.url let openInPlace = urlContext.options.openInPlace let bundleID = urlContext.options.sourceApplication print("Triggered with URL: \(url)") print("Can Open In Place?: \(openInPlace)") print("For Bundle ID: \(bundleID ?? "None")") // get my Root ViewController from window if let rootViewController = self.window?.rootViewController { // currently using just the view if let targetViewController = rootViewController as? ViewController { targetViewController.prepareToLoadSharedPDFDocument(at: url) } // I might use a UINavigationController in the future else if let navigationController = rootViewController as? UINavigationController, let targetViewController = navigationController.viewControllers.first as? ViewController { targetViewController.prepareToLoadSharedPDFDocument(at: url) } } } . . . . ViewController function I broke out the if statement for accessingScope just to make it easier for me the debug and play around with the code in accessingScope == True func loadPDF(fromUrl url: URL) { // If using the File Picker / don't use this // If going through a Share.... we pass the URL and have three outcomes (1, 2a, 2b) // 1. Security scoped resource access NOT needed if from a Share Like Messages or EMail // 2. Security scoped resource access granted/needed from 'Files' App // a. success if in the App's doc directory // b. fail if NOT in the App's doc directory // Set the securty scope variable var accessingScope = false // Log the URLs for debugging print("URL String: \(url.absoluteString)") print("URL Path: \(url.path())") // Check if the URL requires security scoped resource access if url.startAccessingSecurityScopedResource() { accessingScope = true print("Security scoped resource access granted.") } else { print("Security scoped resource access denied or not needed.") } // Stop accessing the scope once everything is compeleted defer { if accessingScope { url.stopAccessingSecurityScopedResource() print("Security scoped resource access stopped.") } } // Make sure the file is still there (it should be in this case) guard FileManager.default.fileExists(atPath: url.path) else { print("File does not exist at URL: \(url)") return } // Let's see if we can open it in place if accessingScope { let fileCoordinator = NSFileCoordinator() var error: NSError? fileCoordinator.coordinate(readingItemAt: url, options: [], error: &error) { (newUrl) in DispatchQueue.main.async { print(url.path()) print(newUrl.path()) if let document = PDFDocument(url: newUrl) { self.pdfView.document = document self.documentFileName = newUrl.deletingPathExtension().lastPathComponent self.fileLoadLocation = newUrl.path() self.updateGUI(pdfLoaded: true) self.setPDFScale(to: self.VM.pdfPageScale, asNewPDF: true) } else { print("Could not load PDF directly from url: \(newUrl)") } } } if let error = error { PRINT("File coordination error: \(error)") } } else { DispatchQueue.main.async { if let document = PDFDocument(url: url) { self.pdfView.document = document self.documentFileName = url.deletingPathExtension().lastPathComponent self.fileLoadLocation = url.path() self.updateGUI(pdfLoaded: true) self.setPDFScale(to: self.VM.pdfPageScale, asNewPDF: true) } else { PRINT("Could not load PDF from url: \(url)") } } } } . . . . Other relevant pList settings I've added are: Supports opening documents in place - YES Document types - PDFs (com.adobe.pdf) UIDocumentBrowserRecentDocumentContentTypes - com.adobe.pdf Application supports iTunes file sharing - YES And iCloud is one for Entitlements with iCloud Container Identifiers Ubiquity Container Identifiers . . . . Thank you in advance!. B
1
0
344
Jun ’24
Sanboxed apps won't open 3rd party filesystem files
I'm having trouble opening files residing on a custom filesystem implemented as a kext via sandboxed apps. Preview.app is one such example. The app launches, but it won't display file contents. In system log files I'm seeing entries related to com.apple.foundation.filecoordination:claims with no error messages to indicate a possible reason why file contents aren't being displayed. Non-sandboxed apps, such as GoogleChrome.app do not exhibit such behaviour. The kext is unsigned and running in an environment with SIP disabled and Security Mode reduced to Permissive. What is required for a 3rd party filesystem kext to integrate with sandboxed apps? Any pointers and/or assistance would be greatly appreciated.
10
2
584
21h
NSFilePresenter Does Not Seem to Work on watchOS on Device
Hi, I submitted a Feedback Report (FB13820685) but I thought I would ask here as well because maybe I am using the framework wrong. I am using NSFilePresenter to monitor changes to a folder. On macOS, iOS (simulator), iOS (device), and watchOS (simulator) it works fine. However when running on watchOS 10.5 on device, it does not appear to work at all. I created a sample project that reproduces this problem. Am I doing something wrong? It seems like this is too basic of a problem for it to be actually broken on all Apple Watches. https://github.com/jeffreybergier/NSFilePresenterBugSampleProject
0
0
212
Jun ’24
open (1) fails with fnfErr while open (2) succeeds on custom filesystem
Hello, I have developed a custom filesystem in golang, that relies on macFUSE. High-level apps on osx (TextEdit, Numbers, Preview) rely on syscall.renamex_np with the flag RENAME_SWAP in order to save edits. In golang, the sys call renamex_np and renameat2 are not supported, thus I had to implement the logic for it it. The discussion opened on the google group for macFUSE can be followed here: https://groups.google.com/g/osxfuse-group/c/Kh0qVRGIVv4 On my mounted filesystem, edits work and performing system calls work. However after I perform a series of edits in TextEdit, and completely exit TextEdit. When I call open (1) on the file I get the following error: The application cannot be opened for an unexpected reason, error=Error Domain=NSOSStatusErrorDomain Code=-43 "fnfErr: File not found" UserInfo={_LSLine=4129, _LSFunction=_LSOpenStuffCallLocal} From the logs of my app, there is no open (2) called on the file. I have tried to (trace) dtruss the open call for Numbers/TextEdit, but when I perform the above scenario, my Mac system freezes and the piped output from dtruss is 0 bytes after rebooting my system. How can I debug my issue? Where can I find more documentation of the order of system calls for open (1)? I couldn't find the source code for renamex_np thus my implementation relied on the linux kernel implementation of renameat2, does renamex_np do something different? I note that, if I open TextEdit for example, and then open my file, there is no problem. Also calling cat for example on the terminal it displays the content correctly. The problem seems to be from open (1). Furthermore, if I perform a rename of the file, open (1) succeeds in opening the file, until I perform at least another edit from a high-level app (that calls rename with the swap flag). Also if I unmount my filesystem and mount it again, open (1) behaves correctly. How can I understand what open (1) is doing under the hood? For the high-level apps I could trace the system calls and figure out why they didn't work, but now I reached a point (scenario) where I can't trace the system calls for open (1) due to my whole system freezing. Any input is appreciated.
8
0
282
Jun ’24
System Integrity Projection (SIP) & app group containers on macOS Sequoia 15
The release notes state the following: To protect users’ privacy, app group containers (in ~/Library/Group Containers) are now protected by System Integrity Protection. This is similar to the protection added to app data containers in macOS Sonoma. An app that’s properly entitled for an app group continues to have access to the app group container. Specifically, the app must use FileManager to get the app group container path and meet one of the following requirements: the app is deployed through Mac App Store; the app group identifier is prefixed with the app’s Team ID; or the app group identifier is authorised by a provisioning profile embedded within the app. If the app doesn’t meet these requirements, the system might present the user a prompt to authorize the app’s use of the app group container. If granted, that consent applies only for the duration of that app instance. This restriction also applies to app extensions, although in that case the system won’t prompt the user for consent but will instead just deny the access. (114586798) We have a helper app which is not sandboxed (due to it requiring Accessibility access/permissions) that accesses our group container. I've tested our helper app with the current beta of macOS Sequoia 15 (24A5264n) and it still works correctly, however I'm not clear if these restrictions are actually enforced in the current beta. I've tried testing for this by accessing the group container via Terminal (with Full Disk Access disabled for Terminal), but did not get any alert mentioned in the notes (or been otherwise restricted). Are these restrictions currently enforced?
1
0
305
Jun ’24
Trouble downgrading to iOS 17 from 18 Developer Beta w/iCloud Save
Hey, I’m an iPhone user of nearly 4 months, and recently, I found out about the iOS 18 developer version. I wanted to try it, so I bought an iCloud subscription and backed up my phone. I downloaded the update overnight through the settings app, and when I woke, it prompted me to restart or along those lines, to which it showed the apple logo and a progress bar as per usual. I didn’t even notice a difference between 17 and 18 until I swiped down to get to the control panel, and instantly, I wanted to switch back. I could no longer see features like my over $300 AirPod’s volume decibel amount; something I use on a regular basis to make sure I’m not developing hearing loss. So, when I got home from school later that day, I looked at the official Apple guide to downgrading to a previous iCloud version. Following the featured steps, I factory reset the iPhone through the General > Transfer or Reset iPhone > Erase all content and settings. I made sure to skip the automatic backup this process created. Then, after it was finished resetting, I followed the standard iPhone setup process until it asked me if I had a backup. I clicked it, signed in with my Apple ID, then clicked “Show All Versions” (as to not backup with an iPhone 18 beta backup) and clicked on my backup from the night prior. Then, it went to the black screen with the Apple logo and a progress bar again, and a few minutes later, the phone booted up. It all seemed normal, up until I saw the iOS 18 control center. I did this whole process again just to make sure I didn’t accidentally mess the procedure up, but I got the same result. Can anyone help me with this? Thanks in advance, Gray (posting this here because usual apple forums remove anything mentioning developer things)
1
0
270
Jun ’24