Create ML

RSS for tag

Create machine learning models for use in your app using Create ML.

Posts under Create ML tag

50 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

CreateML debug log lines?
Where can I find CreateML logs? I'd like to inspect log lines if they exist to diagnose what kind of error the app encounters when I provide it training data for a multi-label image classifier and the UI displays "Data Analysis stopped". I do see some crash reports for "MLRecipeExecutionService" in the Console app which seem related, but I haven't spotted anything useful there yet.
1
1
430
Feb ’24
Swift Playground Bundle can't find Compiled CoreML Model (.mlmodelc)
I have been attempting to debug this for over 10 hours... I am working on implementing Apple's MobileNetV2 CoreML model into a Swift Playgrounds. I performed the following steps Compiled CoreML model in regular Xcode project Moved Compiled CoreML (MobileNetV2.mlmodelc) model to Resources folder of Swift Playground Copy Paste the model class (MobileNetV2.swift) into the Sources folder of Swift Playground Use UIImage extensions to resize and convert UIImage into CVbuffer Implement basic code to run the model. However, every time I run this, it keeps giving me this error: MobileNetV2.swift:100: Fatal error: Unexpectedly found nil while unwrapping an Optional value From the automatically generated model class function: /// URL of model assuming it was installed in the same bundle as this class class var urlOfModelInThisBundle : URL { let bundle = Bundle(for: self) return bundle.url(forResource: "MobileNetV2", withExtension:"mlmodelc")! } The model builds perfectly, this is my contentView Code: import SwiftUI struct ContentView: View { func test() -> String{ // 1. Load the image from the 'Resources' folder. let newImage = UIImage(named: "img") // 2. Resize the image to the required input dimension of the Core ML model // Method from UIImage+Extension.swift let newSize = CGSize(width: 224, height: 224) guard let resizedImage = newImage?.resizeImageTo(size: newSize) else { fatalError("⚠️ The image could not be found or resized.") } // 3. Convert the resized image to CVPixelBuffer as it is the required input // type of the Core ML model. Method from UIImage+Extension.swift guard let convertedImage = resizedImage.convertToBuffer() else { fatalError("⚠️ The image could not be converted to CVPixelBugger") } // 1. Create the ML model instance from the model class in the 'Sources' folder let mlModel = MobileNetV2() // 2. Get the prediction output guard let prediction = try? mlModel.prediction(image: convertedImage) else { fatalError("⚠️ The model could not return a prediction") } // 3. Checking the results of the prediction let mostLikelyImageCategory = prediction.classLabel let probabilityOfEachCategory = prediction.classLabelProbs var highestProbability: Double { let probabilty = probabilityOfEachCategory[mostLikelyImageCategory] ?? 0.0 let roundedProbability = (probabilty * 100).rounded(.toNearestOrEven) return roundedProbability } return("\(mostLikelyImageCategory): \(highestProbability)%") } var body: some View { VStack { let _ = print(test()) Image(systemName: "globe") .imageScale(.large) .foregroundColor(.accentColor) Text("Hello, world!") Image(uiImage: UIImage(named: "img")!) } } } Upon printing my bundle contents, I get these: ["_CodeSignature", "metadata.json", "__PlaceholderAppIcon76x76@2x~ipad.png", "Info.plist", "__PlaceholderAppIcon60x60@2x.png", "coremldata.bin", "{App Name}", "PkgInfo", "Assets.car", "embedded.mobileprovision"] Anything would help 🙏 For additional reference, here are my UIImage extensions in ExtImage.swift: //Huge thanks to @mprecke on github for these UIImage extension function. import Foundation import UIKit extension UIImage { func resizeImageTo(size: CGSize) -> UIImage? { UIGraphicsBeginImageContextWithOptions(size, false, 0.0) self.draw(in: CGRect(origin: CGPoint.zero, size: size)) let resizedImage = UIGraphicsGetImageFromCurrentImageContext()! UIGraphicsEndImageContext() return resizedImage } func convertToBuffer() -> CVPixelBuffer? { let attributes = [ kCVPixelBufferCGImageCompatibilityKey: kCFBooleanTrue, kCVPixelBufferCGBitmapContextCompatibilityKey: kCFBooleanTrue ] as CFDictionary var pixelBuffer: CVPixelBuffer? let status = CVPixelBufferCreate( kCFAllocatorDefault, Int(self.size.width), Int(self.size.height), kCVPixelFormatType_32ARGB, attributes, &pixelBuffer) guard (status == kCVReturnSuccess) else { return nil } CVPixelBufferLockBaseAddress(pixelBuffer!, CVPixelBufferLockFlags(rawValue: 0)) let pixelData = CVPixelBufferGetBaseAddress(pixelBuffer!) let rgbColorSpace = CGColorSpaceCreateDeviceRGB() let context = CGContext( data: pixelData, width: Int(self.size.width), height: Int(self.size.height), bitsPerComponent: 8, bytesPerRow: CVPixelBufferGetBytesPerRow(pixelBuffer!), space: rgbColorSpace, bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue) context?.translateBy(x: 0, y: self.size.height) context?.scaleBy(x: 1.0, y: -1.0) UIGraphicsPushContext(context!) self.draw(in: CGRect(x: 0, y: 0, width: self.size.width, height: self.size.height)) UIGraphicsPopContext() CVPixelBufferUnlockBaseAddress(pixelBuffer!, CVPixelBufferLockFlags(rawValue: 0)) return pixelBuffer } }
5
1
1.3k
Feb ’24
Who will build first app that I can use while I sleep (quite literally)(24/7 immersion)?
In theory, sending signals from iPhone apps to and from the brain with non-invasive technology could be achieved through a combination of brain-computer interface (BCI) technologies, machine learning algorithms, and mobile app development. Brain-Computer Interface (BCI): BCI technology can be used to record brain signals and translate them into commands that can be understood by a computer or a mobile device. Non-invasive BCIs, such as electroencephalography (EEG), can track brain activity using sensors placed on or near the head[6]. For instance, a portable, non-invasive, mind-reading AI developed by UTS uses an AI model called DeWave to translate EEG signals into words and sentences[3]. Machine Learning Algorithms: Machine learning algorithms can be used to analyze and interpret the brain signals recorded by the BCI. These algorithms can learn from large quantities of EEG data to translate brain signals into specific commands[3]. Mobile App Development: A mobile app can be developed to receive these commands and perform specific actions on the iPhone. The app could also potentially send signals back to the brain using technologies like transcranial magnetic stimulation (TMS), which can deliver information to the brain[5]. However, it's important to note that while this technology is theoretically possible, it's still in the early stages of development and faces significant technical and ethical challenges. Current non-invasive BCIs do not have the same level of fidelity as invasive devices, and the practical application of these systems is still limited[1][3]. Furthermore, ethical considerations around privacy, consent, and the potential for misuse of this technology must also be addressed[13]. Sources [1] You can now use your iPhone with your brain after a major breakthrough | Semafor https://www.semafor.com/article/11/01/2022/you-can-now-use-your-iphone-with-your-brain [2] ! Are You A Robot? https://www.sciencedirect.com/science/article/pii/S1110866515000237 [3] Portable, non-invasive, mind-reading AI turns thoughts into text https://techxplore.com/news/2023-12-portable-non-invasive-mind-reading-ai-thoughts.html [4] Elon Musk's Neuralink implants brain chip in first human https://www.reuters.com/technology/neuralink-implants-brain-chip-first-human-musk-says-2024-01-29/ [5] BrainNet: A Multi-Person Brain-to-Brain Interface for Direct Collaboration Between Brains - Scientific Reports https://www.nature.com/articles/s41598-019-41895-7 [6] Brain-computer interfaces and the future of user engagement https://www.fastcompany.com/90802262/brain-computer-interfaces-and-the-future-of-user-engagement [7] Mobile App + Wearable For Neurostimulation - Accion Labs https://www.accionlabs.com/mobile-app-wearable-for-neurostimulation [8] Signal Generation, Acquisition, and Processing in Brain Machine Interfaces: A Unified Review https://www.frontiersin.org/articles/10.3389/fnins.2021.728178/full [9] Mind-reading technology has arrived https://www.vox.com/future-perfect/2023/5/4/23708162/neurotechnology-mind-reading-brain-neuralink-brain-computer-interface [10] Synchron Brain Implant - Breakthrough Allows You to Control Your iPhone With Your Mind - Grit Daily News https://gritdaily.com/synchron-brain-implant-controls-tech-with-the-mind/ [11] Mind uploading - Wikipedia https://en.wikipedia.org/wiki/Mind_uploading [12] BirgerMind - Express your thoughts loudly https://birgermind.com [13] Elon Musk wants to merge humans with AI. How many brains will be damaged along the way? https://www.vox.com/future-perfect/23899981/elon-musk-ai-neuralink-brain-computer-interface [14] Models of communication and control for brain networks: distinctions, convergence, and future outlook https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7655113/ [15] Mind Control for the Masses—No Implant Needed https://www.wired.com/story/nextmind-noninvasive-brain-computer-interface/ [16] Elon Musk unveils Neuralink’s plans for brain-reading ‘threads’ and a robot to insert them https://www.theverge.com/2019/7/16/20697123/elon-musk-neuralink-brain-reading-thread-robot [17] Essa and Kotte https://arxiv.org/pdf/2201.04229.pdf [18] Synchron's Brain Implant Breakthrough Lets Users Control iPhones And iPads With Their Mind https://hothardware.com/news/brain-implant-breakthrough-lets-you-control-ipad-with-your-mind [19] An Apple Watch for Your Brain https://www.thedeload.com/p/an-apple-watch-for-your-brain [20] Toward an information theoretical description of communication in brain networks https://direct.mit.edu/netn/article/5/3/646/97541/Toward-an-information-theoretical-description-of [21] A soft, wearable brain–machine interface https://news.ycombinator.com/item?id=28447778 [22] Portable neurofeedback App https://www.psychosomatik.com/en/portable-neurofeedback-app/ [23] Intro to Brain Computer Interface http://learn.neurotechedu.com/introtobci/
0
1
641
Feb ’24
coreML Hand Pose classification: doesn't appear on the camera.
I created a Hand Pose model using CreateML and integrated it into my SwiftUI project app. While coding, I referred to the Apple Developer documentation app for the necessary code. However, when I ran the app on an iPhone 14, the camera didn't display any effects or finger numbers as expected. note: I've already tested the ML model separately, and it works fine. the code: import CoreML import SceneKit import SwiftUI import Vision import ARKit struct ARViewContainer: UIViewControllerRepresentable { let arViewController: ARViewController let model: modelHand func makeUIViewController(context: UIViewControllerRepresentableContext<ARViewContainer>) -> ARViewController { arViewController.model = model return arViewController } func updateUIViewController(_ uiViewController: ARViewController, context: UIViewControllerRepresentableContext<ARViewContainer>) { // Update the view controller if needed } } class ARViewController: UIViewController, ARSessionDelegate { var frameCounter = 0 let handPosePredictionInterval = 10 var model: modelHand! var effectNode: SCNNode? override func viewDidLoad() { super.viewDidLoad() let arView = ARSCNView(frame: view.bounds) view.addSubview(arView) let session = ARSession() session.delegate = self let configuration = ARWorldTrackingConfiguration() configuration.frameSemantics = .personSegmentationWithDepth arView.session.run(configuration) } func session(_ session: ARSession, didUpdate frame: ARFrame) { let pixelBuffer = frame.capturedImage let handPoseRequest = VNDetectHumanHandPoseRequest() handPoseRequest.maximumHandCount = 1 handPoseRequest.revision = VNDetectHumanHandPoseRequestRevision1 let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, options: [:]) do { try handler.perform([handPoseRequest]) } catch { assertionFailure("Hand Pose Request failed: \(error)") } guard let handPoses = handPoseRequest.results, !handPoses.isEmpty else { return } if frameCounter % handPosePredictionInterval == 0 { if let handObservation = handPoses.first as? VNHumanHandPoseObservation { do { let keypointsMultiArray = try handObservation.keypointsMultiArray() let handPosePrediction = try model.prediction(poses: keypointsMultiArray) let confidence = handPosePrediction.labelProbabilities[handPosePrediction.label]! print("Confidence: \(confidence)") if confidence > 0.9 { print("Rendering hand pose effect: \(handPosePrediction.label)") renderHandPoseEffect(name: handPosePrediction.label) } } catch { fatalError("Failed to perform hand pose prediction: \(error)") } } } } func renderHandPoseEffect(name: String) { switch name { case "One": print("Rendering effect for One") if effectNode == nil { effectNode = addParticleNode(for: "One") } default: print("Removing all particle nodes") removeAllParticleNode() } } func removeAllParticleNode() { effectNode?.removeFromParentNode() effectNode = nil } func addParticleNode(for poseName: String) -> SCNNode { print("Adding particle node for pose: \(poseName)") let particleNode = SCNNode() return particleNode } } struct ContentView: View { let model = modelHand() var body: some View { ARViewContainer(arViewController: ARViewController(), model: model) } } #Preview { ContentView() }
0
0
539
Jan ’24
Need Help with Create ML in Xcode - Unexpected App Closure
Hello Apple Developer community, I hope this message finds you well. I am currently facing an issue with Create ML in Xcode, and I am seeking assistance from the knowledgeable members of this forum. Any help or guidance would be greatly appreciated. Problem Description: I am encountering an unexpected issue when attempting to create a classification model for images using Create ML in Xcode. Upon opening Create ML, the application closes unexpectedly when I choose to create a new image classification model. Steps I Have Taken: I have already tried the following steps to troubleshoot the issue: Updated Xcode and macOS to the latest versions. Restarted Xcode and my computer. Created a new sample project to isolate the issue. Despite these efforts, the problem persists. System Information: Xcode Version: 15.2 macOS Version: Sonoma 14.0 I am on a tight deadline for a project, and resolving this issue quickly is crucial. Your help is invaluable, and I thank you in advance for any support you can provide. Best regards.
1
0
611
Jan ’24
FB13516799: Training Tabular Regression ML Models on large datasets in Xcode 15 continuously "Processing"
Hi, In Xcode 14 I was able to train linear regression models with Create ML using large CSV files (I tested on about 30000 items and 5 features): However, in Xcode 15 (I tested on 15.0.1 and 15.1), the training continuously stays in the "Processing" state: When using a dataset with 900 items, everything works fine. I filed a feedback for this issue: FB13516799. Does anybody else have this issue / can reproduce it?
2
0
753
Jan ’24
DataFrame's Column doesn't support array of dictionary
I'm following Apple WWDC video (https://developer.apple.com/videos/play/wwdc2021/10037/) about how to create a recommendation model. But I'm getting this error when I run the project on that like of code from their tutorial. "Column keywords has element of unsupported type Dictionary<String, Double>." Here is the block of code took from the transcript of WWDC video that cause me issue: func featuresFromMealAndKeywords(meal: String, keywords: [String]) -> [String: Double] { // Capture interactions between content (the dish keywords) and context (meal) by // adding a copy of each keyword modified to include the meal. let featureNames = keywords + keywords.map { meal + ":" + $0 } // For each keyword, create an entry in a dictionary of features with a value of 1.0. return featureNames.reduce(into: [:]) { features, name in features[name] = 1.0 } } var trainingKeywords: [[String: Double]] = [] var trainingTargets: [Double] = [] for item in userPurchasedItems { // Add in the positive example. trainingKeywords.append( featuresFromMealAndKeywords(meal: item.meal, keywords: item.keywords)) trainingTargets.append(1.0) // Add in the negative example. let negativeKeywords = allKeywords.subtracting(item.keywords) trainingKeywords.append( featuresFromMealAndKeywords(meal: item.meal, keywords: Array(negativeKeywords))) trainingTargets.append(-1.0) } // Create the training data. var trainingData = DataFrame() trainingData.append(column: Column(name: "keywords" contents: trainingKeywords)) trainingData.append(column: Column(name: "target", contents: trainingTargets)) // Create the model. let model = try MLLinearRegressor(trainingData: trainingData, targetColumn: "target") Did DataFrame implementation changed since then and doesn't support Dictionary anymore? I'm at lost right now on how to reproduce their example.
4
6
632
Jan ’24
Word Tagging Model- How to change tagging unit
I created a word tagging model in CreateML and am trying to make predictions with it using the following code: let text = "$30.00 7/1/2023" let model = TaggingModel() let input = TaggingModelInput(text: text) guard let output = try? model.prediction(input: input) else { fatalError("Unexpected runtime error.") } However, the output separates "$" and "30.00" as separate tokens as well as "7", "/", "1", "/", etc. Is there any way to make sure prices and dates get grouped together and to simply separate tokens based on whitespace? Any help is appreciated!
1
0
539
Jan ’24
CreateML API train soft lock on 90%
Hello, I'm trying to train a MLImageClassifier dataset using Swift using the function MLImageClassifier.train. It doesn't change the dataset size (I have the same problem with a smaller one), but when the train reaches the 9 completedUnitCount of 10, even if the CPU usage is still high, seems to happen a soft lock that doesn't never brings the model to its completion (or error). The dataset is made of jpg images, using the CreateML app doesn't appear any problem during the training. There is any known issue with CreateML training APIs about part 9 of the process? There is any information about this part of the training job? Thank you
1
0
645
Nov ’23
CoreML: how to use a NeuralNetworkBuilder to make a model updatable
I'm trying to create an updatable model, but this seems possible only by creating from scratch a neural network model and then, using the NeuralNetworkBuilder, call the make_updatable method. But I met a lot of problems on this way for the solution. In this example I try to open a converted ML Model (neural network) using the NeuralNetworkBuilder: import coremltools model = coremltools.models.MLModel("SimpleImageClassifier.mlpackage") spec = model.get_spec() builder = coremltools.models.neural_network.NeuralNetworkBuilder(spec=spec) builder.inspect_layers() But I met this error in the builder instance line: AttributeError: 'NoneType' object has no attribute 'layers' I also tried to define a neural network using the NeuralNetworkBuilder but then what do I have to do with this object? I didn't find a way to save it or convert it. The result I want is simple, the possibility to train more the model on the user device to meet his exigences. However the way to obtain an updatable model seems incomprehensible. In my case, the model should be an image classification. What approach should I follow to achieve this result? Thank you
1
0
769
Nov ’23
NaN value when predict with model resnet
Coremltools: 6.2.0 When I run coreml model in python result is good: {'var_840': array([[-8.15439941e+02, 2.88793579e+02, -3.83110474e+02, -8.95208740e+02, -3.53131561e+02, -3.65339783e+02, -4.94590851e+02, 6.24686813e+01, -5.92614822e+01, -9.67470627e+01, -4.30247498e+02, -9.27047348e+01, 2.19661942e+01, -2.96691345e+02, -4.26566772e+02........ But when I run on xcode so result look like: [-inf,inf,nan,-inf,nan,nan,nan,nan,nan,-inf,-inf,-inf,-inf,-inf,-inf,nan,-inf,-inf,nan,-inf,nan,nan,-inf,nan,-inf,-inf,-inf,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,-inf,nan,nan,nan,nan,-inf,nan,-inf ....... Step1: Convert Resnet50 to coreml: import torch import torchvision # Load a pre-trained version of MobileNetV2 model. torch_model = torchvision.models.resnet50(pretrained=True) # Set the model in evaluation mode. torch_model.eval() # Trace the model with random data. example_input = torch.rand(1, 3, 224, 224) traced_model = torch.jit.trace(torch_model, example_input) out = traced_model(example_input) # Download class labels in ImageNetLabel.txt. # Set the image scale and bias for input image preprocessing. import coremltools as ct image_input = ct.ImageType(shape=example_input.shape, ) # Using image_input in the inputs parameter: # Convert to Core ML using the Unified Conversion API. model = ct.convert( traced_model, inputs=[image_input], compute_units=ct.ComputeUnit.CPU_ONLY, ) # Save the converted model. model.save("resnettest.mlmodel") # Print a confirmation message. print('model converted and saved') Step2: Test model coreml in python: import coremltools as ct import PIL import numpy as np # Load the model model = ct.models.MLModel('/Users/ngoclh/Downloads/resnettest.mlmodel') print(model) img_path = "/Users/ngoclh/gitlocal/DetectCirtochApp/DetectCirtochApp/resources/image.jpg" img = PIL.Image.open(img_path) img = img.resize([224, 224], PIL.Image.ANTIALIAS) coreml_out_dict = model.predict({"input_1" : img}) print(coreml_out_dict) Step3: Test coreml model in Xcode: func getFeature() { do { let deepLab = try VGG_emb.init() //mobilenet_emb.init()//cirtorch_emb.init() let image = UIImage(named: "image.jpg") let pixBuf = image!.pixelBuffer(width: 224, height: 224)! guard let output = try? deepLab.prediction(input_1: pixBuf) else { return } let names = output.featureNames print("ngoc names: ", names) for name in names { let feature = output.featureValue(for: name) print("ngoc feature: ", feature) } } catch { print(error) } }
1
0
620
Nov ’23
!! Assistance needed: Create ML - Scan file with multiple tables on it
Objective: I am in the process of developing an application that utilizes machine learning (Core ML) to interact with photographs of documents, specifically focusing on those containing tables. Step 1: Capturing the Image The application will initiate by allowing users to take photos of documents. The key here is not just any part of the document, but specifically the sections where tables are present. Step 2: Image Analysis through Machine Learning Upon capturing the image, the next phase involves a machine learning model. Using Apple's Create ML tool with Swift, the application will analyze the image. The model's task is two-fold: Identifying the Table: Distinguish the table from other document information, ensuring it recognizes and isolates the table structure within the photograph. Ignoring Irrelevant Information: Concurrently, the model will disregard all non-table content, focusing the application's resources on the table data. Step 3: Data Extraction and Training Once the table is identified, the real work begins. The application will engage in detailed scrutiny, where it's trained to understand and recognize row and column data based on specific datasets. This training will enable the application to 'read' the table accurately, much like a human would, by identifying the organization of information into rows and columns. Step 4: Information Storage Post-analysis, the application will extract this critical data, storing it in a structured format. Each piece of identifiable information from the rows and columns will be systematically organized into a Dictionary or an Object. This structure is not just for immediate use but also efficient for future data operations within the app. Conclusion: Through these sequential steps, the application transitions from merely capturing an image to intelligently recognizing, deciphering, and storing table data from within a physical document. This streamlined process is all courtesy of integrating machine learning into the app's functionality, promising significant efficiency and accuracy in data handling. Realistically, I have not found any good examples out there so I am attempting to create my own ML (with no experience 😅), so any guidance or help would be very much appreciated.
1
1
554
Nov ’23
update already saved model
I followed the video of Composing advanced models with Create ML Components. I have created the model with let urlParameter = URL(fileURLWithPath: "/path/to/model.pkg") let (training, validation) = dataFrame.randomSplit(by: 0.8) let model = try await transformer.fitted(to: DataFrame(training), validateOn: DataFrame(validation)) { event in guard let tAccuracy = event.metrics[.trainingAccuracy] as? Double else { return } print(tAccuracy) } try transformer.write(model, to: url) print("done") Next goal is to read the model and update it with new dataFrame let urlCSV = URL(fileURLWithPath: "path/to/newData.csv") var model = try transformer.read(from: urlParameters) // loading created model let newDataFrame = try DataFrame(contentsOfCSVFile: urlCSV ) // new dataFrame with features and annotations try await transformer.update(&model, with: newDataFrame) // I want to keep previous learned data and update the model with new try transformer.write(model, to: urlParameters) // the model saves but the only last added dataFrame are saved. Previous one just replaced with new one But looks like I only replace old data with new one. **The Question ** How can add new data to model I created without losing old one ?
2
0
558
Nov ’23
preferredMetalDevice shows null for MLBoostedTreeRegressor
I had code that ran 7x faster in Ventura compared to how it runs now in Sonoma. For the basic model training I used let pmst = MLBoostedTreeRegressor.ModelParameters(validation: .split(strategy: .automatic),maxIterations:10000) let model = try MLBoostedTreeRegressor(trainingData: trainingdata, targetColumn: columntopredict, parameters: pmst) Which took around 2 secs in Ventura and now takes between 10 and 14 seconds in Sonoma I have tried to investigate why, and have noticed that when I use I see these results useWatchSPIForScribble: NO, allowLowPrecisionAccumulationOnGPU: NO, allowBackgroundGPUComputeSetting: NO, preferredMetalDevice: (null), enableTestVectorMode: NO, parameters: (null), rootModelURL: (null), profilingOptions: 0, usePreloadedKey: NO, trainWithMLCompute: NO, parentModelName: , modelName: Unnamed_Model, experimentalMLE5EngineUsage: Enable, preparesLazily: NO, predictionConcurrencyHint: 0, Why is the preferred Metal Device null? If I do let devices = MTLCopyAllDevices() for device in devices { config.preferredMetalDevice = device print(device.name) } I can see that the M1 chipset is available but not selected (from reading the literature the default should be nil?) Is this the reason why it is so slow? Is there a way to force a change in the config or elsewhere? Why has the default changed, if it has?
6
0
753
Oct ’23
Core ML Model performance far lower on iOS 17 vs iOS 16 (iOS 17 not using Neural Engine)
Hello, I posted an issue on the coremltools GitHub about my Core ML models not performing as well on iOS 17 vs iOS 16 but I'm posting it here just in case. TL;DR The same model on the same device/chip performs far slower (doesn't use the Neural Engine) on iOS 17 compared to iOS 16. Longer description The following screenshots show the performance of the same model (a PyTorch computer vision model) on an iPhone SE 3rd gen and iPhone 13 Pro (both use the A15 Bionic). iOS 16 - iPhone SE 3rd Gen (A15 Bioinc) iOS 16 uses the ANE and results in fast prediction, load and compilation times. iOS 17 - iPhone 13 Pro (A15 Bionic) iOS 17 doesn't seem to use the ANE, thus the prediction, load and compilation times are all slower. Code To Reproduce The following is my code I'm using to export my PyTorch vision model (using coremltools). I've used the same code for the past few months with sensational results on iOS 16. # Convert to Core ML using the Unified Conversion API coreml_model = ct.convert( model=traced_model, inputs=[image_input], outputs=[ct.TensorType(name="output")], classifier_config=ct.ClassifierConfig(class_names), convert_to="neuralnetwork", # compute_precision=ct.precision.FLOAT16, compute_units=ct.ComputeUnit.ALL ) System environment: Xcode version: 15.0 coremltools version: 7.0.0 OS (e.g. MacOS version or Linux type): Linux Ubuntu 20.04 (for exporting), macOS 13.6 (for testing on Xcode) Any other relevant version information (e.g. PyTorch or TensorFlow version): PyTorch 2.0 Additional context This happens across "neuralnetwork" and "mlprogram" type models, neither use the ANE on iOS 17 but both use the ANE on iOS 16 If anyone has a similar experience, I'd love to hear more. Otherwise, if I'm doing something wrong for the exporting of models for iOS 17+, please let me know. Thank you!
0
1
1k
Oct ’23
CreateML action classifier: wrist joint missing
Hi! I'd like to ask why after training an action classifier model using CreateML, I consistently fail to detect my wrist joint while using the live preview feature, while other joints are detected correctly. What could be causing this result? Thank you to anyone for providing answers. Training Data Sources: YouTube videos, searching keyword: "20min workout at home" All videos display the full body. Training Data Quantity: Model 1: Target action: 1 video / Irrelevant action: 1 video Model 2: Target action: 6 videos / Irrelevant action: 40 videos Both Model 1 and Model 2 fail to detect the wrist joint during live preview.
1
0
575
Sep ’23
[Create ML Components] The transformer is not representable as a CoreML model (ImageReader).
Following this guide https://developer.apple.com/documentation/CreateML/creating-a-multi-label-image-classifier Has anyone been able to export a CoreML model, specifically according to the documentation below? Since there isn't any runnable examples, just snippets, perhaps documentation error? If anyone is already familiar with these pipeline generics, is there something that jumps out about the example transformer used that fails conformance or just factually incorrect? Export the model to use with Vision After you train the model, you can export it as a Core ML model. // Export to Core ML let modelURL = URL(filePath: "/path/to/model") try model.export(to: modelURL)
2
1
641
Sep ’23