Where can I find CreateML logs? I'd like to inspect log lines if they exist to diagnose what kind of error the app encounters when I provide it training data for a multi-label image classifier and the UI displays "Data Analysis stopped".
I do see some crash reports for "MLRecipeExecutionService" in the Console app which seem related, but I haven't spotted anything useful there yet.
Create ML
RSS for tagCreate machine learning models for use in your app using Create ML.
Posts under Create ML tag
50 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
I have been attempting to debug this for over 10 hours...
I am working on implementing Apple's MobileNetV2 CoreML model into a Swift Playgrounds. I performed the following steps
Compiled CoreML model in regular Xcode project
Moved Compiled CoreML (MobileNetV2.mlmodelc) model to Resources folder of Swift Playground
Copy Paste the model class (MobileNetV2.swift) into the Sources folder of Swift Playground
Use UIImage extensions to resize and convert UIImage into CVbuffer
Implement basic code to run the model.
However, every time I run this, it keeps giving me this error:
MobileNetV2.swift:100: Fatal error: Unexpectedly found nil while unwrapping an Optional value
From the automatically generated model class function:
/// URL of model assuming it was installed in the same bundle as this class
class var urlOfModelInThisBundle : URL {
let bundle = Bundle(for: self)
return bundle.url(forResource: "MobileNetV2", withExtension:"mlmodelc")!
}
The model builds perfectly, this is my contentView Code:
import SwiftUI
struct ContentView: View {
func test() -> String{
// 1. Load the image from the 'Resources' folder.
let newImage = UIImage(named: "img")
// 2. Resize the image to the required input dimension of the Core ML model
// Method from UIImage+Extension.swift
let newSize = CGSize(width: 224, height: 224)
guard let resizedImage = newImage?.resizeImageTo(size: newSize) else {
fatalError("⚠️ The image could not be found or resized.")
}
// 3. Convert the resized image to CVPixelBuffer as it is the required input
// type of the Core ML model. Method from UIImage+Extension.swift
guard let convertedImage = resizedImage.convertToBuffer() else {
fatalError("⚠️ The image could not be converted to CVPixelBugger")
}
// 1. Create the ML model instance from the model class in the 'Sources' folder
let mlModel = MobileNetV2()
// 2. Get the prediction output
guard let prediction = try? mlModel.prediction(image: convertedImage) else {
fatalError("⚠️ The model could not return a prediction")
}
// 3. Checking the results of the prediction
let mostLikelyImageCategory = prediction.classLabel
let probabilityOfEachCategory = prediction.classLabelProbs
var highestProbability: Double {
let probabilty = probabilityOfEachCategory[mostLikelyImageCategory] ?? 0.0
let roundedProbability = (probabilty * 100).rounded(.toNearestOrEven)
return roundedProbability
}
return("\(mostLikelyImageCategory): \(highestProbability)%")
}
var body: some View {
VStack {
let _ = print(test())
Image(systemName: "globe")
.imageScale(.large)
.foregroundColor(.accentColor)
Text("Hello, world!")
Image(uiImage: UIImage(named: "img")!)
}
}
}
Upon printing my bundle contents, I get these:
["_CodeSignature", "metadata.json", "__PlaceholderAppIcon76x76@2x~ipad.png", "Info.plist", "__PlaceholderAppIcon60x60@2x.png", "coremldata.bin", "{App Name}", "PkgInfo", "Assets.car", "embedded.mobileprovision"]
Anything would help 🙏
For additional reference, here are my UIImage extensions in ExtImage.swift:
//Huge thanks to @mprecke on github for these UIImage extension function.
import Foundation
import UIKit
extension UIImage {
func resizeImageTo(size: CGSize) -> UIImage? {
UIGraphicsBeginImageContextWithOptions(size, false, 0.0)
self.draw(in: CGRect(origin: CGPoint.zero, size: size))
let resizedImage = UIGraphicsGetImageFromCurrentImageContext()!
UIGraphicsEndImageContext()
return resizedImage
}
func convertToBuffer() -> CVPixelBuffer? {
let attributes = [
kCVPixelBufferCGImageCompatibilityKey: kCFBooleanTrue,
kCVPixelBufferCGBitmapContextCompatibilityKey: kCFBooleanTrue
] as CFDictionary
var pixelBuffer: CVPixelBuffer?
let status = CVPixelBufferCreate(
kCFAllocatorDefault, Int(self.size.width),
Int(self.size.height),
kCVPixelFormatType_32ARGB,
attributes,
&pixelBuffer)
guard (status == kCVReturnSuccess) else {
return nil
}
CVPixelBufferLockBaseAddress(pixelBuffer!, CVPixelBufferLockFlags(rawValue: 0))
let pixelData = CVPixelBufferGetBaseAddress(pixelBuffer!)
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
let context = CGContext(
data: pixelData,
width: Int(self.size.width),
height: Int(self.size.height),
bitsPerComponent: 8,
bytesPerRow: CVPixelBufferGetBytesPerRow(pixelBuffer!),
space: rgbColorSpace,
bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue)
context?.translateBy(x: 0, y: self.size.height)
context?.scaleBy(x: 1.0, y: -1.0)
UIGraphicsPushContext(context!)
self.draw(in: CGRect(x: 0, y: 0, width: self.size.width, height: self.size.height))
UIGraphicsPopContext()
CVPixelBufferUnlockBaseAddress(pixelBuffer!, CVPixelBufferLockFlags(rawValue: 0))
return pixelBuffer
}
}
In theory, sending signals from iPhone apps to and from the brain with non-invasive technology could be achieved through a combination of brain-computer interface (BCI) technologies, machine learning algorithms, and mobile app development.
Brain-Computer Interface (BCI): BCI technology can be used to record brain signals and translate them into commands that can be understood by a computer or a mobile device. Non-invasive BCIs, such as electroencephalography (EEG), can track brain activity using sensors placed on or near the head[6]. For instance, a portable, non-invasive, mind-reading AI developed by UTS uses an AI model called DeWave to translate EEG signals into words and sentences[3].
Machine Learning Algorithms: Machine learning algorithms can be used to analyze and interpret the brain signals recorded by the BCI. These algorithms can learn from large quantities of EEG data to translate brain signals into specific commands[3].
Mobile App Development: A mobile app can be developed to receive these commands and perform specific actions on the iPhone. The app could also potentially send signals back to the brain using technologies like transcranial magnetic stimulation (TMS), which can deliver information to the brain[5].
However, it's important to note that while this technology is theoretically possible, it's still in the early stages of development and faces significant technical and ethical challenges. Current non-invasive BCIs do not have the same level of fidelity as invasive devices, and the practical application of these systems is still limited[1][3]. Furthermore, ethical considerations around privacy, consent, and the potential for misuse of this technology must also be addressed[13].
Sources
[1] You can now use your iPhone with your brain after a major breakthrough | Semafor https://www.semafor.com/article/11/01/2022/you-can-now-use-your-iphone-with-your-brain
[2] ! Are You A Robot? https://www.sciencedirect.com/science/article/pii/S1110866515000237
[3] Portable, non-invasive, mind-reading AI turns thoughts into text https://techxplore.com/news/2023-12-portable-non-invasive-mind-reading-ai-thoughts.html
[4] Elon Musk's Neuralink implants brain chip in first human https://www.reuters.com/technology/neuralink-implants-brain-chip-first-human-musk-says-2024-01-29/
[5] BrainNet: A Multi-Person Brain-to-Brain Interface for Direct Collaboration Between Brains - Scientific Reports https://www.nature.com/articles/s41598-019-41895-7
[6] Brain-computer interfaces and the future of user engagement https://www.fastcompany.com/90802262/brain-computer-interfaces-and-the-future-of-user-engagement
[7] Mobile App + Wearable For Neurostimulation - Accion Labs https://www.accionlabs.com/mobile-app-wearable-for-neurostimulation
[8] Signal Generation, Acquisition, and Processing in Brain Machine Interfaces: A Unified Review https://www.frontiersin.org/articles/10.3389/fnins.2021.728178/full
[9] Mind-reading technology has arrived https://www.vox.com/future-perfect/2023/5/4/23708162/neurotechnology-mind-reading-brain-neuralink-brain-computer-interface
[10] Synchron Brain Implant - Breakthrough Allows You to Control Your iPhone With Your Mind - Grit Daily News https://gritdaily.com/synchron-brain-implant-controls-tech-with-the-mind/
[11] Mind uploading - Wikipedia https://en.wikipedia.org/wiki/Mind_uploading
[12] BirgerMind - Express your thoughts loudly https://birgermind.com
[13] Elon Musk wants to merge humans with AI. How many brains will be damaged along the way? https://www.vox.com/future-perfect/23899981/elon-musk-ai-neuralink-brain-computer-interface
[14] Models of communication and control for brain networks: distinctions, convergence, and future outlook https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7655113/
[15] Mind Control for the Masses—No Implant Needed https://www.wired.com/story/nextmind-noninvasive-brain-computer-interface/
[16] Elon Musk unveils Neuralink’s plans for brain-reading ‘threads’ and a robot to insert them https://www.theverge.com/2019/7/16/20697123/elon-musk-neuralink-brain-reading-thread-robot
[17] Essa and Kotte https://arxiv.org/pdf/2201.04229.pdf
[18] Synchron's Brain Implant Breakthrough Lets Users Control iPhones And iPads With Their Mind https://hothardware.com/news/brain-implant-breakthrough-lets-you-control-ipad-with-your-mind
[19] An Apple Watch for Your Brain https://www.thedeload.com/p/an-apple-watch-for-your-brain
[20] Toward an information theoretical description of communication in brain networks https://direct.mit.edu/netn/article/5/3/646/97541/Toward-an-information-theoretical-description-of
[21] A soft, wearable brain–machine interface https://news.ycombinator.com/item?id=28447778
[22] Portable neurofeedback App https://www.psychosomatik.com/en/portable-neurofeedback-app/
[23] Intro to Brain Computer Interface http://learn.neurotechedu.com/introtobci/
After training my dataset, the training, validation, and testing sets all show 0% in detection accuracy and all my test photos show false negative. The dataset has 1032 photos and 2 classes, and I used Roboflow for the image annotation. For network, I choose full network. If there is any way to fix this?
I created a Hand Pose model using CreateML and integrated it into my SwiftUI project app.
While coding, I referred to the Apple Developer documentation app for the necessary code.
However, when I ran the app on an iPhone 14,
the camera didn't display any effects or finger numbers as expected.
note: I've already tested the ML model separately, and it works fine.
the code:
import CoreML
import SceneKit
import SwiftUI
import Vision
import ARKit
struct ARViewContainer: UIViewControllerRepresentable {
let arViewController: ARViewController
let model: modelHand
func makeUIViewController(context: UIViewControllerRepresentableContext<ARViewContainer>) -> ARViewController {
arViewController.model = model
return arViewController
}
func updateUIViewController(_ uiViewController: ARViewController, context: UIViewControllerRepresentableContext<ARViewContainer>) {
// Update the view controller if needed
}
}
class ARViewController: UIViewController, ARSessionDelegate {
var frameCounter = 0
let handPosePredictionInterval = 10
var model: modelHand!
var effectNode: SCNNode?
override func viewDidLoad() {
super.viewDidLoad()
let arView = ARSCNView(frame: view.bounds)
view.addSubview(arView)
let session = ARSession()
session.delegate = self
let configuration = ARWorldTrackingConfiguration()
configuration.frameSemantics = .personSegmentationWithDepth
arView.session.run(configuration)
}
func session(_ session: ARSession, didUpdate frame: ARFrame) {
let pixelBuffer = frame.capturedImage
let handPoseRequest = VNDetectHumanHandPoseRequest()
handPoseRequest.maximumHandCount = 1
handPoseRequest.revision = VNDetectHumanHandPoseRequestRevision1
let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, options: [:])
do {
try handler.perform([handPoseRequest])
} catch {
assertionFailure("Hand Pose Request failed: \(error)")
}
guard let handPoses = handPoseRequest.results, !handPoses.isEmpty else {
return
}
if frameCounter % handPosePredictionInterval == 0 {
if let handObservation = handPoses.first as? VNHumanHandPoseObservation {
do {
let keypointsMultiArray = try handObservation.keypointsMultiArray()
let handPosePrediction = try model.prediction(poses: keypointsMultiArray)
let confidence = handPosePrediction.labelProbabilities[handPosePrediction.label]!
print("Confidence: \(confidence)")
if confidence > 0.9 {
print("Rendering hand pose effect: \(handPosePrediction.label)")
renderHandPoseEffect(name: handPosePrediction.label)
}
} catch {
fatalError("Failed to perform hand pose prediction: \(error)")
}
}
}
}
func renderHandPoseEffect(name: String) {
switch name {
case "One":
print("Rendering effect for One")
if effectNode == nil {
effectNode = addParticleNode(for: "One")
}
default:
print("Removing all particle nodes")
removeAllParticleNode()
}
}
func removeAllParticleNode() {
effectNode?.removeFromParentNode()
effectNode = nil
}
func addParticleNode(for poseName: String) -> SCNNode {
print("Adding particle node for pose: \(poseName)")
let particleNode = SCNNode()
return particleNode
}
}
struct ContentView: View {
let model = modelHand()
var body: some View {
ARViewContainer(arViewController: ARViewController(), model: model)
}
}
#Preview {
ContentView()
}
Hello Apple Developer community,
I hope this message finds you well. I am currently facing an issue with Create ML in Xcode, and I am seeking assistance from the knowledgeable members of this forum. Any help or guidance would be greatly appreciated.
Problem Description:
I am encountering an unexpected issue when attempting to create a classification model for images using Create ML in Xcode. Upon opening Create ML, the application closes unexpectedly when I choose to create a new image classification model.
Steps I Have Taken:
I have already tried the following steps to troubleshoot the issue:
Updated Xcode and macOS to the latest versions.
Restarted Xcode and my computer.
Created a new sample project to isolate the issue.
Despite these efforts, the problem persists.
System Information:
Xcode Version: 15.2
macOS Version: Sonoma 14.0
I am on a tight deadline for a project, and resolving this issue quickly is crucial. Your help is invaluable, and I thank you in advance for any support you can provide.
Best regards.
I want to use it in the code
I am trying to implement a ML model with Core ML in a playground for a Student Challenge project, but I can not get it to work. I have already tried everything I found online but nothing seems to work (the tutorials where posted long time ago). Anyone knows how to do this with Xcode 15 and the most recent updates?
Hi,
In Xcode 14 I was able to train linear regression models with Create ML using large CSV files (I tested on about 30000 items and 5 features):
However, in Xcode 15 (I tested on 15.0.1 and 15.1), the training continuously stays in the "Processing" state:
When using a dataset with 900 items, everything works fine.
I filed a feedback for this issue: FB13516799.
Does anybody else have this issue / can reproduce it?
I'm following Apple WWDC video (https://developer.apple.com/videos/play/wwdc2021/10037/) about how to create a recommendation model. But I'm getting this error when I run the project on that like of code from their tutorial.
"Column keywords has element of unsupported type Dictionary<String, Double>."
Here is the block of code took from the transcript of WWDC video that cause me issue:
func featuresFromMealAndKeywords(meal: String, keywords: [String]) -> [String: Double] {
// Capture interactions between content (the dish keywords) and context (meal) by
// adding a copy of each keyword modified to include the meal.
let featureNames = keywords + keywords.map { meal + ":" + $0 }
// For each keyword, create an entry in a dictionary of features with a value of 1.0.
return featureNames.reduce(into: [:]) { features, name in
features[name] = 1.0
}
}
var trainingKeywords: [[String: Double]] = []
var trainingTargets: [Double] = []
for item in userPurchasedItems {
// Add in the positive example.
trainingKeywords.append(
featuresFromMealAndKeywords(meal: item.meal, keywords: item.keywords))
trainingTargets.append(1.0)
// Add in the negative example.
let negativeKeywords = allKeywords.subtracting(item.keywords)
trainingKeywords.append(
featuresFromMealAndKeywords(meal: item.meal, keywords: Array(negativeKeywords)))
trainingTargets.append(-1.0)
}
// Create the training data.
var trainingData = DataFrame()
trainingData.append(column: Column(name: "keywords" contents: trainingKeywords))
trainingData.append(column: Column(name: "target", contents: trainingTargets))
// Create the model.
let model = try MLLinearRegressor(trainingData: trainingData, targetColumn: "target")
Did DataFrame implementation changed since then and doesn't support Dictionary anymore? I'm at lost right now on how to reproduce their example.
I created a word tagging model in CreateML and am trying to make predictions with it using the following code:
let text = "$30.00 7/1/2023"
let model = TaggingModel()
let input = TaggingModelInput(text: text)
guard let output = try? model.prediction(input: input) else {
fatalError("Unexpected runtime error.")
}
However, the output separates "$" and "30.00" as separate tokens as well as "7", "/", "1", "/", etc. Is there any way to make sure prices and dates get grouped together and to simply separate tokens based on whitespace? Any help is appreciated!
Hello,
I'm trying to train a MLImageClassifier dataset using Swift using the function MLImageClassifier.train.
It doesn't change the dataset size (I have the same problem with a smaller one), but when the train reaches the 9 completedUnitCount of 10, even if the CPU usage is still high, seems to happen a soft lock that doesn't never brings the model to its completion (or error).
The dataset is made of jpg images, using the CreateML app doesn't appear any problem during the training.
There is any known issue with CreateML training APIs about part 9 of the process?
There is any information about this part of the training job?
Thank you
I'm trying to create an updatable model, but this seems possible only by creating from scratch a neural network model and then, using the NeuralNetworkBuilder, call the make_updatable method.
But I met a lot of problems on this way for the solution.
In this example I try to open a converted ML Model (neural network) using the NeuralNetworkBuilder:
import coremltools
model = coremltools.models.MLModel("SimpleImageClassifier.mlpackage")
spec = model.get_spec()
builder = coremltools.models.neural_network.NeuralNetworkBuilder(spec=spec)
builder.inspect_layers()
But I met this error in the builder instance line:
AttributeError: 'NoneType' object has no attribute 'layers'
I also tried to define a neural network using the NeuralNetworkBuilder but then what do I have to do with this object? I didn't find a way to save it or convert it.
The result I want is simple, the possibility to train more the model on the user device to meet his exigences. However the way to obtain an updatable model seems incomprehensible.
In my case, the model should be an image classification.
What approach should I follow to achieve this result?
Thank you
Coremltools: 6.2.0
When I run coreml model in python result is good:
{'var_840': array([[-8.15439941e+02, 2.88793579e+02, -3.83110474e+02,
-8.95208740e+02, -3.53131561e+02, -3.65339783e+02,
-4.94590851e+02, 6.24686813e+01, -5.92614822e+01,
-9.67470627e+01, -4.30247498e+02, -9.27047348e+01,
2.19661942e+01, -2.96691345e+02, -4.26566772e+02........
But when I run on xcode so result look like:
[-inf,inf,nan,-inf,nan,nan,nan,nan,nan,-inf,-inf,-inf,-inf,-inf,-inf,nan,-inf,-inf,nan,-inf,nan,nan,-inf,nan,-inf,-inf,-inf,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,-inf,nan,nan,nan,nan,-inf,nan,-inf .......
Step1: Convert Resnet50 to coreml:
import torch
import torchvision
# Load a pre-trained version of MobileNetV2 model.
torch_model = torchvision.models.resnet50(pretrained=True)
# Set the model in evaluation mode.
torch_model.eval()
# Trace the model with random data.
example_input = torch.rand(1, 3, 224, 224)
traced_model = torch.jit.trace(torch_model, example_input)
out = traced_model(example_input)
# Download class labels in ImageNetLabel.txt.
# Set the image scale and bias for input image preprocessing.
import coremltools as ct
image_input = ct.ImageType(shape=example_input.shape,
)
# Using image_input in the inputs parameter:
# Convert to Core ML using the Unified Conversion API.
model = ct.convert(
traced_model,
inputs=[image_input],
compute_units=ct.ComputeUnit.CPU_ONLY,
)
# Save the converted model.
model.save("resnettest.mlmodel")
# Print a confirmation message.
print('model converted and saved')
Step2: Test model coreml in python:
import coremltools as ct
import PIL
import numpy as np
# Load the model
model = ct.models.MLModel('/Users/ngoclh/Downloads/resnettest.mlmodel')
print(model)
img_path = "/Users/ngoclh/gitlocal/DetectCirtochApp/DetectCirtochApp/resources/image.jpg"
img = PIL.Image.open(img_path)
img = img.resize([224, 224], PIL.Image.ANTIALIAS)
coreml_out_dict = model.predict({"input_1" : img})
print(coreml_out_dict)
Step3: Test coreml model in Xcode:
func getFeature() {
do {
let deepLab = try VGG_emb.init() //mobilenet_emb.init()//cirtorch_emb.init()
let image = UIImage(named: "image.jpg")
let pixBuf = image!.pixelBuffer(width: 224, height: 224)!
guard let output = try? deepLab.prediction(input_1: pixBuf) else {
return
}
let names = output.featureNames
print("ngoc names: ", names)
for name in names {
let feature = output.featureValue(for: name)
print("ngoc feature: ", feature)
}
} catch {
print(error)
}
}
Objective:
I am in the process of developing an application that utilizes machine learning (Core ML) to interact with photographs of documents, specifically focusing on those containing tables.
Step 1: Capturing the Image
The application will initiate by allowing users to take photos of documents. The key here is not just any part of the document, but specifically the sections where tables are present.
Step 2: Image Analysis through Machine Learning
Upon capturing the image, the next phase involves a machine learning model. Using Apple's Create ML tool with Swift, the application will analyze the image. The model's task is two-fold:
Identifying the Table: Distinguish the table from other document information, ensuring it recognizes and isolates the table structure within the photograph.
Ignoring Irrelevant Information: Concurrently, the model will disregard all non-table content, focusing the application's resources on the table data.
Step 3: Data Extraction and Training
Once the table is identified, the real work begins. The application will engage in detailed scrutiny, where it's trained to understand and recognize row and column data based on specific datasets. This training will enable the application to 'read' the table accurately, much like a human would, by identifying the organization of information into rows and columns.
Step 4: Information Storage
Post-analysis, the application will extract this critical data, storing it in a structured format. Each piece of identifiable information from the rows and columns will be systematically organized into a Dictionary or an Object. This structure is not just for immediate use but also efficient for future data operations within the app.
Conclusion:
Through these sequential steps, the application transitions from merely capturing an image to intelligently recognizing, deciphering, and storing table data from within a physical document. This streamlined process is all courtesy of integrating machine learning into the app's functionality, promising significant efficiency and accuracy in data handling.
Realistically, I have not found any good examples out there so I am attempting to create my own ML (with no experience 😅), so any guidance or help would be very much appreciated.
I followed the video of Composing advanced models with Create ML Components.
I have created the model with
let urlParameter = URL(fileURLWithPath: "/path/to/model.pkg")
let (training, validation) = dataFrame.randomSplit(by: 0.8)
let model = try await transformer.fitted(to: DataFrame(training), validateOn: DataFrame(validation)) { event in
guard let tAccuracy = event.metrics[.trainingAccuracy] as? Double else { return }
print(tAccuracy)
}
try transformer.write(model, to: url)
print("done")
Next goal is to read the model and update it with new dataFrame
let urlCSV = URL(fileURLWithPath: "path/to/newData.csv")
var model = try transformer.read(from: urlParameters) // loading created model
let newDataFrame = try DataFrame(contentsOfCSVFile: urlCSV ) // new dataFrame with features and annotations
try await transformer.update(&model, with: newDataFrame) // I want to keep previous learned data and update the model with new
try transformer.write(model, to: urlParameters) // the model saves but the only last added dataFrame are saved. Previous one just replaced with new one
But looks like I only replace old data with new one.
**The Question **
How can add new data to model I created without losing old one ?
I had code that ran 7x faster in Ventura compared to how it runs now in Sonoma.
For the basic model training I used
let pmst = MLBoostedTreeRegressor.ModelParameters(validation: .split(strategy: .automatic),maxIterations:10000)
let model = try MLBoostedTreeRegressor(trainingData: trainingdata, targetColumn: columntopredict, parameters: pmst)
Which took around 2 secs in Ventura and now takes between 10 and 14 seconds in Sonoma
I have tried to investigate why, and have noticed that when I use
I see these results
useWatchSPIForScribble: NO,
allowLowPrecisionAccumulationOnGPU: NO,
allowBackgroundGPUComputeSetting: NO,
preferredMetalDevice: (null),
enableTestVectorMode: NO,
parameters: (null),
rootModelURL: (null),
profilingOptions: 0,
usePreloadedKey: NO,
trainWithMLCompute: NO,
parentModelName: ,
modelName: Unnamed_Model,
experimentalMLE5EngineUsage: Enable,
preparesLazily: NO,
predictionConcurrencyHint: 0,
Why is the preferred Metal Device null?
If I do
let devices = MTLCopyAllDevices()
for device in devices {
config.preferredMetalDevice = device
print(device.name)
}
I can see that the M1 chipset is available but not selected (from reading the literature the default should be nil?)
Is this the reason why it is so slow? Is there a way to force a change in the config or elsewhere? Why has the default changed, if it has?
Hello,
I posted an issue on the coremltools GitHub about my Core ML models not performing as well on iOS 17 vs iOS 16 but I'm posting it here just in case.
TL;DR
The same model on the same device/chip performs far slower (doesn't use the Neural Engine) on iOS 17 compared to iOS 16.
Longer description
The following screenshots show the performance of the same model (a PyTorch computer vision model) on an iPhone SE 3rd gen and iPhone 13 Pro (both use the A15 Bionic).
iOS 16 - iPhone SE 3rd Gen (A15 Bioinc)
iOS 16 uses the ANE and results in fast prediction, load and compilation times.
iOS 17 - iPhone 13 Pro (A15 Bionic)
iOS 17 doesn't seem to use the ANE, thus the prediction, load and compilation times are all slower.
Code To Reproduce
The following is my code I'm using to export my PyTorch vision model (using coremltools).
I've used the same code for the past few months with sensational results on iOS 16.
# Convert to Core ML using the Unified Conversion API
coreml_model = ct.convert(
model=traced_model,
inputs=[image_input],
outputs=[ct.TensorType(name="output")],
classifier_config=ct.ClassifierConfig(class_names),
convert_to="neuralnetwork",
# compute_precision=ct.precision.FLOAT16,
compute_units=ct.ComputeUnit.ALL
)
System environment:
Xcode version: 15.0
coremltools version: 7.0.0
OS (e.g. MacOS version or Linux type): Linux Ubuntu 20.04 (for exporting), macOS 13.6 (for testing on Xcode)
Any other relevant version information (e.g. PyTorch or TensorFlow version): PyTorch 2.0
Additional context
This happens across "neuralnetwork" and "mlprogram" type models, neither use the ANE on iOS 17 but both use the ANE on iOS 16
If anyone has a similar experience, I'd love to hear more.
Otherwise, if I'm doing something wrong for the exporting of models for iOS 17+, please let me know.
Thank you!
Hi!
I'd like to ask why after training an action classifier model using CreateML, I consistently fail to detect my wrist joint while using the live preview feature, while other joints are detected correctly. What could be causing this result? Thank you to anyone for providing answers.
Training Data Sources:
YouTube videos, searching keyword: "20min workout at home"
All videos display the full body.
Training Data Quantity:
Model 1: Target action: 1 video / Irrelevant action: 1 video
Model 2: Target action: 6 videos / Irrelevant action: 40 videos
Both Model 1 and Model 2 fail to detect the wrist joint during live preview.
Following this guide https://developer.apple.com/documentation/CreateML/creating-a-multi-label-image-classifier
Has anyone been able to export a CoreML model, specifically according to the documentation below? Since there isn't any runnable examples, just snippets, perhaps documentation error? If anyone is already familiar with these pipeline generics, is there something that jumps out about the example transformer used that fails conformance or just factually incorrect?
Export the model to use with Vision
After you train the model, you can export it as a Core ML model.
// Export to Core ML
let modelURL = URL(filePath: "/path/to/model")
try model.export(to: modelURL)