I'm looking for someone who's experienced in machine learning and that might be interested in starting a project.Best regardsSebastian

I'm currently gathering data to train an object detection model using the CreateML app. I'd like to get an understanding if any image augmentation is done automatically during the training process in the CreateML app. eg does the CreateML app Flip, Rotate, Blur or Crop by default when training or should I be providing these images? I know when training an image classifier model you have the option to check boxes for augmentation but I'm trying to understand if I need to provide training data with these augmented styles.

Can anyone clarify which devices or chips the so-called "neural engine" can be used for machine learning model Training? As opposed to inference (or prediction)? And can external libraries such as Tensorflow (perhaps via the browser-based javascript library) access the neural engine in any manner for training or inference?

Can I run inference on the new MacBook Pro with M1 Chips (Apple Silicon) using Keras Models (sometimes PyTorch). These would be computer vision models, some might have custom loss functions or metrics and would have been trained on lets say, Google Colab. If I can perform inference, how do I do that? Also, will the Neural Engines help while performing inference or will it boost training if I have to train on the Mac?

I trained the object detection model into 3 classes using CreateML. Loaded the model into coremltools and performed the prediction, got 'coordinates' and 'confidence' but got no class labels. How can I get class labels? {'coordinates': array([[0.31552333, 0.30995899, 0.38225624, 0.59993058], [0.41404313, 0.5812282 , 0.07037259, 0.06291649], [0.23581643, 0.08297428, 0.18884215, 0.12770388], [0.42899013, 0.18289472, 0.06921174, 0.06600466]]), 'confidence': array([[9.92818241e-06, 9.99987483e-01, 8.74077966e-09], [1.00150883e-05, 1.29828220e-06, 9.98730123e-01], [9.98335183e-01, 1.02320407e-03, 2.00047023e-09], [6.40532759e-04, 1.93859320e-04, 8.27081084e-01]])} def get_feature_size(spec): 		input_description = spec.description.input[0] 		inputSizeImg = input_description.type.imageType.imageSizeRange.widthRange.lowerBound, \ 									 input_description.type.imageType.imageSizeRange.heightRange.lowerBound 		return inputSizeImg loaded_model = coremltools.models.MLModel(modelPath) inputSize_img = get_feature_size(loaded_model.get_spec()) with Image.open(imgPath).resize(input_size_img) as img: out_dict = model.predict({'image':img})

During the launch of M1, Apple mentioned clearly that new M1 has superior performance for running Tensorflow. On the ground, it's not working as expected and their fork has tons of issues and no continuous updates. On the other side, TensorFlow team said clearly that they are not required to attend to ARM64 support. I end in no support from both sides; Apple that is not caring to attend issues in their tensorflow fork & Google that they are not willing to support ARM64. Is there any advice for that?

hello, When I used xcode to generate the model encryption key, an error was reported, the error was 'Failed to Generate Encryption Key and Sign in with you Apple ID in the Apple ID pane in System Preferences and retry '.But I have logged in my apple id in the system preferences, and this error still occurs.I reinstalled xcode and re-logged in to my apple id. This error still exists. Xcode Version 12.4 macOS Catalina 10.15.7 thanks

I'm now running Tensorflow models on my Macbook Air 2020 M1, but I can't find a way to monitor the Neural Engine 16 cores usage to fine tune my ML tasks. The Activity Monitor only reports CPU% and GPU% and I can't find any APIs available on Mach include files in the MacOSX 11.1 sdk or documentation available so I can slap something together from scratch in C. Could anyone point me in some direction as to get a hold of the API for Neural Engine usage. Any indicator I could grab would be a start. It looks like this has been omitted from all sdk documentation and general userland, I've only found a ledger_tag_neural_footprint attribute, which looks memory related, and that's it.

I have a MacBook Pro M1 (16 GB RAM), and testing CreateML's StyleTransfer model training. When I press «Train», it starts processing and fails with error «Could not create buffer with format BGRA -6662». During the «processing» it allocates about 4.5 GB of RAM. I guess it runs out of memory, however, I've closed all other programs and I can see that there's lot of free RAM when it fails. It happens even if I use just 3 small (500*500) images, 1 for each training, content and validation. So, how to fix it?

Is there a Machine Learning API that can take handwriting (either as a bitmap or as a list of points) and convert it to text? I know Scribble can be used to allow handwriting input into text fields, but in this API it is Scribble which controls the rendering of the handwriting. Is there an API where my app can render the handwriting and get information about the text content? In the Keynote demo Craig was able to get text content from a photo of a whiteboard. Are there APIs which would allow an app developer to create something similar?

Dear all, I found the announced built-in sound classifier pretty amazing. I would appreciate it if you could point me to a link or a document that is listed all 300 sound classes mentioned in https://developer.apple.com/videos/play/wwdc2021/10036/. Thank you

Where can I find a comprehensive list of all the classes that the built in Sound Classifier model supports?

Hello! I was wondering if it would be possible for the sample code for the Meal App to be posted. There are some things I'd like to see regarding MLLinearRegressor and how models can be personalized with context and data.

The TabularData DataFrame writeCSV method formats Double (and possibly Integer) numbers that are >= 1000 with a comma thousands separator and surrounds the output in double quotes. Numbers less than 1000 are not double-quoted. If the resulting CSV file is then read by DataFrame contentsOfCSVFile an error is thrown, i.e. a type mismatch. This happens irrespective of whether the types option is set in the CSV read, or not. There is no option in DataFrame writeCSV to specify no thousands separator, nor any other obvious way of preventing the double-quoting. Currently, I "fix" the problem by reading the CSV in Numbers, setting the faulty column(s) to Numeric then exporting back to CSV. Any thoughts on preventing the issue in the first place?

I have watched the 'Classify hand poses and actions with Create ML' from WWDC2021 I was able to create a model using the Create ML app. But I am not sure how to use it in my iOS project. The video above misses out several steps. I have the HandPoseRequest working and can get the MLMultiArray of points. I have added the model file to my project and can create a MLModel instance from it. But the video above shows: let handPosePredition = try model.prediction(poses: keypointsMultiArray) What is the type of model ? How was it created? I cannot find any model type that has a function prediction(poses: MLMultiarray) Thanks.

Most examples, including within documentation, of using CoreML with iOS involve the creation of the Model under Xcode on a Mac and then inclusion of the Xcode generated MLFeatureProvider class into the iOS app and (re)compiling the app.  However, it’s also possible to download an uncompiled model directly into an iOS app  and then compile it (background tasks) - but there’s no MLFeatureProvider class.  The same applies when using CreateML in an iOS app (iOS 15 beta) - there’s no automatically generated MLFeatureProvider.  So how do you get one?  I’ve seen a few queries on here and elsewhere related to this problem, but couldn’t find any clear examples of a solution.  So after some experimentation, here’s my take on how to go about it: Firstly, if you don’t know what features the Model uses, print the model description e.g. print("Model: ",mlModel!.modelDescription). Which gives Model:   inputs: (     "course : String",     "lapDistance : Double",     "cumTime : Double",     "distance : Double",     "lapNumber : Double",     "cumDistance : Double",     "lapTime : Double" ) outputs: (     "duration : Double" ) predictedFeatureName: duration ............ A prediction is created by guard **let durationOutput = try? mlModel!.prediction(from: runFeatures) ** …… where runFeatures is an instance of a class that provides a set of feature names and the value of each feature to be used in making a prediction.  So, for my model that predicts run duration from course, lap number, lap time etc the RunFeatures class is: class RunFeatures : MLFeatureProvider {     var featureNames: Set = ["course","distance","lapNumber","lapDistance","cumDistance","lapTime","cumTime","duration"]     var course : String = "n/a"     var distance : Double = -0.0     var lapNumber : Double = -0.0     var lapDistance : Double = -0.0     var cumDistance : Double = -0.0     var lapTime : Double = -0.0     var cumTime : Double = -0.0          func featureValue(for featureName: String) -> MLFeatureValue? {         switch featureName {         case "distance":             return MLFeatureValue(double: distance)         case "lapNumber":             return MLFeatureValue(double: lapNumber)         case "lapDistance":             return MLFeatureValue(double: lapDistance)         case "cumDistance":             return MLFeatureValue(double: cumDistance)         case "lapTime":             return MLFeatureValue(double: lapTime)         case "cumTime":             return MLFeatureValue(double: cumTime)         case "course":             return MLFeatureValue(string: course)         default:             return MLFeatureValue(double: -0.0)         }     } } Then in my DataModel, prior to prediction, I create an instance of RunFeatures with the input values on which I want to base the prediction: var runFeatures = RunFeatures() runFeatures.distance = 3566.0 runFeatures.lapNumber = 1.0 runFeatures.lapDistance = 1001.0  runFeatures.lapTime = 468.0  runFeatures.cumTime = 468.0  runFeatures.cumDistance = 1001.0  runFeatures.course = "Wishing Well Loop" NOTE there’s no need to provide the output feature (“duration”) here, nor in the featureValue method above but it is required in featureNames. Then get the prediction with guard let durationOutput = try? mlModel!.prediction(from: runFeatures)  Regards, Michaela

DataFrame(contentsOfJSONFile:url) (and it's MLDataTable equivalent) assumes that the rows to be imported are at the root level of the JSON structure. However, I'm downloading a fairly large dataset that has "header" information, such as date created and validity period, with the targeted array at a subsidiary level. The DataFrame initialiser has JSONReadingOptions, but these don't apply to this situation (i.e. there's nothing that helps). It seems that I'm faced with the options of 1) stripping the extraneous data from the JSON file, to leave just the array or 2) decoding the JSON file into a bespoke struct then converting its array into a DataFrame - which removes a lot of the benefits of using DataFrame in the first place. Any thoughts? Cheers, Michaela

Is Apple planning to fix the M1-Pytorch incompatibility? I've tried to build from source but I thought the M1 was supposed to be better with machine learning software.

Create a sample project in Creat ML and choose Activity Classifier. When lowering the sample rate, the preview timeline gets shorter instead of getting longer. Furthermore, it seems the entire preview timeline breaks (can't scroll) if the sample rate is anything other than 50. Try it out: Train a model with sample rate 50, then try training it with sample rate 10.

Could I please ask what is (at least plainly) the deep learning architecture of the Apple's custom pose models available through Vision (for example with the VNDetectHumanBodyPoseRequest)? Or whether it is based on some publicly used architecture (such as ResNet) only with modifications or custom Apple dataset? I was not able to find this information anywhere in the Apple documentation and it would be highly beneficial to know this, as we are using this data in a research about which we want to publish a paper. Thanks beforehand!