Hello, Is there an API available for "Visual Look Up"? https://support.apple.com/en-gb/guide/iphone/iph21c29a1cf/ios

I have MacBook with M1 Pro and I'm trying to use tensorflow_decision_forests. I followed instruction and created new conda env with tenserflow-metal (https://developer.apple.com/metal/tensorflow-plugin/), but it doesn't work right away. I had to downgrade library versions to tenserflow-macos==2.9.0 and tensorflow-metal==0.5.0. nNow i am trying to install tensorflow_decision_forests using pip, however this library requires tenserflow-macos~=2.10.0. Has anyone successfully installed this library?

Hey guys, First post so I'm sorry for making any newbie mistakes. Anyways, I've been trying to get into ML and I wanted to follow a course on it but it requires Tensorflow and I've been trying to get that working on my system. I have the 2021 14" 16GB Macbook Pro with the M1 Pro Chip and I am running Ventura 13.1. I have been following this article as well as digging around about getting Tensorflow working on M1 but to no avail. I managed to get tensorflow-macos installed in my environment as well as tensorflow-metal but when I try to run some sample code in Juyter, I'm getting an error that I do not understand. In Jupyter, when I run: import tensorflow as tf print("Num GPUs Available: ", len(tf.config.experimental.list_physical_devices('GPU'))) I get Num GPUs Available: 1 but when I try to run the rest of the sample code %%time import tensorflow as tf import tensorflow_datasets as tfds print("TensorFlow version:", tf.__version__) print("Num GPUs Available: ", len(tf.config.experimental.list_physical_devices('GPU'))) tf.config.list_physical_devices('GPU') (ds_train, ds_test), ds_info = tfds.load( 'mnist', split=['train', 'test'], shuffle_files=True, as_supervised=True, with_info=True, ) def normalize_img(image, label): """Normalizes images: `uint8` -> `float32`.""" return tf.cast(image, tf.float32) / 255., label batch_size = 128 ds_train = ds_train.map( normalize_img, num_parallel_calls=tf.data.experimental.AUTOTUNE) ds_train = ds_train.cache() ds_train = ds_train.shuffle(ds_info.splits['train'].num_examples) ds_train = ds_train.batch(batch_size) ds_train = ds_train.prefetch(tf.data.experimental.AUTOTUNE) ds_test = ds_test.map( normalize_img, num_parallel_calls=tf.data.experimental.AUTOTUNE) ds_test = ds_test.batch(batch_size) ds_test = ds_test.cache() ds_test = ds_test.prefetch(tf.data.experimental.AUTOTUNE) model = tf.keras.models.Sequential([ tf.keras.layers.Conv2D(32, kernel_size=(3, 3), activation='relu'), tf.keras.layers.Conv2D(64, kernel_size=(3, 3), activation='relu'), tf.keras.layers.MaxPooling2D(pool_size=(2, 2)), # tf.keras.layers.Dropout(0.25), tf.keras.layers.Flatten(), tf.keras.layers.Dense(128, activation='relu'), # tf.keras.layers.Dropout(0.5), tf.keras.layers.Dense(10, activation='softmax') ]) model.compile( loss='sparse_categorical_crossentropy', optimizer=tf.keras.optimizers.Adam(0.001), metrics=['accuracy'], ) model.fit( ds_train, epochs=12, validation_data=ds_test, ) I get TensorFlow version: 2.11.0 Num GPUs Available: 1 Metal device set to: Apple M1 Pro WARNING:tensorflow:AutoGraph could not transform <function normalize_img at 0x14a4cec10> and will run it as-is. Cause: Unable to locate the source code of <function normalize_img at 0x14a4cec10>. Note that functions defined in certain environments, like the interactive Python shell, do not expose their source code. If that is the case, you should define them in a .py source file. If you are certain the code is graph-compatible, wrap the call using @tf.autograph.experimental.do_not_convert. Original error: could not get source code To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert 2022-12-13 13:54:33.658225: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:306] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support. 2022-12-13 13:54:33.658309: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:272] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>) WARNING:tensorflow:AutoGraph could not transform <function normalize_img at 0x14a4cec10> and will run it as-is. Cause: Unable to locate the source code of <function normalize_img at 0x14a4cec10>. Note that functions defined in certain environments, like the interactive Python shell, do not expose their source code. If that is the case, you should define them in a .py source file. If you are certain the code is graph-compatible, wrap the call using @tf.autograph.experimental.do_not_convert. Original error: could not get source code To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert WARNING: AutoGraph could not transform <function normalize_img at 0x14a4cec10> and will run it as-is. Cause: Unable to locate the source code of <function normalize_img at 0x14a4cec10>. Note that functions defined in certain environments, like the interactive Python shell, do not expose their source code. If that is the case, you should define them in a .py source file. If you are certain the code is graph-compatible, wrap the call using @tf.autograph.experimental.do_not_convert. Original error: could not get source code To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert Epoch 1/12 2022-12-13 13:54:34.162300: W tensorflow/tsl/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz 2022-12-13 13:54:34.163015: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled. 2022-12-13 13:54:35.383325: W tensorflow/core/framework/op_kernel.cc:1830] OP_REQUIRES failed at xla_ops.cc:418 : NOT_FOUND: could not find registered platform with id: 0x14a345660 2022-12-13 13:54:35.383350: W tensorflow/core/framework/op_kernel.cc:1830] OP_REQUIRES failed at xla_ops.cc:418 : NOT_FOUND: could not find registered platform with id: 0x14a345660 2022-12-13 13:54:35.389028: W tensorflow/core/framework/op_kernel.cc:1830] OP_REQUIRES failed at xla_ops.cc:418 : NOT_FOUND: could not find registered platform with id: 0x14a345660 2022-12-13 13:54:35.389049: W tensorflow/core/framework/op_kernel.cc:1830] OP_REQUIRES failed at xla_ops.cc:418 : NOT_FOUND: could not find registered platform with id: 0x14a345660 2022-12-13 13:54:35.401250: W tensorflow/core/framework/op_kernel.cc:1830] OP_REQUIRES failed at xla_ops.cc:418 : NOT_FOUND: could not find registered platform with id: 0x14a345660 2022-12-13 13:54:35.401274: W tensorflow/core/framework/op_kernel.cc:1830] OP_REQUIRES failed at xla_ops.cc:418 : NOT_FOUND: could not find registered platform with id: 0x14a345660 2022-12-13 13:54:35.405004: W tensorflow/core/framework/op_kernel.cc:1830] OP_REQUIRES failed at xla_ops.cc:418 : NOT_FOUND: could not find registered platform with id: 0x14a345660 2022-12-13 13:54:35.405025: W tensorflow/core/framework/op_kernel.cc:1830] OP_REQUIRES failed at xla_ops.cc:418 : NOT_FOUND: could not find registered platform with id: 0x14a345660 in addition to the traceback. Can someone help me decipher what's going wrong here?

Hello, I have created a data model with Text Classification which contains many garments with the label clothing, technology equipment with the label technology, etc.... . I have inserted it so that when entering for example "T-shirt" the label clothing is output. But I want it so that when I create an equipment category with the clothing, that it suggests me clothing items, which I add. How do I implement this. In the function you can only use"preditction.label". I program with SwiftUI. func textml() {         do {             let model = try DataModel_Symbols_Consilia(configuration: .init())             let prediction = try model.prediction(text: addProjectVM.title)             addProjectVM.symbol = prediction.label         } catch {             modelOutput = "Something went wrong"         }     } Greetings Janik

Hello Everyone, I am trying to train a hand pose classifier. But I stumble upon the problem where if I give training data above certain number (i.e. ~4000 images). Is it a limit imposed by Apple? because I even tried to reduce the size of images and even dimensions but it always gives me error while training whenever I give training data above certain number. Has anyone else stumbled upon this problem?if so, is there any work around for any number of data can be given to CreateML? Thanks

Hello. I would like to compile the YOLOv3.mlmodel for object detection as a framework and call it from applescript to use it, but I do not know how. The flow I am imagining is as follows Import "YOLOv3.mlmodel" on xcode and create a framework. Load the created framework with applescript's use statement and execute it. Please let me know if you know anything about this.

Greetings! I have a MacBook Pro (2018) running macOS Monterey and was wondering if it could be repurposed for machine learning work. I have a Razer Core X, which would enable me to use AMD Radeon 6900 XT as a eGPU supported configuration. Before doing any shopping, my question is whether the usage of TensorFlow is feasible with this setup. I am aware of tensorflow-metal, but if I understand correctly, only natively supported Mac GPUs can be used with it (not eGPUs)? PlaidML is also an option, but it still not clear which external GPUs are supported. Any guidance on this topic will be appreciated.

hello, When I used xcode to generate the model encryption key, an error was reported, the error was 'Failed to Generate Encryption Key and Sign in with you Apple ID in the Apple ID pane in System Preferences and retry '.But I have logged in my apple id in the system preferences, and this error still occurs.I reinstalled xcode and re-logged in to my apple id. This error still exists. Xcode Version 12.4 macOS Catalina 10.15.7 thanks

The previous question haven't been solved. I send it again. when my video neural network model was running on iphone11, the cpu occupation reached to 31%, is it normal? I am not sure if the value 31% has a serious impact on the other app. So that I would ask for your advice. Previously, I originally expected to lower the cpu percentage by reducing the model op. when I put a model with only 6 biliear_interpolation ops into coreml, the cpu percentage is 20%(equal on cpu, cpuandgpu&all), so I consider if the only coreML scheduling occupy the 20% cpu?

Hi, ich habe nach folgender Anleitung versucht ein Text classifier model zu machen.Allerdings scheitert dies daran, das die evaluation() Funktion nicht funktioniert.Den Error den ich (in SwiftPlayground) bekomme ist: No exact matches in call to instance method 'evaluation' Hier ist der Code, den ich verwende: if #available(iOSApplicationExtension 15.0, *) {         let data = try! MLDataTable(contentsOf:  fileLiteral(resourceName: "TestData.json"))          let (trainingData, testingData) = data.randomSplit(by: 0.8, seed: 5)                  let sentimentClassifier = try! MLTextClassifier(trainingData: trainingData,                                                        textColumn: "text",                                                        labelColumn: "label")         // Training accuracy as a percentage         let trainingAccuracy = (1.0 - sentimentClassifier.trainingMetrics.classificationError) * 100         // Validation accuracy as a percentage         let validationAccuracy = (1.0 - sentimentClassifier.validationMetrics.classificationError) * 100         let evaluationMetrics = sentimentClassifier.evaluation(on: testingData)//Error hier         let evaluationAccuracy = (1.0 - evaluationMetrics.classificationError) * 100                  let metadata = MLModelMetadata(author: "No One",                                        shortDescription: "A model trained to classify movie review sentiment",                                        version: "1.0")         try! sentimentClassifier.write(to: URL(fileURLWithPath: Bundle.main.path(forResource: "MODEL", ofType: "mlmodel")!),                                       metadata: metadata)     } else {         print("dein Gerät erfüllt nicht die Bedingungen für dieses Programm")     } vielen Dank im Voraus (:

I was trying to hand action repetition though HumanBodyActionCounter but seems like its not working. So does HumanBodyActionCounter counts hand action also?

I found out that tf.einsum is extremely slow when running on Apple Silicone. And it's due to tf.einsum being placed on CPU instead of GPU. This causes very low GPU utilization and slowness of the operation itself. I tried forcing it on GPU, but does not work. It can be reproduced with following code that logs device placement for tf.einsum. import tensorflow as tf # https://www.tensorflow.org/guide/gpu#logging_device_placement tf.debugging.set_log_device_placement(True) print(tf.__version__) print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU'))) B=1000 S=200 N=16 K=64 a = tf.random.uniform(shape=[B,N,S,K], minval=0.0, maxval=100.0, dtype=float) b = tf.random.uniform(shape=[B,N,S,K], minval=0.0, maxval=100.0, dtype=float) tf.einsum('BNSK,BNTK->BNTS', a,b) # Notice in the output Einsum is placed on CPU. Output: Executing op Einsum in device /job:localhost/replica:0/task:0/device:CPU:0 2023-03-08 16:44:01.119144: I tensorflow/core/common_runtime/placer.cc:114] inputs_0: (_Arg): /job:localhost/replica:0/task:0/device:CPU:0 2023-03-08 16:44:01.119182: I tensorflow/core/common_runtime/placer.cc:114] inputs_1: (_Arg): /job:localhost/replica:0/task:0/device:CPU:0 2023-03-08 16:44:01.119191: I tensorflow/core/common_runtime/placer.cc:114] Einsum: (Einsum): /job:localhost/replica:0/task:0/device:CPU:0 2023-03-08 16:44:01.119197: I tensorflow/core/common_runtime/placer.cc:114] output_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:CPU:0 If I use tf.matmul, then everything happen on GPU which is what I want. With this limitation I basically have to convert all the code that uses tf.einsum to using tf.matmul and is very inconvenient. Has anyone noticed the same problem? What's your solution?

I'm trying to use tensorflow, but I can't import it. I followed the steps on the Apple website to download and install Tensorflow, and everything appears to be ok, but when I try to import tensorflow, I get some errors. What should I do? Errors: TypeError: Unable to convert function return value to a Python type! The signature was () -> handle RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xf RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xf ImportError: numpy.core._multiarray_umath failed to import ImportError: numpy.core.umath failed to import what I did: conda create --name myenv python conda activate myenv conda install -c apple tensorflow-deps python -m pip install tensorflow-macos python -m pip install tensorflow-metal conda list: Thanks, Ed

Hello, I wanted to hear some opinions on this problem I want to tackle. Currently at my job we have an Endpoint Security sysext app (swift) deployed on 10k+ macs and we are using a custom rule engine we developed to run some rules on the events received by the app. These rules are downloaded by the app. This works great but we wanted to dive into the world of ML and try to use it to detect more complex malware that may be more difficult to detect using rules. We thought of two options to approach this: Periodically collect events from all macs and send them to an api to be stored somewhere and perform the training in the cloud. Somehow, maybe using the ML frameworks provided in Swift, train the model IN the device rather than in the cloud. I know this is a very broad question but I just wanted to hear some suggestions. Thanks in advance.

when my video neural network model was running on iphone11, the cpu occupation reached to 31%, is it normal?

My neural network model has only 6 biliear_interpolation ops running on cpuandgpu, the model's cpu percentage is 20%. And I test my int8model by choosing 3 devices onlyCpu, CpuandGpu, Allunits, the cpu occupation is the same. This is normal? The pecentage is too large?

Hello- what was the data set size for labeling for this application? Also how did you train new data on new environment. For example, at different times of day, different shadows, different objects, etc…

Hi, I am trying to take advantage of my device ANE. I have created a model from torch using coremltools and adapted it until xcode model performance preview indicates it will run on my device ANE. But when i profile my integration into my app, i can see from the com.apple.ane logs the model has been loaded on device: Timestamp Type Process Category Message 00:00.905.087 Debug MLBench (11135) client doLoadModel:options:qos:error:: model[0x2804500c0] : success=1 : progamHandle=10 000 241 581 886: intermediateBufferHandle=10 000 242 143 532 : queueDepth=32 :err= but when i call predict on my model, the ANE is unloaded and the prediction run on CPU: Timestamp Type Process Category Message 00:00.996.015 Debug MLBench (11135) client doUnloadModel:options:qos:error:: model[0x2804500c0]=_ANEModel: { modelURL=file:///var/mobile/Containers/Data/Application/0A9F356B-B8C7-4B86-90A5-6812EF48CC94/tmp/math_custom_trans_decoder_seg_0DB63A47-E84E-4887-A606-BC9986B2C662.mlmodelc/ : key={"isegment":0,"inputs":{"extras":{"shape":[2,1,1,1,1]},"memory":{"shape":[128,5,1,1,1]},"proj_key_seg_in":{"shape":[128,39,1,1,1]},"state_in_k":{"shape":[32,1,1,20,2]},"tgt":{"shape":[5,1,1,1,1]},"state_in_v":{"shape":[32,1,1,20,2]},"pos_enc":{"shape":[128,1,1,1,1]}},"outputs":{"attn_seg":{"shape":[1,5,1,4,1]},"state_out_v":{"shape":[32,2,1,20,2]},"output":{"shape":[292,5,1,1,1]},"state_out_k":{"shape":[32,2,1,20,2]},"extras_tmp":{"shape":[2,1,1,1,1]},"proj_key_seg_in_tmp":{"shape":[128,39,1,1,1]},"attn":{"shape":[1,1,1,5,2]},"proj_key_seg":{"shape":[128,1,1,20,1]}}} : string_id=0x70ac000000015257 : program=_ANEProgramForEvaluation: { programHandle=10000241581886 : intermediateBufferHandle=10000242143532 : queueDepth=32 } : state=3 : programHandle=10000241581886 : intermediateBufferHandle=10000242143532 : queueDepth=32 : attr=... : perfStatsMask=0}  i dont see any obvious error messages in com.apple.ane, com.apple.coreml or com.apple.espresso. where/what should i look for to understand what is going on? and in particular why the ANE model was unloaded? Thank you

Is there any way to train PyTorch models directly on-device on an iPhone via the GPU? PyTorch Mobile docs seems to be completely focused on inference only, as do the iOS app examples (https://github.com/pytorch/ios-demo-app). I did find this article about using MPS backend on Macs (https://developer.apple.com/metal/pytorch/), but not sure if this is at all viable for iOS devices. There's also this prototype article about using iOS GPU for PyTorch mobile (https://pytorch.org/tutorials/prototype/ios_gpu_workflow.html), but it too seems to be focused on inference only. We are attempting to train a large language model on the iPhone 14 and we would like to a) discard intermediate activations and recompute them, and b) manage memory directly to write some intermediate activations to the filesystem and later read them back. We suspect that converting a PyTorch model to CoreML format and using CoreML for training would prevent us from making these low-level modifications, but PyTorch might have the APIs necessary for this. If there's any examples/pointers that anyone can link to that would be great.

Hi, I have been trying to run python scripts on PyCharm but haven't been able to. I have tried different versions of python and a lot of package managers. I haven't been able to run the scripts and haven't been able to pip install tensorflow either.