CoreML Detecting Specific Objects Based On Inputs

I would like to detect specific objects based on a text input for a coreml project. I am currently using ios 14 along with Xcode and using Apple developer object detection template linked here.

For example, can I have a textbox with live capture in the background, and when you put a Banana into the textbox, it only detects bananas, and when you put apples, it only detects apples? Would I have to edit the [VNRecognizedObjectObservation] array? (Basically I just want to control what objects I can detect)

Up vote post of Archie_S Down vote post of Archie_S
959 views
Posted by
Archie_S
Reply
Add a Comment

Replies

This is a neat idea that there's a couple ways you could approach. Two that seem intuitive to me:

  1. Train a few different object detection models for a single class that you want to detect. So one banana model, one apple model, etc. Programmatically switch which model you are using to look for objects based on what's in the textbox.
  2. Train a single model that looks for all of the objects you want to see, and store the predicted output of live capture. Filter the labels seen depending on what's in the textbox to avoid returning any labels that are not currently selected.

There's some tradeoffs in model size/accuracy of the two choices above, but they would both accomplish your task.

Posted by
jcackler
Up vote reply of jcackler Down vote reply of jcackler
Add a Comment