CreateML object detection evaluation


Hey guys I'm a medical student researcher doing some simple work using CreateML and iOS. Im using CreateML and made a model using the object detection, when I get to the evaluation section I’m lost, traditionally for image classification it tells you the precision/accuracy of the model based on its classes but for Object detection it’s different. The section I was confused on was the evaluation/accuracy tab where the Object detection model talks about I/U 50% and Varied I/U and gives us %s for each of those classes. Could you explain to me what the I/U 50% and the Varied I/U represent? I tried searching across the web but almost nobody references those specifically and discusses what they mean. I dont have much of an AI/ML background so its a foreign concept to me. I would greatly appreciate it! Thank you!

Post not yet marked as solved Up vote post of nacrogyn Down vote post of nacrogyn
1.1k views

Replies

Hi!

I have been working on an object detector using the Create ML app and I was confused about the metrics it gave me, as well.

I/U means "Intersection over Union". It is a very simple metric which tells you how good the model is at predicting a bounding box of an object. You can read more about I/U in the "Intersection over Union (IoU) for object detection" article at PyImageSearch.

So:
I/U 50% means how many observations have their I/U score over 50%
Varied I/U is an average I/U score for a data set

Note: it is my understanding of Apple's intention of giving those metrics. What I mean is I am not exactly sure if they meant > or >= 50% in the first case and if it really is an average in the second case, but I am pretty sure it is just the way it works.

Take care!