Create ML model with 96% Training and 0% Validation

Question

Created Jan ’22

Replies 1

Boosts 0

Views 1.4k

Participants 2

I want to detect an image of a dart target (https://commons.wikimedia.org/wiki/File:WA_80_cm_archery_target.svg) in my iOS app.

For that I am creating an object detector with CreateML. I am using the Transfer Learning algorithm and 114 annotated images, the validation data is set to auto.

After 2000 iterations I got the following stats: 96% Training and 0% Validation.

As I understand it, the percentages are the I/U 50% scores (= percentage of intersection over union ratios from the bounding boxes with over 50%).

If the validation data is automatically chosen from the set of images, how can its score be 0%?

Boost

Answer 1

Frameworks Engineer OP

Apple

Feb ’22

I suspect you have an overfitted model (training accuracy high but validation accuracy low). Without looking at the actual data, it's hard to know what is going on, but you can consider have more diverse examples and use the Preview pane in CreateML app to check individual prediction result.

0