Custom Model Not Working Correctly in the Application #56

I created a model that classifies certain objects using yolov8. I noticed that the model is not working properly in my application. While the model works fine in Xcode preview, in the application it either returns the same result with 99% accuracy for each classification or does not provide any result.

In Preview it looks like this:

Predictions:

extension CameraVC : AVCapturePhotoCaptureDelegate {
    
    func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: (any Error)?) {
        
        guard let data = photo.fileDataRepresentation() else {
            return
        }        
        guard let image = UIImage(data: data) else {
            return
        }
            guard let cgImage = image.cgImage else {
                fatalError("Unable to create CIImage")
            }
            let handler = VNImageRequestHandler(cgImage: cgImage,orientation: CGImagePropertyOrientation(image.imageOrientation))
            DispatchQueue.global(qos: .userInitiated).async {
                do {
                    try handler.perform([self.viewModel.detectionRequest])
                } catch {
                    fatalError("Failed to perform detection: \(error)")
                }
   }
lazy var detectionRequest: VNCoreMLRequest = {
        do {
            let model = try VNCoreMLModel(for: bestv720().model)
            let request = VNCoreMLRequest(model: model) { [weak self] request, error in
                self?.processDetections(for: request, error: error)
            }
           request.imageCropAndScaleOption = .centerCrop
            return request
        } catch {
            fatalError("Failed to load Vision ML model: \(error)")
        }
    }()

This is where i print recognized objects:

func processDetections(for request: VNRequest, error: Error?) {
       DispatchQueue.main.async {
           guard let results = request.results as? [VNRecognizedObjectObservation] else {
               return
           }
           var label = ""
           var all_results = []
           var all_confidence = []
           var true_results = []
           var true_confidence = []
           for result in results {
               for i in 0...results.count{
                   all_results.append(result.labels[i].identifier)
                   all_confidence.append(result.labels[i].confidence)
                   for confidence in all_confidence {
                       if confidence as! Float > 0.7 {
                           true_results.append(result.labels[i].identifier)
                           true_confidence.append(confidence)
                       }
                   }
               }
               label = result.labels[0].identifier
           }
           print("True Results " , true_results)
           print("True Confidence ", true_confidence)
           self.output?.updateView(label:label)
       }
   }

I converted the model like this:

from ultralytics import YOLO
model = YOLO(model_path)
model.export(format='coreml', nms=True, imgsz=[720,1280])

@tarikdmrts, at this line, could you confirm reasonable image is stored in cgImage variable in a debugger? I feel that the image conversion from Photo to CGImage might have gone wrong.

let handler = VNImageRequestHandler(cgImage: cgImage,orientation: CGImagePropertyOrientation(image.imageOrientation))

We would like to have a Feedback Assistant report with a reproducible sample code for further analysis. (https://feedbackassistant.apple.com).

Custom Model Not Working Correctly in the Application #56
 
 
Q