Face point are not showing correctly on preview layer in landscape mode in vision framework

Question

Created Jul ’20

Replies 1

Boosts 0

Participants 2

I am trying to detect face using vision API and Camera In portrait mode it is working fine but in land landscape mode it is giving wrong point

Using following code to get face rect

Code Block  let detectFaceRequest = VNDetectFaceLandmarksRequest(completionHandler: showFaceRect)      do {
        try sequenceHandler.perform([detectFaceRequest], on: imageBuffer,orientation: .leftMirrored)
      } catch {
        print(error.localizedDescription)
      }

Convert Vision rect to preview layer rect

Code Block   //Converts face rect from vision api to rect with respect to camera preview
  func convert(rect: CGRect) -> CGRect {
    let origin = previewLayer.layerPointConverted(fromCaptureDevicePoint:rect.origin)
    let size = previewLayer.layerPointConverted(fromCaptureDevicePoint: rect.size.cgPoint) //cgPoint converts rect height width as rect x and y
  return CGRect(origin: origin, size: size.cgSize)
  }

Handling orientation change for preview layer

Code Block      if let connection = self.previewLayer?.connection {
      let currentDevice: UIDevice = UIDevice.current
      let orientation: UIDeviceOrientation = currentDevice.orientation
      let previewLayerConnection : AVCaptureConnection = connection
     
       
      if previewLayerConnection.isVideoOrientationSupported {
        switch (orientation) {
        case .portrait: updatePreviewLayer(layer: previewLayerConnection, orientation: .portrait)
          break
        case .landscapeRight: updatePreviewLayer(layer: previewLayerConnection, orientation: .landscapeLeft)
          break
        case .landscapeLeft: updatePreviewLayer(layer: previewLayerConnection, orientation: .landscapeRight)
          break
        case .portraitUpsideDown: updatePreviewLayer(layer: previewLayerConnection, orientation: .portraitUpsideDown)
          break
        default: updatePreviewLayer(layer: previewLayerConnection, orientation: .portrait)
          break
        }
      }
    }
   private func updatePreviewLayer(layer: AVCaptureConnection, orientation: AVCaptureVideoOrientation) {
    layer.videoOrientation = orientation
    previewLayer.frame = self.cameraView.bounds
    }

Boost

Answer 1

Engineer OP

Apple

Aug ’20

You are handling orientation change for the preview layer, but in your first block you are still calling perform method with orientation set to .leftMirrored. Vision is returning all coordinates relative to that orientation.

0