Object Capture With only manual capturing

Question

Is it possible to capture only manually (automatic off) on object capture api ?

And can I proceed to capturing stage right a way?

Only Object Capture API captures real scale object.

Using AVFoundation or ARKit, I've tried using lidar capturing HEVC or create PhotogrammetrySample, It doesn't create real scale object.

I think, during object capture api, it catches point cloud, intrinsic parameter, and it help mesh to be in real scale.

Does anyone knows 'Object Capture With only manual capturing' or 'Capturing using AVFoundation for real scale mesh'

939

Posted by

HeoJin

Reply

Add a Comment

Answer 1

There is no manual only capture on iOS ObjectCaptureSession API, and a bounding box is required.

You can use a 2 rear camera session of AVFoundation to capture HEIC images with stereo depth embedded instead (which we demonstrated in the previous release of Object Capture) or provide the AVDepthData depth maps in the PhotogrammetrySample. If the depth map is provided correctly, the scale should be correctly computed.

Posted by

Graphics and Games Engineer

Can I achieve this with .. currentCamera = AVCaptureDevice.default(.builtInLiDARDepthCamera, for: .depthData, position: .back) ?

—
HeoJin
Sample code for this is here: https://developer.apple.com/documentation/realitykit/taking_pictures_for_3d_object_capture

—
Graphics and Games Engineer
Thanks for your answer. But what I want is to use builtInLiDARDepthCamera. And though not sure what I've missing, but when I give PhotogrammetrySample Color CVPixelBuffer and correct depth CVPixelBuffer(tried both DisparityFloat32 And DepthFloat32), not exact real scale model shows up. So, I guessed that's because metadata is missing.

With HEIC Image took from ObjectCapturession has metadata including intrinsic parameter, So I think that's the reason why my App doesn't return real scale.

—
HeoJin

Add a Comment

Answer 2

I tried to initialize session with bellow. //MARK: Initialize Camera private func initializeCamera() { print("Initialize Camera") currentCamera = AVCaptureDevice.default(.builtInLiDARDepthCamera, for: .depthData, position: .back)

    currentSession = AVCaptureSession()
    currentSession.sessionPreset = .photo
    do {
        let cameraInput = try AVCaptureDeviceInput(device: currentCamera)
        currentSession.addInput(cameraInput)
    } catch {
        fatalError()
    }
    let videoOutput = AVCaptureVideoDataOutput()
    videoOutput.setSampleBufferDelegate(self,
                                        queue: currentDataOutputQueue)
    videoOutput.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA]
    currentSession.addOutput(videoOutput)
    let depthOutput = AVCaptureDepthDataOutput()
    depthOutput.setDelegate(self, callbackQueue: currentDataOutputQueue)
    depthOutput.isFilteringEnabled = true
    currentSession.addOutput(depthOutput)
    currentPhotoOutput = AVCapturePhotoOutput()
    currentSession.addOutput(currentPhotoOutput)
    currentPhotoOutput.isDepthDataDeliveryEnabled = true
}

Posted by

HeoJin

Add a Comment

Object Capture With only manual capturing

Replies