Thanks for the suggestion. Since posting this I have indeed been able to get the beginnings of a hit test going with the
segmentationBuffer, but then when I try to use the
estimatedDepthData, I run into trouble extracting values.
Here's some of my code:
Code Block let segmentationCols = CVPixelBufferGetWidth(segmentationBuffer) |
let segmentationRows = CVPixelBufferGetHeight(segmentationBuffer) |
let colPosition = screenPosition.x / UIScreen.main.bounds.width * CGFloat(segmentationCols) |
let rowPosition = screenPosition.y / UIScreen.main.bounds.height * CGFloat(segmentationRows) |
|
CVPixelBufferLockBaseAddress(segmentationBuffer, .readOnly) |
guard let baseAddress = CVPixelBufferGetBaseAddress(segmentationBuffer) else { return } |
let bytesPerRow = CVPixelBufferGetBytesPerRow(segmentationBuffer) |
let buffer = baseAddress.assumingMemoryBound(to: UInt8.self) |
let index = Int(colPosition) + Int(rowPosition) * bytesPerRow |
|
let b = buffer[index] |
if let segment = ARFrame.SegmentationClass(rawValue: b), segment == .person, let depthBuffer = frame.estimatedDepthData { |
print("Person!") |
CVPixelBufferLockBaseAddress(depthBuffer, .readOnly) |
|
guard let depthAddress = CVPixelBufferGetBaseAddress(depthBuffer) else { return } |
let depthBytesPerRow = CVPixelBufferGetBytesPerRow(depthBuffer) |
let depthBoundBuffer = depthAddress.assumingMemoryBound(to: Float32.self) |
let depthIndex = Int(colPosition) * Int(rowPosition) |
let depth_b = depthBoundBuffer[depthIndex] |
print(depth_b) |
|
CVPixelBufferUnlockBaseAddress(depthBuffer, .readOnly) |
} |
|
CVPixelBufferUnlockBaseAddress( segmentationBuffer, .readOnly ) |
I strongly suspect that my problems are in line 19 and 20 of my code above, but I can't figure out the right values to find the point I want in the
estimatedDepthData