Hi everyone,
I'm curious about the capabilities of ARKit's object tracking feature. Specifically, I'd like to know:
Is there a size limit for the objects that can be tracked?
Can ARKit differentiate between two objects with the same shape but different models (e.g., different colors)?
Are objects with single colors and generic shapes (like squares or circles) effectively trackable?
Any insights or examples from your experiences would be greatly appreciated!
Thanks in advance.
Vision
RSS for tagApply computer vision algorithms to perform a variety of tasks on input images and video using Vision.
Posts under Vision tag
90 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
I'm seeking insight on why the new VisionOS Barcode Scanning API is categorized as an Enterprise API and restricted only for proprietary and in-house apps.
I understand Apple's focus on privacy and I can see how this restriction could make sense for other Enterprise APIs like main camera access and passthrough screen capture.
Why is barcode scanning restricted from open apps? What makes barcode scanning more of a risk to privacy versus the unrestricted APIs for object tracking, image tracking, or hand tracking?
Vision pro cannot capture recordings over 16kHz to 24kHz at a sampling rate of 48kHz, why? Or can you tell me how to configure it?Vision pro cannot capture recordings over 16kHz to 24kHz at a sampling rate of 48kHz, why? Or can you tell me how to configure it?
"On the latest iOS 18 beta 2, the OCR API,the Translate App and Live Text performs very poorly in recognizing Japanese."
I try to use the new VNCalculateImageAestheticsScoresRequest API.
Code is compiling and running but delivers the same result for every image
Xcode 16 Beta 2 Simulator
Did I missing anything ?
We have a lot of resources made using 3Dmaxs design software. How can the USDZ exported from 3dmax be used in the vision project. The preview in the folder on the Apple computer is correct. However, it is not possible to parse the texture in the xcode project. The difference in USDZ data compared to Blender is that the texture data exported by 3dmax has an additional node graph node. Is there any export plugin support?
Hi All,
I am trying to build a new iOS app by following https://developer.apple.com/videos/play/wwdc2024/10163/?time=67
When I trying to remove all legacy VN I am getting error, I would appreciate if someone can help me get up to speed with the new Vision API
I'm currently streaming synchronised video and depth data from my iPhone 13, using AVFoundation, video set to AVCaptureSession.Preset.vga640x480. When looking at the corresponding images (with depth values mapped to a grey colour map), (both map and image are of size 640x480) it appears the two feeds have different fields of view, with the depth feed zoomed in and angled upwards, and the colour feed more zoomed out. I've looked at the intrinsics from both the depth map, and my colour sample buffer, they are identical.
Does anyone know why this might be?
My setup code is below (shortened):
import AVFoundation
import CoreVideo
class VideoCaptureManager {
private enum SessionSetupResult {
case success
case notAuthorized
case configurationFailed
}
private enum ConfigurationError: Error {
case cannotAddInput
case cannotAddOutput
case defaultDeviceNotExist
}
private let videoDeviceDiscoverySession = AVCaptureDevice.DiscoverySession(deviceTypes: [.builtInTrueDepthCamera],
mediaType: .video,
position: .front)
private let session = AVCaptureSession()
public let videoOutput = AVCaptureVideoDataOutput()
public let depthDataOutput = AVCaptureDepthDataOutput()
private var outputSynchronizer: AVCaptureDataOutputSynchronizer?
private var videoDeviceInput: AVCaptureDeviceInput!
private let sessionQueue = DispatchQueue(label: "session.queue")
private let videoOutputQueue = DispatchQueue(label: "video.output.queue")
private var setupResult: SessionSetupResult = .success
init() {
sessionQueue.async {
self.requestCameraAuthorizationIfNeeded()
}
sessionQueue.async {
self.configureSession()
}
sessionQueue.async {
self.startSessionIfPossible()
}
}
private func requestCameraAuthorizationIfNeeded() {
switch AVCaptureDevice.authorizationStatus(for: .video) {
case .authorized:
break
case .notDetermined:
AVCaptureSession
sessionQueue.suspend()
AVCaptureDevice.requestAccess(for: .video, completionHandler: { granted in
if !granted {
self.setupResult = .notAuthorized
}
self.sessionQueue.resume()
})
default:
setupResult = .notAuthorized
}
}
private func configureSession() {
if setupResult != .success {
return
}
let defaultVideoDevice: AVCaptureDevice? = videoDeviceDiscoverySession.devices.first
guard let videoDevice = defaultVideoDevice else {
print("Could not find any video device")
setupResult = .configurationFailed
return
}
do {
videoDeviceInput = try AVCaptureDeviceInput(device: videoDevice)
} catch {
setupResult = .configurationFailed
return
}
session.beginConfiguration()
session.sessionPreset = AVCaptureSession.Preset.vga640x480
guard session.canAddInput(videoDeviceInput) else {
print("Could not add video device input to the session")
setupResult = .configurationFailed
session.commitConfiguration()
return
}
session.addInput(videoDeviceInput)
if session.canAddOutput(videoOutput) {
session.addOutput(videoOutput)
if let connection = videoOutput.connection(with: .video) {
connection.isCameraIntrinsicMatrixDeliveryEnabled = true
}
else {
print("Cannot setup camera intrinsics")
}
videoOutput.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String: Int(kCVPixelFormatType_32BGRA)]
} else {
print("Could not add video data output to the session")
setupResult = .configurationFailed
session.commitConfiguration()
return
}
if session.canAddOutput(depthDataOutput) {
session.addOutput(depthDataOutput)
depthDataOutput.isFilteringEnabled = false
if let connection = depthDataOutput.connection(with: .depthData) {
connection.isEnabled = true
} else {
print("No AVCaptureConnection")
}
} else {
print("Could not add depth data output to the session")
setupResult = .configurationFailed
session.commitConfiguration()
return
}
let depthFormats = videoDevice.activeFormat.supportedDepthDataFormats
let filtered = depthFormats.filter({
CMFormatDescriptionGetMediaSubType($0.formatDescription) == kCVPixelFormatType_DepthFloat16
})
let selectedFormat = filtered.max(by: {
first, second in CMVideoFormatDescriptionGetDimensions(first.formatDescription).width < CMVideoFormatDescriptionGetDimensions(second.formatDescription).width
})
do {
try videoDevice.lockForConfiguration()
videoDevice.activeDepthDataFormat = selectedFormat
videoDevice.unlockForConfiguration()
} catch {
print("Could not lock device for configuration: \(error)")
setupResult = .configurationFailed
session.commitConfiguration()
return
}
session.commitConfiguration()
}
private func addVideoDeviceInputToSession() throws {
do {
var defaultVideoDevice: AVCaptureDevice?
defaultVideoDevice = AVCaptureDevice.default(
.builtInTrueDepthCamera,
for: .depthData,
position: .front
)
guard let videoDevice = defaultVideoDevice else {
print("Default video device is unavailable.")
setupResult = .configurationFailed
session.commitConfiguration()
throw ConfigurationError.defaultDeviceNotExist
}
let videoDeviceInput = try AVCaptureDeviceInput(device: videoDevice)
if session.canAddInput(videoDeviceInput) {
session.addInput(videoDeviceInput)
} else {
setupResult = .configurationFailed
session.commitConfiguration()
throw ConfigurationError.cannotAddInput
}
}
Hey, is there a way to create a good ground shadow shader? I'm using a ground with an unlit material and I can't get the ground shadow to work properly. If I use a PBR texture it works better, but i can barely see it and I want to control the intensity more.
I can‘t Figure Out How to Get My Earth Entity to Rotate on its Axis. This is a follow up post from a previous Apple Developer forum post.
How would I have the earth (parent) entity rotate CCW underneath the orbiting starship child?
I tried adding the following code block to the RealityView but it is not working:
if let rotatingEarth = starshipEntity.findEntity(named: "Earth") {
rotatingEarth.transform.rotation = simd_quatf.init(angle: 360, axis: SIMD3(x: 0, y: 1, z: 0))
if let animation = try? AnimationResource.generate(with: rotatingEarth as! AnimationDefinition) {
rotatingEarth.playAnimation(animation)
}
}
Any advice on getting the earth to rotate?
I tried reviewing the Hello World WWDC23 project code, but I was unable to understand the complexity and how that sample project got the earth to rotate.
i want to do this for visionOS 1.2. I realize there are some new animation and possible other capabilities coming up in vision 2.0 but I want to try to address this issue in the current released visionOS version.