VisionKit

RSS for tag

Scan documents with the camera on iPhone and iPad devices using VisionKit.

Posts under VisionKit tag

169 Posts

Post

Replies

Boosts

Views

Activity

Getting selected text from ImageAnalysisInteraction on UIImageView
I work on an iOS app that displays images that often contain text, and I'm adding support for ImageAnalysisInteraction as described in this WWDC 2022 session. I have gotten as far as making the interaction show up and being able to select text and get the system selection menu, and even add my own action to the menu via the buildMenuWithBuilder API. But what I really want to do with my custom action is get the selected text and do a custom lookup-like thing to check the text against other content in my app. So how do I get the selected text from an ImageAnalysisInteraction on a UIImageView? The docs show methods to check if there is selected text, but I want to know what the text is.
5
1
2.6k
Aug ’23
Type 'VNRecognizedPointKey' has no member 'thumbTip'
With the release of Xcode 13, a large section of my vision framework processing code became errors and cannot compile. All of these have became deprecated. This is my original code:  do {       // Perform VNDetectHumanHandPoseRequest       try handler.perform([handPoseRequest])       // Continue only when a hand was detected in the frame.       // Since we set the maximumHandCount property of the request to 1, there will be at most one observation.       guard let observation = handPoseRequest.results?.first else {         self.state = "no hand"         return       }       // Get points for thumb and index finger.       let thumbPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyThumb)       let indexFingerPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyIndexFinger)       let middleFingerPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyMiddleFinger)       let ringFingerPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyRingFinger)       let littleFingerPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyLittleFinger)       let wristPoints = try observation.recognizedPoints(forGroupKey: .all)               // Look for tip points.       guard let thumbTipPoint = thumbPoints[.handLandmarkKeyThumbTIP],          let thumbIpPoint = thumbPoints[.handLandmarkKeyThumbIP],          let thumbMpPoint = thumbPoints[.handLandmarkKeyThumbMP],          let thumbCMCPoint = thumbPoints[.handLandmarkKeyThumbCMC] else {         self.state = "no tip"         return       }               guard let indexTipPoint = indexFingerPoints[.handLandmarkKeyIndexTIP],          let indexDipPoint = indexFingerPoints[.handLandmarkKeyIndexDIP],          let indexPipPoint = indexFingerPoints[.handLandmarkKeyIndexPIP],          let indexMcpPoint = indexFingerPoints[.handLandmarkKeyIndexMCP] else {         self.state = "no index"         return       }               guard let middleTipPoint = middleFingerPoints[.handLandmarkKeyMiddleTIP],          let middleDipPoint = middleFingerPoints[.handLandmarkKeyMiddleDIP],          let middlePipPoint = middleFingerPoints[.handLandmarkKeyMiddlePIP],          let middleMcpPoint = middleFingerPoints[.handLandmarkKeyMiddleMCP] else {         self.state = "no middle"         return       }               guard let ringTipPoint = ringFingerPoints[.handLandmarkKeyRingTIP],          let ringDipPoint = ringFingerPoints[.handLandmarkKeyRingDIP],          let ringPipPoint = ringFingerPoints[.handLandmarkKeyRingPIP],          let ringMcpPoint = ringFingerPoints[.handLandmarkKeyRingMCP] else {         self.state = "no ring"         return       }               guard let littleTipPoint = littleFingerPoints[.handLandmarkKeyLittleTIP],          let littleDipPoint = littleFingerPoints[.handLandmarkKeyLittleDIP],          let littlePipPoint = littleFingerPoints[.handLandmarkKeyLittlePIP],          let littleMcpPoint = littleFingerPoints[.handLandmarkKeyLittleMCP] else {         self.state = "no little"         return       }               guard let wristPoint = wristPoints[.handLandmarkKeyWrist] else {         self.state = "no wrist"         return       } ... } Now every line from thumbPoints onwards results in error, I have fixed the first part (not sure if it is correct or not as it cannot compile) to :         let thumbPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.thumb.rawValue)        let indexFingerPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.indexFinger.rawValue)        let middleFingerPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.middleFinger.rawValue)        let ringFingerPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.ringFinger.rawValue)        let littleFingerPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.littleFinger.rawValue)        let wristPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.littleFinger.rawValue) I tried many different things but just could not get the retrieving individual points to work. Can anyone help on fixing this?
2
0
2.2k
Aug ’23
Real-time image processing on passthrough imagery?
Is there a framework that allows for classic image processing operations in real-time from incoming imagery from the front-facing cameras before they are displayed on the OLED screens? Things like spatial filtering, histogram equalization, and image warping. I saw the documentation for the Vision framework, but it seems to address high-level tasks, like object and recognition. Thank you!
1
0
1.2k
Aug ’23
How to gain full control over Apple Vision pro's display and render 2D graph plot on it
How can I achieve full control over Vision Pro's display and effectively render a 2D graph plot on it? I would appreciate guidance on the necessary steps or code snippets. P.s. As per Apple documentation For a more immersive experience, an app can open a dedicated Full Space where only that app’s content will appear. This still does not fulfill the 'flat bounded 2D' requirement as the Spaces provide an unbounded 3D immersive view.
2
0
1.1k
Jul ’23
VNGeneratePersonSegmentationRequest produces mask at different resolution then source image
Trying to use VNGeneratePersonSegmentationRequest.. it seems to work but the output mask isn't at the same resolution as the source image.. so comping the result with the source produces a bad result. Not the full code, but hopefully enough to see what I'm doing. var imageRect = CGRect(x: 0, y: 0, width: image.size.width, height: image.size.height) let imageRef = image.cgImage(forProposedRect: &imageRect, context: nil, hints: nil)! let request = VNGeneratePersonSegmentationRequest() let handler = VNImageRequestHandler(cgImage: imageRef) do { try handler.perform([request]) guard let result = request.results?.first else { return } //Is this the right way to do this? let output = result.pixelBuffer //This ciImage alpha mask is a different resolution than the source image //So I don't know how to combine this with the source to cut out the foreground as they don't line up.. the res it's even the right aspect ratio. let ciImage = CIImage(cvPixelBuffer: output) ..... }
2
0
1.2k
Jul ’23
Can't set VNGenerateImageFeaturePrintRequestRevision consistently, breaking computeDistance between two images
My app uses the Vision framework to find images that are visually similar. Before WWDC23, this code worked predictably to request image observations and compare images: private func observe(cgImage: CGImage?) -> VNFeaturePrintObservation? { var returnVal: VNFeaturePrintObservation? = nil guard let cgImage = cgImage else {return returnVal} let imageRequestHandler = VNImageRequestHandler(cgImage: cgImage, options: [:]) let request = VNGenerateImageFeaturePrintRequest() request.usesCPUOnly = true do { try imageRequestHandler.perform([request]) returnVal = request.results?.first as? VNFeaturePrintObservation } catch { } return returnVal } func similarity(to compareAsset: Asset) -> Float { var dist = Float.infinity if let obs = self.observation, let compareObs = compareAsset.observation { try? obs.computeDistance(&dist, to: compareObs) } return dist } In the new frameworks, there is a new VNGenerateImageFeaturePrintRequestRevision value, and observations made with different request revisions can't be compared. If you try, you get an error: Error Domain=com.apple.vis Code=12 "The revision of the observations do not match" UserInfo={NSLocalizedDescription=The revision of the observations do not match}. The docs state that by explicitly setting the new VNGenerateImageFeaturePrintRequestRevision property on my requests, I can force the request to use a particular version. But I've updated the above code to do this, but explicitly setting the revision of my request doesn't work, and my app still gets tons of errors about mismatched request revisions. Here's the updated code: private func observe(cgImage: CGImage?) -> VNFeaturePrintObservation? { var returnVal: VNFeaturePrintObservation? = nil guard let cgImage = cgImage else {return returnVal} let imageRequestHandler = VNImageRequestHandler(cgImage: cgImage, options: [:]) let request = VNGenerateImageFeaturePrintRequest() if #available(iOS 17.0, *) { request.revision = VNGenerateImageFeaturePrintRequestRevision2 } else { request.revision = VNGenerateImageFeaturePrintRequestRevision1 } request.usesCPUOnly = true do { try imageRequestHandler.perform([request]) returnVal = request.results?.first as? VNFeaturePrintObservation } catch { print("\(type(of: self)) :: \(#function) :: error in observation request \(error)") } return returnVal } func similarity(to compareAsset: Asset) -> Float { var dist = Float.infinity if let obs = self.observation, let compareObs = compareAsset.observation { do { try obs.computeDistance(&dist, to: compareObs) } catch { print("\(type(of: self)) :: \(#function) :: error in simvalue \(error)") if (error as NSError).code == 12 { let revision = obs.requestRevision let compareRevision = compareObs.requestRevision print("\(type(of: self)) :: \(#function) :: simValue req mismatch \(revision) \(compareRevision)") } } } return dist } This breaks my app, and I can't figure out how to reliably force requests to the revision number I need. What am I doing wrong here? Will this behavior sort itself out as the SDK evolves? Thanks y'all
2
1
1.4k
Jun ’23
How to disable VisionKit in WKWebView
I got a crash when right click on an image in a web page loaded by WKWebView. The log and stack trace show that it might related to VisionKit. I think it may related to copy text on image feature. I don't know how to fix it. So I just want to disable VisionKit on WKWebView to check whether it fix the issue or not. Do you know how to do it? Stack trace: The log before crash: [com.apple.VisionKit.processing] Error processing request from MAD on result: Error Domain=NSOSStatusErrorDomain Code=-50 "paramErr: error in user parameter list" UserInfo={NSLocalizedDescription=Error allocating CVPixelBuffer} request: <VKCImageAnalyzerRequest: 0x6000025562b0> requestID: 22 madRequestID: (Not Set) cancelled: NO
1
2
1.5k
Jun ’23
Does VisionKit produce slightly different results when calling from macOS app and iOS app for the same image
I'm using VisionKit framework for text recognition and detecting rectangles in the image. For that, I'm using VNRecognizeText & VNDetectRectangles features of the VisionKit. In macOS and iOS results, I found slight difference in the boundingBox coordinates of the text and the rectangles detected for the same image. Is this expected? Can we do anything to make the results identical? Also, on macOS, when I'm using same features of VisionKit from python (using pyobjc-framework-Vision package), there also i'm getting slightly different results.
1
0
703
Jun ’23
unrecognized selector sent to instance
When trying the code from this Session: https://developer.apple.com/videos/play/wwdc2023/10176/ Specifically: result.generateScaledMaskForImage(forInstances: result.allInstances, from: handler) It throws the error: -[VNInstanceMaskObservation generateScaledMaskForImageForInstances:fromRequestHandler:error:]: unrecognized selector sent to instance Any ideas?
0
0
1.1k
Jun ’23
Recommended machine specifications for Apple Vision Pro application development
Hi One point I would like to ask. How many specs do you need and which Mac would you recommend for Apple Vision Pro development? We use XCode, RealityKit, ARKit, Reality Composer Pro, Unity Editor which supports visionOS development, and MaterialX. If possible, what notebook and desktop models do you recommend? Best regards Sadao Tokuyama https://1planet.co.jp/
2
0
2.4k
Jun ’23
Multiple ImageAnalysisInteractions
Hello there! Really excited about this year's additions to ImageAnalysisInteraction, especially the selectedText. Thank you! My question is if there's a way to have multiple interactions for a given image analysis. For example, instead of being able to make just one text selection, I would like to support multiple. In theory, I know multiple UIInteraction-conforming objects can be added to any view but I'm unsure if this is possible with ImageAnalysisInteraction. I'm willing to create a custom UIInteraction, if you see a potential way forward there (would love to hear any ideas!). Side question/feature request: UITextSelectionDisplayInteraction, which I assume ImageAnalysisInteraction might be using internally, allows handleViews customizations and others. I would be super useful if that was exposed through ImageAnalysisInteraction as well! Looking forward to hearing your ideas. TIA!
0
0
1.3k
Jun ’23
Getting selected text from ImageAnalysisInteraction on UIImageView
I work on an iOS app that displays images that often contain text, and I'm adding support for ImageAnalysisInteraction as described in this WWDC 2022 session. I have gotten as far as making the interaction show up and being able to select text and get the system selection menu, and even add my own action to the menu via the buildMenuWithBuilder API. But what I really want to do with my custom action is get the selected text and do a custom lookup-like thing to check the text against other content in my app. So how do I get the selected text from an ImageAnalysisInteraction on a UIImageView? The docs show methods to check if there is selected text, but I want to know what the text is.
Replies
5
Boosts
1
Views
2.6k
Activity
Aug ’23
Type 'VNRecognizedPointKey' has no member 'thumbTip'
With the release of Xcode 13, a large section of my vision framework processing code became errors and cannot compile. All of these have became deprecated. This is my original code:  do {       // Perform VNDetectHumanHandPoseRequest       try handler.perform([handPoseRequest])       // Continue only when a hand was detected in the frame.       // Since we set the maximumHandCount property of the request to 1, there will be at most one observation.       guard let observation = handPoseRequest.results?.first else {         self.state = "no hand"         return       }       // Get points for thumb and index finger.       let thumbPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyThumb)       let indexFingerPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyIndexFinger)       let middleFingerPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyMiddleFinger)       let ringFingerPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyRingFinger)       let littleFingerPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyLittleFinger)       let wristPoints = try observation.recognizedPoints(forGroupKey: .all)               // Look for tip points.       guard let thumbTipPoint = thumbPoints[.handLandmarkKeyThumbTIP],          let thumbIpPoint = thumbPoints[.handLandmarkKeyThumbIP],          let thumbMpPoint = thumbPoints[.handLandmarkKeyThumbMP],          let thumbCMCPoint = thumbPoints[.handLandmarkKeyThumbCMC] else {         self.state = "no tip"         return       }               guard let indexTipPoint = indexFingerPoints[.handLandmarkKeyIndexTIP],          let indexDipPoint = indexFingerPoints[.handLandmarkKeyIndexDIP],          let indexPipPoint = indexFingerPoints[.handLandmarkKeyIndexPIP],          let indexMcpPoint = indexFingerPoints[.handLandmarkKeyIndexMCP] else {         self.state = "no index"         return       }               guard let middleTipPoint = middleFingerPoints[.handLandmarkKeyMiddleTIP],          let middleDipPoint = middleFingerPoints[.handLandmarkKeyMiddleDIP],          let middlePipPoint = middleFingerPoints[.handLandmarkKeyMiddlePIP],          let middleMcpPoint = middleFingerPoints[.handLandmarkKeyMiddleMCP] else {         self.state = "no middle"         return       }               guard let ringTipPoint = ringFingerPoints[.handLandmarkKeyRingTIP],          let ringDipPoint = ringFingerPoints[.handLandmarkKeyRingDIP],          let ringPipPoint = ringFingerPoints[.handLandmarkKeyRingPIP],          let ringMcpPoint = ringFingerPoints[.handLandmarkKeyRingMCP] else {         self.state = "no ring"         return       }               guard let littleTipPoint = littleFingerPoints[.handLandmarkKeyLittleTIP],          let littleDipPoint = littleFingerPoints[.handLandmarkKeyLittleDIP],          let littlePipPoint = littleFingerPoints[.handLandmarkKeyLittlePIP],          let littleMcpPoint = littleFingerPoints[.handLandmarkKeyLittleMCP] else {         self.state = "no little"         return       }               guard let wristPoint = wristPoints[.handLandmarkKeyWrist] else {         self.state = "no wrist"         return       } ... } Now every line from thumbPoints onwards results in error, I have fixed the first part (not sure if it is correct or not as it cannot compile) to :         let thumbPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.thumb.rawValue)        let indexFingerPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.indexFinger.rawValue)        let middleFingerPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.middleFinger.rawValue)        let ringFingerPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.ringFinger.rawValue)        let littleFingerPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.littleFinger.rawValue)        let wristPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.littleFinger.rawValue) I tried many different things but just could not get the retrieving individual points to work. Can anyone help on fixing this?
Replies
2
Boosts
0
Views
2.2k
Activity
Aug ’23
Real-time image processing on passthrough imagery?
Is there a framework that allows for classic image processing operations in real-time from incoming imagery from the front-facing cameras before they are displayed on the OLED screens? Things like spatial filtering, histogram equalization, and image warping. I saw the documentation for the Vision framework, but it seems to address high-level tasks, like object and recognition. Thank you!
Replies
1
Boosts
0
Views
1.2k
Activity
Aug ’23
How to gain full control over Apple Vision pro's display and render 2D graph plot on it
How can I achieve full control over Vision Pro's display and effectively render a 2D graph plot on it? I would appreciate guidance on the necessary steps or code snippets. P.s. As per Apple documentation For a more immersive experience, an app can open a dedicated Full Space where only that app’s content will appear. This still does not fulfill the 'flat bounded 2D' requirement as the Spaces provide an unbounded 3D immersive view.
Replies
2
Boosts
0
Views
1.1k
Activity
Jul ’23
Parallelizing/MultiProcessing Vision API
First of all this vision api is amazing. the OCR is very accurate. I've been looking to multiprocess using the vision API. I have about 2 million PDFs I want to OCR, and I want to run multiple threads/run parallel processing to OCR each. I tried pyobjc but it does not work so well. Any suggestions on tackling this problem?
Replies
0
Boosts
0
Views
740
Activity
Jul ’23
Vision Pro - Can we see outside through camera, and take picture
Hi, Can VisionPro see outside and let us take a picture from code/iOS/Programming? Eg: like we used to have camera permission on iPad/iPhone to have access to device camera. Thank you
Replies
1
Boosts
0
Views
1.4k
Activity
Jul ’23
VNGeneratePersonSegmentationRequest produces mask at different resolution then source image
Trying to use VNGeneratePersonSegmentationRequest.. it seems to work but the output mask isn't at the same resolution as the source image.. so comping the result with the source produces a bad result. Not the full code, but hopefully enough to see what I'm doing. var imageRect = CGRect(x: 0, y: 0, width: image.size.width, height: image.size.height) let imageRef = image.cgImage(forProposedRect: &imageRect, context: nil, hints: nil)! let request = VNGeneratePersonSegmentationRequest() let handler = VNImageRequestHandler(cgImage: imageRef) do { try handler.perform([request]) guard let result = request.results?.first else { return } //Is this the right way to do this? let output = result.pixelBuffer //This ciImage alpha mask is a different resolution than the source image //So I don't know how to combine this with the source to cut out the foreground as they don't line up.. the res it's even the right aspect ratio. let ciImage = CIImage(cvPixelBuffer: output) ..... }
Replies
2
Boosts
0
Views
1.2k
Activity
Jul ’23
ios17 lift subjects jagged edge and low resolution.
Why iOS 17's lift subject returns jagged edge and low resolution result image ? The output quality is totally different from iOS16. Does this just occur to beta version? or will it be the same in real iOS17 release version ?
Replies
1
Boosts
0
Views
981
Activity
Jul ’23
Can't set VNGenerateImageFeaturePrintRequestRevision consistently, breaking computeDistance between two images
My app uses the Vision framework to find images that are visually similar. Before WWDC23, this code worked predictably to request image observations and compare images: private func observe(cgImage: CGImage?) -> VNFeaturePrintObservation? { var returnVal: VNFeaturePrintObservation? = nil guard let cgImage = cgImage else {return returnVal} let imageRequestHandler = VNImageRequestHandler(cgImage: cgImage, options: [:]) let request = VNGenerateImageFeaturePrintRequest() request.usesCPUOnly = true do { try imageRequestHandler.perform([request]) returnVal = request.results?.first as? VNFeaturePrintObservation } catch { } return returnVal } func similarity(to compareAsset: Asset) -> Float { var dist = Float.infinity if let obs = self.observation, let compareObs = compareAsset.observation { try? obs.computeDistance(&dist, to: compareObs) } return dist } In the new frameworks, there is a new VNGenerateImageFeaturePrintRequestRevision value, and observations made with different request revisions can't be compared. If you try, you get an error: Error Domain=com.apple.vis Code=12 "The revision of the observations do not match" UserInfo={NSLocalizedDescription=The revision of the observations do not match}. The docs state that by explicitly setting the new VNGenerateImageFeaturePrintRequestRevision property on my requests, I can force the request to use a particular version. But I've updated the above code to do this, but explicitly setting the revision of my request doesn't work, and my app still gets tons of errors about mismatched request revisions. Here's the updated code: private func observe(cgImage: CGImage?) -> VNFeaturePrintObservation? { var returnVal: VNFeaturePrintObservation? = nil guard let cgImage = cgImage else {return returnVal} let imageRequestHandler = VNImageRequestHandler(cgImage: cgImage, options: [:]) let request = VNGenerateImageFeaturePrintRequest() if #available(iOS 17.0, *) { request.revision = VNGenerateImageFeaturePrintRequestRevision2 } else { request.revision = VNGenerateImageFeaturePrintRequestRevision1 } request.usesCPUOnly = true do { try imageRequestHandler.perform([request]) returnVal = request.results?.first as? VNFeaturePrintObservation } catch { print("\(type(of: self)) :: \(#function) :: error in observation request \(error)") } return returnVal } func similarity(to compareAsset: Asset) -> Float { var dist = Float.infinity if let obs = self.observation, let compareObs = compareAsset.observation { do { try obs.computeDistance(&dist, to: compareObs) } catch { print("\(type(of: self)) :: \(#function) :: error in simvalue \(error)") if (error as NSError).code == 12 { let revision = obs.requestRevision let compareRevision = compareObs.requestRevision print("\(type(of: self)) :: \(#function) :: simValue req mismatch \(revision) \(compareRevision)") } } } return dist } This breaks my app, and I can't figure out how to reliably force requests to the revision number I need. What am I doing wrong here? Will this behavior sort itself out as the SDK evolves? Thanks y'all
Replies
2
Boosts
1
Views
1.4k
Activity
Jun ’23
Lift subjects from images in your app, give a sample code and app
Lift subjects from images in your app, give a sample code and app
Replies
1
Boosts
1
Views
841
Activity
Jun ’23
How to disable VisionKit in WKWebView
I got a crash when right click on an image in a web page loaded by WKWebView. The log and stack trace show that it might related to VisionKit. I think it may related to copy text on image feature. I don't know how to fix it. So I just want to disable VisionKit on WKWebView to check whether it fix the issue or not. Do you know how to do it? Stack trace: The log before crash: [com.apple.VisionKit.processing] Error processing request from MAD on result: Error Domain=NSOSStatusErrorDomain Code=-50 "paramErr: error in user parameter list" UserInfo={NSLocalizedDescription=Error allocating CVPixelBuffer} request: <VKCImageAnalyzerRequest: 0x6000025562b0> requestID: 22 madRequestID: (Not Set) cancelled: NO
Replies
1
Boosts
2
Views
1.5k
Activity
Jun ’23
Will subject lifting require iOS 17?
Most of the APIs for this show iOS 16 support. Will a full subject lifting implementation require the device to be running iOS 17 or will it run on 16 too?
Replies
1
Boosts
0
Views
1.1k
Activity
Jun ’23
Does VisionKit produce slightly different results when calling from macOS app and iOS app for the same image
I'm using VisionKit framework for text recognition and detecting rectangles in the image. For that, I'm using VNRecognizeText & VNDetectRectangles features of the VisionKit. In macOS and iOS results, I found slight difference in the boundingBox coordinates of the text and the rectangles detected for the same image. Is this expected? Can we do anything to make the results identical? Also, on macOS, when I'm using same features of VisionKit from python (using pyobjc-framework-Vision package), there also i'm getting slightly different results.
Replies
1
Boosts
0
Views
703
Activity
Jun ’23
unrecognized selector sent to instance
When trying the code from this Session: https://developer.apple.com/videos/play/wwdc2023/10176/ Specifically: result.generateScaledMaskForImage(forInstances: result.allInstances, from: handler) It throws the error: -[VNInstanceMaskObservation generateScaledMaskForImageForInstances:fromRequestHandler:error:]: unrecognized selector sent to instance Any ideas?
Replies
0
Boosts
0
Views
1.1k
Activity
Jun ’23
Subject Lifting for SwiftUI
Hi! Wondering if there's a way to add subject lifting to images in my SwiftUI app without relying on UIViewControllerRepresentable and Coordinators to adopt the ImageAnalysisInteraction protocol. Thank you!
Replies
0
Boosts
1
Views
1.2k
Activity
Jun ’23
Recommended machine specifications for Apple Vision Pro application development
Hi One point I would like to ask. How many specs do you need and which Mac would you recommend for Apple Vision Pro development? We use XCode, RealityKit, ARKit, Reality Composer Pro, Unity Editor which supports visionOS development, and MaterialX. If possible, what notebook and desktop models do you recommend? Best regards Sadao Tokuyama https://1planet.co.jp/
Replies
2
Boosts
0
Views
2.4k
Activity
Jun ’23
Animal body poses Vision
Hello guys, I am trying to run this sample project on my Ipad, I got a black screen the camera does not initialize. I tried updating the info.plist and asking for camera permission. I updated all the devices, did someone tried this demo? https://developer.apple.com/documentation/vision/detecting_animal_body_poses_with_vision
Replies
0
Boosts
0
Views
977
Activity
Jun ’23
Multiple ImageAnalysisInteractions
Hello there! Really excited about this year's additions to ImageAnalysisInteraction, especially the selectedText. Thank you! My question is if there's a way to have multiple interactions for a given image analysis. For example, instead of being able to make just one text selection, I would like to support multiple. In theory, I know multiple UIInteraction-conforming objects can be added to any view but I'm unsure if this is possible with ImageAnalysisInteraction. I'm willing to create a custom UIInteraction, if you see a potential way forward there (would love to hear any ideas!). Side question/feature request: UITextSelectionDisplayInteraction, which I assume ImageAnalysisInteraction might be using internally, allows handleViews customizations and others. I would be super useful if that was exposed through ImageAnalysisInteraction as well! Looking forward to hearing your ideas. TIA!
Replies
0
Boosts
0
Views
1.3k
Activity
Jun ’23
How to create world space UI in reality kit using Swift UI libraries?
I have an AR app and soon will be making a VisionKit version. Can we use swift library to create a world space 3D UI with features like Zstack, Buttons, etc? Currently, I am using UI kit to do that.
Replies
1
Boosts
0
Views
1.2k
Activity
Jun ’23
VisionKit customization
Dear All, I would like to kindly ask you about possiblity of customisation of VisionKit framework function:- Is there any way how to change default „scan mode" eg. from Colour to Photo? Thank you very much.
Replies
3
Boosts
0
Views
1.6k
Activity
May ’23