First of all this vision api is amazing. the OCR is very accurate. I've been looking to multiprocess using the vision API. I have about 2 million PDFs I want to OCR, and I want to run multiple threads/run parallel processing to OCR each. I tried pyobjc but it does not work so well. Any suggestions on tackling this problem?

Hi, Can VisionPro see outside and let us take a picture from code/iOS/Programming? Eg: like we used to have camera permission on iPad/iPhone to have access to device camera. Thank you

Trying to use VNGeneratePersonSegmentationRequest.. it seems to work but the output mask isn't at the same resolution as the source image.. so comping the result with the source produces a bad result. Not the full code, but hopefully enough to see what I'm doing. var imageRect = CGRect(x: 0, y: 0, width: image.size.width, height: image.size.height) let imageRef = image.cgImage(forProposedRect: &imageRect, context: nil, hints: nil)! let request = VNGeneratePersonSegmentationRequest() let handler = VNImageRequestHandler(cgImage: imageRef) do { try handler.perform([request]) guard let result = request.results?.first else { return } //Is this the right way to do this? let output = result.pixelBuffer //This ciImage alpha mask is a different resolution than the source image //So I don't know how to combine this with the source to cut out the foreground as they don't line up.. the res it's even the right aspect ratio. let ciImage = CIImage(cvPixelBuffer: output) ..... }

Why iOS 17's lift subject returns jagged edge and low resolution result image ? The output quality is totally different from iOS16. Does this just occur to beta version? or will it be the same in real iOS17 release version ?

My app uses the Vision framework to find images that are visually similar. Before WWDC23, this code worked predictably to request image observations and compare images: private func observe(cgImage: CGImage?) -> VNFeaturePrintObservation? { var returnVal: VNFeaturePrintObservation? = nil guard let cgImage = cgImage else {return returnVal} let imageRequestHandler = VNImageRequestHandler(cgImage: cgImage, options: [:]) let request = VNGenerateImageFeaturePrintRequest() request.usesCPUOnly = true do { try imageRequestHandler.perform([request]) returnVal = request.results?.first as? VNFeaturePrintObservation } catch { } return returnVal } func similarity(to compareAsset: Asset) -> Float { var dist = Float.infinity if let obs = self.observation, let compareObs = compareAsset.observation { try? obs.computeDistance(&dist, to: compareObs) } return dist } In the new frameworks, there is a new VNGenerateImageFeaturePrintRequestRevision value, and observations made with different request revisions can't be compared. If you try, you get an error: Error Domain=com.apple.vis Code=12 "The revision of the observations do not match" UserInfo={NSLocalizedDescription=The revision of the observations do not match}. The docs state that by explicitly setting the new VNGenerateImageFeaturePrintRequestRevision property on my requests, I can force the request to use a particular version. But I've updated the above code to do this, but explicitly setting the revision of my request doesn't work, and my app still gets tons of errors about mismatched request revisions. Here's the updated code: private func observe(cgImage: CGImage?) -> VNFeaturePrintObservation? { var returnVal: VNFeaturePrintObservation? = nil guard let cgImage = cgImage else {return returnVal} let imageRequestHandler = VNImageRequestHandler(cgImage: cgImage, options: [:]) let request = VNGenerateImageFeaturePrintRequest() if #available(iOS 17.0, *) { request.revision = VNGenerateImageFeaturePrintRequestRevision2 } else { request.revision = VNGenerateImageFeaturePrintRequestRevision1 } request.usesCPUOnly = true do { try imageRequestHandler.perform([request]) returnVal = request.results?.first as? VNFeaturePrintObservation } catch { print("\(type(of: self)) :: \(#function) :: error in observation request \(error)") } return returnVal } func similarity(to compareAsset: Asset) -> Float { var dist = Float.infinity if let obs = self.observation, let compareObs = compareAsset.observation { do { try obs.computeDistance(&dist, to: compareObs) } catch { print("\(type(of: self)) :: \(#function) :: error in simvalue \(error)") if (error as NSError).code == 12 { let revision = obs.requestRevision let compareRevision = compareObs.requestRevision print("\(type(of: self)) :: \(#function) :: simValue req mismatch \(revision) \(compareRevision)") } } } return dist } This breaks my app, and I can't figure out how to reliably force requests to the revision number I need. What am I doing wrong here? Will this behavior sort itself out as the SDK evolves? Thanks y'all

Lift subjects from images in your app, give a sample code and app

I got a crash when right click on an image in a web page loaded by WKWebView. The log and stack trace show that it might related to VisionKit. I think it may related to copy text on image feature. I don't know how to fix it. So I just want to disable VisionKit on WKWebView to check whether it fix the issue or not. Do you know how to do it? Stack trace: The log before crash: [com.apple.VisionKit.processing] Error processing request from MAD on result: Error Domain=NSOSStatusErrorDomain Code=-50 "paramErr: error in user parameter list" UserInfo={NSLocalizedDescription=Error allocating CVPixelBuffer} request: <VKCImageAnalyzerRequest: 0x6000025562b0> requestID: 22 madRequestID: (Not Set) cancelled: NO

Most of the APIs for this show iOS 16 support. Will a full subject lifting implementation require the device to be running iOS 17 or will it run on 16 too?

I'm using VisionKit framework for text recognition and detecting rectangles in the image. For that, I'm using VNRecognizeText & VNDetectRectangles features of the VisionKit. In macOS and iOS results, I found slight difference in the boundingBox coordinates of the text and the rectangles detected for the same image. Is this expected? Can we do anything to make the results identical? Also, on macOS, when I'm using same features of VisionKit from python (using pyobjc-framework-Vision package), there also i'm getting slightly different results.

When trying the code from this Session: https://developer.apple.com/videos/play/wwdc2023/10176/ Specifically: result.generateScaledMaskForImage(forInstances: result.allInstances, from: handler) It throws the error: -[VNInstanceMaskObservation generateScaledMaskForImageForInstances:fromRequestHandler:error:]: unrecognized selector sent to instance Any ideas?

Hi! Wondering if there's a way to add subject lifting to images in my SwiftUI app without relying on UIViewControllerRepresentable and Coordinators to adopt the ImageAnalysisInteraction protocol. Thank you!

Hi One point I would like to ask. How many specs do you need and which Mac would you recommend for Apple Vision Pro development? We use XCode, RealityKit, ARKit, Reality Composer Pro, Unity Editor which supports visionOS development, and MaterialX. If possible, what notebook and desktop models do you recommend? Best regards Sadao Tokuyama https://1planet.co.jp/

Hello guys, I am trying to run this sample project on my Ipad, I got a black screen the camera does not initialize. I tried updating the info.plist and asking for camera permission. I updated all the devices, did someone tried this demo? https://developer.apple.com/documentation/vision/detecting_animal_body_poses_with_vision

Hello there! Really excited about this year's additions to ImageAnalysisInteraction, especially the selectedText. Thank you! My question is if there's a way to have multiple interactions for a given image analysis. For example, instead of being able to make just one text selection, I would like to support multiple. In theory, I know multiple UIInteraction-conforming objects can be added to any view but I'm unsure if this is possible with ImageAnalysisInteraction. I'm willing to create a custom UIInteraction, if you see a potential way forward there (would love to hear any ideas!). Side question/feature request: UITextSelectionDisplayInteraction, which I assume ImageAnalysisInteraction might be using internally, allows handleViews customizations and others. I would be super useful if that was exposed through ImageAnalysisInteraction as well! Looking forward to hearing your ideas. TIA!

I have an AR app and soon will be making a VisionKit version. Can we use swift library to create a world space 3D UI with features like Zstack, Buttons, etc? Currently, I am using UI kit to do that.

Dear All, I would like to kindly ask you about possiblity of customisation of VisionKit framework function:- Is there any way how to change default „scan mode" eg. from Colour to Photo? Thank you very much.

Hi All, I am using vision framework for barcode scanning and now i wanted to support UPU s18 4-state barcodes too. Can u pls guide us how i can achive this functionality. TIA A

I am having issues with VNRecognizeTextRequest where the binary images for extracting text from images fails to load. Here is some logs that I have gotten: WARNING: File mapping at offset 80 of file /System/Library/AssetsV2/com_apple_MobileAsset_LinguisticData/d8e5c8c195c5d8c6372e99004e20e5562158a0d4.asset/AssetData/en.lm/fst.dat could not be honored, reading instead. WARNING: File mapping at offset 10400 of file /System/Library/AssetsV2/com_apple_MobileAsset_LinguisticData/d8e5c8c195c5d8c6372e99004e20e5562158a0d4.asset/AssetData/en.lm/fst.dat could not be honored, reading instead.

Hello, Has anyone one of you experienced a change in the UI of the camera feaure in your app since iOS 16 (and in particular iOS 16.3)? What can I do to go back to how it was for all our users (and I supposed this involves pushing a new version of the app?)? Since upgrading to iOS 16.3 (I believe it's when I noticed the change in UI, maybe it was a previous version, though I don't think so), the UI of the camera feature in my app (feature description: the user takes a photo of an object, and if we identify the object, the user is presented with a card telling information about the object) is modified, but without our intervention (as we haven't submitted a new version of our app since December 2021). **The UI of our camera feature before upgrading to iOS 16.3 ** The purpose of the square was to clearly indicate to the user how to capture the object. **The UI of our camera feature since upgrading to iOS 16.3 ** The square is now greyed out and seems to be showing some sort of closed shutter (?). In our user experience, after the user presses the camera button, the image is processed, and if we can't recognize the object, then the user is presented again with the square to take another photo (and if the object is recognized, then the user is presented with a card with a story about the object). Since the upgrade to iOS 16.3, on first launch of the camera, the UI is like shown on the image with the greyed out square, but if the object is not recognized and the Camera reloads (not sure I'm using the proper term as I'm not a developer myself), then UI is like shown on the first image (i.e. like we designed it, unaffected by the modification introduced by iOS 16.3). Has this also happened to anyone of you? What have you done to revert to the way it was? Any recommendation would be greatly appreciated as I don't have an in-house developer and I'm very frustrated with the change in look and feel of my app since this update. Many many thanks

I am buidling out a SwiftUI application which uses DataScannerViewController passed through a UIViewControllerRepresentable. WIth that said, I'd like to create a method to allow the user to change the quality level as they see fit however, every time I attempt to do so, the compiler tells me the below message which obviously means it's not mutable. Cannot assign to property: qualityLevel is a let constant Is there a known way to change this on the fly, or once the ViewController sets it - it's set and cannot be changed?