Dear All, I would like to kindly ask you about possiblity of customisation of VisionKit framework function:- Is there any way how to change default „scan mode" eg. from Colour to Photo? Thank you very much.

With the release of Xcode 13, a large section of my vision framework processing code became errors and cannot compile. All of these have became deprecated. This is my original code:  do {       // Perform VNDetectHumanHandPoseRequest       try handler.perform([handPoseRequest])       // Continue only when a hand was detected in the frame.       // Since we set the maximumHandCount property of the request to 1, there will be at most one observation.       guard let observation = handPoseRequest.results?.first else {         self.state = "no hand"         return       }       // Get points for thumb and index finger.       let thumbPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyThumb)       let indexFingerPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyIndexFinger)       let middleFingerPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyMiddleFinger)       let ringFingerPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyRingFinger)       let littleFingerPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyLittleFinger)       let wristPoints = try observation.recognizedPoints(forGroupKey: .all)               // Look for tip points.       guard let thumbTipPoint = thumbPoints[.handLandmarkKeyThumbTIP],          let thumbIpPoint = thumbPoints[.handLandmarkKeyThumbIP],          let thumbMpPoint = thumbPoints[.handLandmarkKeyThumbMP],          let thumbCMCPoint = thumbPoints[.handLandmarkKeyThumbCMC] else {         self.state = "no tip"         return       }               guard let indexTipPoint = indexFingerPoints[.handLandmarkKeyIndexTIP],          let indexDipPoint = indexFingerPoints[.handLandmarkKeyIndexDIP],          let indexPipPoint = indexFingerPoints[.handLandmarkKeyIndexPIP],          let indexMcpPoint = indexFingerPoints[.handLandmarkKeyIndexMCP] else {         self.state = "no index"         return       }               guard let middleTipPoint = middleFingerPoints[.handLandmarkKeyMiddleTIP],          let middleDipPoint = middleFingerPoints[.handLandmarkKeyMiddleDIP],          let middlePipPoint = middleFingerPoints[.handLandmarkKeyMiddlePIP],          let middleMcpPoint = middleFingerPoints[.handLandmarkKeyMiddleMCP] else {         self.state = "no middle"         return       }               guard let ringTipPoint = ringFingerPoints[.handLandmarkKeyRingTIP],          let ringDipPoint = ringFingerPoints[.handLandmarkKeyRingDIP],          let ringPipPoint = ringFingerPoints[.handLandmarkKeyRingPIP],          let ringMcpPoint = ringFingerPoints[.handLandmarkKeyRingMCP] else {         self.state = "no ring"         return       }               guard let littleTipPoint = littleFingerPoints[.handLandmarkKeyLittleTIP],          let littleDipPoint = littleFingerPoints[.handLandmarkKeyLittleDIP],          let littlePipPoint = littleFingerPoints[.handLandmarkKeyLittlePIP],          let littleMcpPoint = littleFingerPoints[.handLandmarkKeyLittleMCP] else {         self.state = "no little"         return       }               guard let wristPoint = wristPoints[.handLandmarkKeyWrist] else {         self.state = "no wrist"         return       } ... } Now every line from thumbPoints onwards results in error, I have fixed the first part (not sure if it is correct or not as it cannot compile) to :         let thumbPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.thumb.rawValue)        let indexFingerPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.indexFinger.rawValue)        let middleFingerPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.middleFinger.rawValue)        let ringFingerPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.ringFinger.rawValue)        let littleFingerPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.littleFinger.rawValue)        let wristPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.littleFinger.rawValue) I tried many different things but just could not get the retrieving individual points to work. Can anyone help on fixing this?

Hi, do you know if it's possible to handle the flashlight inside DataScannerViewController? I tried with AVCaptureDevice.DiscoverySession, but clearly when torchmode is on DataScannerViewController freeze... Thanks Luca

Hi team, We have an iOS app. Since July 15, 2022, our users met a kind of app crash due to an invalid memory fetch. The time is when Apple released iOS 16 beta officially. After Sep 12, crash count started to increase drastically. The time is Apple released iOS 16 officially. Crash backtrace can be seen as follows. Thread 14 Crashed: 0 libsystem_platform.dylib 0x00000001f8810930 _platform_memmove + 96 1 CoreGraphics 0x00000001adb64104 CGDataProviderCreateWithCopyOfData + 20 2 CoreGraphics 0x00000001adb4cdb4 CGBitmapContextCreateImage + 172 3 VisionKitCore 0x00000001ed813f10 -[VKCRemoveBackgroundResult _createCGImageFromBGRAPixelBuffer:cropRect:] + 348 4 VisionKitCore 0x00000001ed813cc0 -[VKCRemoveBackgroundResult createCGImage] + 156 5 VisionKitCore 0x00000001ed8ab6f8 __vk_cgImageRemoveBackgroundWithDownsizing_block_invoke + 64 6 VisionKitCore 0x00000001ed881474 __63-[VKCRemoveBackgroundRequestHandler performRequest:completion:]_block_invoke.5 + 436 7 MediaAnalysisServices 0x00000001eec58968 __92-[MADService performRequests:onPixelBuffer:withOrientation:andIdentifier:completionHandler:]_block_invoke.38 + 400 8 CoreFoundation 0x00000001abff0a14 __invoking___ + 148 9 CoreFoundation 0x00000001abf9cf2c -[NSInvocation invoke] + 428 10 Foundation 0x00000001a6464d38 __NSXPCCONNECTION_IS_CALLING_OUT_TO_REPLY_BLOCK__ + 16 11 Foundation 0x00000001a64362fc -[NSXPCConnection _decodeAndInvokeReplyBlockWithEvent:sequence:replyInfo:] + 520 12 Foundation 0x00000001a6a10f44 __88-[NSXPCConnection _sendInvocation:orArguments:count:methodSignature:selector:withProxy:]_block_invoke_5 + 188 13 libxpc.dylib 0x00000001f89053e4 _xpc_connection_reply_callout + 124 14 libxpc.dylib 0x00000001f88f8580 _xpc_connection_call_reply_async + 88 15 libdispatch.dylib 0x00000001b340205c _dispatch_client_callout3 + 20 16 libdispatch.dylib 0x00000001b341ff58 _dispatch_mach_msg_async_reply_invoke + 344 17 libdispatch.dylib 0x00000001b340956c _dispatch_lane_serial_drain + 376 18 libdispatch.dylib 0x00000001b340a214 _dispatch_lane_invoke + 436 19 libdispatch.dylib 0x00000001b3414e10 _dispatch_workloop_worker_thread + 652 20 libsystem_pthread.dylib 0x00000001f88a4df8 _pthread_wqthread + 288 21 libsystem_pthread.dylib 0x00000001f88a4b98 start_wqthread + 8 Last but not the least. The users who met this kind of app crash use iOS16+. We think this crash is related to iOS 16 SDK. We're appreciate that you can provide some clues how to fix this kind of crash.

Hi, When using VNFeaturePrintObservation and then computing the distance using two images, the values that it returns varies heavily. When two identical images (same image file) is inputted into function (below) that I have used to compare the images, the distance does not return 0 while it is expected to, since they are identical images. Also, what is the upper limit of computeDistance? I am trying to find the percentage similarity between the two images. (Of course, this cannot be done unless the issue above is resolved). Code that I have used is below func featureprintObservationForImage(image: UIImage) -> VNFeaturePrintObservation? {     let requestHandler = VNImageRequestHandler(cgImage: image.cgImage!, options: [:])     let request = VNGenerateImageFeaturePrintRequest()     request.usesCPUOnly = true // Simulator Testing     do {       try requestHandler.perform([request])       return request.results?.first as? VNFeaturePrintObservation     } catch {       print("Vision Error: \(error)")       return nil     }   }   func compare(origImg: UIImage, drawnImg: UIImage) -> Float? {     let oImgObservation = featureprintObservationForImage(image: origImg)     let dImgObservation = featureprintObservationForImage(image: drawnImg)     if let oImgObservation = oImgObservation {       if let dImgObservation = dImgObservation {         var distance: Float = -1         do {           try oImgObservation.computeDistance(&distance, to: dImgObservation)         } catch {           fatalError("Failed to Compute Distance")         }         if distance == -1 {           return nil         } else {           return distance         }       } else {         print("Drawn Image Observation found Nil")       }     } else {       print("Original Image Observation found Nil")     }     return nil   } Thanks for all the help!

I am having issues with VNRecognizeTextRequest where the binary images for extracting text from images fails to load. Here is some logs that I have gotten: WARNING: File mapping at offset 80 of file /System/Library/AssetsV2/com_apple_MobileAsset_LinguisticData/d8e5c8c195c5d8c6372e99004e20e5562158a0d4.asset/AssetData/en.lm/fst.dat could not be honored, reading instead. WARNING: File mapping at offset 10400 of file /System/Library/AssetsV2/com_apple_MobileAsset_LinguisticData/d8e5c8c195c5d8c6372e99004e20e5562158a0d4.asset/AssetData/en.lm/fst.dat could not be honored, reading instead.

Hi All, I am using vision framework for barcode scanning and now i wanted to support UPU s18 4-state barcodes too. Can u pls guide us how i can achive this functionality. TIA A

I got a crash when right click on an image in a web page loaded by WKWebView. The log and stack trace show that it might related to VisionKit. I think it may related to copy text on image feature. I don't know how to fix it. So I just want to disable VisionKit on WKWebView to check whether it fix the issue or not. Do you know how to do it? Stack trace: The log before crash: [com.apple.VisionKit.processing] Error processing request from MAD on result: Error Domain=NSOSStatusErrorDomain Code=-50 "paramErr: error in user parameter list" UserInfo={NSLocalizedDescription=Error allocating CVPixelBuffer} request: <VKCImageAnalyzerRequest: 0x6000025562b0> requestID: 22 madRequestID: (Not Set) cancelled: NO

I'm using VNDocumentViewController to scan some documents which is working fine. But not infrequently, the app becomes slow after dismissing the VNDocumentViewController. There is no VNDocumentViewController instance allocated, but according to the allocations gathered by instruments, there is a ICDocCamViewController still living and using between 200 and 300 MB. I guess that ICDocCamViewController is an internal component of VNDocumentViewController. Are there any known issues? Unfortunately, I do not see any way to free ICDocCamViewController. 281.37 MB 93.1% 104708 start_wqthread 281.37 MB 93.1% 104708 _pthread_wqthread 256.04 MB 84.7% 42747 _dispatch_workloop_worker_thread 256.03 MB 84.7% 42564 _dispatch_lane_invoke 256.03 MB 84.7% 42564 _dispatch_lane_serial_drain 249.91 MB 82.7% 22258 _dispatch_client_callout 237.01 MB 78.4% 8562 _dispatch_call_block_and_release 236.50 MB 78.3% 3836 __77-[ICDocCamViewController saveCapturedImage:metaData:rects:completionHandler:]_block_invoke_3 236.50 MB 78.3% 3836 -[ICDocCamViewController cropAndFilterImage:rects:filterType:] 236.49 MB 78.3% 3678 +[ICDocCamImageFilters filteredImage:orientation:imageFilterType:] 236.49 MB 78.3% 3678 +[ICDocCamImageFilters colorDocument:orientation:] 236.47 MB 78.3% 3476 -[CIContext(createCGImage) createCGImage:fromRect:] 236.47 MB 78.3% 3476 -[CIContext(_createCGImageInternal) _createCGImage:fromRect:format:premultiplied:colorSpace:deferred:renderCallback:] 236.40 MB 78.2% 2096 -[CIContext(CIRenderDestination) startTaskToRender:fromRect:toDestination:atPoint:error:] 236.40 MB 78.2% 2096 -[CIContext(CIRenderDestination) _startTaskToRender:toDestination:forPrepareRender:forClear:error:] 236.40 MB 78.2% 2096 CI::RenderToBitmap::render(CI::Image*, CI::Context*) const 236.40 MB 78.2% 2096 CI::image_get_bitmap(CI::Context*, CI::Image*, CGRect, CGColorSpace*, CI::Bitmap*, CI::RenderDestination const*) 236.33 MB 78.2% 716 CI::tile_node_graph(CI::Context*, CI::RenderDestination const*, char const*, CI::Node*, CGRect const&, CI::PixelFormat, CI::swizzle_info const&, CI::TileTask* (CI::ProgramNode*, CGRect) block_pointer) 236.33 MB 78.2% 716 CI::recursive_tile(CI::RenderTask*, CI::Context*, CI::RenderDestination const*, char const*, CI::Node*, CGRect const&, CI::PixelFormat, CI::swizzle_info const&, CI::TileTask* (CI::ProgramNode*, CGRect) block_pointer) 236.26 MB 78.2% 212 invocation function for block in CI::image_get_bitmap(CI::Context*, CI::Image*, CGRect, CGColorSpace*, CI::Bitmap*, CI::RenderDestination const*) 236.26 MB 78.2% 212 CI::Context::render(CI::ProgramNode*, CGRect const&) 236.26 MB 78.2% 212 CI::Context::recursive_render(CI::TileTask*, CI::roiKey const&, CI::Node*, bool) 236.26 MB 78.2% 211 CI::Context::recursive_render(CI::TileTask*, CI::roiKey const&, CI::Node*, bool) 236.24 MB 78.2% 196 CI::Context::recursive_render(CI::TileTask*, CI::roiKey const&, CI::Node*, bool) 236.23 MB 78.2% 181 CI::Context::recursive_render(CI::TileTask*, CI::roiKey const&, CI::Node*, bool) 236.21 MB 78.2% 166 CI::Context::recursive_render(CI::TileTask*, CI::roiKey const&, CI::Node*, bool) 236.19 MB 78.2% 151 CI::Context::recursive_render(CI::TileTask*, CI::roiKey const&, CI::Node*, bool) 17.02 KB 0.0% 14 CI::MetalTextureManager::create_intermediate(CI::IntermediateDescriptor const&, unsigned long long, CGRect const&, unsigned long, unsigned long, bool) 128 Bytes 0.0% 1 CI::MetalContext::render_intermediate_node(CI::TileTask*, CI::ProgramNode*, CGRect const&, CI::intermediate_t*, bool, void () block_pointer) 17.02 KB 0.0% 14 CI::MetalTextureManager::create_intermediate(CI::IntermediateDescriptor const&, unsigned long long, CGRect const&, unsigned long, unsigned long, bool) 128 Bytes 0.0% 1 CI::MetalContext::render_intermediate_node(CI::TileTask*, CI::ProgramNode*, CGRect const&, CI::intermediate_t*, bool, void () block_pointer) 17.02 KB 0.0% 14 CI::MetalTextureManager::create_intermediate(CI::IntermediateDescriptor const&, unsigned long long, CGRect const&, unsigned long, unsigned long, bool) 128 Bytes 0.0% 1 CI::MetalContext::render_intermediate_node(CI::TileTask*, CI::ProgramNode*, CGRect const&, CI::intermediate_t*, bool, void () block_pointer) 17.02 KB 0.0% 14 CI::MetalTextureManager::create_intermediate(CI::IntermediateDescriptor const&, unsigned long long, CGRect const&, unsigned long, unsigned long, bool) 128 Bytes 0.0% 1 CI::MetalContext::render_intermediate_node(CI::TileTask*, CI::ProgramNode*, CGRect const&, CI::intermediate_t*, bool, void () block_pointer) 128 Bytes 0.0% 1 CI::MetalContext::render_root_node(CI::TileTask*, CI::ProgramNode*, CGRect const&, void () block_pointer, void () block_pointer) 69.00 KB 0.0% 504 CI::gather_rois_for_program_graph(CI::Context*, char const*, CI::ProgramNode*, CGRect) 71.72 KB 0.0% 1380 CI::prepare_initial_graph(CI::Context*, char const*, CI::Image*, CI::RenderDestination const*, CGRect, CGColorSpace*, CI::PixelFormat, CI::swizzle_info, CI::Affine const&, bool, CI::TextureDescriptor*) 71.72 KB 0.0% 1380 CI::prepare_initial_graph(CI::Context*, char const*, CI::Image*, CI::RenderDestination const*, CGRect, CGColorSpace*, CI::PixelFormat, CI::swizzle_info, CI::Affine const&, bool, CI::TextureDescriptor*) 18.34 KB 0.0% 202 -[CIPaperWash outputImage]

Hello there! Really excited about this year's additions to ImageAnalysisInteraction, especially the selectedText. Thank you! My question is if there's a way to have multiple interactions for a given image analysis. For example, instead of being able to make just one text selection, I would like to support multiple. In theory, I know multiple UIInteraction-conforming objects can be added to any view but I'm unsure if this is possible with ImageAnalysisInteraction. I'm willing to create a custom UIInteraction, if you see a potential way forward there (would love to hear any ideas!). Side question/feature request: UITextSelectionDisplayInteraction, which I assume ImageAnalysisInteraction might be using internally, allows handleViews customizations and others. I would be super useful if that was exposed through ImageAnalysisInteraction as well! Looking forward to hearing your ideas. TIA!

Hello guys, I am trying to run this sample project on my Ipad, I got a black screen the camera does not initialize. I tried updating the info.plist and asking for camera permission. I updated all the devices, did someone tried this demo? https://developer.apple.com/documentation/vision/detecting_animal_body_poses_with_vision

Hi One point I would like to ask. How many specs do you need and which Mac would you recommend for Apple Vision Pro development? We use XCode, RealityKit, ARKit, Reality Composer Pro, Unity Editor which supports visionOS development, and MaterialX. If possible, what notebook and desktop models do you recommend? Best regards Sadao Tokuyama https://1planet.co.jp/

Hi! Wondering if there's a way to add subject lifting to images in my SwiftUI app without relying on UIViewControllerRepresentable and Coordinators to adopt the ImageAnalysisInteraction protocol. Thank you!

When trying the code from this Session: https://developer.apple.com/videos/play/wwdc2023/10176/ Specifically: result.generateScaledMaskForImage(forInstances: result.allInstances, from: handler) It throws the error: -[VNInstanceMaskObservation generateScaledMaskForImageForInstances:fromRequestHandler:error:]: unrecognized selector sent to instance Any ideas?

I'm using VisionKit framework for text recognition and detecting rectangles in the image. For that, I'm using VNRecognizeText & VNDetectRectangles features of the VisionKit. In macOS and iOS results, I found slight difference in the boundingBox coordinates of the text and the rectangles detected for the same image. Is this expected? Can we do anything to make the results identical? Also, on macOS, when I'm using same features of VisionKit from python (using pyobjc-framework-Vision package), there also i'm getting slightly different results.

Most of the APIs for this show iOS 16 support. Will a full subject lifting implementation require the device to be running iOS 17 or will it run on 16 too?