IOSurface

RSS for tag

Share hardware-accelerated buffer data across multiple processes and frameworks using IOSurface.

Posts under IOSurface tag

39 Posts

Post

Replies

Boosts

Views

Activity

NSXPCConnection fails to transport IOSurfaceRef
I'm trying to send an IOSurfaceRef across an NSXPCConnection on osx 10.13 and I'm having trouble with the solution that was provided in the forum thread "Efficiently sending data from an XPC process to the host application." https://developer.apple.com/forums/thread/126716 From that thread: > However, that specific problem got resolved on 10.12 where we introduced a new Objective-C IOSurface object, and that object is transportable directly over NSXPCConnection . So double yay! But it doesn’t seem to work. I have a very simple service protocol that includes (void)sendFrame:(IOSurfaceRef)frame; along with some basic NSString sending methods that successfully transfer across my NSXPCConnection. I have a valid (non-NULL) IOSurface in my app that I send to my helper app with sendFrame, and when the call is executed in the helper, the resulting frame is always NULL. On the other hand, I’ve also tried creating an IOSurface with the (deprecated) kIOSurfaceIsGlobal property and sending the IOSurface’s ID instead with: (void)sendFrameID:(uint32_t)frameID; and [_service sendFrameID:IOSurfaceGetID(surface)]; And on the helper app side, I look up the IO to get an IOSurfaceRef: IOSurfaceRef frame = IOSurfaceLookup(frameID); and it works correctly – I get a valid IOSurface which I can display and see the same pixel contents in both the app and the helper. So what is meant by the new IOSurface object in 10.12 is “transportable directly” over NSXPCConnection? How is it supposed to work? I’m specifically interested in no-copy transfer. Thanks!
4
0
2.9k
Apr ’23
How to convert kCVPixelFormatType_40ARGBLEWideGamut to kCVPixelFormatType_32BGRA
extension MTLTexture { func toCVPixelBufferInBGRA8() -> CVPixelBuffer? { let attrs = [kCVPixelBufferCGImageCompatibilityKey: kCFBooleanTrue, kCVPixelBufferCGBitmapContextCompatibilityKey: kCFBooleanTrue] as CFDictionary var pixelBufferOut: Unmanaged<CVPixelBuffer>? if let iosurface = self.iosurface { CVPixelBufferCreateWithIOSurface(kCFAllocatorDefault, iosurface, attrs, &pixelBufferOut) } guard let pixelBuffer = pixelBufferOut?.takeRetainedValue() else { return nil } CVPixelBufferLockBaseAddress(pixelBuffer, .readOnly) defer { CVPixelBufferUnlockBaseAddress(pixelBuffer, .readOnly) } let width = vImagePixelCount(CVPixelBufferGetWidth(pixelBuffer)) let height = vImagePixelCount(CVPixelBufferGetHeight(pixelBuffer)) let srcBytesPerRow = CVPixelBufferGetBytesPerRow(pixelBuffer) let srcBaseAddress = CVPixelBufferGetBaseAddress(pixelBuffer) var srcBuffer = vImage_Buffer(data: srcBaseAddress, height: height, width: width, rowBytes: srcBytesPerRow) if let src = srcBaseAddress?.assumingMemoryBound(to: UInt16.self) { var n = 0 n = 1 print("srcBaseAddress", src[0 + 4 * n], src[1 + 4 * n], src[2 + 4 * n], src[3 + 4 * n]); n = 1334 print("srcBaseAddress", src[0 + 4 * n], src[1 + 4 * n], src[2 + 4 * n], src[3 + 4 * n]); } let dstBytesPerRow = Int(width) * 5 guard let dstBaseAddress = malloc(Int(height) * dstBytesPerRow) else { return nil } var dstBuffer = vImage_Buffer(data: dstBaseAddress, height: height, width: width, rowBytes: dstBytesPerRow) let error = vImageConvert_ARGB2101010ToARGB8888(&srcBuffer, &dstBuffer, 0, 1023, [0, 1, 2, 3], vImage_Flags(kvImageNoFlags)) guard error == kvImageNoError else { free(dstBaseAddress) return nil } var dstCVPixelBuffer: CVPixelBuffer? let releaseCallback: CVPixelBufferReleaseBytesCallback = {_, pointer in if let pointer = pointer { free(UnsafeMutableRawPointer(mutating: pointer)) } } guard CVPixelBufferCreateWithBytes(nil, Int(width), Int(height), kCVPixelFormatType_32BGRA, dstBaseAddress, dstBytesPerRow, releaseCallback, nil, attrs, &dstCVPixelBuffer) == kCVReturnSuccess else { free(dstBaseAddress) return nil } let dst = dstBaseAddress.assumingMemoryBound(to: UInt8.self) var n = 0 n = 1 print("dstBaseAddress", dst[0 + 4 * n], dst[1 + 4 * n], dst[2 + 4 * n], dst[3 + 4 * n]); n = 1334 print("dstBaseAddress", dst[0 + 4 * n], dst[1 + 4 * n], dst[2 + 4 * n], dst[3 + 4 * n]); print("") return dstCVPixelBuffer } } I have few print statement and you can use the output to see the color value. When the metal texture is red and pixel format is kCVPixelFormatType_40ARGBLEWideGamut, the color value is B=30144 G=25216 R=54592 A=57216. After the conversion, the color in kCVPixelFormatType_32BGRA is 255 126 13 128. But I do not know which is A, R, G, B.
0
0
924
Apr ’23
CMIOExtensionStreamSource supported pixel formats
Hello. I'm working on a Core Media IO camera extension, including a source and sink stream. My application sends sample buffers to the sink stream, which are passed to the source stream. The sample buffers' pixel format is 420v. This format is determined in the application (not the camera extension) by a VTDecompressionSession, which has the image buffer attribute requested: kCVPixelBufferIOSurfaceCoreAnimationCompatibilityKey; and a decoder specification that requests the key kVTVideoDecoderSpecification_EnableHardwareAcceleratedVideoDecoder. However, no client can connect to the extension's source stream. Photo Booth shows a blank screen. I noticed the second argument to the function CMVideoFormatDescriptionCreate, which has type CMVideoCodecType, doesn't explicitly support 420v. In fact there is no 4:2:0 format in the list at all. Comparing the headers, the only format in common between CoreVideo and CoreMedia is 422YpCbCr8... % sed -n -e 's/.*\(kCMVideoCodecType_[a-zA-Z0-9_]*\).*/\1/p' < /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreMedia.framework/Versions/A/Headers/CMFormatDescription.h | sort | uniq > kCMVideoCodecType.txt % sed -n -e 's/.*\(kCVPixelFormatType_[a-zA-Z0-9_]*\).*/\1/p' < /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreVideo.framework/Versions/A/Headers/CVPixelBuffer.h | sort | uniq > kCVPixelFormatType.txt % comm <(sed -e 's/kCMVideoCodecType_//' < kCMVideoCodecType.txt) <(sed -e 's/kCVPixelFormatType_//' < kCVPixelFormatType.txt) What is the guidance for pixel formats in camera extensions? The sample camera extension code produced by Xcode uses kCVPixelFormatType_32BGRA. Should camera extensions all be using this format? That's an extra step of YUV→RGB conversion that I'd rather avoid if possible.
4
0
1.6k
Mar ’23
iOS WKWebView takeSnapshot allocates unnecessary amounts of memory
I'm rapidly capturing screenshots of a webpage loaded into a WKWebView. While working fine in the simulator when running on an iOS device each call to takeSnapshot seems to allocate unnecessary amounts of memory (70+ MiB). Note that the total amount of allocated memory is way less when running in the sim. I did not have this issue prior to upgrading to iOS 16. Is this something that is known to have changed? Also, what does WKWebView base its allocated memory on? It doesn't seem to reduce by capturing smaller screenshots. Thanks.
1
1
1.6k
Oct ’22
Creating MLMultiArray from IOSurface backed CVPixelBuffer fixes strides to multiples of 32
Hi, I have found that when creating a MLMultiArray from an IOSurface backed CVPixelBuffer that the striding across the last dimension gets fixed to a multiple of 32. Say I have some objective-c code that creates a MLMultiArray from a CVPixelBuffer like so; CVPixelBufferRef pixelBuffer = NULL; NSDictionary* pixelBufferAttributes = @{ (id)kCVPixelBufferIOSurfacePropertiesKey: @{} }; // Since shape == [2, 3, 4], width is 4 (= shape[2]) and height is 6 (= shape[0] * shape[1]). CVPixelBufferCreate(kCFAllocatorDefault, 4, 6, kCVPixelFormatType_OneComponent16Half, (__bridge CFDictionaryRef)pixelBufferAttributes, &pixelBuffer); MLMultiArray *multiArray = [[MLMultiArray alloc] initWithPixelBuffer:pixelBuffer shape:@[@2, @3, @4]]; This successfully creates a MLMultiArray from an IOSurface backed CVPixelBuffer. If I inspect the MLMultiArray, I see that the shape and strides are the following; //_shape: [2, 3, 4] //_strides: [96, 32, 1] To me it makes more sense that the strides should be; //_strides: [12,4,1] I assume that this is likely hardcoded to be multiples of 32 somewhere in the MLMultiArray initialisation code and since strides is a read only member variable of MLMultiArray it cannot be changed explicitly. Is it possible to set the striding explicitly in any way, or create an MLMutliArray from IOSurface backed buffer that allows the striding to be set differently ?
1
0
1.8k
Aug ’22
Iphone operating system
Im trying to get the best reasons & understanding to why my iphone 11 pro max is constantly using this operating system on my iphone? * Mozilla/5.0 (iPhone; CPU iPhone OS 14_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Mobile/15E148 Safari/604.1;browserLanguage:en-us;siteLanguage:en-us****
3
0
4.8k
Jul ’22
How do I render a dynamically changing IOSurface to a CALayer or NSView?
I have a background process which is updating an IOSurface-backed CVPixelBuffer at 30fps. I want to render a preview of that pixel buffer in my window, scaled to the size of the NSView that's displaying it. I get a callback every time the pixelbuffer/IOSurface is updated. I've tried using a custom layer-backed NSView and setting the layer contents to the IOsurface -- which works when the view is created but it's never updated unless the window is resized or another window is in front of it. I've tried setting both my view and my layer SetNeedsDisplay(), I've tried changing the layerContentsRedrawPolicy to .onSetNeedsDisplay, I've tried making sure all my content and update code is happening on the UI thread, but I can't get it to dynamically update. Is there a way to bind my layer or view to the IOSurface once and then just have it reflect the updates as they happen, or, if not, at least mark the layer as dirty each frame when it changes? I've pored over the docs but I don't see a lot about the relationship between IOSurface and CALayer.contents, and when in the lifecycle to mark things dirty (especially when updates are happening outside the view). Here's example code: class VideoPreviewThumbnail: NSView, VideoFeedConsumer {   let testCard = TestCardHelper()       override var wantsUpdateLayer: Bool {     get { return true }   }   required init?(coder decoder: NSCoder) {     super.init(coder: decoder)     self.wantsLayer = true     self.layerContentsRedrawPolicy = .onSetNeedsDisplay      &#9;&#9;/* Scale the incoming data to the size of the view */      self.layer?.transform = CATransform3DMakeScale(       (self.layer?.contentsScale)! * self.frame.width / CGFloat(VideoSettings.width),       (self.layer?.contentsScale)! * self.frame.height / CGFloat(VideoSettings.height),       CGFloat(1)) &#9; /* Register us with the content provider */     VideoFeedBrowser.instance.registerConsumer(self)   }       deinit{     VideoFeedBrowser.instance.deregisterConsumer(self)   }       override func updateLayer() { &#9;&#9;/* ideally we woudln't need to do this */     updateLayer(pixelBuffer: VideoFeedBrowser.instance.renderer.pixelBuffer)   }    &#9;/* This gets called every time our pixelbuffer is updated (30fps) */   @objc   func updateFrame(pixelBuffer: CVPixelBuffer) {     updateLayer(pixelBuffer: pixelBuffer)   }       func updateLayer(pixelBuffer: CVPixelBuffer) {     guard let surface = CVPixelBufferGetIOSurface(pixelBuffer)?.takeUnretainedValue() else {      print("pixelbuffer isn't IOsurface backed! noooooo!")       return;     } &#9; /* these don't have any effect */ &#9; //&#9;&#9;self.layer?.setNeedsDisplay() //&#9;&#9;self.setNeedsDisplay(invalidRect: self.visibleRect)     self.layer?.contents = surface   } }
5
0
4.3k
May ’22
Ios 15.3.1 problem
Hello dear kindly I need your support about this problem since I update my iOS 15.3.1 every time I have a call and get waiting I can’t close this sound beep beep beep beep beep only solution to end this sound press on lock bottom so both of call will end or I can reject and answer second call or I can reject only or I can answer and hold So please I need solution for this problem
0
0
779
Feb ’22
com.apple.UIKit.UIPrintInteractionController.printPreviewGeneration (QOS: UNSPECIFIED) EXC_BREAKPOINT 0x00000001809e1ae0
0 libdispatch.dylib _dispatch_assert_queue_fail + 116 1 libdispatch.dylib _dispatch_assert_queue_fail + 198 arrow_right 2 ShareSheet -[SHSheetMainPresenter mainInteractor:didPerformActivityWithResult:] + 72 3 ShareSheet -[SHSheetMainInteractor _didPerformActivityWithResult:] + 88 4 ShareSheet __38-[SHSheetMainInteractor _runActivity:]_block_invoke + 60 5 ShareSheet -[SHSheetActivityPerformer _completePerformingActivityWithState:returnedItems:error:] + 812 6 ShareSheet __57-[SHSheetActivityPerformer performWithCompletionHandler:]_block_invoke + 100 7 ShareSheet -[UIActivity activityDidFinish:items:error:] + 184 8 ShareSheet __41-[UIOpenInIBooksActivity performActivity]_block_invoke + 244 9 UIKitCore __61-[UIPrintInteractionController _endPrintJobWithAction:error:]_block_invoke + 432 10 Foundation -[_NSBarrierOperation main] + 76 11 Foundation __NSOPERATION_IS_INVOKING_MAIN__ + 24 12 Foundation -[NSOperation start] + 804 13 Foundation __NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__ + 24 14 Foundation __NSOQSchedule_f + 184 15 libdispatch.dylib _dispatch_block_async_invoke2 + 148 16 libdispatch.dylib _dispatch_client_callout + 20 17 libdispatch.dylib _dispatch_lane_concurrent_drain + 1000 18 libdispatch.dylib _dispatch_lane_invoke + 504 19 libdispatch.dylib _dispatch_queue_override_invoke + 500 20 libdispatch.dylib _dispatch_root_queue_drain + 396 21 libdispatch.dylib _dispatch_worker_thread2 + 164 22 libsystem_pthread.dylib _pthread_wqthread + 228 23 libsystem_pthread.dylib start_wqthread + 8
0
0
805
Jan ’22
App freezes with GPU timeout (IOAF error 2)
We have an AR project (built in Unity) that's running 24/7 on a number of iPads Pros (12.9, gen 3) in a museum. It's been live for 2.5 years now. We've always had the occasional freeze (on iOS 13 and below), but lately they have gotten exponentially worse (on iOS 14.4.2 and 14.6). The problem manifests as follows, at completely random times: our app keeps running in the background (we can hear sound effects and see database activity continuing), but it stops getting redrawn, causing the screen to freeze on whatever frame was drawn last. We get a gpuEvent log (see here), and when running the app when connected to XCode, we get the following error message a few times: Execution of the command buffer was aborted due to an error during execution. Caused GPU Timeout Error (IOAF code 2) followed by this one, which keeps repeating: Execution of the command buffer was aborted due to an error during execution. Ignored (for causing prior/excessive GPU errors) (IOAF code 4) A couple of potentially related threads: https://developer.apple.com/forums/thread/73225 https://forum.unity.com/threads/iphone-8-a11-gpu-hang-error-command-buffer.500549/ https://github.com/KhronosGroup/MoltenVK/issues/602 Especially the last one seems to describe exactly the same problem as we're having. We've tried to reproduce the freeze with XCode's API and Shader validation options, but that doesn't give a lot of readable info - possibly because we're building with Unity 2018.4? I don't know how to connect this info to the correct Unity shader file, for instance - let alone the offending line of code. Screenshot below. Does anyone have any suggestions as to how we should continue?
2
0
2.4k
Jul ’21
gpuEvent related to IOSurface
Hi, We have an AR project (built in Unity) that's running 24/7 on a number of iPads Pros (12.9, gen 3) in a museum. It's been live for 2.5 years now. We've always had the occasional freeze (on iOS 13 and below), but lately they have gotten exponentially worse (on iOS 14.4.2 and 14.6). The problem manifests as follows, at completely random times: our app keeps running in the background (we can hear sound effects and see database activity continuing), but it stops getting redrawn, causing the screen to freeze on whatever frame was drawn last. We're getting a gpuevent .ips log at freeze time (see below), which has information about an IOSurface being blocked by IOFence. There's very little info online about these logs though, so we're sort of stuck with the analysis. Can anyone here interpret this log better than we can, or does anyone have suggestions as to our next steps? {"bug_type":"284","timestamp":"2021-07-10 10:51:30.00 +0200","os_version":"iPhone OS 14.6 (18F72)","incident_id":"C3ABD477-2A61-4D43-A3B1-25D7CCD706E6"} { "process_name" : "hats", "registers" : { }, "timestamp" : 1625907090, "analysis" : { "iofence_list" : { "iofence_num_iosurfaces" : 1, "iofence_iosurfaces" : [ { "iofence_current_queue" : [ { "iofence_acceleratorid" : 1, "iofence_backtrace" : [ -68148358856, -68148357636, -68144533400, -68144532380, -68144496896, -68144598792, -68144425788, -68144434312 ], "iofence_direction" : 1 } ], "iosurface_id" : 243, "iofence_waiting_queue" : [ { "iofence_acceleratorid" : 2, "iofence_backtrace" : [ -68148358856, -68148357636, -68146420632, -68154559340, -68146423972, -68146427004, -68146495056, -68154290536 ], "iofence_direction" : 2 }, { "iofence_acceleratorid" : 1, "iofence_backtrace" : [ -68148358856, -68148357636, -68144533400, -68144532380, -68144496896, -68144598792, -68144425788, -68144434312 ], "iofence_direction" : 1 } ] } ] }, "fw_ta_substate" : { "slot0" : 0, "slot1" : 0 }, "fw_power_state" : 0, "fw_power_boost_controller" : 0, "guilty_dm" : 1, "fw_power_controller_in_charge" : 0, "fw_cl_state" : { "slot0" : 0 }, "fw_perf_state_lo" : 1, "fw_ta_state" : { "slot0" : 0, "slot1" : 0 }, "signature" : 625, "fw_power_substate" : 4, "command_buffer_trace_id" : 490782964, "fw_perf_state_select" : 0, "restart_reason" : 7, "fw_3d_state" : { "slot0" : 0, "slot1" : 0, "slot2" : 0 }, "fw_gpc_perf_state" : 0, "fw_perf_state_hi" : 1, "fw_power_limit_controller" : 7, "restart_reason_desc" : "blocked by IOFence" } }
1
0
2.3k
Jul ’21
NSXPCConnection fails to transport IOSurfaceRef
I'm trying to send an IOSurfaceRef across an NSXPCConnection on osx 10.13 and I'm having trouble with the solution that was provided in the forum thread "Efficiently sending data from an XPC process to the host application." https://developer.apple.com/forums/thread/126716 From that thread: > However, that specific problem got resolved on 10.12 where we introduced a new Objective-C IOSurface object, and that object is transportable directly over NSXPCConnection . So double yay! But it doesn’t seem to work. I have a very simple service protocol that includes (void)sendFrame:(IOSurfaceRef)frame; along with some basic NSString sending methods that successfully transfer across my NSXPCConnection. I have a valid (non-NULL) IOSurface in my app that I send to my helper app with sendFrame, and when the call is executed in the helper, the resulting frame is always NULL. On the other hand, I’ve also tried creating an IOSurface with the (deprecated) kIOSurfaceIsGlobal property and sending the IOSurface’s ID instead with: (void)sendFrameID:(uint32_t)frameID; and [_service sendFrameID:IOSurfaceGetID(surface)]; And on the helper app side, I look up the IO to get an IOSurfaceRef: IOSurfaceRef frame = IOSurfaceLookup(frameID); and it works correctly – I get a valid IOSurface which I can display and see the same pixel contents in both the app and the helper. So what is meant by the new IOSurface object in 10.12 is “transportable directly” over NSXPCConnection? How is it supposed to work? I’m specifically interested in no-copy transfer. Thanks!
Replies
4
Boosts
0
Views
2.9k
Activity
Apr ’23
Xcode14.3怎么设置成中文界面
怎么切换成中文啊,英文不太好 怎么切换成中文啊,英文不太好
Replies
0
Boosts
0
Views
3.1k
Activity
Apr ’23
How to convert kCVPixelFormatType_40ARGBLEWideGamut to kCVPixelFormatType_32BGRA
extension MTLTexture { func toCVPixelBufferInBGRA8() -> CVPixelBuffer? { let attrs = [kCVPixelBufferCGImageCompatibilityKey: kCFBooleanTrue, kCVPixelBufferCGBitmapContextCompatibilityKey: kCFBooleanTrue] as CFDictionary var pixelBufferOut: Unmanaged<CVPixelBuffer>? if let iosurface = self.iosurface { CVPixelBufferCreateWithIOSurface(kCFAllocatorDefault, iosurface, attrs, &pixelBufferOut) } guard let pixelBuffer = pixelBufferOut?.takeRetainedValue() else { return nil } CVPixelBufferLockBaseAddress(pixelBuffer, .readOnly) defer { CVPixelBufferUnlockBaseAddress(pixelBuffer, .readOnly) } let width = vImagePixelCount(CVPixelBufferGetWidth(pixelBuffer)) let height = vImagePixelCount(CVPixelBufferGetHeight(pixelBuffer)) let srcBytesPerRow = CVPixelBufferGetBytesPerRow(pixelBuffer) let srcBaseAddress = CVPixelBufferGetBaseAddress(pixelBuffer) var srcBuffer = vImage_Buffer(data: srcBaseAddress, height: height, width: width, rowBytes: srcBytesPerRow) if let src = srcBaseAddress?.assumingMemoryBound(to: UInt16.self) { var n = 0 n = 1 print("srcBaseAddress", src[0 + 4 * n], src[1 + 4 * n], src[2 + 4 * n], src[3 + 4 * n]); n = 1334 print("srcBaseAddress", src[0 + 4 * n], src[1 + 4 * n], src[2 + 4 * n], src[3 + 4 * n]); } let dstBytesPerRow = Int(width) * 5 guard let dstBaseAddress = malloc(Int(height) * dstBytesPerRow) else { return nil } var dstBuffer = vImage_Buffer(data: dstBaseAddress, height: height, width: width, rowBytes: dstBytesPerRow) let error = vImageConvert_ARGB2101010ToARGB8888(&srcBuffer, &dstBuffer, 0, 1023, [0, 1, 2, 3], vImage_Flags(kvImageNoFlags)) guard error == kvImageNoError else { free(dstBaseAddress) return nil } var dstCVPixelBuffer: CVPixelBuffer? let releaseCallback: CVPixelBufferReleaseBytesCallback = {_, pointer in if let pointer = pointer { free(UnsafeMutableRawPointer(mutating: pointer)) } } guard CVPixelBufferCreateWithBytes(nil, Int(width), Int(height), kCVPixelFormatType_32BGRA, dstBaseAddress, dstBytesPerRow, releaseCallback, nil, attrs, &dstCVPixelBuffer) == kCVReturnSuccess else { free(dstBaseAddress) return nil } let dst = dstBaseAddress.assumingMemoryBound(to: UInt8.self) var n = 0 n = 1 print("dstBaseAddress", dst[0 + 4 * n], dst[1 + 4 * n], dst[2 + 4 * n], dst[3 + 4 * n]); n = 1334 print("dstBaseAddress", dst[0 + 4 * n], dst[1 + 4 * n], dst[2 + 4 * n], dst[3 + 4 * n]); print("") return dstCVPixelBuffer } } I have few print statement and you can use the output to see the color value. When the metal texture is red and pixel format is kCVPixelFormatType_40ARGBLEWideGamut, the color value is B=30144 G=25216 R=54592 A=57216. After the conversion, the color in kCVPixelFormatType_32BGRA is 255 126 13 128. But I do not know which is A, R, G, B.
Replies
0
Boosts
0
Views
924
Activity
Apr ’23
Crash from IOSSurfaceCreate
I am getting this crash log, where crash is happening from iosCreateNamedProperties. I have attached image of crash log. What could be the reason and how to solve it.
Replies
1
Boosts
0
Views
1.1k
Activity
Apr ’23
CMIOExtensionStreamSource supported pixel formats
Hello. I'm working on a Core Media IO camera extension, including a source and sink stream. My application sends sample buffers to the sink stream, which are passed to the source stream. The sample buffers' pixel format is 420v. This format is determined in the application (not the camera extension) by a VTDecompressionSession, which has the image buffer attribute requested: kCVPixelBufferIOSurfaceCoreAnimationCompatibilityKey; and a decoder specification that requests the key kVTVideoDecoderSpecification_EnableHardwareAcceleratedVideoDecoder. However, no client can connect to the extension's source stream. Photo Booth shows a blank screen. I noticed the second argument to the function CMVideoFormatDescriptionCreate, which has type CMVideoCodecType, doesn't explicitly support 420v. In fact there is no 4:2:0 format in the list at all. Comparing the headers, the only format in common between CoreVideo and CoreMedia is 422YpCbCr8... % sed -n -e 's/.*\(kCMVideoCodecType_[a-zA-Z0-9_]*\).*/\1/p' < /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreMedia.framework/Versions/A/Headers/CMFormatDescription.h | sort | uniq > kCMVideoCodecType.txt % sed -n -e 's/.*\(kCVPixelFormatType_[a-zA-Z0-9_]*\).*/\1/p' < /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreVideo.framework/Versions/A/Headers/CVPixelBuffer.h | sort | uniq > kCVPixelFormatType.txt % comm <(sed -e 's/kCMVideoCodecType_//' < kCMVideoCodecType.txt) <(sed -e 's/kCVPixelFormatType_//' < kCVPixelFormatType.txt) What is the guidance for pixel formats in camera extensions? The sample camera extension code produced by Xcode uses kCVPixelFormatType_32BGRA. Should camera extensions all be using this format? That's an extra step of YUV→RGB conversion that I'd rather avoid if possible.
Replies
4
Boosts
0
Views
1.6k
Activity
Mar ’23
Ios 16.1 crash
no display shown after im updating my iphone xr to 16.1 .. but somehow i can charge the battery just there is no display.. this is happen when i try to play voice note from whatsapp
Replies
1
Boosts
0
Views
1.2k
Activity
Oct ’22
iOS WKWebView takeSnapshot allocates unnecessary amounts of memory
I'm rapidly capturing screenshots of a webpage loaded into a WKWebView. While working fine in the simulator when running on an iOS device each call to takeSnapshot seems to allocate unnecessary amounts of memory (70+ MiB). Note that the total amount of allocated memory is way less when running in the sim. I did not have this issue prior to upgrading to iOS 16. Is this something that is known to have changed? Also, what does WKWebView base its allocated memory on? It doesn't seem to reduce by capturing smaller screenshots. Thanks.
Replies
1
Boosts
1
Views
1.6k
Activity
Oct ’22
Creating MLMultiArray from IOSurface backed CVPixelBuffer fixes strides to multiples of 32
Hi, I have found that when creating a MLMultiArray from an IOSurface backed CVPixelBuffer that the striding across the last dimension gets fixed to a multiple of 32. Say I have some objective-c code that creates a MLMultiArray from a CVPixelBuffer like so; CVPixelBufferRef pixelBuffer = NULL; NSDictionary* pixelBufferAttributes = @{ (id)kCVPixelBufferIOSurfacePropertiesKey: @{} }; // Since shape == [2, 3, 4], width is 4 (= shape[2]) and height is 6 (= shape[0] * shape[1]). CVPixelBufferCreate(kCFAllocatorDefault, 4, 6, kCVPixelFormatType_OneComponent16Half, (__bridge CFDictionaryRef)pixelBufferAttributes, &pixelBuffer); MLMultiArray *multiArray = [[MLMultiArray alloc] initWithPixelBuffer:pixelBuffer shape:@[@2, @3, @4]]; This successfully creates a MLMultiArray from an IOSurface backed CVPixelBuffer. If I inspect the MLMultiArray, I see that the shape and strides are the following; //_shape: [2, 3, 4] //_strides: [96, 32, 1] To me it makes more sense that the strides should be; //_strides: [12,4,1] I assume that this is likely hardcoded to be multiples of 32 somewhere in the MLMultiArray initialisation code and since strides is a read only member variable of MLMultiArray it cannot be changed explicitly. Is it possible to set the striding explicitly in any way, or create an MLMutliArray from IOSurface backed buffer that allows the striding to be set differently ?
Replies
1
Boosts
0
Views
1.8k
Activity
Aug ’22
Iphone operating system
Im trying to get the best reasons & understanding to why my iphone 11 pro max is constantly using this operating system on my iphone? * Mozilla/5.0 (iPhone; CPU iPhone OS 14_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Mobile/15E148 Safari/604.1;browserLanguage:en-us;siteLanguage:en-us****
Replies
3
Boosts
0
Views
4.8k
Activity
Jul ’22
How do I render a dynamically changing IOSurface to a CALayer or NSView?
I have a background process which is updating an IOSurface-backed CVPixelBuffer at 30fps. I want to render a preview of that pixel buffer in my window, scaled to the size of the NSView that's displaying it. I get a callback every time the pixelbuffer/IOSurface is updated. I've tried using a custom layer-backed NSView and setting the layer contents to the IOsurface -- which works when the view is created but it's never updated unless the window is resized or another window is in front of it. I've tried setting both my view and my layer SetNeedsDisplay(), I've tried changing the layerContentsRedrawPolicy to .onSetNeedsDisplay, I've tried making sure all my content and update code is happening on the UI thread, but I can't get it to dynamically update. Is there a way to bind my layer or view to the IOSurface once and then just have it reflect the updates as they happen, or, if not, at least mark the layer as dirty each frame when it changes? I've pored over the docs but I don't see a lot about the relationship between IOSurface and CALayer.contents, and when in the lifecycle to mark things dirty (especially when updates are happening outside the view). Here's example code: class VideoPreviewThumbnail: NSView, VideoFeedConsumer {   let testCard = TestCardHelper()       override var wantsUpdateLayer: Bool {     get { return true }   }   required init?(coder decoder: NSCoder) {     super.init(coder: decoder)     self.wantsLayer = true     self.layerContentsRedrawPolicy = .onSetNeedsDisplay      &#9;&#9;/* Scale the incoming data to the size of the view */      self.layer?.transform = CATransform3DMakeScale(       (self.layer?.contentsScale)! * self.frame.width / CGFloat(VideoSettings.width),       (self.layer?.contentsScale)! * self.frame.height / CGFloat(VideoSettings.height),       CGFloat(1)) &#9; /* Register us with the content provider */     VideoFeedBrowser.instance.registerConsumer(self)   }       deinit{     VideoFeedBrowser.instance.deregisterConsumer(self)   }       override func updateLayer() { &#9;&#9;/* ideally we woudln't need to do this */     updateLayer(pixelBuffer: VideoFeedBrowser.instance.renderer.pixelBuffer)   }    &#9;/* This gets called every time our pixelbuffer is updated (30fps) */   @objc   func updateFrame(pixelBuffer: CVPixelBuffer) {     updateLayer(pixelBuffer: pixelBuffer)   }       func updateLayer(pixelBuffer: CVPixelBuffer) {     guard let surface = CVPixelBufferGetIOSurface(pixelBuffer)?.takeUnretainedValue() else {      print("pixelbuffer isn't IOsurface backed! noooooo!")       return;     } &#9; /* these don't have any effect */ &#9; //&#9;&#9;self.layer?.setNeedsDisplay() //&#9;&#9;self.setNeedsDisplay(invalidRect: self.visibleRect)     self.layer?.contents = surface   } }
Replies
5
Boosts
0
Views
4.3k
Activity
May ’22
iPhone 13 pro max 90 fps not working properly. Pubg global.
The game seems to run smoother and stable in 60fps than in 90 fps option. I believe 90fps is not working. Its getting choppy and heated quite fast as well. Any solution or update coming for that? Thank you.
Replies
0
Boosts
0
Views
1.2k
Activity
Apr ’22
Ios 15.3.1 problem
Hello dear kindly I need your support about this problem since I update my iOS 15.3.1 every time I have a call and get waiting I can’t close this sound beep beep beep beep beep only solution to end this sound press on lock bottom so both of call will end or I can reject and answer second call or I can reject only or I can answer and hold So please I need solution for this problem
Replies
0
Boosts
0
Views
779
Activity
Feb ’22
com.apple.UIKit.UIPrintInteractionController.printPreviewGeneration (QOS: UNSPECIFIED) EXC_BREAKPOINT 0x00000001809e1ae0
0 libdispatch.dylib _dispatch_assert_queue_fail + 116 1 libdispatch.dylib _dispatch_assert_queue_fail + 198 arrow_right 2 ShareSheet -[SHSheetMainPresenter mainInteractor:didPerformActivityWithResult:] + 72 3 ShareSheet -[SHSheetMainInteractor _didPerformActivityWithResult:] + 88 4 ShareSheet __38-[SHSheetMainInteractor _runActivity:]_block_invoke + 60 5 ShareSheet -[SHSheetActivityPerformer _completePerformingActivityWithState:returnedItems:error:] + 812 6 ShareSheet __57-[SHSheetActivityPerformer performWithCompletionHandler:]_block_invoke + 100 7 ShareSheet -[UIActivity activityDidFinish:items:error:] + 184 8 ShareSheet __41-[UIOpenInIBooksActivity performActivity]_block_invoke + 244 9 UIKitCore __61-[UIPrintInteractionController _endPrintJobWithAction:error:]_block_invoke + 432 10 Foundation -[_NSBarrierOperation main] + 76 11 Foundation __NSOPERATION_IS_INVOKING_MAIN__ + 24 12 Foundation -[NSOperation start] + 804 13 Foundation __NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__ + 24 14 Foundation __NSOQSchedule_f + 184 15 libdispatch.dylib _dispatch_block_async_invoke2 + 148 16 libdispatch.dylib _dispatch_client_callout + 20 17 libdispatch.dylib _dispatch_lane_concurrent_drain + 1000 18 libdispatch.dylib _dispatch_lane_invoke + 504 19 libdispatch.dylib _dispatch_queue_override_invoke + 500 20 libdispatch.dylib _dispatch_root_queue_drain + 396 21 libdispatch.dylib _dispatch_worker_thread2 + 164 22 libsystem_pthread.dylib _pthread_wqthread + 228 23 libsystem_pthread.dylib start_wqthread + 8
Replies
0
Boosts
0
Views
805
Activity
Jan ’22
UITextFiled.selectedTextRange bug on iOS 15
Test device version: iPad iOS 15.2、 iPhone iOS15.3。 set UITextFiled.selectedTextRange in -(void)textFieldDidBeginEditing: method. When the selected text range is too long, the text will not be highlighted, but will actually be selected. Text is highlighted when tapped, or be replaced when typing.
Replies
1
Boosts
0
Views
739
Activity
Dec ’21
Downloading app
in Trying to reinstall an app and it keeps saying I have to have a new developer code
Replies
1
Boosts
0
Views
635
Activity
Oct ’21
Iphone 10 1X zoom is taking blurry photos
recently i have purchased iphone x and updated to ios14.03 after which i found that my camera app is taking blurry photos on 1X normal but when i do 2x it takes clear pictures which is causing me a lot of problem i tried to reset all the setting but unable to clear the problem.
Replies
1
Boosts
0
Views
1.3k
Activity
Oct ’21
Is it possible somehow in iOS to prevent screen capture?
Is there's way to prevent taking screenshots entirely while using my app? I need to prevent screen capture by users of my app, for security reasons. The contents I display are confidential and should not be copied onto the device.
Replies
3
Boosts
0
Views
2.3k
Activity
Sep ’21
App freezes with GPU timeout (IOAF error 2)
We have an AR project (built in Unity) that's running 24/7 on a number of iPads Pros (12.9, gen 3) in a museum. It's been live for 2.5 years now. We've always had the occasional freeze (on iOS 13 and below), but lately they have gotten exponentially worse (on iOS 14.4.2 and 14.6). The problem manifests as follows, at completely random times: our app keeps running in the background (we can hear sound effects and see database activity continuing), but it stops getting redrawn, causing the screen to freeze on whatever frame was drawn last. We get a gpuEvent log (see here), and when running the app when connected to XCode, we get the following error message a few times: Execution of the command buffer was aborted due to an error during execution. Caused GPU Timeout Error (IOAF code 2) followed by this one, which keeps repeating: Execution of the command buffer was aborted due to an error during execution. Ignored (for causing prior/excessive GPU errors) (IOAF code 4) A couple of potentially related threads: https://developer.apple.com/forums/thread/73225 https://forum.unity.com/threads/iphone-8-a11-gpu-hang-error-command-buffer.500549/ https://github.com/KhronosGroup/MoltenVK/issues/602 Especially the last one seems to describe exactly the same problem as we're having. We've tried to reproduce the freeze with XCode's API and Shader validation options, but that doesn't give a lot of readable info - possibly because we're building with Unity 2018.4? I don't know how to connect this info to the correct Unity shader file, for instance - let alone the offending line of code. Screenshot below. Does anyone have any suggestions as to how we should continue?
Replies
2
Boosts
0
Views
2.4k
Activity
Jul ’21
gpuEvent related to IOSurface
Hi, We have an AR project (built in Unity) that's running 24/7 on a number of iPads Pros (12.9, gen 3) in a museum. It's been live for 2.5 years now. We've always had the occasional freeze (on iOS 13 and below), but lately they have gotten exponentially worse (on iOS 14.4.2 and 14.6). The problem manifests as follows, at completely random times: our app keeps running in the background (we can hear sound effects and see database activity continuing), but it stops getting redrawn, causing the screen to freeze on whatever frame was drawn last. We're getting a gpuevent .ips log at freeze time (see below), which has information about an IOSurface being blocked by IOFence. There's very little info online about these logs though, so we're sort of stuck with the analysis. Can anyone here interpret this log better than we can, or does anyone have suggestions as to our next steps? {"bug_type":"284","timestamp":"2021-07-10 10:51:30.00 +0200","os_version":"iPhone OS 14.6 (18F72)","incident_id":"C3ABD477-2A61-4D43-A3B1-25D7CCD706E6"} { "process_name" : "hats", "registers" : { }, "timestamp" : 1625907090, "analysis" : { "iofence_list" : { "iofence_num_iosurfaces" : 1, "iofence_iosurfaces" : [ { "iofence_current_queue" : [ { "iofence_acceleratorid" : 1, "iofence_backtrace" : [ -68148358856, -68148357636, -68144533400, -68144532380, -68144496896, -68144598792, -68144425788, -68144434312 ], "iofence_direction" : 1 } ], "iosurface_id" : 243, "iofence_waiting_queue" : [ { "iofence_acceleratorid" : 2, "iofence_backtrace" : [ -68148358856, -68148357636, -68146420632, -68154559340, -68146423972, -68146427004, -68146495056, -68154290536 ], "iofence_direction" : 2 }, { "iofence_acceleratorid" : 1, "iofence_backtrace" : [ -68148358856, -68148357636, -68144533400, -68144532380, -68144496896, -68144598792, -68144425788, -68144434312 ], "iofence_direction" : 1 } ] } ] }, "fw_ta_substate" : { "slot0" : 0, "slot1" : 0 }, "fw_power_state" : 0, "fw_power_boost_controller" : 0, "guilty_dm" : 1, "fw_power_controller_in_charge" : 0, "fw_cl_state" : { "slot0" : 0 }, "fw_perf_state_lo" : 1, "fw_ta_state" : { "slot0" : 0, "slot1" : 0 }, "signature" : 625, "fw_power_substate" : 4, "command_buffer_trace_id" : 490782964, "fw_perf_state_select" : 0, "restart_reason" : 7, "fw_3d_state" : { "slot0" : 0, "slot1" : 0, "slot2" : 0 }, "fw_gpc_perf_state" : 0, "fw_perf_state_hi" : 1, "fw_power_limit_controller" : 7, "restart_reason_desc" : "blocked by IOFence" } }
Replies
1
Boosts
0
Views
2.3k
Activity
Jul ’21