VideoToolbox

RSS for tag

Work directly with hardware-accelerated video encoding and decoding capabilities using VideoToolbox.

Posts under VideoToolbox tag

117 Posts

Post

Replies

Boosts

Views

Activity

Quality of video encoding (VideoToolBox) on M1 Macs
Hi, I think that a visual quality of hardware video encoder (VideoToolBox H.264) in M1 Macs is not great. I compared a software libx264 encoder (Mac) and NVIDIA Geforce (NVENC) on Windows 10 with the same bitrate or constant quality mode producing almost identical file size. I see many visual artefacts on a video encoded with VideoToolBox H.264 (1080p 60 fps) 3300K or CQ56. It's especially noticeable on video with animated photo slideshows and blur effect. Too many artefacts. NVENC and libx264 produce much better visual results with the same file size. I tested in my app and in Handbrake Beta for M1 which supports VideoToolBox H.264 encoding. Apple engineers can contact me for more details.
5
2
12k
Aug ’22
VTDecompressionSessionCreate fails with code -12911 on M1 iPad
In the context of an app that uses a VTDecompressionSession to decode incoming ts streams, the creation of the decompression session always fails for some h264 streams with code -12911 when running on a M1 iPad (iPad Pro, 12.9-inch, 5th generation, iOS 15.5). When the same app is executed on my M1 Mac mini -as My Mac (Designed for iPad)- with the same stream, VTDecompressionSessionCreatesucceeds and the stream can be decoded. Any idea of what could cause this error on the iPad? The code: decompressionSessionCreationStatus = VTDecompressionSessionCreate(allocator: kCFAllocatorDefault,                                                                   formatDescription: videoFormatDescription!,                                                                   decoderSpecification: nil,                                                                   imageBufferAttributes: nil,                                                                   outputCallback: nil,                                                                   decompressionSessionOut: &videoDecompressionSession) if videoDecompressionSession != nil { ... } else {     NSLog("videoDecompressionSession could not be created (status: %d): video format: %@",            decompressionSessionCreationStatus, (videoFormatDescription != nil) ? (CFCopyDescription(videoFormatDescription!) as String) : "{?}") } where videoFormatDescription was previously created by extracting h264 parameter sets and calling CMVideoFormatDescriptionCreateFromH264ParameterSets. Output: videoDecompressionSession could not be created (status: -12911): video format: <CMVideoFormatDescription 0x281894360 [0x1dacc01b8]> { mediaType:'vide'  mediaSubType:'avc1'  mediaSpecific: { codecType: 'avc1' dimensions: 1920 x 1080  }  extensions: {{     CVImageBufferChromaLocationBottomField = Left;     CVImageBufferChromaLocationTopField = Left;     CVImageBufferColorPrimaries = "ITU_R_709_2";     CVImageBufferTransferFunction = "ITU_R_709_2";     CVImageBufferYCbCrMatrix = "ITU_R_709_2";     CVPixelAspectRatio =     {         HorizontalSpacing = 1;         VerticalSpacing = 1;     };     FullRangeVideo = 0;     SampleDescriptionExtensionAtoms =     {         avcC = {length = 119, bytes = 0x01640028 ffe10064 67640028 ad90a470 ... 68ff3cb0 fdf8f800 };     }; }} } Any help on this would be greatly appreciated! :)
2
1
1.7k
Aug ’22
CVPixelBufferGetBytesPerRowOfPlane strange result
Hello everyone. I'm decoding an H264 elementary stream with VideoToolbox. The VTDecompressionSession's output callback returns a CVImageBuffer successfully. While attempting to convert this YUV image to RGB, I prepare a luma vImage_Buffer and chroma vImage_Buffer, but I get strange values from CVPixelBufferGetBytesPerRowOfPlane: CVPixelBufferGetBaseAddressOfPlane(0) 0x10e98c000 CVPixelBufferGetHeightOfPlane(0) 1080 CVPixelBufferGetWidthOfPlane(0) 1440 CVPixelBufferGetBytesPerRowOfPlane(0) 46080 CVPixelBufferGetBaseAddressOfPlane(1) 0x10eb2b000 CVPixelBufferGetHeightOfPlane(1) 540 CVPixelBufferGetWidthOfPlane(1) 720 CVPixelBufferGetBytesPerRowOfPlane(1) 23040 If I use this value (46080) in initializing the vImage_Buffer's rowBytes field, vImageConvert_420Yp8_CbCr8ToARGB8888 crashes. If I simply set rowBytes=width, the conversion is successful. Has anyone seen this before? Notice the value returned is 32 × width. Why?
2
0
2.1k
Aug ’22
Thread 1: EXC_BAD_ACCESS (code=2, address=0x16f6a3ea0)
I am trying to use AVPlayerView Controller But when i try to play the video its crash This is my Code `import UIKit import AVKit import SwiftyJSON import Alamofire import EffyIosFramework class VideoViewController: UIViewController, AVPlayerViewControllerDelegate { var player : AVPlayer! var vc = AVPlayerViewController() var isVideoPlaying = false var instance = VideoViewController() //View Did Load override func viewDidLoad() { super.viewDidLoad() vc.delegate = self } //Play Video func playVideo(url: URL) { player = AVPlayer(url: url) vc.player = player player.automaticallyWaitsToMinimizeStalling = false vc.delegate = self vc.showsPlaybackControls = false vc.videoGravity = AVLayerVideoGravity.resizeAspectFill let playerLayer = AVPlayerLayer(player: player) playerLayer.opacity = 0 self.isVideoPlaying = true self.vc.player?.play() DispatchQueue.main.asyncAfter(wallDeadline: .now()+0.2) { self.vc.view.frame = self.view.window!.bounds self.vc.view.backgroundColor = .clear DispatchQueue.main.async { self.vc.view.addSubview(self.view) } self.view.window?.addSubview(self.vc.view) } } }
2
0
3.1k
May ’22
videotoolbox h264 decoder regression in iOS 15 / Apple Silicon
We have ffmpeg-based code that runs on both iOS and Macos. We've noticed, that with the upgrade from iOS 14 -> iOS 15 videotoolbox decoder started failing to process h264 stream. Today we've finally dug a bit deeper into this, and it seems the failure is similar on iOS 15, and on MacOS 12.3 -- but only on Apply Silicon for the latter. The same software and input work fine on Intel Mac. The failure is explained in some detail here: https://trac.ffmpeg.org/ticket/9713 and here: https://trac.ffmpeg.org/ticket/9016 It is quite easily reproduced with ffmpeg
1
0
1.5k
May ’22
VideoToolbox regression with iOS 15.4
Hi We use VideoToolbox to encode our videos and upload them on our servers. But since iOS 15.4, all videos uploaded have a really bad quality. It is because the bitrate is very low. Apparently, when we create ou VTCompressionSessionRef object, we have to change the kVTCompressionPropertyKey_DataRateLimits and put it much higher than before, but if we put it too high, it does not work either. It is very weird. Has someone the same problem ? Best regards
5
0
2.9k
Apr ’22
About LTR support in VideoToolbox Encoding
Hi there, When I enable LTR by setting "kVTCompressionPropertyKey_EnableLTR" to be true, and get LTR refresh frame by setting ForceLTRRefresh to be true in VTCompressionSessionEncodeFrame() call. Everything goes well, but I found I cannot know which encoded frame is LTR Refresh frame, I try to find out, so dumpped the attachments of the CMSampleBuffer frame, I found the normal P frame's attachment like this: { DependsOnOthers = 1; EarlierDisplayTimesAllowed = 0; EncodedFrameAvgQP = 35; IsDependedOnByOthers = 1; NotSync = 1; RequireLTRAcknowledgementToken = 9177; TemporalID = 0; } And LTR refresh frame may look like this: { DependsOnOthers = 1; EarlierDisplayTimesAllowed = 0; EncodedFrameAvgQP = 38; FECGroupID = 0; FECLastFrameInGroup = 1; FECLevelOfProtection = 1; IsDependedOnByOthers = 1; NotSync = 1; ReferenceWasRefreshed = 1; RequireLTRAcknowledgementToken = 9180; TemporalID = 0; } The laster has 4 more key-value pairs than the former: FECGroupID = 0; FECLastFrameInGroup = 1; FECLevelOfProtection = 1; ReferenceWasRefreshed = 1; The key "ReferenceWasRefreshed" is very noticeable, So question 1: May this key signify current frame is a LTR Refresh frame? (But It's weird I didn't find any documents about this key.) Question 2: If this frame is LTR refresh frame, How can I know which frame this LTR frame refers to? More specifily, the corresponding 'AcknowledgementToken'. Question 3: This question is independent. I found that if I didn't set 'kVTEncodeFrameOptionKey_AcknowledgedLTRTokens' for a while (keep the tokens be empty), I cannot get the 'AcknowledgementToken' from encoded CMSampleBuffer for a while too, that's to say this key is not contained in the attachments of CMSampleBuffer. What should I do to make the encoded frame always contains the AcknowledgementToken? Thanks very much!
0
0
1.5k
Mar ’22
VTDecompressionSessionCreate fails if instantiated with predefined constant from SDK
VTDecompressionSessionCreate fails if instantiated with predefined constant - kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder. However it is succeeded if replaced with literal - CFSTR( "RequireHardwareAcceleratedVideoDecoder" ). Literally they have same value. Sample data (sps and pps) identical for both cases. Could you help please? Build command: clang -x objective-c -target x86_64-apple-macos10.14 -framework Foundation -framework CoreMedia -framework VideoToolbox -o vt main.m main.m
0
0
783
Mar ’22
Mac Pro: Two h.264 HW Encoders?
Hey folks - While working with OBS Studio, I noticed that when it's running on my 2019 Mac Pro 7,1, it lists two hardware h.264 encoders available. If I check that same version of OBS on my 2018-era Macbook Pro (with discrete AMD GPU), it only lists a single harrdware h.264 encoder. My Mac Pro has the single AMD Radeon Pro Vega II GPU (not the dual). It's not really clear why the API is presenting two hardware encoders, and if there are actually any differences between the two. One of the OBS devs wrote me a quick-and-dirty C program to dump out the list of encoders. The program looks like: #include <stdlib.h> #include <CoreFoundation/CoreFoundation.h> #include <VideoToolbox/VideoToolbox.h> #include <VideoToolbox/VTVideoEncoderList.h> #include <CoreMedia/CoreMedia.h> int main() { CFArrayRef encoder_list; VTCopyVideoEncoderList(NULL, &encoder_list); CFIndex size = CFArrayGetCount(encoder_list); for (CFIndex i = 0; i < size; i++) { CFDictionaryRef encoder_dict = CFArrayGetValueAtIndex(encoder_list, i); #define VT_PRINT(key, name) \ CFStringRef name##_ref = CFDictionaryGetValue(encoder_dict, key); \ CFIndex name##_len = CFStringGetLength(name##_ref); \ char *name = malloc(name##_len + 1); \ memset(name, 0, name##_len + 1); \ CFStringGetFileSystemRepresentation(name##_ref, name, name##_len); VT_PRINT(kVTVideoEncoderList_EncoderName, name); printf("Name: %s\n", name); VT_PRINT(kVTVideoEncoderList_DisplayName, dn); printf("Display Name: %s\n", dn); VT_PRINT(kVTVideoEncoderList_EncoderID, id); printf("Id: %s\n", id); printf("=========================\n"); } CFRelease(encoder_list); exit(0); } Executing that on my Mac Pro, I see, among the output: Name: Apple H.264 (HW) Display Name: Apple H.264 (HW) Id: com.apple.videotoolbox.videoencoder.h264.gva.100000abc ========================= Name: Apple H.264 (HW) Display Name: Apple H.264 (HW) Id: com.apple.videotoolbox.videoencoder.h264.gva When I run it on the laptop: ========================= Name: Apple H.264 (HW) Display Name: Apple H.264 (HW) Id: com.apple.videotoolbox.videoencoder.h264.gva My question is: what's the difference between those two hardware encoders listed for the Mac Pro? Thanks for any guidance. :-)
1
0
2.0k
Mar ’22
iOS camera encoding
Does iOS camera app use VideoToolBox framework's method e.g. VTCompressionSessionEncodeFrame() for encoding? Or has apple their own separate APIs apart from open APIs available for 3rd party developers in VideoToolBox?
0
0
990
Mar ’22
Why is my video resource invalid for PHAssetChangeRequest.creationRequestForAssetFromVideo(atFileURL:)
I am processing an H264 encoded video stream from a non-apple IoT device. I want to record bits of this video stream. I'm getting an error when I try to save to the photo gallery: The operation couldn’t be completed. (PHPhotosErrorDomain error 3302.) My Code, let me know if I need to share more:   private func beginRecording() {     self.handlePhotoLibraryAuth()     self.createFilePath()     guard let videoOutputURL = self.outputURL,        let vidWriter = try? AVAssetWriter(outputURL: videoOutputURL, fileType: AVFileType.mp4),        self.formatDesc != nil else {          print("Warning: No Format For Video")          return        }     let vidInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: nil, sourceFormatHint: self.formatDesc)     guard vidWriter.canAdd(vidInput) else {       print("Error: Cant add video writer input")       return     }           let sourcePixelBufferAttributes = [       kCVPixelBufferPixelFormatTypeKey as String: NSNumber(value: kCVPixelFormatType_32ARGB),       kCVPixelBufferWidthKey as String: "1280",       kCVPixelBufferHeightKey as String: "720"] as [String : Any]           self.videoWriterInputPixelBufferAdaptor = AVAssetWriterInputPixelBufferAdaptor(       assetWriterInput: vidInput,       sourcePixelBufferAttributes: sourcePixelBufferAttributes)     vidInput.expectsMediaDataInRealTime = true     vidWriter.add(vidInput)     guard vidWriter.startWriting() else {       print("Error: Cant write with vid writer")       return     }     vidWriter.startSession(atSourceTime: CMTimeMake(value: self.videoFrameCounter, timescale: self.videoFPS))     self.videoWriter = vidWriter     self.videoWriterInput = vidInput     print("Recording Video Stream")   } Save the Video   private func saveRecordingToPhotoLibrary() {     let fileManager = FileManager.default     guard fileManager.fileExists(atPath: self.path) else {       print("Error: The file: \(self.path) not exists, so cannot move this file camera roll")       return     }     print("The file: \(self.path) has been save into documents folder, and is ready to be moved to camera roll") This is what Fails     PHPhotoLibrary.shared().performChanges({       PHAssetChangeRequest.creationRequestForAssetFromVideo(atFileURL: URL(fileURLWithPath: self.path))     }) { completed, error in       guard completed else {         print ("Error: Cannot move the video \(self.path) to camera roll, error: \(String(describing: error?.localizedDescription))")         return       }       print("Video \(self.path) has been moved to camera roll")     }   }  When Recording ends we save the video   private func endRecording() {     guard let vidInput = videoWriterInput, let vidWriter = videoWriter else {       print("Error, no video writer or video input")       return     }     vidInput.markAsFinished()     if !vidInput.isReadyForMoreMediaData {       vidWriter.finishWriting {         print("Finished Recording")         guard vidWriter.status == .completed else {           print("Warning: The Video Writer status is not completed, status: \(vidWriter.status.rawValue)")           print(vidWriter.error.debugDescription)           return         }         print("VideoWriter status is completed")         self.saveRecordingToPhotoLibrary()       }     }   }
3
0
7.5k
Dec ’21
Quality of video encoding (VideoToolBox) on M1 Macs
Hi, I think that a visual quality of hardware video encoder (VideoToolBox H.264) in M1 Macs is not great. I compared a software libx264 encoder (Mac) and NVIDIA Geforce (NVENC) on Windows 10 with the same bitrate or constant quality mode producing almost identical file size. I see many visual artefacts on a video encoded with VideoToolBox H.264 (1080p 60 fps) 3300K or CQ56. It's especially noticeable on video with animated photo slideshows and blur effect. Too many artefacts. NVENC and libx264 produce much better visual results with the same file size. I tested in my app and in Handbrake Beta for M1 which supports VideoToolBox H.264 encoding. Apple engineers can contact me for more details.
Replies
5
Boosts
2
Views
12k
Activity
Aug ’22
VTDecompressionSessionCreate fails with code -12911 on M1 iPad
In the context of an app that uses a VTDecompressionSession to decode incoming ts streams, the creation of the decompression session always fails for some h264 streams with code -12911 when running on a M1 iPad (iPad Pro, 12.9-inch, 5th generation, iOS 15.5). When the same app is executed on my M1 Mac mini -as My Mac (Designed for iPad)- with the same stream, VTDecompressionSessionCreatesucceeds and the stream can be decoded. Any idea of what could cause this error on the iPad? The code: decompressionSessionCreationStatus = VTDecompressionSessionCreate(allocator: kCFAllocatorDefault,                                                                   formatDescription: videoFormatDescription!,                                                                   decoderSpecification: nil,                                                                   imageBufferAttributes: nil,                                                                   outputCallback: nil,                                                                   decompressionSessionOut: &videoDecompressionSession) if videoDecompressionSession != nil { ... } else {     NSLog("videoDecompressionSession could not be created (status: %d): video format: %@",            decompressionSessionCreationStatus, (videoFormatDescription != nil) ? (CFCopyDescription(videoFormatDescription!) as String) : "{?}") } where videoFormatDescription was previously created by extracting h264 parameter sets and calling CMVideoFormatDescriptionCreateFromH264ParameterSets. Output: videoDecompressionSession could not be created (status: -12911): video format: <CMVideoFormatDescription 0x281894360 [0x1dacc01b8]> { mediaType:'vide'  mediaSubType:'avc1'  mediaSpecific: { codecType: 'avc1' dimensions: 1920 x 1080  }  extensions: {{     CVImageBufferChromaLocationBottomField = Left;     CVImageBufferChromaLocationTopField = Left;     CVImageBufferColorPrimaries = "ITU_R_709_2";     CVImageBufferTransferFunction = "ITU_R_709_2";     CVImageBufferYCbCrMatrix = "ITU_R_709_2";     CVPixelAspectRatio =     {         HorizontalSpacing = 1;         VerticalSpacing = 1;     };     FullRangeVideo = 0;     SampleDescriptionExtensionAtoms =     {         avcC = {length = 119, bytes = 0x01640028 ffe10064 67640028 ad90a470 ... 68ff3cb0 fdf8f800 };     }; }} } Any help on this would be greatly appreciated! :)
Replies
2
Boosts
1
Views
1.7k
Activity
Aug ’22
CVPixelBufferGetBytesPerRowOfPlane strange result
Hello everyone. I'm decoding an H264 elementary stream with VideoToolbox. The VTDecompressionSession's output callback returns a CVImageBuffer successfully. While attempting to convert this YUV image to RGB, I prepare a luma vImage_Buffer and chroma vImage_Buffer, but I get strange values from CVPixelBufferGetBytesPerRowOfPlane: CVPixelBufferGetBaseAddressOfPlane(0) 0x10e98c000 CVPixelBufferGetHeightOfPlane(0) 1080 CVPixelBufferGetWidthOfPlane(0) 1440 CVPixelBufferGetBytesPerRowOfPlane(0) 46080 CVPixelBufferGetBaseAddressOfPlane(1) 0x10eb2b000 CVPixelBufferGetHeightOfPlane(1) 540 CVPixelBufferGetWidthOfPlane(1) 720 CVPixelBufferGetBytesPerRowOfPlane(1) 23040 If I use this value (46080) in initializing the vImage_Buffer's rowBytes field, vImageConvert_420Yp8_CbCr8ToARGB8888 crashes. If I simply set rowBytes=width, the conversion is successful. Has anyone seen this before? Notice the value returned is 32 × width. Why?
Replies
2
Boosts
0
Views
2.1k
Activity
Aug ’22
Thread 1: EXC_BAD_ACCESS (code=2, address=0x16f6a3ea0)
I am trying to use AVPlayerView Controller But when i try to play the video its crash This is my Code `import UIKit import AVKit import SwiftyJSON import Alamofire import EffyIosFramework class VideoViewController: UIViewController, AVPlayerViewControllerDelegate { var player : AVPlayer! var vc = AVPlayerViewController() var isVideoPlaying = false var instance = VideoViewController() //View Did Load override func viewDidLoad() { super.viewDidLoad() vc.delegate = self } //Play Video func playVideo(url: URL) { player = AVPlayer(url: url) vc.player = player player.automaticallyWaitsToMinimizeStalling = false vc.delegate = self vc.showsPlaybackControls = false vc.videoGravity = AVLayerVideoGravity.resizeAspectFill let playerLayer = AVPlayerLayer(player: player) playerLayer.opacity = 0 self.isVideoPlaying = true self.vc.player?.play() DispatchQueue.main.asyncAfter(wallDeadline: .now()+0.2) { self.vc.view.frame = self.view.window!.bounds self.vc.view.backgroundColor = .clear DispatchQueue.main.async { self.vc.view.addSubview(self.view) } self.view.window?.addSubview(self.vc.view) } } }
Replies
2
Boosts
0
Views
3.1k
Activity
May ’22
videotoolbox h264 decoder regression in iOS 15 / Apple Silicon
We have ffmpeg-based code that runs on both iOS and Macos. We've noticed, that with the upgrade from iOS 14 -> iOS 15 videotoolbox decoder started failing to process h264 stream. Today we've finally dug a bit deeper into this, and it seems the failure is similar on iOS 15, and on MacOS 12.3 -- but only on Apply Silicon for the latter. The same software and input work fine on Intel Mac. The failure is explained in some detail here: https://trac.ffmpeg.org/ticket/9713 and here: https://trac.ffmpeg.org/ticket/9016 It is quite easily reproduced with ffmpeg
Replies
1
Boosts
0
Views
1.5k
Activity
May ’22
webview.allowsInlineMediaPlayback
I am developing a hybrid app, I am using java script and html, to be able to compile it to xcode I use capacitor, the problem is that my app includes videos but I cannot block the native ios player, I want to block it. webview.allowsInlineMediaPlayback = yes; I found this, the problem is that it only blocks it for ipad, not for iphones.
Replies
1
Boosts
0
Views
1.8k
Activity
May ’22
VideoToolbox regression with iOS 15.4
Hi We use VideoToolbox to encode our videos and upload them on our servers. But since iOS 15.4, all videos uploaded have a really bad quality. It is because the bitrate is very low. Apparently, when we create ou VTCompressionSessionRef object, we have to change the kVTCompressionPropertyKey_DataRateLimits and put it much higher than before, but if we put it too high, it does not work either. It is very weird. Has someone the same problem ? Best regards
Replies
5
Boosts
0
Views
2.9k
Activity
Apr ’22
About LTR support in VideoToolbox Encoding
Hi there, When I enable LTR by setting "kVTCompressionPropertyKey_EnableLTR" to be true, and get LTR refresh frame by setting ForceLTRRefresh to be true in VTCompressionSessionEncodeFrame() call. Everything goes well, but I found I cannot know which encoded frame is LTR Refresh frame, I try to find out, so dumpped the attachments of the CMSampleBuffer frame, I found the normal P frame's attachment like this: { DependsOnOthers = 1; EarlierDisplayTimesAllowed = 0; EncodedFrameAvgQP = 35; IsDependedOnByOthers = 1; NotSync = 1; RequireLTRAcknowledgementToken = 9177; TemporalID = 0; } And LTR refresh frame may look like this: { DependsOnOthers = 1; EarlierDisplayTimesAllowed = 0; EncodedFrameAvgQP = 38; FECGroupID = 0; FECLastFrameInGroup = 1; FECLevelOfProtection = 1; IsDependedOnByOthers = 1; NotSync = 1; ReferenceWasRefreshed = 1; RequireLTRAcknowledgementToken = 9180; TemporalID = 0; } The laster has 4 more key-value pairs than the former: FECGroupID = 0; FECLastFrameInGroup = 1; FECLevelOfProtection = 1; ReferenceWasRefreshed = 1; The key "ReferenceWasRefreshed" is very noticeable, So question 1: May this key signify current frame is a LTR Refresh frame? (But It's weird I didn't find any documents about this key.) Question 2: If this frame is LTR refresh frame, How can I know which frame this LTR frame refers to? More specifily, the corresponding 'AcknowledgementToken'. Question 3: This question is independent. I found that if I didn't set 'kVTEncodeFrameOptionKey_AcknowledgedLTRTokens' for a while (keep the tokens be empty), I cannot get the 'AcknowledgementToken' from encoded CMSampleBuffer for a while too, that's to say this key is not contained in the attachments of CMSampleBuffer. What should I do to make the encoded frame always contains the AcknowledgementToken? Thanks very much!
Replies
0
Boosts
0
Views
1.5k
Activity
Mar ’22
VTDecompressionSessionCreate fails if instantiated with predefined constant from SDK
VTDecompressionSessionCreate fails if instantiated with predefined constant - kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder. However it is succeeded if replaced with literal - CFSTR( "RequireHardwareAcceleratedVideoDecoder" ). Literally they have same value. Sample data (sps and pps) identical for both cases. Could you help please? Build command: clang -x objective-c -target x86_64-apple-macos10.14 -framework Foundation -framework CoreMedia -framework VideoToolbox -o vt main.m main.m
Replies
0
Boosts
0
Views
783
Activity
Mar ’22
Mac Pro: Two h.264 HW Encoders?
Hey folks - While working with OBS Studio, I noticed that when it's running on my 2019 Mac Pro 7,1, it lists two hardware h.264 encoders available. If I check that same version of OBS on my 2018-era Macbook Pro (with discrete AMD GPU), it only lists a single harrdware h.264 encoder. My Mac Pro has the single AMD Radeon Pro Vega II GPU (not the dual). It's not really clear why the API is presenting two hardware encoders, and if there are actually any differences between the two. One of the OBS devs wrote me a quick-and-dirty C program to dump out the list of encoders. The program looks like: #include <stdlib.h> #include <CoreFoundation/CoreFoundation.h> #include <VideoToolbox/VideoToolbox.h> #include <VideoToolbox/VTVideoEncoderList.h> #include <CoreMedia/CoreMedia.h> int main() { CFArrayRef encoder_list; VTCopyVideoEncoderList(NULL, &encoder_list); CFIndex size = CFArrayGetCount(encoder_list); for (CFIndex i = 0; i < size; i++) { CFDictionaryRef encoder_dict = CFArrayGetValueAtIndex(encoder_list, i); #define VT_PRINT(key, name) \ CFStringRef name##_ref = CFDictionaryGetValue(encoder_dict, key); \ CFIndex name##_len = CFStringGetLength(name##_ref); \ char *name = malloc(name##_len + 1); \ memset(name, 0, name##_len + 1); \ CFStringGetFileSystemRepresentation(name##_ref, name, name##_len); VT_PRINT(kVTVideoEncoderList_EncoderName, name); printf("Name: %s\n", name); VT_PRINT(kVTVideoEncoderList_DisplayName, dn); printf("Display Name: %s\n", dn); VT_PRINT(kVTVideoEncoderList_EncoderID, id); printf("Id: %s\n", id); printf("=========================\n"); } CFRelease(encoder_list); exit(0); } Executing that on my Mac Pro, I see, among the output: Name: Apple H.264 (HW) Display Name: Apple H.264 (HW) Id: com.apple.videotoolbox.videoencoder.h264.gva.100000abc ========================= Name: Apple H.264 (HW) Display Name: Apple H.264 (HW) Id: com.apple.videotoolbox.videoencoder.h264.gva When I run it on the laptop: ========================= Name: Apple H.264 (HW) Display Name: Apple H.264 (HW) Id: com.apple.videotoolbox.videoencoder.h264.gva My question is: what's the difference between those two hardware encoders listed for the Mac Pro? Thanks for any guidance. :-)
Replies
1
Boosts
0
Views
2.0k
Activity
Mar ’22
iOS camera encoding
Does iOS camera app use VideoToolBox framework's method e.g. VTCompressionSessionEncodeFrame() for encoding? Or has apple their own separate APIs apart from open APIs available for 3rd party developers in VideoToolBox?
Replies
0
Boosts
0
Views
990
Activity
Mar ’22
Why is my video resource invalid for PHAssetChangeRequest.creationRequestForAssetFromVideo(atFileURL:)
I am processing an H264 encoded video stream from a non-apple IoT device. I want to record bits of this video stream. I'm getting an error when I try to save to the photo gallery: The operation couldn’t be completed. (PHPhotosErrorDomain error 3302.) My Code, let me know if I need to share more:   private func beginRecording() {     self.handlePhotoLibraryAuth()     self.createFilePath()     guard let videoOutputURL = self.outputURL,        let vidWriter = try? AVAssetWriter(outputURL: videoOutputURL, fileType: AVFileType.mp4),        self.formatDesc != nil else {          print("Warning: No Format For Video")          return        }     let vidInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: nil, sourceFormatHint: self.formatDesc)     guard vidWriter.canAdd(vidInput) else {       print("Error: Cant add video writer input")       return     }           let sourcePixelBufferAttributes = [       kCVPixelBufferPixelFormatTypeKey as String: NSNumber(value: kCVPixelFormatType_32ARGB),       kCVPixelBufferWidthKey as String: "1280",       kCVPixelBufferHeightKey as String: "720"] as [String : Any]           self.videoWriterInputPixelBufferAdaptor = AVAssetWriterInputPixelBufferAdaptor(       assetWriterInput: vidInput,       sourcePixelBufferAttributes: sourcePixelBufferAttributes)     vidInput.expectsMediaDataInRealTime = true     vidWriter.add(vidInput)     guard vidWriter.startWriting() else {       print("Error: Cant write with vid writer")       return     }     vidWriter.startSession(atSourceTime: CMTimeMake(value: self.videoFrameCounter, timescale: self.videoFPS))     self.videoWriter = vidWriter     self.videoWriterInput = vidInput     print("Recording Video Stream")   } Save the Video   private func saveRecordingToPhotoLibrary() {     let fileManager = FileManager.default     guard fileManager.fileExists(atPath: self.path) else {       print("Error: The file: \(self.path) not exists, so cannot move this file camera roll")       return     }     print("The file: \(self.path) has been save into documents folder, and is ready to be moved to camera roll") This is what Fails     PHPhotoLibrary.shared().performChanges({       PHAssetChangeRequest.creationRequestForAssetFromVideo(atFileURL: URL(fileURLWithPath: self.path))     }) { completed, error in       guard completed else {         print ("Error: Cannot move the video \(self.path) to camera roll, error: \(String(describing: error?.localizedDescription))")         return       }       print("Video \(self.path) has been moved to camera roll")     }   }  When Recording ends we save the video   private func endRecording() {     guard let vidInput = videoWriterInput, let vidWriter = videoWriter else {       print("Error, no video writer or video input")       return     }     vidInput.markAsFinished()     if !vidInput.isReadyForMoreMediaData {       vidWriter.finishWriting {         print("Finished Recording")         guard vidWriter.status == .completed else {           print("Warning: The Video Writer status is not completed, status: \(vidWriter.status.rawValue)")           print(vidWriter.error.debugDescription)           return         }         print("VideoWriter status is completed")         self.saveRecordingToPhotoLibrary()       }     }   }
Replies
3
Boosts
0
Views
7.5k
Activity
Dec ’21