ReplayKit

RSS for tag

Record or stream video from the screen and audio from the app and microphone using ReplayKit.

Posts under ReplayKit tag

111 Posts

Post

Replies

Boosts

Views

Activity

Capturing External Object Images via Vision Pro Passthrough Camera with Enterprise APIs
We are currently working with the Enterprise APIs for visionOS 2 and have successfully obtained the necessary entitlements for passthrough camera access. Our goal is to capture images of external real-world objects using the passthrough camera of the Vision Pro, not just take screenshots or screen captures. Our specific use case involves: 1. Accessing the raw passthrough camera feed. 2. Capturing high-resolution images of objects in the real world through the camera. 3. Processing and saving these images for further analysis within our custom enterprise app. We would greatly appreciate any guidance, tutorials, or sample code that could help us achieve this functionality. If there are specific APIs or best practices for handling real-world image capture via passthrough cameras with the Enterprise APIs, please let us know.
0
0
675
Oct ’24
RPScreenRecorder startCapture issues on generated file
Hello all, This is my first post on the developer forums. I am developing an app that records the screen of my app, using AVAssetWriter and RPScreenRecorder startCapture. Everything is working as it should on most cases. There are some seemingly random times where the file generated is of some kb and it is corrupted. There seems to be no pattern on what the device is or the iOS version is. It can happen on various phones and iOS versions. The steps I have followed in order to create the file are: configuring the AssetWritter videoAssetWriter = try? AVAssetWriter(outputURL: url!, fileType: AVFileType.mp4) let size = UIScreen.main.bounds.size let width = (Int(size.width / 4)) * 4 let height = (Int(size.height / 4)) * 4 let videoOutputSettings: Dictionary<String, Any> = [ AVVideoCodecKey : AVVideoCodecType.h264, AVVideoWidthKey : width, AVVideoHeightKey : height ] videoInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: videoOutputSettings) videoInput?.expectsMediaDataInRealTime = true guard let videoInput = videoInput else { return } if videoAssetWriter?.canAdd(videoInput) ?? false { videoAssetWriter?.add(videoInput) } let audioInputsettings = [ AVFormatIDKey: Int(kAudioFormatMPEG4AAC), AVSampleRateKey: 12000, AVNumberOfChannelsKey: 1, AVEncoderAudioQualityKey: AVAudioQuality.high.rawValue ] audioInput = AVAssetWriterInput(mediaType: .audio, outputSettings: audioInputsettings) audioInput?.expectsMediaDataInRealTime = true guard let audioInput = audioInput else { return } if videoAssetWriter?.canAdd(audioInput) ?? false { videoAssetWriter?.add(audioInput) } The urlForVideo returns the URL to the documentDirectory, after appending and creating the folders needed. This part seems to be working as it should as the directories are created and the video file exists on them. Start the recording if RPScreenRecorder.shared().isRecording { return } RPScreenRecorder.shared().startCapture(handler: { [weak self] sample, bufferType, error in if let error = error { onError?(error.localizedDescription) } else { if (!RPScreenRecorder.shared().isMicrophoneEnabled) { RPScreenRecorder.shared().stopCapture { error in if let error = error { return } } onError?("Microphone was not enabled") } else { succesCompletion?() succesCompletion = nil self?.processSampleBuffer(sample, with: bufferType) } } }) { error in if let error = error { onError?(error.localizedDescription) } } Process the sampleBuffers guard CMSampleBufferDataIsReady(sampleBuffer) else { return } DispatchQueue.main.async { [weak self] in switch sampleBufferType { case .video: self?.handleVideoBaffer(sampleBuffer) case .audioMic: self?.add(sample: sampleBuffer, to: self?.audioInput) self?.audioInput) default: break } } // The add function from above fileprivate func add(sample: CMSampleBuffer, to writerInput: AVAssetWriterInput?) { if writerInput?.isReadyForMoreMediaData ?? false { writerInput?.append(sample) } // The handleVideoBaffer function from above fileprivate func handleVideoBaffer(_ sampleBuffer: CMSampleBuffer) { if self.videoAssetWriter?.status == AVAssetWriter.Status.unknown { self.videoAssetWriter?.startWriting() self.videoAssetWriter?.startSession(atSourceTime: CMSampleBufferGetPresentationTimeStamp(sampleBuffer)) } else { if (self.videoInput?.isReadyForMoreMediaData) ?? false { if self.videoAssetWriter?.status == AVAssetWriter.Status.writing { self.videoInput?.append(sampleBuffer) } } } } } Finally the stop recording func stopRecording(completion: @escaping (URL?, URL?, Error?) -> Void) { RPScreenRecorder.shared().stopCapture { error in if let error = error { completion(nil, nil, error) return } self.finish { videoURL, _ in completion(videoURL, nil, nil) } } } // The finish function mentioned above fileprivate func finish(completion: @escaping (URL?, URL?) -> Void) { let dispatchGroup = DispatchGroup() dispatchGroup.enter() finishRecordVideo { dispatchGroup.leave() } dispatchGroup.notify(queue: .main) { print("Finish with url:\(String(describing: self.urlForVideo()))") completion(self.urlForVideo(), nil) } } // The finishRecordVideo mentioned above fileprivate func finishRecordVideo(completion: @escaping ()-> Void) { videoInput?.markAsFinished() audioInput?.markAsFinished() videoAssetWriter?.finishWriting { if let writer = self.videoAssetWriter { if writer.status == .completed { completion() } else if writer.status == .failed { // Print the error to find out what went wrong if let error = writer.error { print("Video asset writing failed with error: \(error.localizedDescription). Url: \(writer.outputURL.path)") } else { print("Video asset writing failed, but no error description available.") } completion() }else { completion() } } } } What could it be the reason of the corrupted files generated? This issue has never happened to my devices so there is no way to debug using xcode. Also there are no errors popping out on the logs. Can you spot any issues on the code that can create this kind of issue? Do you have any suggestions on the problem at hand? Thanks
0
0
707
Sep ’24
Record Entire Phone Screen Functionality IOS Xcode 15.4
Hi guys. I am currently working on an App where one of the functionalities is a screen recording function. The function should record the ENTIRE phone screen and be able to upload it to a Firestore database. I am currently having issues with the screen recording. I used the ReplayKit package and I am able to screen record just the screen of the App and not the entire phone screen. Essentially, the screen recording function should allow a user to record other apps screens that the user goes to for about 30 seconds. I am unable to figure out how to record the entire screen. Does anybody know way (package, API, or process) for me to achieve such functionality. Thank you in advance, Hanav Modasiya
1
0
777
Aug ’24
IOS BroadcastExtension
Hello, I'm new here, I was developing a screen recording extension for an IOS application, I used the RPSBroadcastSampleHandler livekit as a basis, in tests a few months ago it worked, but after the long wait for publishing authorization the extension stopped working, I noticed which is not just mine but screen sharing from Google Meet, Zoom Mettings and others also don't work, I tested it on iPhone 14 pro and iPhone 6s, nothing worked, the option to select the extension appears but when clicking "start sharing" nothing happens and after a few seconds the sharing button returns to "start sharing", the same behavior in all tested apps, does anyone know what happens? Did you change the way you record and no app has updated? Internal error in IOS? Nothing logs in terminal just doesn't work.
1
0
1.2k
Jul ’24
RPBroadcastSampleHandler crashes while processing payload
We have an app with a broadcast extension with a RPBroadcastSampleHandler. The implementation is working fine, however for quite some users the extension suddenly crashes during the broadcast. The stacktrace stacktrace of the crashing thread always looks like the shortened sample below. (Full crash reports and stack traces are attached to the submitted Feedbacks.) Looking at the stacktrace none of our code is running, just ReplayKit code handling XPC messages at that moment: Thread: #0 0x00000001e2cf342c in __pthread_kill () #1 0x00000001f6a51c0c in pthread_kill () #2 0x00000001a1bfaba0 in abort () #3 0x00000001a9e38588 in malloc_vreport () #4 0x00000001a9e35430 in malloc_zone_error () [...] #18 0x0000000218ac91bc in -[RPBroadcastSampleHandler processPayload:completion:] () #19 0x0000000198b81360 in __NSXPCCONNECTION_IS_CALLING_OUT_TO_EXPORTED_OBJECT_S2__ () Is anyone aware of there issues with ReplayKit? Are there known workarounds? Could anything we're doing affect crashes like this? Would greatly appreciate it if anyone from Apple DTS could look into this and flag the below Feedbacks to the relevant teams! Feedback IDs: FB13949098, FB13949188
0
0
799
Jun ’24
iOS to Android H264 encoding issue.
I'm trying to cast the screen from an iOS device to an Android device. I'm leveraging ReplayKit on iOS to capture the screen and VideoToolbox for compressing the captured video data into H.264 format using CMSampleBuffers. Both iOS and Android are configured for H.264 compression and decompression. While screen casting works flawlessly within the same platform (iOS to iOS or Android to Android), I'm encountering an error ("not in avi mode") on the Android receiver when casting from iOS. My research suggests that the underlying container formats for H.264 might differ between iOS and Android. Data transmission over the TCP socket seems to be functioning correctly. My question is: Is there a way to ensure a common container format for H.264 compression and decompression across iOS and Android platforms? Here's a breakdown of the iOS sender details: Device: iPhone 13 mini running iOS 17 Development Environment: Xcode 15 with a minimum deployment target of iOS 16 Screen Capture: ReplayKit for capturing the screen and obtaining CMSampleBuffers Video Compression: VideoToolbox for H.264 compression Compression Properties: kVTCompressionPropertyKey_ConstantBitRate: 6144000 (bitrate) kVTCompressionPropertyKey_ProfileLevel: kVTProfileLevel_H264_Main_AutoLevel (profile and level) kVTCompressionPropertyKey_MaxKeyFrameInterval: 60 (maximum keyframe interval) kVTCompressionPropertyKey_RealTime: true (real-time encoding) kVTCompressionPropertyKey_Quality: 1 (lowest quality) NAL Unit Handling: Custom header is added to NAL units Android Receiver Details: Device: RedMi 7A running Android 10 Video Decoding: MediaCodec API for receiving and decoding the H.264 stream
0
0
1.3k
May ’24
Replaykit Broadcast finishing unexpectedly:Attempted to start an invalid broadcast session
I'm currently working on live screen broadcasting app which allows the user's to record their screen to save a mp4 video. I write video file by AVAssetWriter, and it works fine. But, when there is 1GB-2BG of storage space remaining on the device, errors such as "Attempted to start an invalid broadcast session" frequently occur, and video files cannot be played due to not call assetWriter.finishWriting(). Occur on device: iPhone se3 iPhone 12 pro max iPhone 13 iPad 19 iPad air 5 I have tried the movieFragmentInterval of AVAssetWriter to write movie fragments , set shouldOptimizeForNetworkUse true/false , not working.The video can not be played. I want to known how to observe or catch this error? Thanks!
0
0
777
Jan ’24
Replaykit sometimes will be auto stop screen record !!!
I just used replaykit to achieve simple screen recording,But sometimes there may be frequent occurrences of automatic stopping of recording, i don`t why ? have anyone konw this bug ? use Api: func startCapture(handler captureHandler: ((CMSampleBuffer, RPSampleBufferType, Error?) -> Void)?, completionHandler: ((Error?) -> Void)? = nil) open func stopCapture(handler: ((Error?) -> Void)? = nil) `` The prerequisite for monitoring the automatic stop recording method is that you have already started screen recording and have not actively called to stop recording let publisher = recorder.publisher(for: .isRecording) let cancelBag = Subscribers.Sink<Bool, Never>(receiveCompletion: {[weak self] complete in /// }, receiveValue: {[weak self] value in /// }) publisher.subscribe(cancelBag)
0
0
697
Nov ’23
RPScreenRecorder: Recording video
Hello everyone. I am trying to make a small utility that, in the context of digital forensics, logs the desktop. The utility is to be started via shell like this : "./nemeapp start path_to_file" and be terminated in "./nemeapp stop". The code I wrote is: import Foundation import ReplayKit let arguments = CommandLine.arguments guard arguments.count == 4 else { print("Utilizzo: nome_script start|stop percorso_file include_audio(true|false)") exit(0) } let command = arguments[1] let filePath = arguments[2] let includeAudio = arguments[3] == "true" switch command { case "start": startScreenRecording(filePath: filePath, includeAudio: includeAudio) case "stop": stopScreenRecording() default: print("Comando non riconosciuto. Utilizzo: nome_script start|stop percorso_file include_audio(true|false)") } func startScreenRecording(filePath: String, includeAudio: Bool) { if RPScreenRecorder.shared().isAvailable { RPScreenRecorder.shared().startRecording(handler: { error in if let unwrappedError = error { print("Errore durante l'avvio della registrazione: \(unwrappedError.localizedDescription)") } else { print("La registrazione dello schermo è stata avviata correttamente. Il file verrà salvato in: \(filePath)") } }) } else { print("La registrazione dello schermo non è disponibile.") } } func stopScreenRecording() { RPScreenRecorder.shared().stopRecording { previewViewController, error in if let unwrappedError = error { print("Errore durante l'arresto della registrazione: \(unwrappedError.localizedDescription)") } else { print("La registrazione dello schermo è stata interrotta correttamente.") } } } Unfortunately, the code returns no error message. Only when I give the stop command does it tell me that the registration never started. I can't even figure out if it is a permissions issue.
0
0
806
Oct ’23
iOS 17 RPSystemBroadcastPickerView not working
My existing code is working properly in iOS < 17 devices it records the iPhone screen and records audio as well simultaneously, but in iOS 17 devices the screen recording video is captured for only 2 seconds and then stops automatically, As its an extension, i don't have logs to debug the issue. I have tested the same code in other iPhones and OS less than 17, its working fine but in iOS 17 devices this issue is coming. @try { NSLog(@“initAssesWriter”); NSError *error = nil; CGRect screenRect = [[UIScreen mainScreen] bounds]; _videoWriter = [[AVAssetWriter alloc] initWithURL: _filePath fileType:AVFileTypeMPEG4 error:&error]; NSParameterAssert(_videoWriter); //Configure video NSDictionary* videoCompressionProps = [NSDictionary dictionaryWithObjectsAndKeys: [NSNumber numberWithDouble:2048*1024.0], AVVideoAverageBitRateKey, nil ]; NSDictionary* videoSettings = [NSDictionary dictionaryWithObjectsAndKeys: AVVideoCodecTypeH264, AVVideoCodecKey, [NSNumber numberWithInt:screenRect.size.width * 4], AVVideoWidthKey, [NSNumber numberWithInt:screenRect.size.height * 4], AVVideoHeightKey, videoCompressionProps, AVVideoCompressionPropertiesKey, nil]; _writerInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo outputSettings:videoSettings] ; _writerInput.expectsMediaDataInRealTime = YES; NSParameterAssert(_writerInput); NSParameterAssert([_videoWriter canAddInput:_writerInput]); [_videoWriter addInput:_writerInput]; AudioChannelLayout acl; bzero( &acl, sizeof(acl)); acl.mChannelLayoutTag = kAudioChannelLayoutTag_Mono; NSDictionary* audioOutputSettings = [NSDictionary dictionaryWithObjectsAndKeys: [ NSNumber numberWithInt: kAudioFormatMPEG4AAC], AVFormatIDKey, [ NSNumber numberWithInt: 1 ], AVNumberOfChannelsKey, [ NSNumber numberWithFloat: 44100.0 ], AVSampleRateKey, [ NSData dataWithBytes: &acl length: sizeof( AudioChannelLayout ) ], AVChannelLayoutKey, [ NSNumber numberWithInt: 64000 ], AVEncoderBitRateKey, nil]; _audioWriterInput = [AVAssetWriterInput assetWriterInputWithMediaType: AVMediaTypeAudio outputSettings: audioOutputSettings ]; _audioWriterInput.expectsMediaDataInRealTime = YES; // seems to work slightly better NSParameterAssert(_audioWriterInput); NSParameterAssert([_videoWriter canAddInput:_audioWriterInput]); [_videoWriter addInput:_audioWriterInput]; [_videoWriter setMovieFragmentInterval:CMTimeMake(1, 600)]; [_videoWriter startWriting]; } @catch (NSException *exception) { } @finally { } -(void)processSampleBuffer:(CMSampleBufferRef)sampleBuffer withType:(RPSampleBufferType)sampleBufferType{ @try { if(!_isRecordingStarted){ [_videoWriter startSessionAtSourceTime:CMSampleBufferGetPresentationTimeStamp(sampleBuffer)]; _isRecordingStarted = YES; [self saveFlurryLogs:@"Assest writer Start Recording" Details:@""]; NSLog(@"CMSampleBufferGetPresentationTimeStamp"); } } @catch (NSException *exception) { [self saveFlurryLogs:@"Recording Start Execption" Details:exception.description]; } @finally { } @try { switch (sampleBufferType) { case RPSampleBufferTypeVideo: // Handle video sample buffer if([_writerInput isReadyForMoreMediaData]){ [_writerInput appendSampleBuffer:sampleBuffer]; NSLog(@"writing matadata Video"); } break; case RPSampleBufferTypeAudioApp: // Handle audio sample buffer for app audio break; case RPSampleBufferTypeAudioMic: if([_audioWriterInput isReadyForMoreMediaData]){ [_audioWriterInput appendSampleBuffer:sampleBuffer]; NSLog(@"writing matadata Audio"); } // Handle audio sample buffer for mic audio break; default: break; } } @catch (NSException *exception) { [self saveFlurryLogs:@"Packet Write Execption" Details:exception.description]; } @finally { } }
1
1
1.3k
Oct ’23
ReplayKit and kAudioUnitSubType_VoiceProcessingIO
We find that ReplayKit as of iOS 9.3.2 (haven't tested in iOS 10) is unable to record audio output when the audio unit sub type is kAudioUnitSubType_VoiceProcessingIO and ReplayKit is used via startRecordingWithMicrophoneEnabled(false). When we switch to using sub type kAudioUnitSubType_RemoteIO, ReplayKit works as expected. This is for a live video app where viewers receive live video and audio. We do not use AVPlayer (AVPlayer is the only incompatibility listed in the docs).Are there any workarounds that will allow use of kAudioUnitSubType_VoiceProcessingIO with ReplayKit?Thanks,Robert
2
0
1.5k
Jun ’23
Capturing External Object Images via Vision Pro Passthrough Camera with Enterprise APIs
We are currently working with the Enterprise APIs for visionOS 2 and have successfully obtained the necessary entitlements for passthrough camera access. Our goal is to capture images of external real-world objects using the passthrough camera of the Vision Pro, not just take screenshots or screen captures. Our specific use case involves: 1. Accessing the raw passthrough camera feed. 2. Capturing high-resolution images of objects in the real world through the camera. 3. Processing and saving these images for further analysis within our custom enterprise app. We would greatly appreciate any guidance, tutorials, or sample code that could help us achieve this functionality. If there are specific APIs or best practices for handling real-world image capture via passthrough cameras with the Enterprise APIs, please let us know.
Replies
0
Boosts
0
Views
675
Activity
Oct ’24
RPScreenRecorder startCapture issues on generated file
Hello all, This is my first post on the developer forums. I am developing an app that records the screen of my app, using AVAssetWriter and RPScreenRecorder startCapture. Everything is working as it should on most cases. There are some seemingly random times where the file generated is of some kb and it is corrupted. There seems to be no pattern on what the device is or the iOS version is. It can happen on various phones and iOS versions. The steps I have followed in order to create the file are: configuring the AssetWritter videoAssetWriter = try? AVAssetWriter(outputURL: url!, fileType: AVFileType.mp4) let size = UIScreen.main.bounds.size let width = (Int(size.width / 4)) * 4 let height = (Int(size.height / 4)) * 4 let videoOutputSettings: Dictionary<String, Any> = [ AVVideoCodecKey : AVVideoCodecType.h264, AVVideoWidthKey : width, AVVideoHeightKey : height ] videoInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: videoOutputSettings) videoInput?.expectsMediaDataInRealTime = true guard let videoInput = videoInput else { return } if videoAssetWriter?.canAdd(videoInput) ?? false { videoAssetWriter?.add(videoInput) } let audioInputsettings = [ AVFormatIDKey: Int(kAudioFormatMPEG4AAC), AVSampleRateKey: 12000, AVNumberOfChannelsKey: 1, AVEncoderAudioQualityKey: AVAudioQuality.high.rawValue ] audioInput = AVAssetWriterInput(mediaType: .audio, outputSettings: audioInputsettings) audioInput?.expectsMediaDataInRealTime = true guard let audioInput = audioInput else { return } if videoAssetWriter?.canAdd(audioInput) ?? false { videoAssetWriter?.add(audioInput) } The urlForVideo returns the URL to the documentDirectory, after appending and creating the folders needed. This part seems to be working as it should as the directories are created and the video file exists on them. Start the recording if RPScreenRecorder.shared().isRecording { return } RPScreenRecorder.shared().startCapture(handler: { [weak self] sample, bufferType, error in if let error = error { onError?(error.localizedDescription) } else { if (!RPScreenRecorder.shared().isMicrophoneEnabled) { RPScreenRecorder.shared().stopCapture { error in if let error = error { return } } onError?("Microphone was not enabled") } else { succesCompletion?() succesCompletion = nil self?.processSampleBuffer(sample, with: bufferType) } } }) { error in if let error = error { onError?(error.localizedDescription) } } Process the sampleBuffers guard CMSampleBufferDataIsReady(sampleBuffer) else { return } DispatchQueue.main.async { [weak self] in switch sampleBufferType { case .video: self?.handleVideoBaffer(sampleBuffer) case .audioMic: self?.add(sample: sampleBuffer, to: self?.audioInput) self?.audioInput) default: break } } // The add function from above fileprivate func add(sample: CMSampleBuffer, to writerInput: AVAssetWriterInput?) { if writerInput?.isReadyForMoreMediaData ?? false { writerInput?.append(sample) } // The handleVideoBaffer function from above fileprivate func handleVideoBaffer(_ sampleBuffer: CMSampleBuffer) { if self.videoAssetWriter?.status == AVAssetWriter.Status.unknown { self.videoAssetWriter?.startWriting() self.videoAssetWriter?.startSession(atSourceTime: CMSampleBufferGetPresentationTimeStamp(sampleBuffer)) } else { if (self.videoInput?.isReadyForMoreMediaData) ?? false { if self.videoAssetWriter?.status == AVAssetWriter.Status.writing { self.videoInput?.append(sampleBuffer) } } } } } Finally the stop recording func stopRecording(completion: @escaping (URL?, URL?, Error?) -> Void) { RPScreenRecorder.shared().stopCapture { error in if let error = error { completion(nil, nil, error) return } self.finish { videoURL, _ in completion(videoURL, nil, nil) } } } // The finish function mentioned above fileprivate func finish(completion: @escaping (URL?, URL?) -> Void) { let dispatchGroup = DispatchGroup() dispatchGroup.enter() finishRecordVideo { dispatchGroup.leave() } dispatchGroup.notify(queue: .main) { print("Finish with url:\(String(describing: self.urlForVideo()))") completion(self.urlForVideo(), nil) } } // The finishRecordVideo mentioned above fileprivate func finishRecordVideo(completion: @escaping ()-> Void) { videoInput?.markAsFinished() audioInput?.markAsFinished() videoAssetWriter?.finishWriting { if let writer = self.videoAssetWriter { if writer.status == .completed { completion() } else if writer.status == .failed { // Print the error to find out what went wrong if let error = writer.error { print("Video asset writing failed with error: \(error.localizedDescription). Url: \(writer.outputURL.path)") } else { print("Video asset writing failed, but no error description available.") } completion() }else { completion() } } } } What could it be the reason of the corrupted files generated? This issue has never happened to my devices so there is no way to debug using xcode. Also there are no errors popping out on the logs. Can you spot any issues on the code that can create this kind of issue? Do you have any suggestions on the problem at hand? Thanks
Replies
0
Boosts
0
Views
707
Activity
Sep ’24
Screen Mirroring iOS to Android Smart TV
Hi, i have try with RPBroadcastSampleHandler Broadcast Extension and RPSystemBroadcastPickerView but i don't understand how can i mirror my iOS screen to Android smart TV?
Replies
0
Boosts
0
Views
905
Activity
Sep ’24
Record Entire Phone Screen Functionality IOS Xcode 15.4
Hi guys. I am currently working on an App where one of the functionalities is a screen recording function. The function should record the ENTIRE phone screen and be able to upload it to a Firestore database. I am currently having issues with the screen recording. I used the ReplayKit package and I am able to screen record just the screen of the App and not the entire phone screen. Essentially, the screen recording function should allow a user to record other apps screens that the user goes to for about 30 seconds. I am unable to figure out how to record the entire screen. Does anybody know way (package, API, or process) for me to achieve such functionality. Thank you in advance, Hanav Modasiya
Replies
1
Boosts
0
Views
777
Activity
Aug ’24
iOS Replaykit exportClipToURL has poor quality video
When I use ReplayKit's exportClipToURL function on iOS to capture a 15-second replay, the resulting video quality is poor, with snowy artifacts on the damaged visuals and audio distortion.
Replies
1
Boosts
0
Views
751
Activity
Jul ’24
IOS BroadcastExtension
Hello, I'm new here, I was developing a screen recording extension for an IOS application, I used the RPSBroadcastSampleHandler livekit as a basis, in tests a few months ago it worked, but after the long wait for publishing authorization the extension stopped working, I noticed which is not just mine but screen sharing from Google Meet, Zoom Mettings and others also don't work, I tested it on iPhone 14 pro and iPhone 6s, nothing worked, the option to select the extension appears but when clicking "start sharing" nothing happens and after a few seconds the sharing button returns to "start sharing", the same behavior in all tested apps, does anyone know what happens? Did you change the way you record and no app has updated? Internal error in IOS? Nothing logs in terminal just doesn't work.
Replies
1
Boosts
0
Views
1.2k
Activity
Jul ’24
How to disable screen record/capture?
how do I stop screen recorder/capture on iOS 14. have any native API for this?. have privacy/security concern for my app so kindly help me. Thanks
Replies
1
Boosts
0
Views
934
Activity
Jun ’24
RPBroadcastSampleHandler crashes while processing payload
We have an app with a broadcast extension with a RPBroadcastSampleHandler. The implementation is working fine, however for quite some users the extension suddenly crashes during the broadcast. The stacktrace stacktrace of the crashing thread always looks like the shortened sample below. (Full crash reports and stack traces are attached to the submitted Feedbacks.) Looking at the stacktrace none of our code is running, just ReplayKit code handling XPC messages at that moment: Thread: #0 0x00000001e2cf342c in __pthread_kill () #1 0x00000001f6a51c0c in pthread_kill () #2 0x00000001a1bfaba0 in abort () #3 0x00000001a9e38588 in malloc_vreport () #4 0x00000001a9e35430 in malloc_zone_error () [...] #18 0x0000000218ac91bc in -[RPBroadcastSampleHandler processPayload:completion:] () #19 0x0000000198b81360 in __NSXPCCONNECTION_IS_CALLING_OUT_TO_EXPORTED_OBJECT_S2__ () Is anyone aware of there issues with ReplayKit? Are there known workarounds? Could anything we're doing affect crashes like this? Would greatly appreciate it if anyone from Apple DTS could look into this and flag the below Feedbacks to the relevant teams! Feedback IDs: FB13949098, FB13949188
Replies
0
Boosts
0
Views
799
Activity
Jun ’24
iOS to Android H264 encoding issue.
I'm trying to cast the screen from an iOS device to an Android device. I'm leveraging ReplayKit on iOS to capture the screen and VideoToolbox for compressing the captured video data into H.264 format using CMSampleBuffers. Both iOS and Android are configured for H.264 compression and decompression. While screen casting works flawlessly within the same platform (iOS to iOS or Android to Android), I'm encountering an error ("not in avi mode") on the Android receiver when casting from iOS. My research suggests that the underlying container formats for H.264 might differ between iOS and Android. Data transmission over the TCP socket seems to be functioning correctly. My question is: Is there a way to ensure a common container format for H.264 compression and decompression across iOS and Android platforms? Here's a breakdown of the iOS sender details: Device: iPhone 13 mini running iOS 17 Development Environment: Xcode 15 with a minimum deployment target of iOS 16 Screen Capture: ReplayKit for capturing the screen and obtaining CMSampleBuffers Video Compression: VideoToolbox for H.264 compression Compression Properties: kVTCompressionPropertyKey_ConstantBitRate: 6144000 (bitrate) kVTCompressionPropertyKey_ProfileLevel: kVTProfileLevel_H264_Main_AutoLevel (profile and level) kVTCompressionPropertyKey_MaxKeyFrameInterval: 60 (maximum keyframe interval) kVTCompressionPropertyKey_RealTime: true (real-time encoding) kVTCompressionPropertyKey_Quality: 1 (lowest quality) NAL Unit Handling: Custom header is added to NAL units Android Receiver Details: Device: RedMi 7A running Android 10 Video Decoding: MediaCodec API for receiving and decoding the H.264 stream
Replies
0
Boosts
0
Views
1.3k
Activity
May ’24
how can replaykit recognizes that another app on my phone is turning on recording
I have the conflict when i use replaykit. There is other app or system use the recording. I want my app can recognize them and let them use the replaykit preferentially.
Replies
0
Boosts
0
Views
743
Activity
Apr ’24
Screen Recording Immersive Space in VisionOS
I am using ReplayKit's RPScreenRecorder to record my app. When I use it in a mixed immersive space, nothing is actually recorded. The video is entirely blank. Is this a feature or a bug? I am trying to record everything the user sees, including passthrough. Is there another way to do this?
Replies
0
Boosts
0
Views
842
Activity
Mar ’24
Replaykit Broadcast finishing unexpectedly:Attempted to start an invalid broadcast session
I'm currently working on live screen broadcasting app which allows the user's to record their screen to save a mp4 video. I write video file by AVAssetWriter, and it works fine. But, when there is 1GB-2BG of storage space remaining on the device, errors such as "Attempted to start an invalid broadcast session" frequently occur, and video files cannot be played due to not call assetWriter.finishWriting(). Occur on device: iPhone se3 iPhone 12 pro max iPhone 13 iPad 19 iPad air 5 I have tried the movieFragmentInterval of AVAssetWriter to write movie fragments , set shouldOptimizeForNetworkUse true/false , not working.The video can not be played. I want to known how to observe or catch this error? Thanks!
Replies
0
Boosts
0
Views
777
Activity
Jan ’24
Replaykit sometimes will be auto stop screen record !!!
I just used replaykit to achieve simple screen recording,But sometimes there may be frequent occurrences of automatic stopping of recording, i don`t why ? have anyone konw this bug ? use Api: func startCapture(handler captureHandler: ((CMSampleBuffer, RPSampleBufferType, Error?) -> Void)?, completionHandler: ((Error?) -> Void)? = nil) open func stopCapture(handler: ((Error?) -> Void)? = nil) `` The prerequisite for monitoring the automatic stop recording method is that you have already started screen recording and have not actively called to stop recording let publisher = recorder.publisher(for: .isRecording) let cancelBag = Subscribers.Sink<Bool, Never>(receiveCompletion: {[weak self] complete in /// }, receiveValue: {[weak self] value in /// }) publisher.subscribe(cancelBag)
Replies
0
Boosts
0
Views
697
Activity
Nov ’23
RPScreenRecorder: Recording video
Hello everyone. I am trying to make a small utility that, in the context of digital forensics, logs the desktop. The utility is to be started via shell like this : "./nemeapp start path_to_file" and be terminated in "./nemeapp stop". The code I wrote is: import Foundation import ReplayKit let arguments = CommandLine.arguments guard arguments.count == 4 else { print("Utilizzo: nome_script start|stop percorso_file include_audio(true|false)") exit(0) } let command = arguments[1] let filePath = arguments[2] let includeAudio = arguments[3] == "true" switch command { case "start": startScreenRecording(filePath: filePath, includeAudio: includeAudio) case "stop": stopScreenRecording() default: print("Comando non riconosciuto. Utilizzo: nome_script start|stop percorso_file include_audio(true|false)") } func startScreenRecording(filePath: String, includeAudio: Bool) { if RPScreenRecorder.shared().isAvailable { RPScreenRecorder.shared().startRecording(handler: { error in if let unwrappedError = error { print("Errore durante l'avvio della registrazione: \(unwrappedError.localizedDescription)") } else { print("La registrazione dello schermo è stata avviata correttamente. Il file verrà salvato in: \(filePath)") } }) } else { print("La registrazione dello schermo non è disponibile.") } } func stopScreenRecording() { RPScreenRecorder.shared().stopRecording { previewViewController, error in if let unwrappedError = error { print("Errore durante l'arresto della registrazione: \(unwrappedError.localizedDescription)") } else { print("La registrazione dello schermo è stata interrotta correttamente.") } } } Unfortunately, the code returns no error message. Only when I give the stop command does it tell me that the registration never started. I can't even figure out if it is a permissions issue.
Replies
0
Boosts
0
Views
806
Activity
Oct ’23
same problem with HLS Video not working on 16.4 simulator and 17.0.1 and at the device with 17.0.3
we have just a black screen and don't have audio and video !
Replies
0
Boosts
0
Views
761
Activity
Oct ’23
iOS 17 RPSystemBroadcastPickerView not working
My existing code is working properly in iOS < 17 devices it records the iPhone screen and records audio as well simultaneously, but in iOS 17 devices the screen recording video is captured for only 2 seconds and then stops automatically, As its an extension, i don't have logs to debug the issue. I have tested the same code in other iPhones and OS less than 17, its working fine but in iOS 17 devices this issue is coming. @try { NSLog(@“initAssesWriter”); NSError *error = nil; CGRect screenRect = [[UIScreen mainScreen] bounds]; _videoWriter = [[AVAssetWriter alloc] initWithURL: _filePath fileType:AVFileTypeMPEG4 error:&error]; NSParameterAssert(_videoWriter); //Configure video NSDictionary* videoCompressionProps = [NSDictionary dictionaryWithObjectsAndKeys: [NSNumber numberWithDouble:2048*1024.0], AVVideoAverageBitRateKey, nil ]; NSDictionary* videoSettings = [NSDictionary dictionaryWithObjectsAndKeys: AVVideoCodecTypeH264, AVVideoCodecKey, [NSNumber numberWithInt:screenRect.size.width * 4], AVVideoWidthKey, [NSNumber numberWithInt:screenRect.size.height * 4], AVVideoHeightKey, videoCompressionProps, AVVideoCompressionPropertiesKey, nil]; _writerInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo outputSettings:videoSettings] ; _writerInput.expectsMediaDataInRealTime = YES; NSParameterAssert(_writerInput); NSParameterAssert([_videoWriter canAddInput:_writerInput]); [_videoWriter addInput:_writerInput]; AudioChannelLayout acl; bzero( &acl, sizeof(acl)); acl.mChannelLayoutTag = kAudioChannelLayoutTag_Mono; NSDictionary* audioOutputSettings = [NSDictionary dictionaryWithObjectsAndKeys: [ NSNumber numberWithInt: kAudioFormatMPEG4AAC], AVFormatIDKey, [ NSNumber numberWithInt: 1 ], AVNumberOfChannelsKey, [ NSNumber numberWithFloat: 44100.0 ], AVSampleRateKey, [ NSData dataWithBytes: &acl length: sizeof( AudioChannelLayout ) ], AVChannelLayoutKey, [ NSNumber numberWithInt: 64000 ], AVEncoderBitRateKey, nil]; _audioWriterInput = [AVAssetWriterInput assetWriterInputWithMediaType: AVMediaTypeAudio outputSettings: audioOutputSettings ]; _audioWriterInput.expectsMediaDataInRealTime = YES; // seems to work slightly better NSParameterAssert(_audioWriterInput); NSParameterAssert([_videoWriter canAddInput:_audioWriterInput]); [_videoWriter addInput:_audioWriterInput]; [_videoWriter setMovieFragmentInterval:CMTimeMake(1, 600)]; [_videoWriter startWriting]; } @catch (NSException *exception) { } @finally { } -(void)processSampleBuffer:(CMSampleBufferRef)sampleBuffer withType:(RPSampleBufferType)sampleBufferType{ @try { if(!_isRecordingStarted){ [_videoWriter startSessionAtSourceTime:CMSampleBufferGetPresentationTimeStamp(sampleBuffer)]; _isRecordingStarted = YES; [self saveFlurryLogs:@"Assest writer Start Recording" Details:@""]; NSLog(@"CMSampleBufferGetPresentationTimeStamp"); } } @catch (NSException *exception) { [self saveFlurryLogs:@"Recording Start Execption" Details:exception.description]; } @finally { } @try { switch (sampleBufferType) { case RPSampleBufferTypeVideo: // Handle video sample buffer if([_writerInput isReadyForMoreMediaData]){ [_writerInput appendSampleBuffer:sampleBuffer]; NSLog(@"writing matadata Video"); } break; case RPSampleBufferTypeAudioApp: // Handle audio sample buffer for app audio break; case RPSampleBufferTypeAudioMic: if([_audioWriterInput isReadyForMoreMediaData]){ [_audioWriterInput appendSampleBuffer:sampleBuffer]; NSLog(@"writing matadata Audio"); } // Handle audio sample buffer for mic audio break; default: break; } } @catch (NSException *exception) { [self saveFlurryLogs:@"Packet Write Execption" Details:exception.description]; } @finally { } }
Replies
1
Boosts
1
Views
1.3k
Activity
Oct ’23
Excluding selected Views from screen recording using Replaykit
I have in app screen recording functionality in my app and while recording I want to exclude PHPickerViewController/UIDocumentPickerViewController from being recorded if opened in the app during recording. How can I achieve this ? Is there any configuration available in ReplayKit using which I can achieve this behaviour?
Replies
1
Boosts
0
Views
752
Activity
Oct ’23
Screen capture app or API for ios
I want to develop the application where I can able to share screen (display)from iphone(or from iphone to iphone) for remote support(or any other method) for this this I need screen captute application or API or system call. If any app is there in iphone or app store or any screen sharing client which can give me screen capture support.
Replies
0
Boosts
0
Views
904
Activity
Sep ’23
ReplayKit error: Code=-5833 "Failed to start due to audio/video capture failure"
This error occurs when my app exits the background after starting the screen recording operation. If this error occurs, how can it be avoided or solved? Every time, the process is killed to continue to be enabled
Replies
2
Boosts
0
Views
2k
Activity
Jul ’23
ReplayKit and kAudioUnitSubType_VoiceProcessingIO
We find that ReplayKit as of iOS 9.3.2 (haven't tested in iOS 10) is unable to record audio output when the audio unit sub type is kAudioUnitSubType_VoiceProcessingIO and ReplayKit is used via startRecordingWithMicrophoneEnabled(false). When we switch to using sub type kAudioUnitSubType_RemoteIO, ReplayKit works as expected. This is for a live video app where viewers receive live video and audio. We do not use AVPlayer (AVPlayer is the only incompatibility listed in the docs).Are there any workarounds that will allow use of kAudioUnitSubType_VoiceProcessingIO with ReplayKit?Thanks,Robert
Replies
2
Boosts
0
Views
1.5k
Activity
Jun ’23