ReplayKit

RSS for tag

Record or stream video from the screen and audio from the app and microphone using ReplayKit.

Posts under ReplayKit tag

111 Posts

Post

Replies

Boosts

Views

Activity

Where is WWDC 2016 session 601 "Go Live with ReplayKit"?
I'm developing and application that will use ReplayKit's Broadcast API to stream the screen of a iOS device. For that I'll need to create the Client App and the Broadcaster App. The WWDC videos about ReplayKit mention WWDC 2016 session 601 "Go Live with ReplayKit" but I can't find it. Can someone point me to that video or to its transcript?. Also, any other resource about the Broadcast API will be appreciated. Most of the samples and documentation that I've found on the web are for just streaming of the app's own screen. Thank you.
1
0
1.1k
Dec ’21
Broadcast Extension receives broadcastFinished prematurely
I have a Broadcast Extension that's running a system wide screen capture. I can start the broadcast and receive buffers and the like.. Buffers are successfully getting to our back and and distributed to all the peers (webrtc). The problem is when I navigate from the host app to apps that play video with audio. And then try to go landscape, sometimes during the rotation, the system will send a broadcastFinished() message to my RPBroadcastSampleHandler subclass. It doesn't occur all the time, but when it does .. it looks like the rotation glitches some. Hard to explain. But I've noticed that it only sends the finished message when it glitches. I would love to get a screen grab of it occurring, but my capture is running.. A quick google/forum search has revealed not much help. Has anyone heard of such a thing?
0
0
515
Dec ’21
ReplayKit: Select window to record?
I have a simple macOS app, with two windows. I would like the user to be able recored each window using a button on the window. I can record the main window and save that to a file using ReplayKit - so far, so good. But I can only record the main window - even if I have all ReplayKit code in the second NSViewController, its the main widow thats being recorded. It is possible to tell ReplayKit what window to record somehow?
2
0
1.2k
Nov ’21
Is there a way to notify the user of important information during Screen recording?
On iOS15.1, if users set Settings > Notifications > Screen Sharing > Notifications Off, our application cannot display notification during screen recording. I find this setting very useful, but it means that there is no way to display important messages such as no audio. Is there a way around this with any notifications?
0
0
757
Nov ’21
Share the current screen to another user in the call.
My goal is to implement a screen sharing function like Skype, Discord etc. My biggest problem in that implementation is that when you have an active RPScreenRecorder.shared() screen recording and then leave the app, the screen recording stops. For my goal it is necessary to be able to share the screen even if the app is in background. For example: The local user shares his current screen to another user, then exits the app to show the other user his Photo Library in realtime. Problem 1: I have came across ReplayKit and "RPBroadcastSampleHandler", but this extension is only called when I explicitly run the app with the "ScreenShare" scheme that I created. When I try to run the app on the normal/main scheme the extension functions aren't called. Problem 2: I can't call my pod functionalities from the "RPBroadcastSampleHandler" class. It just says "-PodName- not found" when I try to import the pod I need to use. There are no full documentations on that topic and I would really appreciate anybody who can explain to me what I am doing wrong.
0
0
854
Nov ’21
AVAssetWriterInput.append giving error "An Unknown Error Occurred (-12706)"
I am simply trying to capture the screen, app audio and mic audio. The app audio and mic audio independently work fine but when combined some unknown error is thrown. Following are the methods to start the capture of screen and processSampleBuffer. Method to setup writers for screen capture func startCapture() { _filename = UUID().uuidString let videoPath = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!.appendingPathComponent("\(_filename).mp4") let writer = try! AVAssetWriter(outputURL: videoPath, fileType: .mp4) let screen = UIScreen.main.bounds let screenBounds = screen.size let videoCompressionPropertys = [ AVVideoAverageBitRateKey: screenBounds.width * screenBounds.height * 10.1 ] let videoSettings: [String: Any] = [ AVVideoCodecKey: AVVideoCodecType.h264, AVVideoWidthKey: screenBounds.width, AVVideoHeightKey: screenBounds.height, AVVideoCompressionPropertiesKey: videoCompressionPropertys ] let input = AVAssetWriterInput(mediaType: .video, outputSettings: videoSettings) input.expectsMediaDataInRealTime = true if writer.canAdd(input) { writer.add(input) } // Add the app audio input var acl = AudioChannelLayout() memset(&acl, 0, MemoryLayout<AudioChannelLayout>.size) acl.mChannelLayoutTag = kAudioChannelLayoutTag_Mono; let audioOutputSettings: [String: Any] = [ AVFormatIDKey: kAudioFormatMPEG4AAC, AVSampleRateKey : 44100, AVNumberOfChannelsKey : 1, AVEncoderBitRateKey : 128000, AVChannelLayoutKey : Data(bytes: &acl, count: MemoryLayout<AudioChannelLayout>.size)] let audioInput = AVAssetWriterInput(mediaType: AVMediaType.audio, outputSettings: audioOutputSettings) audioInput.expectsMediaDataInRealTime = true if (writer.canAdd(audioInput)) { writer.add(audioInput) } // Add the mic audio input let audioOutputSettings1: [String: Any] = [ AVFormatIDKey: kAudioFormatMPEG4AAC, AVSampleRateKey : 24000, AVNumberOfChannelsKey : 1, AVEncoderAudioQualityKey: AVAudioQuality.high.rawValue] let micAudioInput = AVAssetWriterInput(mediaType: AVMediaType.audio, outputSettings: audioOutputSettings1) micAudioInput.expectsMediaDataInRealTime = true if (writer.canAdd(micAudioInput)) { writer.add(micAudioInput) } writer.startWriting() _audioAssetWriterInput = audioInput _micAssetWriterInput = micAudioInput _assetWriterInput = input _assetWriter = writer } processSampleBuffer override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) { if startTime == nil { startTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer) _assetWriter!.startSession(atSourceTime: CMTime.zero) } if sampleBufferType == RPSampleBufferType.video { if let _assetWriterInput = _assetWriterInput { if _assetWriterInput.isReadyForMoreMediaData { let appended = _assetWriterInput.append(sampleBuffer) print(appended) if !appended { let status = _assetWriter?.status let error = _assetWriter?.error print("cannot append video") } } } } if sampleBufferType == RPSampleBufferType.audioApp { if let _assetWriterInput = _audioAssetWriterInput { if _assetWriterInput.isReadyForMoreMediaData { let appended = _assetWriterInput.append(sampleBuffer) print(appended) if !appended { let status = _assetWriter?.status let error = _assetWriter?.error print("cannot append app audio") } } } } if sampleBufferType == RPSampleBufferType.audioMic { if let _assetWriterInput = _micAssetWriterInput { if _assetWriterInput.isReadyForMoreMediaData { let appended = _assetWriterInput.append(sampleBuffer) print(appended) if !appended { let status = _assetWriter?.status let error = _assetWriter?.error print("cannot append mic audio") } } } } if shouldEnd { _finishWriters() } }
0
0
1.2k
Oct ’21
anyone know how to rotate the CMSampleBuffer to landscape
Has anyone figured this out.It seems replaykit 2 always delevers frames at a fixed resolution of 886 x 1918I've tried various solutions to rotate the screen but so far none work.Using the code below crashes in the rotate stepvImageRotate90_ARGB8888(&amp;srcBuffer, &amp;destBuffer, factor, &amp;color, vImage_Flags(0)) let flags = CVPixelBufferLockFlags(rawValue: 0) guard kCVReturnSuccess == CVPixelBufferLockBaseAddress(srcPixelBuffer, flags) else { return nil } defer { CVPixelBufferUnlockBaseAddress(srcPixelBuffer, flags) } guard let srcData = CVPixelBufferGetBaseAddress(srcPixelBuffer) else { print("Error: could not get pixel buffer base address") return nil } let sourceWidth = CVPixelBufferGetWidth(srcPixelBuffer) let sourceHeight = CVPixelBufferGetHeight(srcPixelBuffer) var destWidth = sourceHeight var destHeight = sourceWidth var color = UInt8(0) if factor % 2 == 0 { destWidth = sourceWidth destHeight = sourceHeight } let srcBytesPerRow = CVPixelBufferGetBytesPerRow(srcPixelBuffer) var srcBuffer = vImage_Buffer(data: srcData, height: vImagePixelCount(sourceHeight), width: vImagePixelCount(sourceWidth), rowBytes: srcBytesPerRow) let destBytesPerRow = destWidth*4 guard let destData = malloc(destHeight*destBytesPerRow) else { print("Error: out of memory") return nil } var destBuffer = vImage_Buffer(data: destData, height: vImagePixelCount(destHeight), width: vImagePixelCount(destWidth), rowBytes: destBytesPerRow) let error = vImageRotate90_ARGB8888(&amp;srcBuffer, &amp;destBuffer, factor, &amp;color, vImage_Flags(0)) if error != kvImageNoError { print("Error:", error) free(destData)
3
0
5.7k
Oct ’21
When using startCaptureWithHandler, iPhone APP can’t rotate.
When using ReplayKit => RPScreenRecorder => startCaptureWithHandler, iPhone APP can’t rotate.
Replies
0
Boosts
0
Views
681
Activity
Dec ’21
Where is WWDC 2016 session 601 "Go Live with ReplayKit"?
I'm developing and application that will use ReplayKit's Broadcast API to stream the screen of a iOS device. For that I'll need to create the Client App and the Broadcaster App. The WWDC videos about ReplayKit mention WWDC 2016 session 601 "Go Live with ReplayKit" but I can't find it. Can someone point me to that video or to its transcript?. Also, any other resource about the Broadcast API will be appreciated. Most of the samples and documentation that I've found on the web are for just streaming of the app's own screen. Thank you.
Replies
1
Boosts
0
Views
1.1k
Activity
Dec ’21
Broadcast Extension receives broadcastFinished prematurely
I have a Broadcast Extension that's running a system wide screen capture. I can start the broadcast and receive buffers and the like.. Buffers are successfully getting to our back and and distributed to all the peers (webrtc). The problem is when I navigate from the host app to apps that play video with audio. And then try to go landscape, sometimes during the rotation, the system will send a broadcastFinished() message to my RPBroadcastSampleHandler subclass. It doesn't occur all the time, but when it does .. it looks like the rotation glitches some. Hard to explain. But I've noticed that it only sends the finished message when it glitches. I would love to get a screen grab of it occurring, but my capture is running.. A quick google/forum search has revealed not much help. Has anyone heard of such a thing?
Replies
0
Boosts
0
Views
515
Activity
Dec ’21
ReplayKit: Select window to record?
I have a simple macOS app, with two windows. I would like the user to be able recored each window using a button on the window. I can record the main window and save that to a file using ReplayKit - so far, so good. But I can only record the main window - even if I have all ReplayKit code in the second NSViewController, its the main widow thats being recorded. It is possible to tell ReplayKit what window to record somehow?
Replies
2
Boosts
0
Views
1.2k
Activity
Nov ’21
I want to delete the RPScreenRecorder-startCapture confirmation message.
When RPScreenRecorder - startCapture is performed, the user is prompted to allow screen capture. Every time you open an app and run it, it will ask you. Is it possible not to ask after the second launch?
Replies
1
Boosts
1
Views
746
Activity
Nov ’21
Is there a way to notify the user of important information during Screen recording?
On iOS15.1, if users set Settings > Notifications > Screen Sharing > Notifications Off, our application cannot display notification during screen recording. I find this setting very useful, but it means that there is no way to display important messages such as no audio. Is there a way around this with any notifications?
Replies
0
Boosts
0
Views
757
Activity
Nov ’21
Share the current screen to another user in the call.
My goal is to implement a screen sharing function like Skype, Discord etc. My biggest problem in that implementation is that when you have an active RPScreenRecorder.shared() screen recording and then leave the app, the screen recording stops. For my goal it is necessary to be able to share the screen even if the app is in background. For example: The local user shares his current screen to another user, then exits the app to show the other user his Photo Library in realtime. Problem 1: I have came across ReplayKit and "RPBroadcastSampleHandler", but this extension is only called when I explicitly run the app with the "ScreenShare" scheme that I created. When I try to run the app on the normal/main scheme the extension functions aren't called. Problem 2: I can't call my pod functionalities from the "RPBroadcastSampleHandler" class. It just says "-PodName- not found" when I try to import the pod I need to use. There are no full documentations on that topic and I would really appreciate anybody who can explain to me what I am doing wrong.
Replies
0
Boosts
0
Views
854
Activity
Nov ’21
AVAssetWriterInput.append giving error "An Unknown Error Occurred (-12706)"
I am simply trying to capture the screen, app audio and mic audio. The app audio and mic audio independently work fine but when combined some unknown error is thrown. Following are the methods to start the capture of screen and processSampleBuffer. Method to setup writers for screen capture func startCapture() { _filename = UUID().uuidString let videoPath = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!.appendingPathComponent("\(_filename).mp4") let writer = try! AVAssetWriter(outputURL: videoPath, fileType: .mp4) let screen = UIScreen.main.bounds let screenBounds = screen.size let videoCompressionPropertys = [ AVVideoAverageBitRateKey: screenBounds.width * screenBounds.height * 10.1 ] let videoSettings: [String: Any] = [ AVVideoCodecKey: AVVideoCodecType.h264, AVVideoWidthKey: screenBounds.width, AVVideoHeightKey: screenBounds.height, AVVideoCompressionPropertiesKey: videoCompressionPropertys ] let input = AVAssetWriterInput(mediaType: .video, outputSettings: videoSettings) input.expectsMediaDataInRealTime = true if writer.canAdd(input) { writer.add(input) } // Add the app audio input var acl = AudioChannelLayout() memset(&acl, 0, MemoryLayout<AudioChannelLayout>.size) acl.mChannelLayoutTag = kAudioChannelLayoutTag_Mono; let audioOutputSettings: [String: Any] = [ AVFormatIDKey: kAudioFormatMPEG4AAC, AVSampleRateKey : 44100, AVNumberOfChannelsKey : 1, AVEncoderBitRateKey : 128000, AVChannelLayoutKey : Data(bytes: &acl, count: MemoryLayout<AudioChannelLayout>.size)] let audioInput = AVAssetWriterInput(mediaType: AVMediaType.audio, outputSettings: audioOutputSettings) audioInput.expectsMediaDataInRealTime = true if (writer.canAdd(audioInput)) { writer.add(audioInput) } // Add the mic audio input let audioOutputSettings1: [String: Any] = [ AVFormatIDKey: kAudioFormatMPEG4AAC, AVSampleRateKey : 24000, AVNumberOfChannelsKey : 1, AVEncoderAudioQualityKey: AVAudioQuality.high.rawValue] let micAudioInput = AVAssetWriterInput(mediaType: AVMediaType.audio, outputSettings: audioOutputSettings1) micAudioInput.expectsMediaDataInRealTime = true if (writer.canAdd(micAudioInput)) { writer.add(micAudioInput) } writer.startWriting() _audioAssetWriterInput = audioInput _micAssetWriterInput = micAudioInput _assetWriterInput = input _assetWriter = writer } processSampleBuffer override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) { if startTime == nil { startTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer) _assetWriter!.startSession(atSourceTime: CMTime.zero) } if sampleBufferType == RPSampleBufferType.video { if let _assetWriterInput = _assetWriterInput { if _assetWriterInput.isReadyForMoreMediaData { let appended = _assetWriterInput.append(sampleBuffer) print(appended) if !appended { let status = _assetWriter?.status let error = _assetWriter?.error print("cannot append video") } } } } if sampleBufferType == RPSampleBufferType.audioApp { if let _assetWriterInput = _audioAssetWriterInput { if _assetWriterInput.isReadyForMoreMediaData { let appended = _assetWriterInput.append(sampleBuffer) print(appended) if !appended { let status = _assetWriter?.status let error = _assetWriter?.error print("cannot append app audio") } } } } if sampleBufferType == RPSampleBufferType.audioMic { if let _assetWriterInput = _micAssetWriterInput { if _assetWriterInput.isReadyForMoreMediaData { let appended = _assetWriterInput.append(sampleBuffer) print(appended) if !appended { let status = _assetWriter?.status let error = _assetWriter?.error print("cannot append mic audio") } } } } if shouldEnd { _finishWriters() } }
Replies
0
Boosts
0
Views
1.2k
Activity
Oct ’21
anyone know how to rotate the CMSampleBuffer to landscape
Has anyone figured this out.It seems replaykit 2 always delevers frames at a fixed resolution of 886 x 1918I've tried various solutions to rotate the screen but so far none work.Using the code below crashes in the rotate stepvImageRotate90_ARGB8888(&amp;srcBuffer, &amp;destBuffer, factor, &amp;color, vImage_Flags(0)) let flags = CVPixelBufferLockFlags(rawValue: 0) guard kCVReturnSuccess == CVPixelBufferLockBaseAddress(srcPixelBuffer, flags) else { return nil } defer { CVPixelBufferUnlockBaseAddress(srcPixelBuffer, flags) } guard let srcData = CVPixelBufferGetBaseAddress(srcPixelBuffer) else { print("Error: could not get pixel buffer base address") return nil } let sourceWidth = CVPixelBufferGetWidth(srcPixelBuffer) let sourceHeight = CVPixelBufferGetHeight(srcPixelBuffer) var destWidth = sourceHeight var destHeight = sourceWidth var color = UInt8(0) if factor % 2 == 0 { destWidth = sourceWidth destHeight = sourceHeight } let srcBytesPerRow = CVPixelBufferGetBytesPerRow(srcPixelBuffer) var srcBuffer = vImage_Buffer(data: srcData, height: vImagePixelCount(sourceHeight), width: vImagePixelCount(sourceWidth), rowBytes: srcBytesPerRow) let destBytesPerRow = destWidth*4 guard let destData = malloc(destHeight*destBytesPerRow) else { print("Error: out of memory") return nil } var destBuffer = vImage_Buffer(data: destData, height: vImagePixelCount(destHeight), width: vImagePixelCount(destWidth), rowBytes: destBytesPerRow) let error = vImageRotate90_ARGB8888(&amp;srcBuffer, &amp;destBuffer, factor, &amp;color, vImage_Flags(0)) if error != kvImageNoError { print("Error:", error) free(destData)
Replies
3
Boosts
0
Views
5.7k
Activity
Oct ’21
Check users using Record Screen from Access Control or third party app
I use RelayKit to record screen in my app, but I want to detect if user use Record Screen from Access Control or third party app. I tried UIScreen.capturedDidChangeNotification but it doesn't check if I'm recording the screen in my app, or using Record Screen from Access Control. Do you have any ideas
Replies
1
Boosts
0
Views
907
Activity
Oct ’21
Screen broadcast get invalid broadcast session error
While i'm sharing my screen, I get an error message “Live broadcast to: has stopped due to: Attempted to start an invalid broadcast session”. Could I know what is the cause of this error What can I do to fix this error in the future?
Replies
0
Boosts
0
Views
648
Activity
Oct ’21