I am reaching out regarding an issue with my Apple FairPlay Streaming Certificate. To generate the certificate signing request (CSR), I used the following OpenSSL commands:
openssl genrsa -out private_key.pem 1024 openssl req -new -key private_key.pem -out request.csr
However, according to the guide provided by Apple and instructions from my DRM provider, I should have used:
openssl genrsa -aes256 -out privatekey.pem 1024 openssl req -new -sha1 -key privatekey.pem -out certreq.csr -subj "/CN=SubjectName /OU=OrganizationalUnit /O=Organization /C=US"
I suspect this discrepancy might be causing the issue with my FairPlay certificate. After obtaining the fairplay.cer file and importing it into Keychain Access, I noticed the following:
When I expand the certificate in Keychain Access, I can only see a public key and no private key.
As a result, I am unable to export the certificate as a .p12 file, as this option is disabled.
As per my DRM provider's instructions, I need to export the certificate along with the corresponding private key as a .p12 file with a password. Since the private key is not visible in Keychain Access, I am unable to proceed further.
I have read the FairPlay Streaming Overview but could not find any reasons as to why this issue is occurring or guidance on the procedure to revoke a certificate.
Additionally, I came across the terms and conditions which mentioned reaching out to product-security at Apple for assistance in revoking corrupt certificates. However, despite reaching out, I have not received a response.
Any help on how to proceed will be great!
Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.
Selecting any option will automatically load the page
Post
Replies
Boosts
Views
Activity
Hello,
Using ShazamKit, based on a shazam catalog result, would it be possible to detect the audio-recorded FPS (speed)?
I'm thinking that the shazam catalog which was created from an audio file can be used to compare the speed of a live recorded audio.
Thank you!
Hi,
I am trying to enable the default MIDINetworkSession in a Catalyst app on MacOS like this:
MIDINetworkSession.default().isEnabled = true
MIDINetworkSession.default().connectionPolicy = .anyone
In the AppSandbox I have both incoming and outgoing network connections enabled. And I also added the NSLocalNetworkUsageDescription key to the info.plist. Bonjour services are also added to the info.plist:
NSBonjourServices
_apple-midi._udp.
Nevertheless the session stays disabled. Running the same code works just fine on iOS.
Is there any special setup I need to make on MacOS to enable the MIDINetworkSession?
Thanks!
Hi All, I'm working on a camera system extension where the main app is supposed to transfer a video stream using IOSurface memory sharing to the cam extension.
I have built a sample app that does contains all the logic, but without a camera extension. So I'm essentially using IOSurface to render a video in one SwiftUI view and show the result in another SwiftUI view. Just for testing purposes. And everything works fine so far.
Now, when moving the receiver code to the camera extensions, I'm having problems in accessing the IOSurface via ID. I am sharing the IOSurface ID via UserDefaults. I know from the logs the ID is correctly transferred.
Here is the code that uses IOSurfaceLookup to get the IOSurface. But this fails with the given message. The error message prints the surface ID which is the correct one. I know this from the main app where I get the ID and print it as well.
private var surfaceId: Int = -1 {
didSet {
logger.info("surfaceId has changed")
if surfaceId == -1 {
stopReceivingFrames()
ioSurface = nil
} else {
guard let surface = IOSurfaceLookup(IOSurfaceID(surfaceId)) else {
logger.error("failed to lookup IOSurface with ID: \(self.surfaceId)")
return
}
self.ioSurface = surface
logger.info("surface set, now starting receiving frames")
startReceivingFrames()
}
}
}
My gut feeling says that this issue might be related to some missing entitlement, sandboxing. In general, I have a working camera extension. I'm just not able to render a video in the main app, and send it over to the camera extension to overlay another web cam.
Both, the main app and camera extension are in the same XCode workspace and share the same AppGroup.
In short, my actual questions are:
Is there any entitlement required for using IOSurface between app and camera system extension?
Is using IOSurface actually possible in system extensions?
Is there any specific setting/requirement that I need to handle to make this work?
Topic:
Media Technologies
SubTopic:
Photos & Camera
Tags:
IOSurface
System Extensions
Camera
Core Media
Hi. I am working on an audio app for iOS. I have implemented UI and handling which allows the user to change playback rate of audio. When the user selects a different rate, I update the rate property on my AVQueuePlayer. This is working well on device.
When I use Airplay, it works for some devices and not for others. Some devices won't change playback rate and will always play at 1x speed.
Is this possibly a limitation of those 3rd-party devices? Or is there something I'm missing/should check? Would love to get playback rate changes working across all Airplay devices with our app.
Kind regards.
Getting MatchError "MATCH_ATTEMPT_FAILED" everytime when matchstream on Android Studio Java+Kotlin project. My project reads the samples from the mic input using audioRecord class and sents them to the Shazamkit to matchstream. I created a kotlin class to handle to Shazamkit. The audioRecord is build to be mono and 16 bit.
My Kotlin Class
class ShazamKitHelper {
val shazamScope = CoroutineScope(Dispatchers.IO + SupervisorJob())
lateinit var streaming_session: StreamingSession
lateinit var signature: Signature
lateinit var catalog: ShazamCatalog
fun createStreamingSessionAsync(developerTokenProvider: DeveloperTokenProvider, readBufferSize: Int, sampleRate: AudioSampleRateInHz
): CompletableFuture<Unit>{
return CompletableFuture.supplyAsync {
runBlocking {
runCatching {
shazamScope.launch {
createStreamingSession(developerTokenProvider,readBufferSize,sampleRate)
}.join()
}.onFailure { throwable ->
}.getOrThrow()
}
}
}
private suspend fun createStreamingSession(developerTokenProvider:DeveloperTokenProvider,readBufferSize: Int,sampleRateInHz: AudioSampleRateInHz) {
catalog = ShazamKit.createShazamCatalog(developerTokenProvider)
streaming_session = (ShazamKit.createStreamingSession(
catalog,
sampleRateInHz,
readBufferSize
) as ShazamKitResult.Success).data
}
fun startMatching() {
val audioData = sharedAudioData ?: return // Return if sharedAudioData is null
CoroutineScope(Dispatchers.IO).launch {
runCatching {
streaming_session.matchStream(audioData.data, audioData.meaningfulLengthInBytes, audioData.timestampInMs)
}.onFailure { throwable ->
Log.e("ShazamKitHelper", "Error during matchStream", throwable)
}
}
}
@JvmField
var sharedAudioData: AudioData? = null;
data class AudioData(val data: ByteArray, val meaningfulLengthInBytes: Int, val timestampInMs: Long)
fun startListeningForMatches() {
CoroutineScope(Dispatchers.IO).launch {
streaming_session.recognitionResults().collect { matchResult ->
when (matchResult) {
is MatchResult.Match -> {
val match = matchResult.matchedMediaItems
println("Match found: ${match.get(0).title} by ${match.get(0).artist}")
}
is MatchResult.NoMatch -> {
println("No match found")
}
is MatchResult.Error -> {
val error = matchResult.exception
println("Match error: ${error.message}")
}
}
}
}
}
}
My code in java reads the samples from a thread:
shazam_create_session();
while (audioRecord.getRecordingState() == AudioRecord.RECORDSTATE_RECORDING){
if (shazam_session_created){
byte[] buffer = new byte[288000];//max_shazam_seconds * sampleRate * 2];
audioRecord.read(buffer,0,buffer.length,AudioRecord.READ_BLOCKING);
helper.sharedAudioData = new ShazamKitHelper.AudioData(buffer,buffer.length,System.currentTimeMillis());
helper.startMatching();
if (!listener_called){
listener_called = true;
helper.startListeningForMatches();
}
} else{
SystemClock.sleep(100);
}
}
private void shazam_create_session() {
MyDeveloperTokenProvider provider = new MyDeveloperTokenProvider();
AudioSampleRateInHz sample_rate = AudioSampleRateInHz.SAMPLE_RATE_48000;
if (sampleRate == 44100)
sample_rate = AudioSampleRateInHz.SAMPLE_RATE_44100;
CompletableFuture<Unit> future = helper.createStreamingSessionAsync(provider, 288000, sample_rate);
future.thenAccept(result -> {
shazam_session_created = true;
});
future.exceptionally(throwable -> {
Toast.makeText(mine, "Failure", Toast.LENGTH_SHORT).show();
return null;
});
}
I Implemented the developer token in java as follows
public static class MyDeveloperTokenProvider implements DeveloperTokenProvider {
DeveloperToken the_token = null;
@NonNull
@Override
public DeveloperToken provideDeveloperToken() {
if (the_token == null){
try {
the_token = generateDeveloperToken();
return the_token;
} catch (NoSuchAlgorithmException | InvalidKeySpecException e) {
throw new RuntimeException(e);
}
} else{
return the_token;
}
}
@NonNull
private DeveloperToken generateDeveloperToken() throws NoSuchAlgorithmException, InvalidKeySpecException {
PKCS8EncodedKeySpec priPKCS8 = new PKCS8EncodedKeySpec(Decoders.BASE64.decode(p8));
PrivateKey appleKey = KeyFactory.getInstance("EC").generatePrivate(priPKCS8);
Instant now = Instant.now();
Instant expiration = now.plus(Duration.ofDays(90));
String jwt = Jwts.builder()
.header().add("alg", "ES256").add("kid", keyId).and()
.issuer(teamId)
.issuedAt(Date.from(now))
.expiration(Date.from(expiration))
.signWith(appleKey) // Specify algorithm explicitly
.compact();
return new DeveloperToken(jwt);
}
}
Does Phase support creating new sound events at runtime? Is that implemented in the plugin for Unity as well? Does Phase support Unity's addressable system, are they compatible?
The following is my playground code. Any of the apple audio units show the plugin view, however anything else (i.e. kontakt, spitfire, etc.) does not. It does not error, just where the visual is expected is blank.
import AppKit
import PlaygroundSupport
import AudioToolbox
import AVFoundation
import CoreAudioKit
let manager = AVAudioUnitComponentManager.shared()
let description = AudioComponentDescription(componentType: kAudioUnitType_MusicDevice,
componentSubType: 0,
componentManufacturer: 0,
componentFlags: 0,
componentFlagsMask: 0)
var deviceComponents = manager.components(matching: description)
var names = deviceComponents.map{$0.name}
let pluginName: String = "AUSampler" // This works
//let pluginName: String = "Kontakt" // This does not
var plugin = deviceComponents.filter{$0.name.contains(pluginName)}.first!
print("Plugin name: \(plugin.name)")
var customViewController:NSViewController?
AVAudioUnit.instantiate(with: plugin.audioComponentDescription, options: []){avAudioUnit, error in
var ilip = avAudioUnit!.auAudioUnit.isLoadedInProcess
print("Loaded in process: \(ilip)")
guard error == nil else {
print("Error: \(error!.localizedDescription)")
return
}
print("AudioUnit successfully created.")
let audioUnit = avAudioUnit!.auAudioUnit
audioUnit.requestViewController{ vc in
if let viewCtrl = vc {
customViewController = vc
var b = vc?.view.bounds
PlaygroundPage.current.liveView = vc
print("Successfully added view controller.")
}else{
print("Failed to load controller.")
}
}
}
Hi I'm working on a project that require video frame PTS to be consistent between original video and a transcoded one. It's working fairly well on regular mp4, however if I set preferredOutputSegmentInterval to have generate a fMP4 output, even I specified the initialSegmentStartTime as 0, it always add one frame pts offset to all frames.
For example: if I use the code sample provided by Apple: https://developer.apple.com/videos/play/wwdc2020/10011/?time=406, useffprobe -select_streams v:0 -show_entries packet=pts_time -of csv ~/Downloads/fmp4/prog_index.m3u8 to display the pts of the output, it doesn't start from 0, but has some one frame pts offset. I also tried open with MP4Box, it also shows the first frames dts and cts are not start from 0.
However, if I use AVAssetReader to read the same output video, and get the PTS from 1st frame, it's returning 0. So I can't use it to calculate the pts difference between 2 videos neither.
Can I get some help to understand why there is difference between AVAssetWriter/Reader fMP4's pts and others like ffprobe?
In our app we have implemented a AVAssetResourceLoaderDelegate to handle encrypted downloaded files. We have it working on all iOS versions but we are seeing issues on iOS 15 (15.8.3) with large files (> 1 Gb). We have so far seen two cases where either the load method on the AVURLAsset fails early and throws an unknown error code or starts requesting more data than the device has available RAM. The CPU usage is almost always over 100%, even after pausing playback. The memory issue can happen even though the player has successfully started playback.
When running this on devices running iOS 16 and above we set the isEntireLengthAvailableOnDemand to true on the AVAssetResourceLoadingContentInformationRequest. This seems to be key to solving the issue those devices that support it. If we set the property to false we see the same memory issue as on iOS 15.
So we have a solution for iOS 16 and upwards but are at a loss for how to handle iOS 15. Is there something we have overlooked or is it in fact an issue with that iOS version?
Hello,
I'm writing a program to create CMAF compliant HLS files, with encryption.
I have a copy of ISO_IEC_23001-7_2023 to attempt to follow the spec.
I am following the 1:9 pattern encryption using CBCS, so for every 16 bytes of encrypted NAL unit data (of type 1 and 5), there's 144 bytes of clear data.
When testing my output in Safari with 'identity' keys Quickly Diagnosing Content Key and IV Issues, Safari will request the identity key from my test server and first few bytes of the CMAF renditions, but will not play and console gives away no clues to the error.
I am setting the subsample bytesofclear/protected data in the senc boxes. What I'm not sure of, is whether HLS/Safari/iOS acknowledges the senc/saiz/saio boxes of the MP4. There are other third party packagers Bento4, who suggest that they do not:
those clients ignore the explicit encryption
layout metadata found in saio/saiz boxes, and instead rely purely on the
video slice header size to determine the portions of the sample that is
encrypted
So now I'm fairly sure I need to decipher the video slice header size, and apply the protected blocks from that point on.
My question is, is that all there is to it? And is there a better way to debug my output? mediastreamvalidator will only work against unencrypted variants (which I'm outputting okay).
Thanks in advance!
Topic:
Media Technologies
SubTopic:
Streaming
Tags:
FairPlay Streaming
HTTP Live Streaming
AVFoundation
AVAudioEngine and AVAudioSession
Welcome! I will start off with the terms AVAudioEngineImpl::Initialize(NSError**).
Why? I want to make those who run into this issue have to possibility to find this post through Search Engines!
This is short small breakdown based on what I observed while trying to use these two Components. It's not a guide that goes into all the details.
If you're trying to figure out how to fix a crash, you may can find a common way to fix it, in this post!
Is it possible to use AVAudioEngine and AVAudioSession together?
The answer is yes.
But you will face challenges regarding it. Mostly AVAudioEngine. Whatever you're trying to do, it will take a lot of testing. I don't know how it will be with an IDE. But with just .app and iPhone it will take some testing. Or a lot of testing.
Something that helped me fixing a crash was, this here: https://developer.apple.com/documentation/avfaudio/audio_engine/audio_units/using_voice_processing
This example Project by Apple, uses both AVAudioEngine and AVAudioSession.
How can I fix AVAudioEngineImpl::Initialize(NSError**) ?
I think this depends. If you're lucky and have a crash log, you may can find clues, but the stack trace sometimes doesn't really help either.
I will mention common cases that I encountered though.
inputNode
https://developer.apple.com/documentation/avfaudio/avaudioengine/1386063-inputnode
You need an inputNode apparently. You need to access it or else I think there won't be one. And if there isn't one, AVAudioEngine.start will most likely crash.
The audio engine creates a singleton on demand when first accessing this variable.
Doing this has prevented this common issue for me.
.prepare deallocates and can cause a crash if you restart your AudioEngine
Another issue I faced was handling .prepare wrong. You don't need .prepare. But if you use installTap or other things, I think you need it.
Here is a common thing to note.
If you had previous initialized inputNode. Those could be gone after using .prepare.
You have to ensure you're accessing AVAudioEngine.inputNode again before calling .start() or whatever node you need.
The Voice Processing Project, does this by creating a Managing Controller for AVAudioEngine with a sort of "setup" function, which ensures that everything is ready, before .prepare and .start get called.
AVAudioSession's setCategory
You have to experiment with it. The crashes can be very weird. Sometimes your App will only crash once, and then only after you install it again, or if you start it up.
You are actually able to use .setActive and .setCategory with AVAduioEngine. Just do not try to do .setActive(false) before you've stopped the AudioEngine, as it will fail.
Sometimes I'd run into an issue with .setActive(true) so you really have to experiment if leaving that part out resolves the issue or not.
try session.setCategory(.multiRoute, mode: .default, options: [.defaultToSpeaker, .mixWithOthers])
Experiment with it. But these .multiRoute and .mixWithOthers have allowed me to use AVAudioEngine to make a test recording. And I can even switch the Data Sources and Polar Patterns without any issues.
Sometimes you can get away without setting .setActive at all. Not sure if AVAudioEngine does it automatically.
Short Summary
If you use .prepare and then .stop, make sure to initialize things like .inputNode before calling .prepare and .start again. (THIS CAN BE DIFFERENT)
Only call .setActive(false) after you used .stop. Otherwise I believe it has no chance to stop it.
AVAudioSession setCategory is important. Ensure you use mixRoutes or experiment with all the modes.
If you manage to solve your crash, you'll be able to indeed change the Data Sources and Polar Patterns and more!
Use isRunning before using .start, this will save you from another crash. If you use .start while it's already running, I think try and catch won't save you here, you have to ensure you're not starting it twice.
I hope that this short breakdown will help you to resolve your crash. If you get deeper into AVAudioEngine and AVAudioSession, you'll probably face more crashes. I yet, need to figure out how to solve them. I have a lot of trouble to put my Testing App on my iPhone, so I am sorry if this guide didn't cover every detail of it.
A HUGE tip from me is to check the Documentations. As example, when I read the Documentation for inputNode I learned why my app crashed, it's because I never accessed and initialized one.
The Developer Documentation can be a little bit of a laberynth, and I strongly recommend you to read every property you try to access if you believe they cause issues. And I also recommend to find example Projects like the Voice Processing ones. As there aren't any Code Examples in the Documentation.
Here we are focusing to change the cookie at every 120 seconds while playing , in apple avplayer we can't modify cookie after initialisation due to that we followed the approach to using " Resource loader delegate " to pass cookie as a header value .
What I notice is that the playlist file (.m3u8) gets downloaded correctly. Then video file (.m4a) some chunks also gets downloaded. I know that the .ts file is downloaded because I can see the GET request completing on the web server with status 200. I also set a breakpoint at the following line:
loadingRequest.dataRequest?.respond(with: data)
immediately got error from avplayer status as
"The operation could not be completed. An unknown error occurred (-12881) From core media"
Need confirmation on why I am unable to load HLS using resource loader.
is it possible to update cookie value while paying continuously on avplayer.
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view.
let urlString = "localhost://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel.ism/.m3u8"
guard let url = URL(string: urlString) else {
print("Invalid URL")
return
}
//Create cookie to prepare for player asset
let cookie = HTTPCookie(properties: [
.name: "dazn-token",
.value: "cookie value",
.domain: url.host() ?? "",
.path: "/",
.discard: true
])
//Create cookie key to set AVURLAsset
let options = [AVURLAssetHTTPCookiesKey: [cookie]]
let asset = AVURLAsset(url: url,options: options)
proxy = ReverseProxyResourceLoader()
proxy?.cookie = "exampleCookie"
// Set resource loader delegate to moniter the chunks
asset.resourceLoader.setDelegate(proxy, queue: DispatchQueue.global())
// Load asset keys asynchronously (e.g., "playable")
let keys = ["playable"]
// Initialize the AVPlayer with the URL
let playerItem = AVPlayerItem(asset: asset)
self.player = AVPlayer(playerItem: playerItem)
playerItem.addObserver(self, forKeyPath: "status", options: [.new, .initial], context: nil)
// Observe 'error' property (if needed)
playerItem.addObserver(self, forKeyPath: "error", options: [.new], context: nil)
let contentKeySessionDelegate = ContentKeyDelegate()
// Initialize AVContentKeySession
let contentKeySession = AVContentKeySession(keySystem: .clearKey)
self.contentKeySession = contentKeySession
contentKeySession.setDelegate(contentKeySessionDelegate, queue: DispatchQueue.main)
// Associate the asset with the content key session
contentKeySession.addContentKeyRecipient(asset)
// Create a layer for the AVPlayer and add it to the view
playerLayer = AVPlayerLayer(player: player)
playerLayer?.frame = view.bounds
playerLayer?.videoGravity = .resizeAspect
if let playerLayer = playerLayer {
view.layer.addSublayer(playerLayer)
}
NotificationCenter.default.addObserver(
self,
selector: #selector(playerDidFinishPlaying),
name: .AVPlayerItemDidPlayToEndTime,
object: player?.currentItem
)
// Start playback
player?.play()
}
// Update cookie when ever needed
func updateCookie() {
proxy?.cookie = "update exampleCookie"
}
@objc private func playerDidFinishPlaying(notification: Notification) {
print("Playback finished!")
// Optionally, handle end-of-playback actions here
}
//
// ReverseProxyResourceLoader.swift
// HLSDemo
//
// Created by Gajje.Venkatarao on 12/12/24.
//
import Foundation
import AVKit
import AVFoundation
class ReverseProxyResourceLoader: NSObject, AVAssetResourceLoaderDelegate {
var cookie = ""
func resourceLoader(_ resourceLoader: AVAssetResourceLoader, shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest) -> Bool {
resourceLoader.preloadsEligibleContentKeys = true
guard let interceptedURL = loadingRequest.request.url else {
loadingRequest.finishLoading(with: NSError(domain: "ReverseProxy", code: -1, userInfo: [NSLocalizedDescriptionKey: "Invalid URL"]))
return false
}
if interceptedURL.scheme == "skd" {
print("Token updated Cookie:", interceptedURL )
return false
}
var components = URLComponents(url: interceptedURL, resolvingAgainstBaseURL: false)
components?.scheme = "https" // Replace with the original scheme
guard let originalURL = components?.url else {
loadingRequest.finishLoading(with: NSError(domain: "ReverseProxy", code: -1, userInfo: [NSLocalizedDescriptionKey: "Failed to map URL"]))
loadingRequest.finishLoading()
return false
}
var request = URLRequest(url: originalURL)
request.httpMethod = "GET"
if let storeCoockie = HTTPCookie(properties: [
.name: "dazn-token",
.value: cookie,
.domain: originalURL.host ?? "",
.path: "/",
.discard: true
]){
HTTPCookieStorage.shared.setCookie(storeCoockie)
}
let headers = loadingRequest.request.allHTTPHeaderFields ?? [:]
for (key, value) in headers {
request.addValue(value, forHTTPHeaderField: key)
}
request.addValue(cookie, forHTTPHeaderField: "Cookie")
URLSession.shared.configuration.httpShouldSetCookies = true
request.httpShouldHandleCookies = true
let task = (URLSession.shared.dataTask(with: originalURL) { data, response, error in
if let error = error {
print("Error Received:", error)
loadingRequest.finishLoading(with: error)
return
}
print(originalURL)
guard let data = data , let url = response?.url else {
loadingRequest.finishLoading(with: NSError(domain: "ReverseProxy", code: -1, userInfo: [NSLocalizedDescriptionKey: "No data received"]))
return
}
loadingRequest.dataRequest?.respond(with: data)
loadingRequest.finishLoading()
} as URLSessionDataTask)
task.resume()
return true
}
}
Example project
I am having trouble accessing the lastPlayedData for any given album or track using MusicKit. The value is always nil, both on numerous albums and tracks I tested.
Afaik this is not a property that has to be fetched separately like tracks for example.
I am running this on my physical iPhone 12 18.1.1 with Xcode 16.1. The albums and tracks have definitely been played multiple times before. The app has permission to the library using MusicAuthorization.request()
This post mentions the same problem but offers no solution.
Thanks for any help
I'm trying to setup a listener for kAudioProcessPropertyIsRunningOutput but it's never triggered. I get calls for kAudioProcessPropertyIsRunning and kAudioProcessPropertyDevices but not for kAudioProcessPropertyIsRunningInput or kAudioProcessPropertyIsRunningOutput.
class MyDelegate: PropertyListenerDelegate {
func propertiesChanged(properties: [AudioObjectPropertyAddress]) {
print(properties)
}
}
var myDelegate = MyDelegate()
var processes = try AudioHardwareSystem.shared.processes
for process in processes {
process.delegates += [myDelegate]
try process.addListener(forProperties: [AudioObjectPropertyAddress(mSelector: kAudioPropertyWildcardPropertyID, mScope: kAudioObjectPropertyScopeWildcard, mElement: kAudioObjectPropertyElementWildcard)])
}
Xcode 16.1
macOS 15.0.1
Hello all,
This is my first post on the developer forums.
I am developing an app that records the screen of my app, using AVAssetWriter and RPScreenRecorder startCapture.
Everything is working as it should on most cases. There are some seemingly random times where the file generated is of some kb and it is corrupted. There seems to be no pattern on what the device is or the iOS version is. It can happen on various phones and iOS versions.
The steps I have followed in order to create the file are:
configuring the AssetWritter
videoAssetWriter = try? AVAssetWriter(outputURL: url!, fileType: AVFileType.mp4)
let size = UIScreen.main.bounds.size
let width = (Int(size.width / 4)) * 4
let height = (Int(size.height / 4)) * 4
let videoOutputSettings: Dictionary<String, Any> = [
AVVideoCodecKey : AVVideoCodecType.h264,
AVVideoWidthKey : width,
AVVideoHeightKey : height
]
videoInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: videoOutputSettings)
videoInput?.expectsMediaDataInRealTime = true
guard let videoInput = videoInput else { return }
if videoAssetWriter?.canAdd(videoInput) ?? false {
videoAssetWriter?.add(videoInput)
}
let audioInputsettings = [
AVFormatIDKey: Int(kAudioFormatMPEG4AAC),
AVSampleRateKey: 12000,
AVNumberOfChannelsKey: 1,
AVEncoderAudioQualityKey: AVAudioQuality.high.rawValue
]
audioInput = AVAssetWriterInput(mediaType: .audio, outputSettings: audioInputsettings)
audioInput?.expectsMediaDataInRealTime = true
guard let audioInput = audioInput else { return }
if videoAssetWriter?.canAdd(audioInput) ?? false {
videoAssetWriter?.add(audioInput)
}
The urlForVideo returns the URL to the documentDirectory, after appending and creating the folders needed. This part seems to be working as it should as the directories are created and the video file exists on them.
Start the recording
if RPScreenRecorder.shared().isRecording { return }
RPScreenRecorder.shared().startCapture(handler: { [weak self] sample, bufferType, error in
if let error = error {
onError?(error.localizedDescription)
} else {
if (!RPScreenRecorder.shared().isMicrophoneEnabled) {
RPScreenRecorder.shared().stopCapture { error in
if let error = error { return }
}
onError?("Microphone was not enabled")
}
else {
succesCompletion?()
succesCompletion = nil
self?.processSampleBuffer(sample, with: bufferType)
}
}
}) { error in
if let error = error {
onError?(error.localizedDescription)
}
}
Process the sampleBuffers
guard CMSampleBufferDataIsReady(sampleBuffer) else { return }
DispatchQueue.main.async { [weak self] in
switch sampleBufferType {
case .video:
self?.handleVideoBaffer(sampleBuffer)
case .audioMic:
self?.add(sample: sampleBuffer, to: self?.audioInput)
self?.audioInput)
default:
break
}
}
// The add function from above
fileprivate func add(sample: CMSampleBuffer, to writerInput: AVAssetWriterInput?) {
if writerInput?.isReadyForMoreMediaData ?? false {
writerInput?.append(sample)
}
// The handleVideoBaffer function from above
fileprivate func handleVideoBaffer(_ sampleBuffer: CMSampleBuffer) {
if self.videoAssetWriter?.status == AVAssetWriter.Status.unknown {
self.videoAssetWriter?.startWriting()
self.videoAssetWriter?.startSession(atSourceTime: CMSampleBufferGetPresentationTimeStamp(sampleBuffer))
} else {
if (self.videoInput?.isReadyForMoreMediaData) ?? false {
if self.videoAssetWriter?.status == AVAssetWriter.Status.writing {
self.videoInput?.append(sampleBuffer)
}
}
}
}
}
Finally the stop recording
func stopRecording(completion: @escaping (URL?, URL?, Error?) -> Void) {
RPScreenRecorder.shared().stopCapture { error in
if let error = error {
completion(nil, nil, error)
return
}
self.finish { videoURL, _ in
completion(videoURL, nil, nil)
}
}
}
// The finish function mentioned above
fileprivate func finish(completion: @escaping (URL?, URL?) -> Void) {
let dispatchGroup = DispatchGroup()
dispatchGroup.enter()
finishRecordVideo {
dispatchGroup.leave()
}
dispatchGroup.notify(queue: .main) {
print("Finish with url:\(String(describing: self.urlForVideo()))")
completion(self.urlForVideo(), nil)
}
}
// The finishRecordVideo mentioned above
fileprivate func finishRecordVideo(completion: @escaping ()-> Void) {
videoInput?.markAsFinished()
audioInput?.markAsFinished()
videoAssetWriter?.finishWriting {
if let writer = self.videoAssetWriter {
if writer.status == .completed {
completion()
}
else if writer.status == .failed {
// Print the error to find out what went wrong
if let error = writer.error {
print("Video asset writing failed with error: \(error.localizedDescription). Url: \(writer.outputURL.path)")
} else {
print("Video asset writing failed, but no error description available.")
}
completion()
}else {
completion()
}
}
}
}
What could it be the reason of the corrupted files generated? This issue has never happened to my devices so there is no way to debug using xcode. Also there are no errors popping out on the logs.
Can you spot any issues on the code that can create this kind of issue? Do you have any suggestions on the problem at hand?
Thanks
I explored several methods to trigger a 35mm camera connected via USB:
1- ICCameraDevice: Unable to make it work with Canon cameras (details).
2- Canon's EDSDK: Works but is complex to implement.
3- gPhoto2 (command-line): Simple to use but requires gPhoto2 to be installed.
In your opinion, what is the most efficient way to trigger and download images via USB from Canon cameras?
I tried configuring the preferredForwardBufferDuration on devices using 4G and Wi-Fi, and in these cases, AVPlayer works correctly according to the configured buffer duration. However, when the device is connected to a 5G network, the configuration value no longer works.
For example, if I set preferredForwardBufferDuration to 30 seconds, AVPlayer preloads with a buffer of over 100 seconds. I’m not sure how to resolve this, as it’s causing issues with my system.
I am experiencing an issue while recording audio using AVAudioEngine with the installTap method. I convert the AVAudioPCMBuffer to Data and send it to a UDP server. However, when I receive the Data and play it back, there is continuous crackling noise during playback.
I am sending audio data using this library "https://github.com/mindAndroid/swift-rtp" by creating packet and send it.
Please help me resolve this issue. I have attached the code reference that I am currently using.
Thank you.
ViewController.swift
Hi:
I am working with the ObjectCapture frameworks and sample code.
Everything works great.
We are trying to go from capturing 12MP images as in the sample code to capturing 48MP 6048 × 8064 images.
We can't seem to get it to work.
Any advice here?
Topic:
Media Technologies
SubTopic:
Photos & Camera