Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

All subtopics
Posts under Media Technologies topic

Post

Replies

Boosts

Views

Activity

PHPhotoLibrary availability detection/notification
Hi, I'm having difficulties figuring out how I can reliably detect if the macOS system PHPhotoLibrary is available. If I place the system Photo Library on an external drive and then eject it, other apps on startup, such as Photos, will tell me that the library isn't available. How can I replicate this behavior? The only API I can find for this detection on startup is "PHPhotoLibrary.shared().unavailabilityReason". However this always returns nil. Another strange behavior is if I register a PHPhotoLibraryAvailabilityObserver class on startup when the library is available and then eject the drive, I do get a notification via photoLibraryDidBecomeUnavailable, but then directly after the call the app is terminated. This prevents the app to perform any kind of graceful termination. Is this the expected behavior? It would make sense that it's up to the developer to decide what happens if the library becomes unavailable. Thanks
0
0
528
Nov ’24
AirPods Pro issue during a VoIP call,
Case-ID: 10075936 PLATFORM AND VERSION iOS Development environment: Xcode Xcode15, macOS macOS 14.5 Run-time configuration: iOS iOS18.0.1 DESCRIPTION OF PROBLEM Our customer experienced an one-way audio issue when switching from the built-in microphone to AirPods Pro (model: A2084, version: 6F21) during a VoIP call. The issue occurred when the customer's voice could not be heard by the other party, but the customer could hear the other party's voice. STEPS TO REPRODUCE Here are the details: After the issue occurred, subsequent VoIP calls experienced the same issue when using AirPods Pro, but the issue did not occur when using the built-in microphone. The issue could only be resolved by restarting the system, and killing the app did not work. Log and code analysis: In WebRTC, it listens for AVAudioSessionRouteChangeNotification. In the above scenario, when webrtc receives the route change notification, it will print the audio session configuration information. At this point, the input channel count was 0, which was abnormal: [Webrtc] (RTCLogging.mm:33): (audio_device_ios.mm:535 HandleValidRouteChange): RTC_OBJC_TYPE(RTCAudioSession): { category: AVAudioSessionCategoryPlayAndRecord categoryOptions: 128 mode: AVAudioSessionModeVoiceChat isActive: 1 sampleRate: 48000.00 IOBufferDuration: 0.020000 outputNumberOfChannels: 2 inputNumberOfChannels: 0 outputLatency: 0.021500 inputLatency: 0.005000 outputVolume: 0.600000 isPreferredSpeaker: 0 isCallkit: 0 } If app tries to call API, setPreferredInputNumberOfChannels at this point, it will fail with an error code of -50: setConfiguration:active:shouldSetActive:error:]): Failed to set preferred input number of channels(1): The operation couldn’t be completed. (OSStatus error -50.) Our questions: When AVAudioSession is active, the category and mode are as expected. Why is the input channel count 0? Assuming that the AVAudioSession state is abnormal at this point, why does killing the app not resolve the issue, and why does the system need to be restarted to resolve the issue? Is it possible that the category and mode of the AVAudioSession fetched by the app is currently wrong? Does it need to be reset again each time the callkit is started if the category and mode fetched are the same as the values to be set?
2
0
549
Nov ’24
mp3 audio plays on my device and simulator but some users have issue
Hi I just released an app which is live. i have a strange issue: while the audio files in the app play fine on my device, but some users are unable to hear. One friend said it played yesterday but not today. Any idea why? The files are mp3, I see them in Build Phase, and in the project obviously. Here's the audio view code, thank you! import AVFoundation struct MeditationView: View { @State private var player: AVAudioPlayer? @State private var isPlaying = false @State private var selectedMeditation: String? var isiPad = UIDevice.current.userInterfaceIdiom == .pad let columns = [GridItem(.flexible()),GridItem(.flexible())] let tracks = ["Intro":"intro.mp3", "Peace" : "mysoundbath1.mp3", "Serenity" : "mysoundbath2.mp3", "Relax" : "mysoundbath3.mp3"] var body: some View { VStack{ VStack{ VStack{ Image("dhvani").resizable().aspectRatio(contentMode: .fit) .frame(width: 120) Text("Enter the world of Dhvani soundbath sessions, click lotus icon to play.") .font(.custom("Times New Roman", size: 20)) .lineLimit(nil) .multilineTextAlignment(.leading) .fixedSize(horizontal: false, vertical: true) .italic() .foregroundStyle(Color.ashramGreen) .padding() } LazyVGrid(columns:columns, spacing:10){ ForEach(tracks.keys.sorted(),id:\.self){ track in Button { self.playMeditation(named: tracks[track]!) } label: { Image("lotus") .resizable() .frame(width: 40,height: 40) .background(Color.ashramGreen) .cornerRadius(10) } Text(track) .font(.custom("Times New Roman", size: 22)) .foregroundStyle(Color.ashramGreen) .italic() } } HStack(spacing:20) { Button(action: { self.togglePlayPause() }) { Image(systemName: isPlaying ? "playpause.fill" : "play.fill") .resizable() .frame(width: 20, height: 20) .foregroundColor(Color.ashramGreen) } Button(action: { self.stopMeditation() }) { Image(systemName: "stop.fill") .resizable() .frame(width: 20, height: 20) .foregroundColor(Color.ashramGreen) } } }.padding() .background(Color.ashramBeige) .cornerRadius(20) Spacer() //video play VStack{ Text("Chant") .font(.custom("Times New Roman", size: 24)) .foregroundStyle(Color.ashramGreen) .padding(5) WebView(urlString: "https://www.youtube.com/embed/ny3TqP9BxzE") .frame(height: isiPad ? 400 : 200) .cornerRadius(10) .padding() Text("Courtesy Sri Ramanasramam").font(.footnote).italic() } }.background(Color.ashramBeige) } //View func playMeditation(named name: String) { if let url = Bundle.main.url(forResource: name, withExtension: nil) { do { player = try AVAudioPlayer(contentsOf: url) player?.play() isPlaying = true } catch { print("Error playing meditation") } } } func togglePlayPause() { if let player = player { if player.isPlaying { player.pause() isPlaying = false } else { player.play() isPlaying = true } } } func stopMeditation() { player?.stop() isPlaying = false } } #Preview { MeditationView() }
1
0
346
Nov ’24
How to save 4K60 ProRes Log Video Internally on iPhone internally?
Hello Apple Engineers, Specific Issue: I am working on a video recording feature in my SwiftUI app, and I am trying to record 4K60 video in ProRes Log format using the iPhone's internal storage. Here's what I have tried so far: I am using AVCaptureSession with AVCaptureMovieFileOutput and configuring the session to support 4K resolution and ProRes codec. The sessionPreset is set to .inputPriority, and the video device is configured with settings such as disabling HDR to prepare for Log. However, when attempting to record 4K60 ProRes video, I get the error: "Capturing 4k60 with ProRes codec on this device is supported only on external storage device." This error seems to imply that 4K60 ProRes recording is restricted to external storage devices. But I am trying to achieve this internally on devices such as the iPhone 15 Pro Max, which has native support for ProRes encoding. Here are my questions: Is it technically possible to record 4K60 ProRes Log video internally on supported iPhones (for example: iPhone 15 Pro Max)? There are some 3rd apps (i.e. Blackmagic 👍🏻) that can save 4K60 ProRes Log video on iPhone internally. If internal saving is supported, what additional configuration is needed for the AVCaptureSession or other technique to bypass this limitation? If anyone has successfully saved 4K60 ProRes Log video on iPhone internal storage, your guidance would be highly appreciated. Thank you for your help!
0
0
648
Nov ’24
AudioQueue error 561145187
our app meet a wired problem for online version. more and more user get 561145187 when try to call this code: AudioQueueNewInput(&self->_recordFormat, inputBufferHandler, (__bridge void *)(self), NULL, NULL, 0, &self->_audioQueue)" I search for several weeks, but nothing help. we sum up all issues devices, found some similarity: only happens on iPad OS 14.0 + occurred when app started or wake from background (we call the code when app received "UIApplicationDidBecomeActiveNotification") Any Idea why this happens?
1
0
1.2k
Nov ’24
No charts genres for some storefronts
Hello, This is about the Get Catalog Top Charts Genres endpoint : GET https://api.music.apple.com/v1/catalog/{storefront}/genres I noticed that for some storefronts, no genre is returned. You can try with the following storefront values : France (fr) Poland (pl) Kyrgyzstan (kg) Uzbekistan (uz) Turkmenistan (tm) Is that a bug or is it on purpose ? Thank you.
0
0
432
Dec ’24
AVASSETREADER and AVAssetWriter: ideal settings
Hello, To create a test project, I want to understand how the video and audio settings would look for a destination video whose content comes from a source video. I obtained the output from the source video in the audio and video tracks as follows: let audioSettings = [ AVFormatIDKey: kAudioFormatLinearPCM, AVSampleRateKey: 44100, AVNumberOfChannelsKey: 2 ] as [String : Any] var audioOutput = AVAssetReaderTrackOutput(track: audioTrack!, outputSettings: audioSettings) // Video let videoSettings = [ kCVPixelBufferPixelFormatTypeKey: kCVPixelFormatType_32BGRA, kCVPixelBufferWidthKey: videoTrack!.naturalSize.width, kCVPixelBufferHeightKey: videoTrack!.naturalSize.height ] as [String: Any] var videoOutput = AVAssetReaderTrackOutput(track: videoTrack!, outputSettings: videoSettings) With this, I'm obtaining the CMSampleBuffer using AVAssetReader.copyNextSampleBuffer . How can I add it to the destination video? Should I use a while loop, considering I already have the AVAssetWriter set up? Something like this: while let buffer = videoOutput.copyNextSampleBuffer() { if let imgBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) { let frame = imgBuffer as CVPixelBuffer let time = CMSampleBufferGetPresentationTimeStamp(sampleBuffer) adaptor.append(frame, withMediaTime: time) } } Lastly, regarding the destination video. How should the AVAssetWriterInput for audio and PixelBuffer of the destination video be set up? Provide an example, something like: let audioSettings = […] as [String: Any] Looking forward to your response.
5
0
600
Dec ’24
why does tripple camera take photo faster than single camera device?
I found this phenomenon, and it can be reproduced stably. If I use a triple-camera to take a photo, if the picture is moving, or I move the phone, let's assume it moves horizontally, when I aim at an object, I press the shutter, which is called time T. At this time, the picture in the viewfinder is T0, and the photo produced is about T+100ms. If I use a single-camera to take a photo, use the same speed to move the phone to move the picture, and press the shutter when aiming at the same object, the photo produced is about T+400ms later. Let me describe the problem I encountered in another way. Suppose a pile of cards are placed horizontally on the table, and the cards are written with numbers from left to right, 0,1,2,3,4,5,6... Now aim the camera at the number 0, and then move to the right at a uniform speed. The numbers pass through the camera's viewfinder and continue to increase. When aiming at the number 5, press the shutter. If it is a triple-camera, the photo obtained will probably show 6, while if it is taken with a single-camera, the photo obtained will be about 9. This means the triple camera can capture photos faster, but why is this the case? Any explanation?
2
0
495
Dec ’24
How to add CIFilter to AVCaptureDeferredPhotoProxy
Hello, I m trying to implement deferred photo processing in my photo capture app. After I take a photo, I pass it through a CIFilter, now with the Deferred Photo Processing where would I pass the resulting photo through the CIFilter? Since there is no way for me to know when the system has finished processing a photo. If I have to do it in my app foreground every time, how do I prevent a scenario, where the user takes a photo, heads straight to the Photos App and sees the image without the filter?
2
0
620
Dec ’24
AVMIDIPlayer not working for all instruments
Hi, I test AVMIDIPlayer in order to replace classes written based on AVAudioEngine with callbacks functions sending MIDI events to test, I use an NSMutableData filled with: the MIDI header a track for time signature a track containing a few midi events. I then create an instance of the AVMIDIPlayer using the data Everything works fine for some instrument (00 … 20) or 90 but not for other instruments 60, 70, … The MiDI header and the time signature track are based on the MIDI.org sample, https://midi.org/standard-midi-files-specification RP-001_v1-0_Standard_MIDI_Files_Specification_96-1-4.pdf the midi events are: UInt8 trkEvents[] = { 0x00, 0xC0, instrument, // Tubular bell 0x00, 0x90, 0x4C, 0xA0, // Note 4C 0x81, 0x40, 0x48, 0xB0, // TS + Note 48 0x00, 0xFF, 0x2F, 0x00}; // End for (UInt8 i=0; i<3; i++) { printf("0x%X ", trkEvents[i]); } printf("\n"); [_midiTempData appendBytes:trkEvents length:sizeof(trkEvents)]; A template application is used to change the instrument in a NSTextField I was wondering if specifics are required for some instruments? The interface header: #import <AVFoundation/AVFoundation.h> NS_ASSUME_NONNULL_BEGIN @interface TestMIDIPlayer : NSObject @property (retain) NSMutableData *midiTempData; @property (retain) NSURL *midiTempURL; @property (retain) AVMIDIPlayer *midiPlayer; - (void)createTest:(UInt8)instrument; @end NS_ASSUME_NONNULL_END The implementation: #pragma mark - typedef struct _MThd { char magic[4]; // = "MThd" UInt8 headerSize[4]; // 4 Bytes, MSB first. Always = 00 00 00 06 UInt8 format[2]; // 16 bit, MSB first. 0; 1; 2 Use 1 UInt8 trackCount[2]; // 16 bit, MSB first. UInt8 division[2]; // }MThd; MThd MThdMake(void); void MThdPrint(MThd *mthd) ; typedef struct _MIDITrackHeader { char magic[4]; // = "MTrk" UInt8 trackLength[4]; // Ignore, because it is occasionally wrong. } Track; Track TrackMake(void); void TrackPrint(Track *track) ; #pragma mark - C Functions MThd MThdMake(void) { MThd mthd = { "MThd", {0, 0, 0, 6}, {0, 1}, {0, 0}, {0, 0} }; MThdPrint(&mthd); return mthd; } void MThdPrint(MThd *mthd) { char *ptr = (char *)mthd; for (int i=0;i<sizeof(MThd); i++, ptr++) { printf("%X", *ptr); } printf("\n"); } Track TrackMake(void) { Track track = { "MTrk", {0, 0, 0, 0} }; TrackPrint(&track); return track; } void TrackPrint(Track *track) { char *ptr = (char *)track; for (int i=0;i<sizeof(Track); i++, ptr++) { printf("%X", *ptr); } printf("\n"); } @implementation TestMIDIPlayer - (id)init { self = [super init]; printf("%s %p\n", __FUNCTION__, self); if (self) { _midiTempData = nil; _midiTempURL = [[NSURL alloc]initFileURLWithPath:@"midiTempUrl.mid"]; _midiPlayer = nil; [self createTest:0x0E]; NSLog(@"_midiTempData:%@", _midiTempData); } return self; } - (void)dealloc { [_midiTempData release]; [_midiTempURL release]; [_midiPlayer release]; [super dealloc]; } - (void)createTest:(UInt8)instrument { /* MIDI Header */ [_midiTempData release]; _midiTempData = nil; _midiTempData = [[NSMutableData alloc]initWithCapacity:1024]; MThd mthd = MThdMake(); MThd *ptrMthd = &mthd; ptrMthd->trackCount[1] = 2; ptrMthd->division[1] = 0x60; MThdPrint(ptrMthd); [_midiTempData appendBytes:ptrMthd length:sizeof(MThd)]; /* Track Header Time signature */ Track track = TrackMake(); Track *ptrTrack = &track; ptrTrack->trackLength[3] = 0x14; [_midiTempData appendBytes:ptrTrack length:sizeof(track)]; UInt8 trkEventsTS[]= { 0x00, 0xFF, 0x58, 0x04, 0x04, 0x04, 0x18, 0x08, // Time signature 4/4; 18; 08 0x00, 0xFF, 0x51, 0x03, 0x07, 0xA1, 0x20, // tempo 0x7A120 = 500000 0x83, 0x00, 0xFF, 0x2F, 0x00 }; // End [_midiTempData appendBytes:trkEventsTS length:sizeof(trkEventsTS)]; /* Track Header Track events */ ptrTrack->trackLength[3] = 0x0F; [_midiTempData appendBytes:ptrTrack length:sizeof(track)]; UInt8 trkEvents[] = { 0x00, 0xC0, instrument, // Tubular bell 0x00, 0x90, 0x4C, 0xA0, // Note 4C 0x81, 0x40, 0x48, 0xB0, // TS + Note 48 0x00, 0xFF, 0x2F, 0x00}; // End for (UInt8 i=0; i<3; i++) { printf("0x%X ", trkEvents[i]); } printf("\n"); [_midiTempData appendBytes:trkEvents length:sizeof(trkEvents)]; [_midiTempData writeToURL:_midiTempURL atomically:YES]; dispatch_async(dispatch_get_main_queue(), ^{ if (!_midiPlayer.isPlaying) [self midiPlay]; }); } - (void)midiPlay { NSError *error = nil; _midiPlayer = [[AVMIDIPlayer alloc]initWithData:_midiTempData soundBankURL:nil error:&error]; if (_midiPlayer) { [_midiPlayer prepareToPlay]; [_midiPlayer play:^{ printf("Midi Player ended\n"); [_midiPlayer stop]; [_midiPlayer release]; _midiPlayer = nil; }]; } } @end Call from AppDelegate - (IBAction)actionInstrument:(NSTextField*)sender { [_testMidiplayer createTest:(UInt8)sender.intValue]; }
1
0
435
Dec ’24
Swift iOS CallKit audio resource contention
I noticed the following behavior with CallKit when receiving a VolP push notification: When the app is in the foreground and a CallKit incoming call banner appears, pressing the answer button directly causes the speaker indicator in the CallKit interface to turn on. However, the audio is not actually activated (the iPhone's orange microphone indicator does not light up). In the same foreground scenario, if I expand the CallKit banner before answering the call, the speaker indicator does not turn on, but the orange microphone indicator does light up, and audio works as expected. When the app is in the background or not running, the incoming call banner works as expected: I can answer the call directly without expanding the banner, and the speaker does not turn on automatically. The orange microphone indicator lights up as it should. Why is there a difference in behavior between answering directly from the banner versus expanding it first when the app is in the foreground? Is there a way to ensure consistent audio activation behavior across these scenarios?
0
0
384
Dec ’24
Swift iOS CallKit audio resource contention
I noticed the following behavior with CallKit when receiving a VolP push notification: When the app is in the foreground and a CallKit incoming call banner appears, pressing the answer button directly causes the speaker indicator in the CallKit interface to turn on. However, the audio is not actually activated (the iPhone's orange microphone indicator does not light up). In the same foreground scenario, if I expand the CallKit banner before answering the call, the speaker indicator does not turn on, but the orange microphone indicator does light up, and audio works as expected. When the app is in the background or not running, the incoming call banner works as expected: I can answer the call directly without expanding the banner, and the speaker does not turn on automatically. The orange microphone indicator lights up as it should. Why is there a difference in behavior between answering directly from the banner versus expanding it first when the app is in the foreground? Is there a way to ensure consistent audio activation behavior across these scenarios? I tried reconfiguring the audio when answering a call, but an error occurred during setActive, preventing the configuration from succeeding. let audioSession = AVAudioSession.sharedInstance() do { try audioSession.setActive(false) try audioSession.setCategory(.playAndRecord, mode: .voiceChat, options: [.defaultToSpeaker]) try audioSession.setActive(true, options: []) } catch { print("Failed to activate audio session: \(error)") } action.fulfill() } Error Domain=NSOSStatusErrorDomain Code=561017449 "Session activation failed" UserInfo={NSLocalizedDescription=Session activation failed}
1
0
520
Dec ’24
Fairplay 4k streams decode errors - no video, audio only
HLS live streaming 4k is not displaying any video, but is streaming audio. Getting the following errors in the console where it shows that it is failing to decode every frame. Can I get some help as to what these error codes refer to and why it would fail to decode? 08:30:42.675879-0800 videocodecd AppleAVD: AppleAVDDecodeFrameInternal(): avdDec - Frame# 3588, DecodeFrame failed with error: 0x196 08:30:42.675908-0800 videocodecd AppleAVD: AppleAVDDisplayCallback(): Asking fig to drop frame # 3588 with err -12909 - internalStatus: 315 08:30:42.697412-0800 videocodecd AppleAVD: AppleAVDDecodeFrameResponse(): Frame# 3589 DecodeFrame failed with error 0x00000196 08:30:42.697876-0800 videocodecd AppleAVD: AppleAVDDecodeFrameInternal(): failed - error: 406
0
0
402
Dec ’24
AVMIDIPlayer ignores initial track volume settings on first playback
Issue Description When playing certain MIDI files using AVMIDIPlayer, the initial volume settings for individual tracks are being ignored during the first playback. This results in all tracks playing at the same volume level, regardless of their specified volume settings in the MIDI file. Steps to Reproduce Load a MIDI file that contains different volume settings for multiple tracks Start playback using AVMIDIPlayer Observe that all tracks play at the same volume level, ignoring their individual volume settings Current Behavior All tracks play at the same volume level during initial playback Track volume settings specified in the MIDI file are not being respected This behavior consistently occurs on first playback of affected MIDI files Expected Behavior Each track should play at its specified volume level from the beginning Volume settings in the MIDI file should be respected from the first playback Workaround I discovered that the correct volume settings can be restored by: Starting playback of the MIDI file Setting the currentPosition property to (current time - 1 second) After this operation, all tracks play at their intended volume levels However, this is not an ideal solution as it requires manual intervention and may affect the playback experience. Questions Is there a way to ensure the track volume settings are respected during the initial playback? Is this a known issue with AVMIDIPlayer? Are there any configuration settings or alternative approaches that could resolve this issue? Technical Details iOS Version: 18.1.1 (22B91) Xcode Version: 16.1 (16B40)
2
0
301
Dec ’24
Best way to cache a inifinite scroll view of videos
Hi, Im working on a app with a infinite scrollable video similar to Tiktok or instagram reels. I initially thought it would be a good idea to cache videos in the file system but after reading this post it seems like it is not recommended to cache videos on the file system: https://forums.developer.apple.com/forums/thread/649810#:~:text=If%20the%20videos%20can%20be%20reasonably%20cached%20in%20RAM%20then%20we%20would%20recommend%20that.%20Regularly%20caching%20video%20to%20disk%20contributes%20to%20NAND%20wear The reason I am hesitant to cache videos to memory is because this will add up pretty quickly and increase memory pressure for my app. After seeing the amount of documents and data storage that instagram stores, its obvious they are caching videos on the file system. So I was wondering what is the updated best practice for caching for these kind of apps?
2
0
498
Dec ’24
AVSpeechSynthesizer - just not working on 15.1.1
So get a swift file and put this in it import Foundation import AVFoundation let synthesizer = AVSpeechSynthesizer() let utterance = AVSpeechUtterance(string: "Hello, testing speech synthesis on macOS.") if let voice = AVSpeechSynthesisVoice(identifier: "com.apple.voice.compact.en-GB.Daniel") { utterance.voice = voice print("Using voice: \(voice.name), \(voice.language)") } else { print("Daniel voice not found on macOS.") } synthesizer.speak(utterance) I get no speech output and this log output Error reading languages in for local resources. Error reading languages in for local resources. Using voice: Daniel, en-GB Program ended with exit code: 0 Why? and whats with "Error reading languages in for local resources." ?
3
2
939
Dec ’24
SFCustomLanguageModelData.CustomPronunciation and X-SAMPA string conversion
Can anyone please guide me on how to use SFCustomLanguageModelData.CustomPronunciation? I am following the below example from WWDC23 https://wwdcnotes.com/documentation/wwdcnotes/wwdc23-10101-customize-ondevice-speech-recognition/ While using this kind of custom pronunciations we need X-SAMPA string of the specific word. There are tools available on the web to do the same Word to IPA: https://openl.io/ IPA to X-SAMPA: https://tools.lgm.cl/xsampa.html But these tools does not seem to produce the same kind of X-SAMPA strings used in demo, example - "Winawer" is converted to "w I n aU @r". While using any online tools it gives - "/wI"nA:w@r/".
4
2
829
Dec ’24