Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

All subtopics

Post

Replies

Boosts

Views

Activity

Voice recorder app recording in dual mono instead of stereo
Hi y'all, After getting mono recording working, I want to differentiate my app from the standard voice memos to allow for stereo recording. I followed this tutorial (https://developer.apple.com/documentation/avfaudio/capturing_stereo_audio_from_built-in_microphones) to get my voice recorder to record stereo audio. However, when I look at the waveform in Audacity, both channels are the same. If I look at the file info after sharing it, it says the file is in stereo. I don't exactly know what's going on here. What I suspect is happening is that the recorder is only using one microphone. Here is the relevant part of my recorder: // MARK: - Initialization override init() { super.init() do { try configureAudioSession() try enableBuiltInMicrophone() try setupAudioRecorder() } catch { // If any errors occur during initialization, // terminate the app with a fatalError. fatalError("Error: \(error)") } } // MARK: - Audio Session and Recorder Configuration private func enableBuiltInMicrophone() throws { let audioSession = AVAudioSession.sharedInstance() let availableInputs = audioSession.availableInputs guard let builtInMicInput = availableInputs?.first(where: { $0.portType == .builtInMic }) else { throw Errors.NoBuiltInMic } do { try audioSession.setPreferredInput(builtInMicInput) } catch { throw Errors.UnableToSetBuiltInMicrophone } } private func configureAudioSession() throws { let audioSession = AVAudioSession.sharedInstance() do { try audioSession.setCategory(.record, mode: .default, options: [.allowBluetooth]) try audioSession.setActive(true) } catch { throw Errors.FailedToInitSessionError } } private func setupAudioRecorder() throws { let date = Date() let dateFormatter = DateFormatter() dateFormatter.locale = Locale(identifier: "en_US_POSIX") dateFormatter.dateFormat = "yyyy-MM-dd, HH:mm:ss" let timestamp = dateFormatter.string(from: date) self.recording = Recording(name: timestamp) guard let fileURL = recording?.returnURL() else { fatalError("Failed to create file URL") } self.currentURL = fileURL print("Recording URL: \(fileURL)") do { let audioSettings: [String: Any] = [ AVFormatIDKey: Int(kAudioFormatMPEG4AAC), AVLinearPCMIsNonInterleaved: false, AVSampleRateKey: 44_100.0, AVNumberOfChannelsKey: isStereoSupported ? 2 : 1, AVLinearPCMBitDepthKey: 16, AVEncoderAudioQualityKey: AVAudioQuality.max.rawValue ] audioRecorder = try AVAudioRecorder(url: fileURL, settings: audioSettings) } catch { throw Errors.UnableToCreateAudioRecorder } audioRecorder.delegate = self audioRecorder.prepareToRecord() } //MARK: update orientation public func updateOrientation(withDataSourceOrientation orientation: AVAudioSession.Orientation = .front, interfaceOrientation: UIInterfaceOrientation) async throws { let session = AVAudioSession.sharedInstance() guard let preferredInput = session.preferredInput, let dataSources = preferredInput.dataSources, let newDataSource = dataSources.first(where: { $0.orientation == orientation }), let supportedPolarPatterns = newDataSource.supportedPolarPatterns else { return } isStereoSupported = supportedPolarPatterns.contains(.stereo) if isStereoSupported { try newDataSource.setPreferredPolarPattern(.stereo) } try preferredInput.setPreferredDataSource(newDataSource) try session.setPreferredInputOrientation(interfaceOrientation.inputOrientation) } Here is the relevant part of my SwiftUI view: RecordView() .onAppear {             Task {                 if await AVAudioApplication.requestRecordPermission() {                     // The user grants access. Present recording interface.                     print("Permission granted")                 } else {                     // The user denies access. Present a message that indicates                     // that they can change their permission settings in the                     // Privacy & Security section of the Settings app.                     model.showAlert.toggle()                 }                 try await recorder.updateOrientation(interfaceOrientation: deviceOrientation)             }         }         .onReceive(NotificationCenter.default.publisher(for: UIDevice.orientationDidChangeNotification)) { _ in                     if let windowScene = UIApplication.shared.connectedScenes.first as? UIWindowScene,                        let orientation = windowScene.windows.first?.windowScene?.interfaceOrientation {                         deviceOrientation = orientation                         Task {                             do {                                 try await recorder.updateOrientation(interfaceOrientation: deviceOrientation)                             } catch {                                 throw Errors.UnableToUpdateOrientation                             }                         }                     }                 } Here is the full repo: https://github.com/aabagdi/MemoMan/tree/MemoManStereo Thanks for any leads!
1
0
45
5h
Input location of AVAudioSession are different between iPhone
Position of AVAudioSession is different when I use the speaker. try session.setCategory(.playAndRecord, mode: .voiceChat, options: []) try session.overrideOutputAudioPort(.speaker) try session.setActive(true) let route = session.currentRoute route.inputs.forEach{ input in print(input.selectedDataSource?.location) } In iPhone 11(iOS 17.5.1), AVAudioSessionLocation: Lower In iPhone 7 Plus(iOS 15.8.2), AVAudioSessionLocation: Upper What causes this difference in behavior?
0
0
76
21h
AVAudioSessionErrorCodeCannotInterruptOthers
We are to judge the AVAudioSessionInterruptionOptionShouldResume, to restore the audio playback. We have been online for a long time and have been able to resume audio playback normally. But recently we've had a lot of user feedback as to why the audio won't resume playing. Based on this feedback, we checked and found that there were some apps that did not play audio but occupied audio all the time. For example, when a user was using the wechat app, after sending a voice message, we received a notification to resume audio playback, and wechat did not play audio either. But we resume play times wrong AVAudioSessionErrorCodeCannotInterruptOthers. After that, we gave feedback to the wechat app and fixed the problem. But we still have some users feedback this problem, we do not know which app is maliciously occupying audio, so we do not know which aspect to troubleshoot the problem. We pay close attention to user feedback and hope it can help us solve user experience problems.
0
0
49
21h
Exporting Audio with Scaled Segments Causes copyNextSampleBuffer to Hang
I am trying to export an AVMutableComposition with a single audio track. This track has a scaled AVCompositionTrackSegment to simulate speed changes up to 20x. I need to use AVAssetWriter and AVAssetReader classes for this task. When I scale the source duration up to a maximum of 5x, everything works fine. However, when I scale it to higher speeds, such as 20x, the app hangs on the copyNextSampleBuffer method. I'm not sure why this is happening and how to prevent it. Also, this often happens if the exported audio track has segments with different speeds. (The duration of the audio file in the example is 47 seconds.) Example of code: class Export { func startExport() { let inputURL = Bundle.main.url(forResource: "Piano", withExtension: ".m4a")! let documentsDirectory = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first! let outputURL = documentsDirectory.appendingPathComponent("Piano20x.m4a") try? FileManager.default.removeItem(at: outputURL) print("Output URL: \(outputURL)") changeAudioSpeed(inputURL: inputURL, outputURL: outputURL, speed: 20) } func changeAudioSpeed(inputURL: URL, outputURL: URL, speed: Float) { let urlAsset = AVAsset(url: inputURL) guard let assetTrack = urlAsset.tracks(withMediaType: .audio).first else { return } let composition = AVMutableComposition() let compositionAudioTrack = composition.addMutableTrack(withMediaType: .audio, preferredTrackID: kCMPersistentTrackID_Invalid) do { try compositionAudioTrack?.insertTimeRange(CMTimeRangeMake(start: .zero, duration: assetTrack.timeRange.duration), of: assetTrack, at: .zero) } catch { print("Failed to insert audio track: \(error)") return } let scaledDuration = CMTimeMultiplyByFloat64(assetTrack.timeRange.duration, multiplier: Double(1.0 / speed)) compositionAudioTrack?.scaleTimeRange(CMTimeRangeMake(start: .zero, duration: assetTrack.timeRange.duration), toDuration: scaledDuration) print("Scaled audio from \(assetTrack.timeRange.duration.seconds)sec to \(scaledDuration.seconds) sec") compositionAudioTrack?.segments do { let compositionAudioTracks = composition.tracks(withMediaType: .audio) let assetReader = try AVAssetReader(asset: composition) let audioSettings: [String: Any] = [ AVFormatIDKey: kAudioFormatLinearPCM, AVSampleRateKey: 44100, AVNumberOfChannelsKey: 2, AVLinearPCMBitDepthKey: 16, AVLinearPCMIsBigEndianKey: false, AVLinearPCMIsFloatKey: false, AVLinearPCMIsNonInterleaved: false ] let readerOutput = AVAssetReaderAudioMixOutput(audioTracks: compositionAudioTracks, audioSettings: audioSettings) assetReader.add(readerOutput) let assetWriter = try AVAssetWriter(outputURL: outputURL, fileType: .m4a) let writerInput = AVAssetWriterInput(mediaType: .audio, outputSettings: audioSettings) assetWriter.add(writerInput) assetReader.startReading() assetWriter.startWriting() assetWriter.startSession(atSourceTime: .zero) let conversionQueue = DispatchQueue(label: "ConversionQueue") writerInput.requestMediaDataWhenReady(on: conversionQueue) { while writerInput.isReadyForMoreMediaData { if let sampleBuffer = readerOutput.copyNextSampleBuffer() { // APP hangs here!!! writerInput.append(sampleBuffer) } else { writerInput.markAsFinished() assetWriter.finishWriting { print("Export completed successfully") } break } } } } catch { print("Failed with error: \(error)") } } }
0
0
49
1d
Is one step folder creation possible in Finder ?
I need to disallow folder renaming on my NSFileProviderReplicatedExtension extension as my foreign API system changes the asset id when a folder gets renamed and I can't retrieve or compute the new ID. For this reason I disable .allowsRenaming on folders for this volume but the Finder will still trigger the two step create "untitled folder" / rename when creating new folders. I can see that existing folders can't be renamed on the volume so the capability seems properly managed by my extension. Is there a way to prevent the creation of the "untitled folder" folder when creating new folders in Finder on a folder with renaming disabled ?
0
0
53
1d
API to get AVSpeechSynthesisVoice that are available to download (via Settings.app) but not yet downloaded to a device
Here is the use case, I have a language learning app that uses AVSpeechSynthesizer ❤️. When a user listens to a phrase with the AVSpeechSynthesizer using a AVSpeechSynthesisVoice with a AVSpeechSynthesisVoiceQuality of default it sounds much much worse than voices with enhanced or premium, really affecting usability. There appears to be no API for the app to know if there are enhanced or premium voices available to download (via Settings.app) but not yet downloaded to a device. The only API I could find is AVSpeechSynthesisVoice.speechVoices() which returns all available voices on the device, but not a full list of voices available via download. So the app cannot know if it should inform the user "hey this voice your listening to is a much lower quality than enhanced or premium, go to settings and download the enhanced or premium version". Any ideas? Do I need to send in an enhancement request via Feedback Assistant? Thank you for helping my users ears and helping them turn speech synthesis voice quality up to 11 when it's available to them with just a couple of extra taps! (I suppose the best workaround is to display a warning every time the user is using a default quality voice, I wonder what % of voices have enhanced or premium versions...)
0
0
89
1d
Issue with iPhone QR Code Contact Preview Displaying Email Instead of Organization
Post Content: Hi everyone, I’m encountering an issue with how iPhone displays contact information from a vCard QR code in the contact preview. When I scan the QR code with my iPhone camera, the contact preview shows the email address between the name and the contact image, instead of displaying the organization name. Here’s the structure of the vCard I’m using: BEGIN:VCARD VERSION:3.0 FN:Ahmad Rana N:Rana;Ahmad;;; ORG:Company 3 TEL;TYPE=voice,msg:+1234567890 EMAIL:a(at the rate)gmail.com URL:https://example.com IMPP:facebook:fb END:VCARD What I Expect: When I scan it with camera and in the contact preview before creating the camera I want organization name between name and image of the preview but I get email instead of ogrganization name. If only organisation is passed then it displays correctly but when I pass email it displayed email in between. Steps I’ve Taken: Verified the vCard structure to ensure it follows the standard format. Reordered the fields in the vCard to prioritize the organization name and job title. Tested with a simplified vCard containing only the name, organization, and email. Despite these efforts, the email address continues to be displayed in the contact preview between the name and the contact image, while the organization name is not shown as expected. Question: How can I ensure that the organization name is displayed correctly in the contact preview on iPhone when scanning a QR code? Are there specific rules or best practices for field prioritization in vCards that I might be missing? I would appreciate any insights or suggestions on how to resolve this issue. Thank you!
0
0
109
2d
Updating metadata properties of a DNG (or other) format image
Hi, I'm looking to update the metadata properties of a DNG image stored on disc, saving to a new file. Using ImageIO's CGImageSource and CGImageDestination classes, I run into a problem where by the destination doesn't support the type of the source. For example: let imageSourceOptions = [kCGImageSourceShouldCache: false] as CFDictionary if let cgImageSource = CGImageSourceCreateWithURL(sourceURL as CFURL, imageSourceOptions), let type = CGImageSourceGetType(cgImageSource) { guard let imageDestination = CGImageDestinationCreateWithURL(destinationURL as CFURL, type, 1, nil) else { fatalError("Unable to create image destination") } // Code to update properties and write out to destination url } } When this code is executed I get the following errors on the command line when trying to create the destination: 2024-06-30 11:52:25.531530+0100 ABC[7564:273101] [ABC] findWriterForTypeAndAlternateType:119: *** ERROR: unsupported output file format 'com.adobe.raw-image' 2024-06-30 11:52:25.531661+0100 ABC[7564:273101] [ABC] CGImageDestinationCreateWithURL:4429: *** ERROR: CGImageDestinationCreateWithURL: failed to create 'CGImageDestinationRef' I don't see a way to create a destination directly from a source? Yes, the code works for say a JPEG file but I want it to work for any image format that CGImageSource can work with?
0
0
88
2d
Getting the amount of audio loaded in an AVPlayer
I have AVPlayer loading an MP3 from a URL. While it is playing, how can I tell how much of the MP3 has been actually downloaded so far? I have tried using item.loadedTimeRanges, but it does not seem to be accurate. I get a few notifications but it usualy stops sending notifications around 80 seconds and doesn't keep up to even the current position of the player. Any idea? Also, is there any way to get the total duration of the audio? All the methods I've tried return NAN.
0
0
87
3d
How to create equalizer for HLS along with AVPlayer?
I saw equalizer in apps like Musi and Spotify. I think (but not sure) they use HLS streaming. If so, how to implement such an equalizer for HLS? I searched and tried several approaches but so far none works, like: AVAudioEngine seems only support local file; Download .ts and merge into .mp3 to make it local can not guarantee real time effect; MTAudioProcessingTap needs the audio track. For remote .mp3 I can extract the audio track but not for HLS. Any suggestion?
0
0
106
4d
launch app by scan NFC Tags
My app need a specific scene that play a video when my iPhone close to NFC Tags. and my app can read the data from NFC Tags, the data will tell us which kind of video can be play. I tried to write URLScheme or Universal Link in NFC Tags, but all this ways will pop up notifications. not launch my app and play a video, how can I design my app. please give me some advice, thanks!
1
0
110
4d
Improving object separation with live depth data
Hey, I'm building a portrait mode into my camera app but I'm having trouble with matching the quality of Apples native camera implementation. I'm streaming the depth data and applying a CIMaskedVariableBlur to the video stream which works quite well but the definition of the object in focus looks quite bad in some scenarios. See comparison below with Apples UI + depth data. What I don't quite understand is how Apple is able to do such a good cutout around my hand assuming it has similar depth data to what I am receiving. You can see in the depth image that my hand is essentially the same colour as parts of background, and this shows in the blur preview - but Apple gets around this. Does anyone have any ideas? Thanks!
0
0
101
4d
FairPlay+AVC+AC-3 on iOS
We found that when enabled FairPlay to streams that uses AVC video codec and AC-3 + MP4 audio codec, the playback fails on some devices with CoreMedia error 1718449215 after started playback for about a second. Successful playback devices: iPhone SE Failed playback devices: iPhone 12 Pro, iPhone 14 Pro
0
0
67
4d
-[AVCaptureSession init] has a watchdog issue in iOS 17
My app has encountered many watchdog issues on iOS 17, with stack traces as follows: Attributed: Call stack 0: mach_msg2_trap (in libsystem_kernel.dylib) + 7 mach_msg2_internal (in libsystem_kernel.dylib) + 79 mach_msg_overwrite (in libsystem_kernel.dylib) + 435 mach_msg (in libsystem_kernel.dylib) + 23 _dispatch_mach_send_and_wait_for_reply (in libdispatch.dylib) + 543 dispatch_mach_send_with_result_and_wait_for_reply (in libdispatch.dylib) + 59 xpc_connection_send_message_with_reply_sync (in libxpc.dylib) + 263 FigXPCConnectionSendSyncMessageCreatingReply (in CoreMedia) + 291 FigXPCRemoteClientSendSyncMessageCreatingReply (in CoreMedia) + 47 FigCaptureSessionRemoteCreate (in CMCapture) + 131 -[AVCaptureSession _createFigCaptureSession] (in AVFCapture) + 123 -[AVCaptureSession _initWithMediaEnvironment:] (in AVFCapture) + 619 -[AVCaptureSession init] (in AVFCapture) + 415 We also have many iOS 16 users, but have never encountered a watchdog issue with the AVCaptureSession init method in iOS 16. Is there any change in iOS 17 that could have caused this? How can I avoid this issue? The complete stack trace is attached avfoundation-watchdog.txt
1
0
109
5d
ERROR 1852797029
I have a new iPhone 15 PRO and some new Usb-C Earphones too, both are 3 days old. Since the first day I have been having this Error 1852797029. I can be listening to music on Apple Music for a while but when I stop it and a while passes without resuming playback, when I resume it it gives me this error and I have to close the App and disconnecting and connecting the earphones again. It's very annoying and I'm very angry that this is happening to me from day one. With both devices completely new. Does anyone have a solution other than connecting and disconnecting the earphones?
1
0
125
6d
Ability to hide/show tvOS AVPlayerViewController's progress bar
I'm working on streaming tvOS app and as you know there are mostly two type of video streams - live and vod. AVPlayerViewController handles these types of streams by showing respective playback controls. Recently I got a task to implement synchronous vod playback(syncVod), it's when we need to simulate live playback while actual vod stream playback. In order to simulate live playback below things needs to be handled: Disabling scrubbing via remote. (Done. playerVc.requiresLinearPlayback = true) Disabling info panel view w/play "From beginning" button. (Done, playerVc.playbackControlsIncludeInfoViews = false) Disabling play/pause button.(Done, not ideally though. On rate change observer - if player.rate == 0 && playbackMode == .syncVod { player.play() return }). Why not ideal solution - tapping on remote causes quite short hiccup in playback - but playback resumes, no actual pause happens. Hiding progress bar and time labels. :( Point #4 is the main problem, we can't hide progress bar and it's related UI elements(time labels) particularly, but only hide all playback controls - playerVc.showsPlaybackControls = false. The thing is I have custom buttons in transportBarCustomMenuItems and hiding all playback controls is not the right option for me. Implementing custom playback controls panel is kind of heavy lift, but as of now it seems the only proper way of implementing syncVod playback ideally. Did anyone face similar issue and could resolve it w/out implementing custom playback controls panel ? Is there way to hide progress bar only in tvOS AVPlayerViewController?
0
0
95
6d