Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

Post

Replies

Boosts

Views

Activity

Is AVPlayerViewController not supported on macCatalyst?
When building an application that can be built on iOS using macCatalyst, a link error like the one below will occur. Undefined symbol: OBJC_CLASS$_AVPlayerViewController The AVPlayerViewController documentation seems to support macCatalyst, but what is the reality? [AVPlayerViewController](https://developer.apple.com/documentation/avkit/avplayerviewcontroller? language=objc) Each version of the environment is as follows. Xcode 16.2 macOS deployment target: macOS 10.15 iOS deployment target: iOS 13.0 Thank you for your support.
1
0
93
19h
HDR video metadata
On an iOS 18 phone, I use AVCaptureSession to capture HDR with x420 format. The output CMSampleBuffer is HLG colorspace, the propagated attachments contain kCVImageBufferAmbientViewingEnvironmentKey and kCVImageBufferSceneIlluminationKey. Now I use CAMetalLayer to render the CVPixelBuffer to the screen, but the brightness is brighter than AVSampleBufferDisplayLayer. Here is my code. - (void)_updateColorSpaceIfNeed:(CVPixelBufferRef)pixelBuffer { CAMetalLayer *layer = (CAMetalLayer *)_mtkView.layer; if (![layer isKindOfClass:CAMetalLayer.class]) return; layer.wantsExtendedDynamicRangeContent = YES; CFDataRef ambientViewingEnvironment = (CFDataRef)CVBufferCopyAttachment(pixelBuffer, kCVImageBufferAmbientViewingEnvironmentKey, NULL); NSData *data = (__bridge NSData *)ambientViewingEnvironment; if (ambientViewingEnvironment) CFRelease(ambientViewingEnvironment); CAEDRMetadata *metadata = [CAEDRMetadata HLGMetadataWithAmbientViewingEnvironment:data]; // CAEDRMetadata *metadata = [CAEDRMetadata HLGMetadata]; layer.EDRMetadata = metadata; layer.pixelFormat = MTLPixelFormatRGBA16Float; CGColorSpaceRef colorspace = CGColorSpaceCreateWithName(kCGColorSpaceITUR_2100_HLG); layer.colorspace = colorspace; if (colorspace) CGColorSpaceRelease(colorspace); } Why does the CAEDRMetadata class have "HLGMetadataWithAmbientViewingEnvironment:" and "HLGMetadata" methods, but does not provide the "HLGMetadataWithAmbientViewingEnvironment:sceneIllumination" method? I want to know how kCVImageBufferAmbientViewingEnvironmentKey and kCVImageBufferSceneIlluminationKey affect tone mapping. Is there any documentation I can refer to?
0
0
75
1d
How to reduce CMSampleBuffer volume
Hello, Basically, I am reading and writing an asset. To simplify, I am just reading the asset and rewriting it into an output video without any modifications. However, I want to add a fade-out effect to the last three seconds of the output video. I don’t know how to do this. So far, before adding the CMSampleBuffer to the output video, I tried reducing its volume using an extension on CMSampleBuffer. In the extension, I passed 0.4 for testing, aiming to reduce the video's overall volume by 60%. My question is: How can I directly adjust the volume of a CMSampleBuffer? Here is the extension: extension CMSampleBuffer { func adjustVolume(by factor: Float) -> CMSampleBuffer? { guard let blockBuffer = CMSampleBufferGetDataBuffer(self) else { return nil } var length = 0 var dataPointer: UnsafeMutablePointer<Int8>? guard CMBlockBufferGetDataPointer(blockBuffer, atOffset: 0, lengthAtOffsetOut: nil, totalLengthOut: &length, dataPointerOut: &dataPointer) == kCMBlockBufferNoErr else { return nil } guard let dataPointer = dataPointer else { return nil } let sampleCount = length / MemoryLayout<Int16>.size dataPointer.withMemoryRebound(to: Int16.self, capacity: sampleCount) { pointer in for i in 0..<sampleCount { let sample = Float(pointer[i]) pointer[i] = Int16(sample * factor) } } return self } }
0
0
134
2d
Issues After Switching to v3
I've been having issues with authorization after switching from v1 to v3, have tried some of the suggestions provided by someone in a reply to a post of mine. Have tried to reach out to Apple Support a few times as well, though I haven't received any support that has helped me to move forward. I have tested my token at https://jwt.io and I'm getting a "Signature Verified", tried multiple browsers in private/incognito mode, now when I try to sign into my Apple Account to test my player I am receiving an error that stats "There is a Problem Connecting. There May be an Issue with Your Network." (which is not the case). I have tried everything I can think of, I'm at a loss and would appreciate any help to get my project moving forward. This is what I am seeing in the browser developer console (using Firefox): Authorization failed: AUTHORIZATION_ERROR: Unauthorized MKError https://js-cdn.music.apple.com/musickit/v3/musickit.js:13 authorize https://js-cdn.music.apple.com/musickit/v3/musickit.js:28 asyncGeneratorStep$w https://js-cdn.music.apple.com/musickit/v3/musickit.js:28 _next https://js-cdn.music.apple.com/musickit/v3/musickit.js:28 media.mydomain.com:398:19 https://media.mydomain.com/:398
5
0
308
1w
MusicKit - Authorization failed: AUTHORIZATION_ERROR: Unauthorized
Hello, I am having difficulties with configuring MusicKit correctly for my web app that I am building, seeking assistance with the issues I am having. Would greatly appreciate any help! After allowing access to the following, "Access Request media.mydomain.com would like to access Apple Music, media library, and listening activity for myemail'@icloud.com.", I get a popup error that states, "Authorization failed. Please try again.". Following is the information that is given in developer console: [Error] Failed to load resource: the server responded with a status of 403 () (webPlayerLogout, line 0) [Error] Authorization failed: AUTHORIZATION_ERROR: Unauthorized (anonymous function) (media.mydomain.com:398)
6
0
517
Dec ’24
Call Limit of the Apple Music API
We are planning to develop an application using the Apple Music API. We would like to design our system based on the details of the rate limits mentioned below and have a few questions: https://developer.apple.com/documentation/applemusicapi/generating-developer-tokens#Request-Rate-Limiting Regarding the Catalog API (/v1/catalog/*), we understand that server-side caching is enabled, making it less likely to reach the rate limit. Is this understanding correct? (Excluding the search API) For APIs like the Library API (/v1/me/library/*), where responses vary by user, we assume they are more likely to reach the rate limit. Is this correct? We plan to implement optimizations to minimize unnecessary API calls. Given this, would the current Music API be able to handle a significant increase in users? (Assuming a DAU of around 100,000 to 1,000,000) If the API cannot support this scale, would it be allowed under Apple’s policy to cache responses from the Catalog API (/v1/catalog/*) via our proxy server to avoid hitting the rate limit? The third question is the one we most want to confirm.
0
0
127
5d
SFSpeechRecognizer throws User denied access to speech recognition
I have created an app where you can speak using SFSpeechRecognizer and it will recognize you speech into text, translate it and then return it back using speech synthesis. All locales for SFSpeechRecognizer and switching between them work fine when the app is in the foreground but after I turn off my screen(the app is still running I just turned off the screen) and try to create new recognitionTask it it receives this error inside the recognition task: User denied access to speech recognition. The weird thing about this is it only happens with some languages. The error happens with Croatian or Hungarian locale for speech recognition but doesn't with English or Spanish locale.
0
0
160
6d
HDR10 MVHECV can not play on Safari
Hi, just generated a HDR10 MVHEVC file, mediainfo is below: Color range : Limited Color primaries : BT.2020 Transfer characteristics : PQ Matrix coefficients : BT.2020 non-constant Codec configuration box : hvcC+lhvC then generate the segment files with below command: mediafilesegmenter --iso-fragmented -t 4 -f av_1 av_new_1.mov then upload the segment files and prog_index.m3u8 to web server. just find that can not play the HLS stream on Safari... the url is http://ip/vod/prog_index.m3u8 just checked that if i remove the tag Transfer characteristics : PQ when generating the MVHEVC file. above same mediafilesegmenter command and upload the files to web server. the new version of HLS stream is can play on Safari... Is there any way to play HLS PQ video on Safari. thanks.
2
1
638
May ’24
Musickit Media player missing output device selection
Hi All, I am working on a DJ playout app (MACOS). The app has a few AVAudioPlayerNode's combined with the ApplicationMusicPlayer from Musickit. I can route the output of the AVaudioPlayer to a hardware device so that the audio files are directed to their own dedicated output on my Mac. The ApplicationMusicPlayer is following the default output and this is pretty annoying. Has anyone found a solution to chain the ApplicationMusicPlayer and get it set to a output device? Thanks Pancras
2
0
372
Dec ’24
AVPlayer error: Too many open files
For some users in production, there's a high probability that after launching the App, using AVPlayer to play any local audio resources results in the following error. Restarting the App doesn't help. issue: [error: Error Domain=AVFoundationErrorDomain Code=-11800 "这项操作无法完成" UserInfo={NSLocalizedFailureReason=发生未知错误(24), NSLocalizedDescription=这项操作无法完成, NSUnderlyingError=0x30311f270 {Error Domain=NSPOSIXErrorDomain Code=24 "Too many open files"}} I've checked the code, and there aren't actually multiple AVPlayers playing simultaneously. What could be causing this?
0
0
195
2w
How to Implement Screen Mirroring in iOS for Google TV?
I am developing an iOS application that supports screen mirroring to Google TV (or Chromecast with Google TV). My goal is to mirror the iPhone/iPad screen in real time to a Google TV device. What I Have Tried So Far I have explored multiple approaches but haven't found a direct way to achieve low-latency screen mirroring. Here are some of my findings: Google Cast SDK: Google Cast SDK is primarily designed for casting media (videos, images, audio) rather than real-time mirroring. It supports custom receiver applications, but there are no direct APIs for full screen mirroring. Casting a recorded video is possible, but it introduces latency and is not real-time. ReplayKit for Screen Capture: RPScreenRecorder.shared().startCapture(handler: ...) allows capturing the iPhone screen as a video stream. However, sending this stream to Google TV in real time is a challenge. I could potentially encode the video as HLS and stream it, but the delay is significant. RTSP/UDP Streaming: Some third-party libraries support RTSP/UDP streaming for real-time screen sharing. Google TV does not natively support RTSP, making this approach difficult. My Questions: Is it possible to achieve real-time screen mirroring on Google TV using Google Cast SDK? Does Google TV support WebRTC or any low-latency streaming protocol that can be used from iOS? Are there any alternative approaches to mirror an iOS screen to Google TV with minimal latency? I would appreciate any guidance, code examples, or references to relevant documentation.
0
1
209
2w
Question regarding CarPlay Integration for a Note/Voice Recording App
Hello everyone, I am currently working on an app project aimed at users who want to quickly and easily capture their ideas and notes while on the go. The basic concept is to develop an iOS app where users can store both typed notes and voice recordings – essentially a "brain dump" solution. The core functionality (storing, editing, synchronizing via CloudKit, etc.) will be handled within the iOS app. In addition, I plan to integrate a CarPlay extension that allows the driver to start and stop a recording – ideally through a minimalist interface featuring a large record button and a "Done" button. Since the iPhone is often not within immediate reach in the car, the CarPlay integration should serve as a quick trigger to initiate the recording in the iOS app. My questions are as follows: Has anyone had experience implementing a CarPlay extension for an app that primarily handles notes and voice recordings, rather than falling into the traditional categories like navigation, audio, or communication? Has such a concept ever been approved by Apple, or are there known hurdles and guidelines that must be observed? Are there alternative approaches to implementing CarPlay integration in this context in a compliant and effective manner? I would greatly appreciate any feedback, shared experiences, and tips on best practices. Thank you in advance and best regards!
1
0
257
3w
How to detect a song end?
I'm playing library items (MPMediaItem) and apple music tracks (Track) in MPMusicPlayerApplicationController.applicationQueuePlayer, but I can't use the actual Queue functionality because I can't figure out how to get both media types into the same queue. If there's a way to get both types in a single queue, that would solve my problem, but I've given up on that one. Because I can't use a queue, I have to be able to detect when a song ends so that I can put the next song in the queue and play it. The only way I can figure out to detect when a song ends is by watching the playBackState, and I've actually got that pretty much working, but it's really ugly, because you get playBackState of paused when a song ends, and when a bluetooth speaker disconnects, etc. The only answer I've been able to find on the internet is to watch the MPMusicPlayerControllerNowPlayingItemDidChange, and when that fires, and the nowPlayingItem is NIL, a song ends.. but that's not the case. When a song ends, the nowPlayingItem remains the same. There's got to be an answer to this problem, right?
13
3
4.9k
Aug ’21
ApplicationMusicPlayer / MediaPlayer Refuses to Play
We use BassDSDPlayer / SFBAudioEngine to play just about any file, but playing Apple Music is failing. All subscriptions are up to date. We stop the SFBAudioEngine and the BassDSDPlayer before playing Apple Music to no avail. PRINTS: Supported files in /Users/dorian/Music/Music/Media.localized/Music/4: 28364 Apple Music is authorized and can play catalog. Resetting default output device... Releasing BassDSDPlayer audio device... BassDSDPlayer: Audio device released. STOPPED sfbAudioDevice Default output device is ID: 76 applicationQueuePlayer _establishConnectionIfNeeded timeout [ping did not pong] applicationQueuePlayer _establishConnectionIfNeeded timeout [ping did not pong] Player State - After resetting output: Playback Status: stopped Queue Count: 0 No track is playing. Music player reset successfully. BassDSDPlayer: Audio device released. Default output device set successfully: 76 Default output device is ID: 76 Default output device set successfully: 76 Default output device ID: 76 Validated PlayParameters for track: squabble up PlayParameters: PlayParameters(id: 1781270321, kind: "song", isLibrary: nil, catalogID: nil, libraryID: nil, deviceLocalID: nil, rawValues: [:]) Starting playback... Player State - After playback: Playback Status: stopped Queue Count: 1 No track is playing. Notification BASS DSD NSConcreteNotification 0x600007ce2b00 {name = kUpdateSongInfo; object = { AlbumTitle = GNX; ArtistName = "Kendrick Lamar"; SongArtwork = "<NSImage 0x6000041b7ca0 Size={300, 300} RepProvider=<NSImageArrayRepProvider: 0x600003518770, reps:(\n "NSBitmapImageRep 0x600009ed9dc0 Size={300, 300} ColorSpace=(not yet loaded) BPS=8 BPP=(not yet loaded) Pixels=300x300 Alpha=NO Planar=NO Format=(not yet loaded) CurrentBacking=nil (faulting) CGImageSource=0x600007ce15c0"\n)>>"; SongLength = "157.992"; SongTitle = "squabble up"; Source = AppleMusic; }} Apple Music track loaded: squabble up by Kendrick Lamar Player State - Before play: Playback Status: stopped Queue Count: 1 No track is playing. prepareToPlay failed [no target descriptor] NSError Code: 1, Domain: MPMusicPlayerControllerErrorDomain Player State - After play: Playback Status: stopped Queue Count: 1 No track is playing. func playAppleMusicTracks(tracks: [Track]) { AppleMusicManager.shared.isAuthorizedAndReadyForPlayback { isAuthorized in guard isAuthorized else { print("Apple Music authorization or capabilities insufficient for playback.") return } print("Resetting default output device...") self.stopSFBAudioDevice() self.resetMusicPlayer() self.resetAudioSystem() self.ensureOutputDeviceReady() Task { for track in tracks { guard self.validatePlayParameters(for: track) else { continue } do { try await ApplicationMusicPlayer.shared.queue.insert(track, position: .afterCurrentEntry) guard !ApplicationMusicPlayer.shared.queue.entries.isEmpty else { print("Queue is empty after queuing. Playback cannot proceed.") return } self.notifyAppleMusicTrackInfo(track) } catch { print("Error starting playback: \(error)") if let nsError = error as NSError? { print("NSError Code: \(nsError.code), Domain: \(nsError.domain)") } } } MusicKitWrapper.shared.logPlayerState(message: "After playback") } } } @objc public class MusicKitWrapper: NSObject { @objc public static let shared = MusicKitWrapper() private let player = ApplicationMusicPlayer.shared // Play the current track @objc public func play() { guard !player.queue.entries.isEmpty else { print("Queue is empty. Cannot start playback.") return } logPlayerState(message: "Before play") Task { do { try await player.prepareToPlay() try await player.play() print("Playback started successfully.") } catch { if let nsError = error as NSError? { print("NSError Code: \(nsError.code), Domain: \(nsError.domain)") } } logPlayerState(message: "After play") } } Any help would be appreciated. Thanks!
5
0
386
Dec ’24
Need help with run screen record like root user
Hi, On macOS Sonoma and earlier versions, I used my script along with a LaunchDaemon to continuously record my screen. However, after upgrading to macOS Sequoia, the LaunchDaemon no longer works for screen recording. It only works when I use a LaunchAgent. For my workflow, using a LaunchDaemon and running the ffmpeg process as root is much more efficient than running it as a regular user. Does anyone know how to run a script in the background using a LaunchDaemon on macOS Sequoia? Here are my LaunchDaemon plist and script: LaunchDaemon plist: &lt;?xml version="1.0" encoding="UTF-8"?&gt; &lt;!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"&gt; &lt;?xml version="1.0" encoding="UTF-8"?&gt; &lt;!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"&gt; &lt;plist version="1.0"&gt; &lt;dict&gt; &lt;key&gt;Label&lt;/key&gt; &lt;string&gt;local.ScreenRecord&lt;/string&gt; &lt;key&gt;Disabled&lt;/key&gt; &lt;false/&gt; &lt;key&gt;RunAtLoad&lt;/key&gt; &lt;true/&gt; &lt;key&gt;KeepAlive&lt;/key&gt; &lt;true/&gt; &lt;key&gt;Nice&lt;/key&gt; &lt;integer&gt;-20&lt;/integer&gt; &lt;key&gt;ProgramArguments&lt;/key&gt; &lt;array&gt; &lt;string&gt;/bin/bash&lt;/string&gt; &lt;string&gt;/usr/local/distrib/record.bash&lt;/string&gt; &lt;/array&gt; &lt;/dict&gt; &lt;/plist&gt; Script ## !!! START CONFIGURATION !!! # FFMPEG_LOC="/usr/local/distrib/ffmpeg" FFMPEG_INPUT="-hide_banner -f avfoundation -capture_cursor 1 -pixel_format uyvy422" FFMPEG_VSET="-codec libx264 -r 22 -crf 26 -preset veryfast -b:v 5M -maxrate 7M -bufsize 14M" TIME_REC="-t 600" FOLDER_REC="/usr/local/distrib/REC/" # ## !!! END CONFIGURATION !!! # # Find number id monitor MON_ID=$($FFMPEG_LOC -hide_banner -f avfoundation -list_devices true -i "" 2&gt;&amp;1 | awk -F'[]|[]' '/Capture\ screen/ {print $4}') # if [ -n "$MON_ID" ] then # Time for name DATE_REC=$(date +"%m-%d-%Y_%H-%M-%S") # Number of output video file OUTPUT_NUM=0 # Full command to record FFMPEG_FULL_COMMAND="" for MON_NUM in $MON_ID do FFMPEG_FULL_COMMAND=""$FFMPEG_FULL_COMMAND" "$FFMPEG_INPUT" -i "$MON_NUM" "$FFMPEG_VSET" "$TIME_REC" -map "$OUTPUT_NUM" "$FOLDER_REC""$DATE_REC"_Mon"$MON_NUM".mkv" let OUTPUT_NUM=$OUTPUT_NUM+1 done $FFMPEG_LOC $FFMPEG_FULL_COMMAND else echo "No monitors" sleep 5 fi
1
0
272
Jan ’25
Cannot load iTunesLibrary on macOS Sequoia 15.1
I use the iTunes Library framework in one of my apps, starting with macOS Sequoia 15.1 i can't create the ITLibrary object anymore with the following error: Connection to amplibraryd was interrupted. clientName:iTunesLibrary(ITLibraryLoader) Error connecting to the server via proxy object Error Domain=NSCocoaErrorDomain Code=4097 "connection to service named com.apple.amp.library.framework" UserInfo={NSDebugDescription=connection to service named com.apple.amp.library.framework} configure failed: Error Domain=NSCocoaErrorDomain Code=4097 "connection to service named com.apple.amp.library.framework" UserInfo={NSDebugDescription=connection to service named com.apple.amp.library.framework} I created a new sandboxed macOS app, added the music folder read permission and it reproduced the error: import SwiftUI import iTunesLibrary @main struct ITLibraryLoaderApp: App { var body: some Scene { WindowGroup { ContentView() } } } struct ContentView: View { var body: some View { Button("Load ITLibrary") { loadLibrary() } } func loadLibrary() { do { let _ = try ITLibrary(apiVersion: "1.0", options: .none) } catch { print(error) } } } I restarted my developer machine and the music app with no luck.
2
0
654
Nov ’24