Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

All subtopics

Post

Replies

Boosts

Views

Activity

How to create equalizer for HLS along with AVPlayer?
I saw equalizer in apps like Musi and Spotify. I think (but not sure) they use HLS streaming. If so, how to implement such an equalizer for HLS? I searched and tried several approaches but so far none works, like: AVAudioEngine seems only support local file; Download .ts and merge into .mp3 to make it local can not guarantee real time effect; MTAudioProcessingTap needs the audio track. For remote .mp3 I can extract the audio track but not for HLS. Any suggestion?
0
0
148
1w
launch app by scan NFC Tags
My app need a specific scene that play a video when my iPhone close to NFC Tags. and my app can read the data from NFC Tags, the data will tell us which kind of video can be play. I tried to write URLScheme or Universal Link in NFC Tags, but all this ways will pop up notifications. not launch my app and play a video, how can I design my app. please give me some advice, thanks!
1
0
169
1w
Improving object separation with live depth data
Hey, I'm building a portrait mode into my camera app but I'm having trouble with matching the quality of Apples native camera implementation. I'm streaming the depth data and applying a CIMaskedVariableBlur to the video stream which works quite well but the definition of the object in focus looks quite bad in some scenarios. See comparison below with Apples UI + depth data. What I don't quite understand is how Apple is able to do such a good cutout around my hand assuming it has similar depth data to what I am receiving. You can see in the depth image that my hand is essentially the same colour as parts of background, and this shows in the blur preview - but Apple gets around this. Does anyone have any ideas? Thanks!
0
0
146
1w
FairPlay+AVC+AC-3 on iOS
We found that when enabled FairPlay to streams that uses AVC video codec and AC-3 + MP4 audio codec, the playback fails on some devices with CoreMedia error 1718449215 after started playback for about a second. Successful playback devices: iPhone SE Failed playback devices: iPhone 12 Pro, iPhone 14 Pro
0
0
107
1w
Bluetooth audio becomes choppy on iOS with entitlement error but works just fine on MacCatalyst
I am converting some old objective-C code deployed on ios 12 to swift in a WKWebView app. Im also developing the app for Mac via MacCatalyst. the issue im experiencing relates to a programmable learning bot that is programmed via block coding and the app facilitates the read and writes back and forth. the audio works via a A2DP connection the user sets manually in their settings, while the actual movement of the robot is controlled via a BLE connection. Currently the code works as intended on MacCatalyst, while on iPhone, the audio being sent back to the robot is very choppy and sometimes doesn't play at all. I apologize for the length of this, but there is a bit to unpack here. First, I know there has been a few threads posted about this issue, this one that seems similar but went unsolved https://forums.developer.apple.com/forums/thread/740354 as well as this one where apple says it is "log noise" https://forums.developer.apple.com/forums/thread/742739 However I just find it hard to believe that this issue seems to be log noise in this case. Mac Catalyst uses a legacy header file for WebKit, and im wondering if that could be part of the issue here.I have enable everything relating to bluetooth in my info plist file as the developer documents say. In my app sandbox for mac catalyst I have the permissions set for bluetooth as well there. Here are snippets of my read and write function func readFunction(session: String){ // Wait if we are still waiting to hear from the robot if self.serialRxBuf == ""{ self.emptyReadCount += 1 } if (!self.serialRxWaiting){ return } // Make sure we are waiting for the correct session if (Int(session) != self.serialRxSession){ return } self.serialRxWaiting = false self.serialRxSession += 1 let buf = self.serialRxBuf self.serialRxBuf = "" print("sending Read: \(buf)") self.MainWebView.evaluateJavaScript(""" if (serialPort.onRead) { serialPort.onRead("\(buf)"); } serialPort.onRead = null; """ ,completionHandler: nil) } // ----- Write function for javascript bluetooth interface ----- func writeFunction(buf: String) -> Bool { emptyReadCount = 0 if((self.blePeripheral == nil) || (self.bleCharacteristic == nil) || self.blePeripheral?.state != .connected){ print("write result: bad state, peripheral, or connection ") // in case we recieve an error that will freeze react side, safely navigate and clear bluetooth information. if MainWebView.canGoBack{ MainWebView.reload() showDisconnectedAlert() self.centralManager = nil // we will just start over next time self.blePeripheral = nil self.bleCharacteristic = nil self.connectACD2Failed() return false } return false } var data = Data() var byteStr = "" for i in stride(from: 0, to: buf.count, by: 2) { let startIndex = buf.index(buf.startIndex, offsetBy: i) let endIndex = buf.index(startIndex, offsetBy: 2) byteStr = String(buf[startIndex..<endIndex]) let byte = UInt8(byteStr, radix: 16)! data.append(byte) } guard let connectedCharacteristic = self.bleCharacteristic else { print("write result: Failure to assign bleCharacteristic") return false } print("sending bleWrite: \(String(describing: data))") self.blePeripheral.writeValue(data, for: connectedCharacteristic, type: .withoutResponse) print("write result: True") return true } Here is what the log looks like when running on mac catalyst, which works just fine sending bleWrite: 20 bytes write result: True sending Read: sending Read: 55AA55AA0B0040469EE6000000000000000000ED sending bleWrite: 20 bytes write result: True sending Read: sending Read: sending Read: 55AA55AA0B0040469EE6000000000000000000ED sending bleWrite: 20 bytes write result: True sending Read: 55AA55AA0B0040469EE6000000000000000000ED sending bleWrite: 20 bytes write result: True sending Read: 55AA55AA0B0040EDCB09000000000000000000ED sending bleWrite: 20 bytes write result: True sending Read: sending Read: 55AA55AA0B00407A7B96000000000000000000ED sending bleWrite: 20 bytes write result: True Error acquiring assertion: <Error Domain=RBSServiceErrorDomain Code=1 "(originator doesn't have entitlement com.apple.runningboard.assertions.webkit AND originator doesn't have entitlement com.apple.multitasking.systemappassertions)" UserInfo={NSLocalizedFailureReason=(originator doesn't have entitlement com.apple.runningboard.assertions.webkit AND originator doesn't have entitlement com.apple.multitasking.systemappassertions)}> 0x12c0380e0 - ProcessAssertion::acquireSync Failed to acquire RBS assertion 'WebKit Media Playback' for process with PID=36540, error: Error Domain=RBSServiceErrorDomain Code=1 "(originator doesn't have entitlement com.apple.runningboard.assertions.webkit AND originator doesn't have entitlement com.apple.multitasking.systemappassertions)" UserInfo={NSLocalizedFailureReason=(originator doesn't have entitlement com.apple.runningboard.assertions.webkit AND originator doesn't have entitlement com.apple.multitasking.systemappassertions)} and here is the log from when we are running the code on iPhone (trying to save space here) I apologize for the length of this post, however submitting a test project to apple developer support just isn't possible with the device thats in use. Any help at all is appreciated. i've looked at every permission, entitlement, background processing, and tried every solution that I could find to no avail.
0
0
253
1w
-[AVCaptureSession init] has a watchdog issue in iOS 17
My app has encountered many watchdog issues on iOS 17, with stack traces as follows: Attributed: Call stack 0: mach_msg2_trap (in libsystem_kernel.dylib) + 7 mach_msg2_internal (in libsystem_kernel.dylib) + 79 mach_msg_overwrite (in libsystem_kernel.dylib) + 435 mach_msg (in libsystem_kernel.dylib) + 23 _dispatch_mach_send_and_wait_for_reply (in libdispatch.dylib) + 543 dispatch_mach_send_with_result_and_wait_for_reply (in libdispatch.dylib) + 59 xpc_connection_send_message_with_reply_sync (in libxpc.dylib) + 263 FigXPCConnectionSendSyncMessageCreatingReply (in CoreMedia) + 291 FigXPCRemoteClientSendSyncMessageCreatingReply (in CoreMedia) + 47 FigCaptureSessionRemoteCreate (in CMCapture) + 131 -[AVCaptureSession _createFigCaptureSession] (in AVFCapture) + 123 -[AVCaptureSession _initWithMediaEnvironment:] (in AVFCapture) + 619 -[AVCaptureSession init] (in AVFCapture) + 415 We also have many iOS 16 users, but have never encountered a watchdog issue with the AVCaptureSession init method in iOS 16. Is there any change in iOS 17 that could have caused this? How can I avoid this issue? The complete stack trace is attached avfoundation-watchdog.txt
1
0
162
1w
How to migrate PHPhotoLibraryChangeObserver to Swift 6?
I have the following code: extension AssetGridViewController: PHPhotoLibraryChangeObserver { func photoLibraryDidChange(_ changeInstance: PHChange) { Task { @MainActor in guard let changes = changeInstance.changeDetails(for: fetchResult) else { return } fetchResult = changes.fetchResultAfterChanges } } } With Swift 6, this generates a compilation error: Main actor-isolated instance method 'photoLibraryDidChange' cannot be used to satisfy nonisolated protocol requirement. The error includes to fix-it suggestions: Adding nonisolated to the function (nonisolated func photoLibraryDidChange(_ changeInstance: PHChange)) Adding @preconcurrency to the protocol conformance (extension AssetGridViewController: @preconcurrency PHPhotoLibraryChangeObserver {) Both options generate a runtime error: EXC_BREAKPOINT (code=1, subcode=0x105b7c400). For context, AssetGridViewController is a regular UIViewController. Any ideas on how to fix this?
2
0
166
2w
ERROR 1852797029
I have a new iPhone 15 PRO and some new Usb-C Earphones too, both are 3 days old. Since the first day I have been having this Error 1852797029. I can be listening to music on Apple Music for a while but when I stop it and a while passes without resuming playback, when I resume it it gives me this error and I have to close the App and disconnecting and connecting the earphones again. It's very annoying and I'm very angry that this is happening to me from day one. With both devices completely new. Does anyone have a solution other than connecting and disconnecting the earphones?
1
0
197
2w
Ability to hide/show tvOS AVPlayerViewController's progress bar
I'm working on streaming tvOS app and as you know there are mostly two type of video streams - live and vod. AVPlayerViewController handles these types of streams by showing respective playback controls. Recently I got a task to implement synchronous vod playback(syncVod), it's when we need to simulate live playback while actual vod stream playback. In order to simulate live playback below things needs to be handled: Disabling scrubbing via remote. (Done. playerVc.requiresLinearPlayback = true) Disabling info panel view w/play "From beginning" button. (Done, playerVc.playbackControlsIncludeInfoViews = false) Disabling play/pause button.(Done, not ideally though. On rate change observer - if player.rate == 0 && playbackMode == .syncVod { player.play() return }). Why not ideal solution - tapping on remote causes quite short hiccup in playback - but playback resumes, no actual pause happens. Hiding progress bar and time labels. :( Point #4 is the main problem, we can't hide progress bar and it's related UI elements(time labels) particularly, but only hide all playback controls - playerVc.showsPlaybackControls = false. The thing is I have custom buttons in transportBarCustomMenuItems and hiding all playback controls is not the right option for me. Implementing custom playback controls panel is kind of heavy lift, but as of now it seems the only proper way of implementing syncVod playback ideally. Did anyone face similar issue and could resolve it w/out implementing custom playback controls panel ? Is there way to hide progress bar only in tvOS AVPlayerViewController?
0
0
153
2w
Disabling the Ability to use recorder while using my app
I have an application which is based on a video streaming service. A critical thing about this app is that these videos must be secured all the way and can't be pirated. The current problem I have is that anyone can use voice recorder to capture my videos audio which isn't suitable this app purpose. My question is Is there anyway that I can disable using any voice recording apps (or the microphone) and detect if someone tried to while streaming videos from my app? Thanks in advance
0
1
132
2w
IOSurface vs. IOSurfaceRef on Catalyst
I have an IOSurface and I want to turn that into a CIImage. However, the constructor of CIImage takes a IOSurfaceRef instead of a IOSurface. On most platforms, this is not an issue because the two types are toll-free bridgeable... except for Mac Catalyst, where this fails. I observed the same back in Xcode 13 on macOS. But there I could force-cast the IOSurface to a IOSurfaceRef: let image = CIImage(ioSurface: surface as! IOSurfaceRef) This cast fails at runtime on Catalyst. I found that unsafeBitCast(surface, to: IOSurfaceRef.self) actually works on Catalyst, but it feels very wrong. Am I missing something? Why aren't the types bridgeable on Catalyst? Also, there should ideally be an init for CIImage that takes an IOSurface instead of a ref.
2
1
147
2w
Disable iOS Screen Mirroring for Apps
Hello Apple, I am concerned about the new iOS Screen Mirroring that is available on iOS. I have an app that is only meant to be viewed on iPhones (not Macs or Computers, due to security reasons. I am assuming that Screen Mirroring is using AirPlay underneath, otherwise is there an API being planned or coming that can disable this functionality or is there a way for my app to opt out out of iOS Screen Mirroring? Thanks.
1
0
239
2w
Thread safety of AudioObject APIs
Are the AudioObject APIs (such as AudioObjectGetPropertyData, AudioObjectSetPropertyData, etc.) thread-safe? Meaning, for the same AudioObjectID is it safe to do things like: Get a property in one thread while setting the same property in another thread Set the same property in two different threads Add and remove property listeners in different threads Put differently, is there any internal synchronization or mutex for this kind of usage or is the burden on the caller? I was unable to find any documentation either way which makes me think that the APIs are not thread-safe.
0
2
164
2w
Music is not being played through BLE connected headphone on iPhone
I have created a demo iOS app to create BLE connection with surrounding headphone. I am able to connect to headphone successfully through my demo iOS app. I can also see in iPhone Bluetooth Setting that headphone is connected but when i am playing music from Spotify/YouTube then music is not being played through headphone. It is still using iPhone speakers. First i am scanning sarounding bluetooth Devices through CBCentralManager and then connecting one of the found device. cBCenteralManager.scanForPeripherals(withServices: nil, options: nil) For connecting: cBCenteralManager.connect(peripheral, options: nil) Do i need to make any code changes while connecting via BLE? I am expecting when i am connecting to headphone via my Demo app. Same connection is visible in iPhone Bluetooth setting too then when i play music on spotify/youtube then sound should be played on headphone and not on iPhone speakers.
2
0
184
2w
ImmersiveSpaceを切り替えるとAVAudioPlayerで再生していたBGMの音が聞こえなくなる / When switching ImmersiveSpace, background music played by AVAudioPlayer is not heard.
【手順】 1.アプリを起動する。 2.ImmersiveSpace1がopenされ、3Dオブジェクトのアニメーションが再生される。 3.アニメーションが終了するとImmersiveSpace1をdismissしてImmersiveSpace2をopenする。 【期待値】 ImmersiveSpace1をopenするとBGMが再生され、ImmersiveSpace2がopenしても引き続きBGMが再生されていること。 【結果】 ImmersiveSpace1をopenするとBGMが再生され、ImmersiveSpace2がopenするとBGMの再生が止まる。 【環境】 ・実機(VisionOS2)にて発生。 ・シミュレータでは発生しない。 ・Xcode:Version 15.2 (15C500b) 【ログ】 ImmersiveSpace2をopenした際に実機で出力されている。シミュレータでは出力されない。 AVAudioSession_iOS.mm:2223 Server returned an error from destroySession:. Error Domain=NSCocoaErrorDomain Code=4099 "The connection to service with pid 39 named com.apple.audio.AudioSession was invalidated from this process." UserInfo={NSDebugDescription=The connection to service with pid 39 named com.apple.audio.AudioSession was invalidated from this process.} 【Procedure】 Launch the application. ImmersiveSpace1 is opened and the animation of the 3D object is played. When the animation finishes, ImmersiveSpace1 is dismissed and ImmersiveSpace2 is opened. 【Expected value】 When ImmersiveSpace1 is opened, the background music should play, and when ImmersiveSpace2 is opened, the background music should continue to play. 【Result】 When ImmersiveSpace1 is opened, the BGM is played, and when ImmersiveSpace2 is opened, the BGM stops playing. 【Environment】 This problem occurs on an actual machine (VisionOS2). It does not occur on the simulator. Xcode: Version 15.2 (15C500b) 【Log】 Output on actual device when ImmersiveSpace is opened. It is not output on the simulator. AVAudioSession_iOS.mm:2223 Server returned an error from destroySession:. Error Domain=NSCocoaErrorDomain Code=4099 "The connection to service with pid 39 named com.apple.audio.AudioSession was invalidated from this UserInfo={NSDebugDescription=The connection to service with pid 39 named com.apple.audio.AudioSession was invalidated from this AudioSession was invalidated from this process.}
0
2
167
2w