Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

All subtopics

Post

Replies

Boosts

Views

Activity

How to correctly load video selected with PHPickerViewController?
Hello! I am playing around with the PHPickerViewController and so far I was able to get the selected images by loading them into UIImage instances but I don't know how to get the selected video. Below is the relevant implementation of the method: func picker(_ picker: PHPickerViewController, didFinishPicking results: [PHPickerResult]): let provider = result.itemProvider guard provider.hasItemConformingToTypeIdentifier(AVFileType.mov.rawValue) else { return } 						provider.loadItem(forTypeIdentifier: AVFileType.mov.rawValue, options: nil) { (fileURL, error) in 								if let error = error { 										print(error) 										return 								} 								guard let videoURL = fileURL as? URL else { return } 								DispatchQueue.main.async { 										let fm = FileManager.default 										let destination = fm.temporaryDirectory.appendingPathComponent("video123.mov") 										try! fm.copyItem(at: videoURL, to: destination) 										let playerVC = AVPlayerViewController() 										playerVC.player = AVPlayer(url: destination) 										self.present(playerVC, animated: true, completion: nil) 								} 						} I get crash trying to copy the item. It says the source file does not exists but the path looks real to me. "The file “3C2BCCBC-4474-491B-90C2-93DF848AADF5.mov” couldn’t be opened because there is no such file." I tried it without copying first and just passing the URL to AVPlayer but nothing would play. I am testing this on a simulator. Thanks for help!
13
0
7.5k
Jul ’20
Could not load some videos
Hello there! I am trying to use PHPickerViewController to load videos, but I got a problem: I could load some videos only not all. I refer to the existing thread - https://developer.apple.com/forums/thread/652695, but dosen't work. This is the code I persent PHPickerViewController var config = PHPickerConfiguration() config.selectionLimit = 1 config.filter = .videos config.preferredAssetRepresentationMode = .current let picker = PHPickerViewController(configuration: config) picker.delegate = self present(picker, animated: true, completion: nil) Below is the relevant implementation of the method: func picker(_ picker: PHPickerViewController, didFinishPicking results: [PHPickerResult]): picker.dismiss(animated: true, completion: nil) for result in results {   result.itemProvider.loadFileRepresentation(forTypeIdentifier: UTType.movie.identifier) { (url, error) in     if let error = error {       print(error)       return     }     guard let url = url else { return }     let fileName = "\(Date().timeIntervalSince1970).\(url.pathExtension)"     let newUrl = URL(fileURLWithPath: NSTemporaryDirectory() + fileName)     try? FileManager.default.copyItem(at: url, to: newUrl)     DispatchQueue.main.async {       self.playVideo(newUrl)     }   } } Before I print error in line 5, Xcode printed 3 lines of error: [AXRuntimeCommon] Unknown client: TestPHPicker [default] [ERROR] Could not create a bookmark: NSError: Cocoa 257 "The file couldn’t be opened because you don’t have permission to view it." } Error copying file type public.movie. Error: Error Domain=NSItemProviderErrorDomain Code=-1000 "Cannot load representation of type public.movie" UserInfo={NSLocalizedDescription=Cannot load representation of type public.movie, NSUnderlyingError=0x283a4a610 {Error Domain=NSCocoaErrorDomain Code=4101 "Couldn’t communicate with a helper application." UserInfo={NSUnderlyingError=0x283a48b10 {Error Domain=PHAssetExportRequestErrorDomain Code=2 "(null)" UserInfo={NSUnderlyingError=0x283a4a550 {Error Domain=CloudPhotoLibraryErrorDomain Code=82 "Failed to download CPLResourceTypeOriginal" UserInfo=0x28219b300 (not displayed)}}}}}} And I print error in line 5: Error Domain=NSItemProviderErrorDomain Code=-1000 "Cannot load representation of type public.movie" UserInfo={NSLocalizedDescription=Cannot load representation of type public.movie, NSUnderlyingError=0x283a4a610 {Error Domain=NSCocoaErrorDomain Code=4101 "Couldn’t communicate with a helper application." UserInfo={NSUnderlyingError=0x283a48b10 {Error Domain=PHAssetExportRequestErrorDomain Code=2 "(null)" UserInfo={NSUnderlyingError=0x283a4a550 {Error Domain=CloudPhotoLibraryErrorDomain Code=82 "Failed to download CPLResourceTypeOriginal" UserInfo=0x28219b300 (not displayed)}}}}}} For some videos I can load successfully, and some videos I got error. I don't know why this happened. I am testing this on an iPhone X iOS 14.0(18A373). Xcode 12.0 (12A7209). Thanks for help!
6
0
2.9k
Sep ’20
Ios flashlight specification
We are making application of ios. The application is for Disaster prevention in Japan. Because, Recentry in Japan has many Disaster in all season. We spplied to Local government in Japan and using general public and they are using well. One of the local government request to us, they want also supply to deaf person and help whe disaster occur. Local government already has disaster prevention broadcast but it is using loudspeaker. Then, when rains heavily, it can not hear to general public. And our application is from disaster prevention broadcast and forwarding to ios smartphone. It is helpfull to general public well. We are making new application it is not only prevention broadcast text but flashlight with iphone. But after making application, the flashlight is lighting only open our application. like beloe link. But, if Deaf person using it maybe he is not notice well. Our application is already has viblation function but Deaf person put his smartphone in his bag maybe I think he never noticed that Heavy rain and tsunami. Here is our application https://apps.apple.com/jp/app/cosmocast/id1247774270?mt=8 Here is application rule for flashlight. https://stackoverflow.com/questions/32136442/iphone-flashlight-not-working-while-app-is-in-background/32137434 I want help Deaf person and also senior citizens for Heavy rain and tsunami by our application. We'd like to make a flash light at the same time as the push notification arrived. Does anyone know a good way? Thank you and Best regards. Tomita.
1
0
681
Oct ’20
iOS 14 Photo caption - how to access in metadata??
I use the following code to parse Photo metadata and this works well. However, I am unable to pull the new iOS 14 "caption" from this metadata (it worked in early iOS 14 betas, but has since stopped working in the GM.) Does anyone know how I can get the caption data from a PHAsset? Thanks! Stephen         let options = PHContentEditingInputRequestOptions()         options.isNetworkAccessAllowed = true         asset.requestContentEditingInput(with: options, completionHandler: {(contentEditingInput, _) -> Void in             if let url = contentEditingInput?.fullSizeImageURL {                 let fullImage = CIImage(contentsOf: url)                                  // get all the metadata                 self.allPhotoMetadata = fullImage?.properties ?? [:]                                  // {TIFF}                 if let tiffDict = self.allPhotoMetadata["{TIFF}"] as? [String:Any] {                     if tiffDict["Make"] != nil {                         self.cameraData[cameraKeys.make] = tiffDict["Make"]                     }                     if tiffDict["Model"] != nil {                         self.cameraData[cameraKeys.model] = tiffDict["Model"]                     }                     if tiffDict["ImageDescription"] != nil {                         self.imageData[imageKeys.caption] = tiffDict["ImageDescription"]                     }                 }                                  // {IPTC}                 if let iptcDict = self.allPhotoMetadata["{IPTC}"] as? [String:Any] {                     // if we didn't find a caption in the TIFF dict, try to get it from IPTC data                     // first try, Caption/Abtract, then ArtworkContentDescription                     if self.imageData[imageKeys.caption] == nil {                         if iptcDict["Caption/Abstract"] != nil {                             self.imageData[imageKeys.caption] = iptcDict["ArtworkContentDescription"]                         } else if iptcDict["ArtworkContentDescription"] != nil {                             self.imageData[imageKeys.caption] = iptcDict["ArtworkContentDescription"]                         }                     }                 }             }         })     }
9
2
3.5k
Oct ’20
iOS 14 - HLS metadata not showing user defined URL frames within id3 tag
Hello, I am using HLS audio stream and I get metadata to show the song info, artist info etc., in the app using AVPlayer. In iOS 14, I see the artwork data (which embeds in WXXX frame of id3 tag) is coming over across along with other metadata. The same stream is working fine and showing artwork data in iOS 13. Does anything changed in iOS 14 which effects this metadata? Please let me know Thank you
4
0
1.5k
Dec ’20
AVAssetWriter leading to huge kernel_task memory usage
I am currently working on a macOS app which will be creating very large video files with up to an hour of content. However, generating the images and adding them to AVAssetWriter leads to VTDecoderXPCService using 16+ GB of memory and the kernel-task using 40+ GB (the max I saw was 105GB). It seems like the generated video is not streamed onto the disk but rather written to memory for it to be written to disk all at once. How can I force it to stream the data to disk while the encoding is happening? Btw. my app itself consistently needs around 300MB of memory, so I don't think I have a memory leak here. Here is the relevant code: func analyse() { 				self.videoWritter = try! AVAssetWriter(outputURL: outputVideo, fileType: AVFileType.mp4) 				let writeSettings: [String: Any] = [ 						AVVideoCodecKey: AVVideoCodecType.h264, 						AVVideoWidthKey: videoSize.width, 						AVVideoHeightKey: videoSize.height, 						AVVideoCompressionPropertiesKey: [ 								AVVideoAverageBitRateKey: 10_000_000, 						] 				] 				self.videoWritter!.movieFragmentInterval = CMTimeMake(value: 60, timescale: 1) 				self.frameInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: writeSettings) 				self.frameInput?.expectsMediaDataInRealTime = true 				self.videoWritter!.add(self.frameInput!) 				if self.videoWritter!.startWriting() == false { 						print("Could not write file: \(self.videoWritter!.error!)") 						return 				} } func writeFrame(frame: Frame) { 				/* some more code to determine the correct frame presentation time stamp */ 				let newSampleBuffer = self.setTimeStamp(frame.sampleBuffer, newTime: self.nextFrameStartTime!) 				self.frameInput!.append(newSampleBuffer) 				/* some more code */ } func setTimeStamp(_ sample: CMSampleBuffer, newTime: CMTime) -> CMSampleBuffer { 				var timing: CMSampleTimingInfo = CMSampleTimingInfo( 						duration: CMTime.zero, 						presentationTimeStamp: newTime, 						decodeTimeStamp: CMTime.zero 				) 				var newSampleBuffer: CMSampleBuffer? 				CMSampleBufferCreateCopyWithNewTiming( 						allocator: kCFAllocatorDefault, 					 sampleBuffer: sample, 					 sampleTimingEntryCount: 1, 					 sampleTimingArray: &timing, 					 sampleBufferOut: &newSampleBuffer 			 ) 				return	newSampleBuffer! 		} My specs: MacBook Pro 2018 32GB Memory macOS Big Sur 11.1
5
0
2k
Dec ’20
ERROR_FAILED_TO_VERIFY_JWT error when trying to link with Apple Music
I want to use Apple Music using MusicKit. I've created MusicKit identifier and private key by following all the steps mentioned here: https://help.apple.com/developer-account/#/devce5522674 I have my Team ID, the Music key ID and the private key (.p8 file) with me. I tried to generate the required JWT token by using many scripts that were mentioned in similar questions asked here, but none of them have worked so far. When I try to authenticate, I'm getting an error ERROR_FAILED_TO_VERIFY_JWT. Ref: Codes I've tried to generate the JWT token from: Thread 130168 - https://developer.apple.com/forums/thread/130168 A nice and simple blog by Lee Martin [Blog - Creating an Apple Music API Token] Thread 79074 - https://developer.apple.com/forums/thread/79074 Many other links But sadly, none of them worked and I had to post this question. I have tried the sample Android app: https://developer.apple.com/download/more/?=Android%20MusicKit After authentication, the page goes on with an infinite loader, which I believe must be because of the same error ERROR_FAILED_TO_VERIFY_JWT Also tried with this HTML page: <html> <head> <meta name="apple-music-developer-token" content="the-JWT-token-generated-using-the-reference-links"> <meta name="apple-music-app-name" content="My App Name"> <meta name="apple-music-app-build" content="1978.4.1"> </head> <body> <button id="apple-music-authorize"></button> <button id="apple-music-unauthorize"></button> </body> <script src="link-to-musickit.js"></script> <script> let music = MusicKit.getInstance(); &#9;&#9;music.player.play(); &#9;&#9;music.authorize().then(function() { &#9;&#9;&#9;&#9;music.player.play(); &#9;&#9;}); &#9;&#9;music.authorize().then(function() { &#9;&#9;&#9;&#9; music.api.library.albums.then(function(cloudAlbums) { &#9;&#9;&#9;&#9;&#9;&#9;&#9;// user's cloudAlbums &#9;&#9;&#9;&#9;}); &#9;&#9;}); </script> </html>
4
0
2.6k
Feb ’21
Video Quality selection in HLS streams
Hello there, in our team we were requested to add the possibility to manually select the video quality. I know that HLS is an adaptive stream and that depending on the network condition it choose the best quality that fits to the current situation. I also tried some setting with preferredMaximumResolution and preferredPeakBitRate but none of them worked once the user was watching the steam. I also tried something like replacing the currentPlayerItem with the new configuration but anyway this only allowed me to downgrade the quality of the video. When I wanted to set it for example to 4k it did not change to that track event if I set a very high values to both params mentioned above. My question is if there is any method which would allow me to force certain quality from the manifest file. I already have some kind of extraction which can parse the manifest file and provide me all the available information but I couldn't still figure out how to make the player reproduce specific stream with my desired quality from the available playlist.
4
0
6.7k
Mar ’21
What causes "issue_type = overload" in coreaudiod with USB audio interface?
I have a USB audio interface that is causing kernel traps and the audio output to "skip" or dropout every few seconds. This behavior occurs with a completely fresh install of Catalina as well as Big Sur with the stock Music app on a 2019 MacBook Pro 16 (full specs below). The Console logs show coreaudiod got an error from a kernel trap, a "USB Sound assertion" in AppleUSBAudio/AppleUSBAudio-401.4/KEXT/AppleUSBAudioDevice.cpp at line 6644, and the Music app "skipping cycle due to overload." I've added a short snippet from Console logs around the time of the audio skip/drop out. The more complete logs are at this gist: https://gist.github.com/djflux/08d9007e2146884e6df1741770de5105 I've also opened a Feedback Assistant ticket (FB9037528): https://feedbackassistant.apple.com/feedback/9037528 Does anyone know what could be causing this issue? Thanks for any help. Cheers, Flux aka Andy. Hardware Overview:  Model Name: MacBook Pro  Model Identifier: MacBookPro16,1  Processor Name: 8-Core Intel Core i9  Processor Speed: 2.4 GHz  Number of Processors: 1  Total Number of Cores: 8  L2 Cache (per Core): 256 KB  L3 Cache: 16 MB  Hyper-Threading Technology: Enabled  Memory: 64 GB  System Firmware Version: 1554.80.3.0.0 (iBridge: 18.16.14347.0.0,0) System Software Overview: System Version: macOS 11.2.3 (20D91)  Kernel Version: Darwin 20.3.0  Boot Volume: Macintosh HD  Boot Mode: Normal  Computer Name: mycomputername  User Name: myusername  Secure Virtual Memory: Enabled  System Integrity Protection: Enabled USB interface: Denon DJ DS1 Snippet of Console logs error 21:07:04.848721-0500 coreaudiod HALS_IOA1Engine::EndWriting: got an error from the kernel trap, Error: 0xE00002D7 default 21:07:04.848855-0500 Music HALC_ProxyIOContext::IOWorkLoop: skipping cycle due to overload default 21:07:04.857903-0500 kernel USB Sound assertion (Resetting engine due to error returned in Read Handler) in /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/AppleUSBAudio/AppleUSBAudio-401.4/KEXT/AppleUSBAudioDevice.cpp at line 6644 ... default 21:07:05.102746-0500 coreaudiod Audio IO Overload inputs: 'private' outputs: 'private' cause: 'Unknown' prewarming: no recovering: no default 21:07:05.102926-0500 coreaudiod   CAReportingClient.mm:508  message {   HostApplicationDisplayID = "com.apple.Music";   cause = Unknown;   deadline = 2615019;   "input_device_source_list" = Unknown;   "input_device_transport_list" = USB;   "input_device_uid_list" = "AppleUSBAudioEngine:Denon DJ:DS1:000:1,2";   "io_buffer_size" = 512;   "io_cycle" = 1;   "is_prewarming" = 0;   "is_recovering" = 0;   "issue_type" = overload;   lateness = "-535";   "output_device_source_list" = Unknown;   "output_device_transport_list" = USB;   "output_device_uid_list" = "AppleUSBAudioEngine:Denon DJ:DS1:000:1,2"; }: (null)
37
12
16k
Apr ’21
MusicKit: developer token request failed
The MusicKit video states that you just enable "MusicKit" in your application identifier and "you're done!" Ok, so I did that, and I'm seeing the following error when trying to run a song query: [DataRequesting] Failed retrieving MusicKit tokens: Error Domain=ICErrorDomain Code=-8200 "Media API Token Service's response was invalid (status code: Unauthorized (401))." UserInfo={NSDebugDescription=Media API Token Service's response was invalid (status code: Unauthorized (401))., NSUnderlyingError=0x6000023a0c60 {Error Domain=AMSErrorDomain Code=301 "Invalid Status Code" UserInfo={NSLocalizedDescription=Invalid Status Code, AMSURL=https://sf-api-token-service.itunes.apple.com/apiToken?REDACTED, AMSStatusCode=401, AMSServerPayload={ status = verificationFailure; }, NSLocalizedFailureReason=The response has an invalid status code}}}. Throwing .developerTokenRequestFailed. Is this just broken on Apple's side? Is there some other magic string that needs to be added to the plist other than NSAppleMusicUsageDescription?
7
0
3.3k
Jun ’21
Upstream Service Error when using MusicDataRequest
Hey there! I'm trying to use MusicDataRequest to fetch the contents of a user's library. Most of the documented endpoints I've tried seem to be working as expected, but the /me/library/artists and /me/library/albums endpoints are consistenty giving me a 500 Upstream Service Error. Here's an example of my code, and the resulting error: let url = URL(string: "https://api.music.apple.com/v1/me/library/albums")! let request = MusicDataRequest(urlRequest: URLRequest(url: url)) do { let response = try await request.response()     let string = String(data: response.data, encoding: .utf8)!     print("success: \(string)") } catch {     print("error: \(error)") } MusicDataRequest.Error(   status: 500,   code: 50001,   title: "Upstream Service Error",   detailText: "Error fetching library content",   id: "5OFXMJAGNU2WCTDKNAYYP4BJXI",   originalResponse: MusicDataResponse(    data: 153 bytes,    urlResponse: <NSHTTPURLResponse: 0x0000000280f04dc0>   ) ) If I replace /albums with /songs or /playlists in the above code everything works as expected. Is there something I'm missing from the albums and artists requests? Or is this a bug with the API?
12
0
2.9k
Jul ’21
How to get the album info for a song fetched by MusicDataRequest?
I'm slowly learning the new MusicKit beta for swift. I've learned to successfully retrieve tracks of type Song using MusicDataRequest, using the following: ... let countryCode = try await MusicDataRequest.currentCountryCode if let url = URL(string: "https://api.music.apple.com/v1/catalog/\(countryCode)/songs?filter[isrc]=\(isrc)") {   let dataRequest = MusicDataRequest(urlRequest: URLRequest(url: url))   let dataResponse = try await dataRequest.response() ... However, when I decode the data, there does not seem to be any album information that I can see. I've tried adding includes=albums to the URL, but I don't think that's the right approach, because when I veiw the Song struct in MusicKit, I don't see a reference to an Album type anywhere. Any advice on how to retrieve the album information would be most appreciated. Thanks.
7
0
2.8k
Aug ’21
How to detect a song end?
I'm playing library items (MPMediaItem) and apple music tracks (Track) in MPMusicPlayerApplicationController.applicationQueuePlayer, but I can't use the actual Queue functionality because I can't figure out how to get both media types into the same queue. If there's a way to get both types in a single queue, that would solve my problem, but I've given up on that one. Because I can't use a queue, I have to be able to detect when a song ends so that I can put the next song in the queue and play it. The only way I can figure out to detect when a song ends is by watching the playBackState, and I've actually got that pretty much working, but it's really ugly, because you get playBackState of paused when a song ends, and when a bluetooth speaker disconnects, etc. The only answer I've been able to find on the internet is to watch the MPMusicPlayerControllerNowPlayingItemDidChange, and when that fires, and the nowPlayingItem is NIL, a song ends.. but that's not the case. When a song ends, the nowPlayingItem remains the same. There's got to be an answer to this problem, right?
12
3
4.0k
Aug ’21
How to insert timed metadata (id3) into live HLS files with Apple's mediastreamsegmenter and ffmpeg
I am trying to insert timed metadata (id3) into a live HLS stream created with Apple's mediastreamsegmenter tool. I am getting the video from an ffmpeg stream, here is the command I run to test from an existing file: ffmpeg -re -i vid1.mp4 -vcodec libx264 -acodec aac -f mpegts - | mediastreamsegmenter -f /Users/username/Sites/video -s 10 -y test -m -M 4242 -l log.txt To inject metadata, I run this command: id3taggenerator -text '{"x":"data dan","y":"36"}' -a localhost:4242 This setup creates the expected .ts files and I can play back the video/audio with no issues. However the metadata I am attempting to insert does not work in the final file. I know the metadata is there in some form, when I file-compare a no-metadata version of the video to one I injected metadata into, I can see the ID3 tags within the binary data. Bad File Analysis When I analyze the generated files using ffmpeg: ffmpeg -i video1.ts the output I get is: [mpegts @ 0x7fb00a008200] start time for stream 2 is not set in estimate_timings_from_pts[mpegts @ 0x7fb00a008200] stream 2 : no TS found at start of file, duration not set[mpegts @ 0x7fb00a008200] Could not find codec parameters for stream 2 (Audio: mp3, 0 channels): unspecified frame sizeConsider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options Input #0, mpegts, from 'video1.ts':  Duration: 00:00:10.02, start: 0.043444, bitrate: 1745 kb/s  Program 1   Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 848x464 [SAR 1:1 DAR 53:29], 30 fps, 30 tbr, 90k tbn, 60 tbc   Stream #0:1[0x101] Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 130 kb/s  No Program   Stream #0:2[0x102]: Audio: mp3, 0 channels Note how the third stream (stream #0:2) is marked as mp3...this is incorrect! Also it says "No Program", instead of being in "Program 1". When I analyze a properly encoded video file with inserted ID3 metadata that I created with Apple's mediafilesegmenter tool, the analysis shows a "timed_id3" track and this metadata track works properly in my web browser. Good File Analysis ffmpeg -i video1.ts —Input #0, mpegts, from 'video1.ts':  Duration: 00:00:10.08, start: 19.984578, bitrate: 1175 kb/s  Program 1   Stream #0:0[0x101]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 848x464, 30 fps, 30 tbr, 90k tbn, 180k tbc  Stream #0:1[0x102]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 67 kb/s  Stream #0:2[0x103]: Data: timed_id3 (ID3  / 0x20334449) I must use mediastreamsegmenter because that is required for live streams. Does anyone know how I can get timed ID3 metadata into a live HLS stream properly?
2
0
2.2k
Sep ’21
iPhone 13 / 13 mini builtInUltraWideCamera trouble
I ran into a strange problem. A camera app using AVFoundation, I use the following code; captureDevice = AVCaptureDevice.default(AVCaptureDevice.DeviceType.builtInUltraWideCamera, for: AVMediaType.video, position: .back) then, let isAutoFocusSupported = captureDevice.isFocusModeSupported(.autoFocus) "isAutoFocusSupported" should be "true". For iPhone 13 pro, it is "true". But for 13 / 13 mini, it is "false". Why?
4
0
1.6k
Oct ’21