Hi All,
We're a studio building an app and as part of a scene we have a 3D asset with a smoke particle emitter and a curved mesh that plays video. I notice that when the video alone is played or the particle effect alone is done then the scene works fine but the frame rate drops drastically when both are turned on.
How do I solve this because this is an important storytelling feature.
Video
RSS for tagIntegrate video and other forms of moving visual media into your apps.
Posts under Video tag
63 Posts
Selecting any option will automatically load the page
Post
Replies
Boosts
Views
Activity
context:Explore low-latency video encoding with VideoToolbox https://developer.apple.com/videos/play/wwdc2021/10158/
I see above post said HEVC VideoToolbox can support SVC, temporal layering of streams in low-latency mode with all P frames.
my question is that Does HEVC VideoToolbox support temporal layering of streams with B-frames ?? thanks
iPhoneで撮影した映像をブラウザのアプリへ送信して画面に映す機能を持ったアプリを開発しています。
iPhone 17 Pro, 17 Pro Maxでこのアプリを利用するとブラウザ側に表示される映像が緑一色や、緑がメインのカラフルな映像になってしまいます。
調べてみると17Proと17ProMaxで超広角カメラと望遠カメラの画素数が変更になっている（1200万画素→4800万画素）ためエンコーディングで失敗しているのではないかと疑っています。
なんでも情報下さい。
環境情報
WebRTCライブラリ： GoogleWebRTC バージョン 1.1 (CocoaPodsで導入)
シグナリングサーバー： AWS Kinesis Video Streams
問題が発生するデバイス：
モデル名: iPhone18,1, OS: 26.0
モデル名: iPhone18,1, OS: 26.1
問題が発生しないデバイス：
iPhone17,5 以前の多数のモデル
モデル名: iPhone18,1, OS: 26.0
モデル名: iPhone18,3, OS: 26.0
I’m encountering a consistent crash in WebKit when using WKWebView to play a YouTube playlist in my iOS app. Playback starts successfully, but the web process terminates during the second video in the playlist. This only occurs on physical devices, not in the simulator.
Here’s a simplified Swift example of my setup:
import SwiftUI
import WebKit
struct ContentView: View {
private let playlistID = "PLig2mjpwQBZnghraUKGhCqc9eAy0UbpDN"
var body: some View {
YouTubeWebView(playlistID: playlistID)
.edgesIgnoringSafeArea(.all)
}
}
struct YouTubeWebView: UIViewRepresentable {
let playlistID: String
func makeUIView(context: Context) -> WKWebView {
let config = WKWebViewConfiguration()
config.allowsInlineMediaPlayback = true
let webView = WKWebView(frame: .zero, configuration: config)
webView.scrollView.isScrollEnabled = true
let html = """
<!doctype html>
<html>
<head>
<meta name="viewport" content="initial-scale=1.0, maximum-scale=1.0">
<style>body,html{height:100%;margin:0;background:#000}iframe{width:100%;height:100%;border:0}</style>
</head>
<body>
<iframe
src="https://www.youtube-nocookie.com/embed/videoseries?list=\(playlistID)&controls=1&rel=0&playsinline=1&iv_load_policy=3"
frameborder="0"
allow="encrypted-media; picture-in-picture; fullscreen"
webkit-playsinline
allowfullscreen
></iframe>
</body>
</html>
"""
webView.loadHTMLString(html, baseURL: nil)
return webView
}
func updateUIView(_ uiView: WKWebView, context: Context) {}
}
#Preview {
ContentView()
}
Observed behavior:
First video plays without issue.
Web process crashes when the second video in the playlist starts.
Console logs show WebProcessProxy::didClose and repeated memory status messages.
Using ProcessAssertion or background activity does not prevent the crash.
Only occurs on physical devices; simulators do not reproduce the issue.
Questions:
Is there something I should change or add in my WKWebView setup or HTML/iframe to prevent the crash when playing the second video in a playlist on physical iOS devices?
Is there an officially supported way to limit memory or prevent WebKit from terminating the web process during multi-video playback?
Are there recommended patterns for playing YouTube playlists in a WKWebView on iOS without risking crashes?
Any tips for debugging or configuring WKWebView to make it more stable for continuous playlist playback?
Thanks in advance for any guidance!
I'm at my wit's end with a problem I'm facing while developing an app. The app is designed to send video captured on an iPhone to a browser application for real-time display.
While it works on many older iPhone models, whenever I test it on an iPhone 17 Pro or 17 Pro Max, the video displayed in the browser becomes a solid green screen, or a colorful, garbled image that's mostly green.
I've been digging into this, and my main suspicion is an encoding failure. It seems the resolution of the ultra-wide and telephoto cameras was significantly increased on the 17 Pro and Pro Max (from 12MP to 48MP), and I think this might be overwhelming the encoder.
I'm really hoping someone here has encountered a similar issue or has any suggestions. I'm open to any information or ideas you might have. Please help!
Environment Information:
WebRTC Library: GoogleWebRTC Version 1.1 (via CocoaPods)
Signaling Server: AWS Kinesis Video Streams
Problem Occurs on:
Model: iPhone18,1, OS: 26.0
Model: iPhone18,1, OS: 26.1
Works Fine on:
Many models before iPhone17,5
Model: iPhone18,1, OS: 26.0
Model: iPhone18,3, OS: 26.0
The same H265 encrypted Fairplay content can be played in all Apple devices except A1625.
The clear H265 content is played in A1625.
The question is: will this model (A1625) support H265 Fairplay encrypted content?
A ticket was created here:
https://discussions.apple.com/thread/255658006?sortBy=best
Topic:
Media Technologies
SubTopic:
Streaming
Tags:
FairPlay Streaming
Media
Video
HTTP Live Streaming
In visionOS, I'm trying to create an immersive environment which would feature several spheres in which immersive movies are visible. I'm starting from a sample code which creates a sphere, sets an immersive movie as its material, and opens it as an immersive environment. This works fine.
But if I create a sphere in an open immersive environment using Reality Composer Pro and sets its material to an immersive movie, I can see the movie on the sphere while I move outside of it but if I try to get inside the sphere, it disappears. What would be the right way of doing this ?
Our app plays TS files on an iPhone.
The app fragments the TS files, creates an M3U8 playlist, converts them to HLS(HTTP Live Streaming), and then uses AVPlayer to play the video content.
On a device running iOS 26, after starting playback and seeking, restarting playback causes the video and audio to be out of sync (by about 2-3 seconds depending on the situation).
This also occurs on iPadOS/macOS 26.
This issue was not observed prior to iOS 18.
We are trying to fix this issue on the app side, but we have the following questions:
The behavior of AVPlayer is different between iOS 26 and previous versions. Has there been any change that could be considered? Or is it a bug?
We tried pausing before seeking, but it didn’t seem to have any effect. Are there any APIs or workarounds that can improve this?
We would appreciate it if you could tell us any other helpful documents or URLs.
When we embed some of the youtube videos are unable to load in the Mobile app but at the same time it works in Website. I need to allow it in both places.
I have tried both embed and native sdk for youtube in iOS.
I would like to play videos in webM format on my iPhone. I understand that it is basically impossible to play videos in webM format on an iPhone, but is there any way to display videos in webM format? I would like to know if there is an official Swift SDK or development kit released by Apple. Or if there are any third-party products, please let me know.
I'm capturing video stream from GoPro camera (via UDP MPEG-TS packets) but unable to play in iOS app. can someone provide a source code to decode MPEG-TS data to CMSampleBuffer.
I'm working on a media app that would like to be able to tell if the TV connected to tvOS is running at 59.94hz or 60.00hz, so it can optimize a video stream. It looks like the best I can currently do is to check if the user has Match Content Rate enabled, and based on that, when calling displayManager.preferredDisplayCriteria to change video modes, I could guess which rate their TV might be in. It's not very ideal, because not all TVs support both of these rates, and my request for 59.94 might end up as 60 and vice versa.
I dug around and can't find any available method in UIScreen to get this info. The odd thing is, the data is right there in currentMode when I look in the debugger, but it seems to be in a private or undocumented class. Is there any way to get at it?
Does anyone have a template of an Apple Projected Media Profile Format Description or a File of a Stereo wideFOV video?
Use case I have 2 compatible cameras that I stereo sync and I want to move the projection information from the compatible video to the Spatial video that combines them.
Every version I can come up with crashes the AVP and when viewing as Spatial in Tahoe I just get a black screen.
I'm developing an app that uses the SwiftUI .photosPicker modifier to allow the user to open videos from their photos. While many videos successfully load, one video results in the following errors occurring:
Error loading com.apple.quicktime-movie: <decode: bad range for [%@] got [offs:100 len:1229 within:0]>
Error loading public.movie: <decode: bad range for [%@] got [offs:87 len:1229 within:0]>
"The operation couldn’t be completed. (CoreTransferable.TransferableSupportError error 0.)"
I was able to isolate the line of code within the Transferable where this occurs to be the following:
try FileManager.default.copyItem(at: received.file, to: destination)
Is there something that I can do to ensure the app can reliably open any video?
The entire transferable struct is as follows:
import Foundation
import CoreTransferable
import UniformTypeIdentifiers
struct Video: Transferable {
let url: URL
let filename: String
static var transferRepresentation: some TransferRepresentation {
FileRepresentation(contentType: .mpeg4Movie) { video in
SentTransferredFile(video.url)
} importing: { received in
try Video.transfer(from: received)
}
FileRepresentation(contentType: .quickTimeMovie) { video in
SentTransferredFile(video.url)
} importing: { received in
try Video.transfer(from: received)
}
FileRepresentation(contentType: .avi) { video in
SentTransferredFile(video.url)
} importing: { received in
try Video.transfer(from: received)
}
FileRepresentation(contentType: .mpeg) { video in
SentTransferredFile(video.url)
} importing: { received in
try Video.transfer(from: received)
}
FileRepresentation(contentType: .mpeg2Video) { video in
SentTransferredFile(video.url)
} importing: { received in
try Video.transfer(from: received)
}
FileRepresentation(contentType: .video) { video in
SentTransferredFile(video.url)
} importing: { received in
try Video.transfer(from: received)
}
FileRepresentation(contentType: .movie) { video in
SentTransferredFile(video.url)
} importing: { received in
try Video.transfer(from: received)
}
}
static func transfer(from received: ReceivedTransferredFile) throws -> Video {
let destination = FileManager.default.temporaryDirectory.appendingPathComponent(received.file.lastPathComponent)
if FileManager.default.fileExists(atPath: destination.path) {
try FileManager.default.removeItem(at: destination)
}
try FileManager.default.copyItem(at: received.file, to: destination)
return Video(url: destination, filename: received.file.lastPathComponent)
}
}
our app is live, and it appears that since the ios 18 update - the VideoMaterial renders pink / purple color instead of the video (picture attached). the audio is rendered properly.
we found that it occurs on old devices: iPhone 11 & iPhone SE 2020.
I've found this thread of Andy Jazz on stackoverflow:
Steps to Reproduce:
Create a plane for the video screen.
Apply a VideoMaterial using AVPlayerItem.
Anchor the model entity to an ARImageAnchor.
Expected Outcome:
The video should play as a material on the plane in RealityKit.
Actual Outcome:
On iOS 18, the plane appears pink, indicating the VideoMaterial isn’t applied.
What I’ve Tried:
-Verified the video URL is correct.
-Checked that the AVPlayerItem and VideoMaterial are initialised correctly.
-Ensured the AVPlayer is playing the video.
I also tried different formats (mov / mp4 / m4v), and verifying that the video's status is readyToPlay.
any suggestions?
Starting in iOS 18.4, (and still in the iOS 18.5 beta), the AVPlayer seems to freeze when we:
Replace the current AVPlayerItem, ReplaceCurrentItemWithPlayerItem and then:
Call Seek very shortly afterwards (seekToTime:toleranceBefore:toleranceAfter: / seek(to:))
And then subsequent calls to play after have no effect. However, it feels scrubbing to see after works and also changing the playback rate (i.e. fast forward) tends to clear up the frozen state.
Our primary workflow involves video playback, replacing video to show new clips and in some cases seeking to specific frames. This appears to only be occurring while streaming video, reports are all that local downloaded video playback remains fine.
This same code path has worked without issue on 17.x and 18.3.2 and for years before that.
What is particularly strange is that time observers log that video is still playing or feeding frames. The reported status is ReadyToPlay, IsLikelyToKeepUp is true, and there are no indications of stalling or buffering.
A similar issue is true for our web application in Safari. While on Sonoma and Safari 17.x, there is no issue. When you update to macOS Sequoia 15.4.1 and Safari 18.4, you begin observing a similar freezing. The same does not occur on Chrome or other tested browsers.
There appears to be in the release notes for Safari 18.4, an interesting "fix" note that seems similar to what we are now experiencing:
https://developer.apple.com/documentation/safari-release-notes/safari-18_4-release-notes
"Fixed an issue where playback doesn’t always resume after a seek. (140097993)"
"Fixed playing video generating non-monotonic ‘timeupdate’ events. (142275184) (FB16222910)"
"Fixed websites calling play() during a seek() is allowed by the specification so that the play event is fired even if the seek hasn’t completed. (142517488)"
"Fixed seek not completing for WebM under some circumstances. (143372794)"
"Fixed MediaRecorderPrivateEncoder writing frames out of order. (143956063)"
Guys has anyone here used the PlayVideoIntent protocol while implementing app intents?
If yes can you please walk me though what purpose it solves and what features and functionality I can unlock with it?
Link to apple's documentation -> https://developer.apple.com/documentation/appintents/playvideointent
Topic:
App & System Services
SubTopic:
Automation & Scripting
Tags:
Video
App Intents
Apple Intelligence
I believe I have created a videoMaterial and assigned it to a mesh with code I found in the Developer's Documentation but Im getting this error.
"Trailing closure passed to parameter of type 'String' that does not accept a closure"
I have attached a photo of the code and where the error happens.
Any help will greatly be appreciated.
How can I setup correctly AVSampleBufferDisplayLayer for video display when I have input picture format kCVPixelFormatType_32BGRA?
Currently video i visible in simulator, but not iPhone, miss I something?
Render code:
var pixelBuffer: CVPixelBuffer?
let attrs: [String: Any] = [
kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA,
kCVPixelBufferWidthKey as String: width,
kCVPixelBufferHeightKey as String: height,
kCVPixelBufferBytesPerRowAlignmentKey as String: width * 4,
kCVPixelBufferIOSurfacePropertiesKey as String: [:]
]
let status = CVPixelBufferCreateWithBytes(
nil,
width,
height,
kCVPixelFormatType_32BGRA,
img,
width * 4,
nil,
nil,
attrs as CFDictionary,
&pixelBuffer
)
guard status == kCVReturnSuccess, let pb = pixelBuffer else { return }
var formatDesc: CMVideoFormatDescription?
CMVideoFormatDescriptionCreateForImageBuffer(
allocator: nil,
imageBuffer: pb,
formatDescriptionOut: &formatDesc
)
guard let format = formatDesc else { return }
var timingInfo = CMSampleTimingInfo(
duration: .invalid,
presentationTimeStamp: currentTime,
decodeTimeStamp: .invalid
)
var sampleBuffer: CMSampleBuffer?
CMSampleBufferCreateForImageBuffer(
allocator: kCFAllocatorDefault,
imageBuffer: pb,
dataReady: true,
makeDataReadyCallback: nil,
refcon: nil,
formatDescription: format,
sampleTiming: &timingInfo,
sampleBufferOut: &sampleBuffer
)
if let sb = sampleBuffer {
if CMSampleBufferGetPresentationTimeStamp(sb) == .invalid {
print("Invalid video timestamp")
}
if (displayLayer.status == .failed) {
displayLayer.flush()
}
DispatchQueue.main.async { [weak self] in
guard let self = self else {
print("Lost reference to self drawing")
return
}
displayLayer.enqueue(sb)
}
frameIndex += 1
}
I use startCaptureWithHandler to record screen and AVAssetWriter appendSampleBuffer: to save audio and video ,but when played the saved file audio and video are out of sync.
I don t know if it s a AVAssetWriterInputr setup problem，here is my code
NSDictionary *audioCompressionSettings = @{
AVEncoderBitRatePerChannelKey : @(64000),
AVFormatIDKey : @(kAudioFormatMPEG4AAC),
AVNumberOfChannelsKey : @(2),
AVSampleRateKey : @(44100) };
AVAssetWriterInput *audioAssetWriterInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeAudio outputSettings:audioCompressionSettings];
audioAssetWriterInput.expectsMediaDataInRealTime = YES;
[_assetWriter addInput:audioAssetWriterInput];
NSDictionary *videoCompressSetting = @{AVVideoAverageBitRateKey:@(screenWidth*screenHeight*5),
AVVideoMaxKeyFrameIntervalKey:@(30),
AVVideoProfileLevelKey : AVVideoProfileLevelH264MainAutoLevel};
NSDictionary *codecSetting = @{AVVideoCodecKey:AVVideoCodecTypeH264,
AVVideoScalingModeKey : AVVideoScalingModeResize,
AVVideoWidthKey:@(screenWidth*2),
AVVideoHeightKey:@(screenHeight*2),
AVVideoCompressionPropertiesKey:videoCompressSetting
};
AVAssetWriterInput* videoAssetWriterInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo outputSettings:codecSetting];
videoAssetWriterInput.expectsMediaDataInRealTime = YES;
[_assetWriter addInput:videoAssetWriterInput];