-
使用 Foveated Streaming 将沉浸式内容融入 visionOS
了解 Foveated Streaming 如何以高保真质量将远程渲染的场景传输到 Apple Vision Pro。探索这个框架如何通过完全无线的方式,将 visionOS 原生功能与第三方流媒体技术相结合,并通过 OpenXR 场景和 NVIDIA CloudXR 进行演示。了解 Foveated Streaming 框架、与 NVIDIA CloudXR SDK 的集成,以及动态注视点流式传输技术如何在保护隐私的同时带来种种好处。
章节
- 0:00 - Introduction
- 4:08 - How Foveated Streaming works
- 4:46 - Set up the streaming endpoint
- 5:18 - Create a visionOS receiver app
- 8:02 - Integrate with the streaming endpoint
- 11:28 - Measure performance
- 11:56 - Enhance with visionOS features
- 13:56 - Next steps
资源
- Analyzing the performance of a foveated streaming session
- Establishing foveated streaming sessions with Apple Vision Pro
- Streaming a CloudXR application to Apple Vision Pro with foveation
- Creating a foveated streaming client on visionOS
- Foveated Streaming
- StreamingSession: Streaming immersive content from a CloudXR™ application to visionOS and iOS
相关视频
WWDC26
-
搜索此视频…
-
-
6:03 - Connect to a streaming endpoint
// Connect to a streaming endpoint import SwiftUI import FoveatedStreaming struct ConnectView: View { let session: FoveatedStreamingSession var body: some View { Button("Connect") { Task { try await session.connect() } } } } -
6:44 - Display a Foveated Streaming session in your immersive space
// Display a Foveated Streaming session in your immersive space import SwiftUI import FoveatedStreaming @main struct FoveatedStreamingSampleApp: App { private let session = FoveatedStreamingSession() var body: some SwiftUI.Scene { ImmersiveSpace(foveatedStreaming: session) } } -
6:55 - Compose SwiftUI content with Foveated Streaming
// Compose SwiftUI content with Foveated Streaming import SwiftUI import FoveatedStreaming @main struct FoveatedStreamingSampleApp: App { private let session = FoveatedStreamingSession() private let appModel = AppModel() var body: some SwiftUI.Scene { Window("Main", id: appModel.mainWindowId) { ContentView(session: session) .environment(appModel) .environment(session) // ... } ImmersiveSpace(foveatedStreaming: session) { SpatialContainer { ReopenMainWindowView().environment(appModel) TransformStreamWidgetView().environment(session) } } } } -
13:42 - Compose RealityKit content with Foveated Streaming
// Compose RealityKit content with Foveated Streaming import SwiftUI import RealityKit import FoveatedStreaming @main struct FoveatedStreamingSampleApp: App { private let session = FoveatedStreamingSession() private let appModel = AppModel() var body: some SwiftUI.Scene { ImmersiveSpace(foveatedStreaming: session) { RealityView { content in // ... } } } }
-
-
- 0:00 - Introduction
Overview of Foveated Streaming — a new framework in visionOS 26.4 that lets Apple Vision Pro connect to a PC and stream immersive OpenXR content with eye-tracking-based video optimization.
- 4:08 - How Foveated Streaming works
Learn the architecture: the visionOS receiver app uses the FoveatedStreaming framework, the streaming endpoint implements the Foveated Streaming Protocol, and NVIDIA CloudXR handles the OpenXR runtime and Wi-Fi/cloud streaming.
- 4:46 - Set up the streaming endpoint
Set up your Windows streaming endpoint using open-source sample code on GitHub, which includes a reference implementation of the Foveated Streaming Protocol and an example OpenXR application.
- 5:18 - Create a visionOS receiver app
Build your visionOS receiver app using FoveatedStreamingSession to connect and pair with the endpoint, present streamed content in an ImmersiveSpace, and enrich the UI with SwiftUI windows and progressive immersion.
- 8:02 - Integrate with the streaming endpoint
Configure your OpenXR client with the Foveated Streaming Protocol for authentication and pairing, and integrate the NVIDIA CloudXR runtime to handle input data, hand tracking, depth buffers, and alpha channel compositing.
- 11:28 - Measure performance
Use the Foveated Streaming instrument in Xcode to measure your stream bandwidth, pose latency, and frame rate, and diagnose performance issues before shipping.
- 11:56 - Enhance with visionOS features
Enhance your app with message channels to exchange data between the visionOS and OpenXR apps, ARKit to anchor virtual content to the physical world, and RealityKit to composite native 3D objects alongside the streamed content.
- 13:56 - Next steps
Download sample code on GitHub and developer.apple.com to get started, set up your own receiver app, and integrate your OpenXR client with the Foveated Streaming Protocol.