-
Usa streaming foveado para llevar contenido envolvente a visionOS
Descubre cómo el streaming foveado transmite escenas renderizadas de forma remota a Apple Vision Pro con total fidelidad. Explora cómo este framework combina las capacidades nativas de visionOS con tecnologías de streaming de terceros de forma totalmente inalámbrica, tal y como se muestra en una escena de OpenXR y NVIDIA CloudXR. Obtén información sobre el framework FoveatedStreaming, su integración con el SDK de NVIDIA CloudXR y cómo el streaming foveado dinámico ofrece ventajas sin dejar de preservar la privacidad.
Capítulos
- 0:00 - Introduction
- 4:08 - How Foveated Streaming works
- 4:46 - Set up the streaming endpoint
- 5:18 - Create a visionOS receiver app
- 8:02 - Integrate with the streaming endpoint
- 11:28 - Measure performance
- 11:56 - Enhance with visionOS features
- 13:56 - Next steps
Recursos
- Analyzing the performance of a foveated streaming session
- Establishing foveated streaming sessions with Apple Vision Pro
- Streaming a CloudXR application to Apple Vision Pro with foveation
- Creating a foveated streaming client on visionOS
- Foveated Streaming
- StreamingSession: Streaming immersive content from a CloudXR™ application to visionOS and iOS
Videos relacionados
WWDC26
-
Buscar este video…
-
-
6:03 - Connect to a streaming endpoint
// Connect to a streaming endpoint import SwiftUI import FoveatedStreaming struct ConnectView: View { let session: FoveatedStreamingSession var body: some View { Button("Connect") { Task { try await session.connect() } } } } -
6:44 - Display a Foveated Streaming session in your immersive space
// Display a Foveated Streaming session in your immersive space import SwiftUI import FoveatedStreaming @main struct FoveatedStreamingSampleApp: App { private let session = FoveatedStreamingSession() var body: some SwiftUI.Scene { ImmersiveSpace(foveatedStreaming: session) } } -
6:55 - Compose SwiftUI content with Foveated Streaming
// Compose SwiftUI content with Foveated Streaming import SwiftUI import FoveatedStreaming @main struct FoveatedStreamingSampleApp: App { private let session = FoveatedStreamingSession() private let appModel = AppModel() var body: some SwiftUI.Scene { Window("Main", id: appModel.mainWindowId) { ContentView(session: session) .environment(appModel) .environment(session) // ... } ImmersiveSpace(foveatedStreaming: session) { SpatialContainer { ReopenMainWindowView().environment(appModel) TransformStreamWidgetView().environment(session) } } } } -
13:42 - Compose RealityKit content with Foveated Streaming
// Compose RealityKit content with Foveated Streaming import SwiftUI import RealityKit import FoveatedStreaming @main struct FoveatedStreamingSampleApp: App { private let session = FoveatedStreamingSession() private let appModel = AppModel() var body: some SwiftUI.Scene { ImmersiveSpace(foveatedStreaming: session) { RealityView { content in // ... } } } }
-
-
- 0:00 - Introduction
Overview of Foveated Streaming — a new framework in visionOS 26.4 that lets Apple Vision Pro connect to a PC and stream immersive OpenXR content with eye-tracking-based video optimization.
- 4:08 - How Foveated Streaming works
Learn the architecture: the visionOS receiver app uses the FoveatedStreaming framework, the streaming endpoint implements the Foveated Streaming Protocol, and NVIDIA CloudXR handles the OpenXR runtime and Wi-Fi/cloud streaming.
- 4:46 - Set up the streaming endpoint
Set up your Windows streaming endpoint using open-source sample code on GitHub, which includes a reference implementation of the Foveated Streaming Protocol and an example OpenXR application.
- 5:18 - Create a visionOS receiver app
Build your visionOS receiver app using FoveatedStreamingSession to connect and pair with the endpoint, present streamed content in an ImmersiveSpace, and enrich the UI with SwiftUI windows and progressive immersion.
- 8:02 - Integrate with the streaming endpoint
Configure your OpenXR client with the Foveated Streaming Protocol for authentication and pairing, and integrate the NVIDIA CloudXR runtime to handle input data, hand tracking, depth buffers, and alpha channel compositing.
- 11:28 - Measure performance
Use the Foveated Streaming instrument in Xcode to measure your stream bandwidth, pose latency, and frame rate, and diagnose performance issues before shipping.
- 11:56 - Enhance with visionOS features
Enhance your app with message channels to exchange data between the visionOS and OpenXR apps, ARKit to anchor virtual content to the physical world, and RealityKit to composite native 3D objects alongside the streamed content.
- 13:56 - Next steps
Download sample code on GitHub and developer.apple.com to get started, set up your own receiver app, and integrate your OpenXR client with the Foveated Streaming Protocol.