Sample Project for WWDC24 10092 Metal with Passthrough?

Question

KTRosenberg OP

Created 1w

Replies 2

Boosts 1

Participants 2

It’s great that we’ll be able to use Metal custom renderers in passthrough mode on visionOS.

https://developer.apple.com/wwdc24/10092

This is a lot of complicated set-up, however. It’s also unclear how occlusion and custom algorithms / raytracing will work in tandem with scene understanding. May we have a project template and/or sample? Preferably with the C api and not just swift. This would be much-appreciated and helpful to everyone who wants this set-up. I’d like to see the whole process.

Thank you for introducing this feature!

Answered by DTS Engineer in 790217022

Please enjoy Interacting with virtual content blended with passthrough !

Send us an enhancement request for a C-language-based Compositor Services sample and we'll gladly consider it.

Boost

Answer 1

DTS Engineer OP

Apple

1w

Accepted Answer

Recommended

Please enjoy Interacting with virtual content blended with passthrough !

Send us an enhancement request for a C-language-based Compositor Services sample and we'll gladly consider it.

1

Answer 2

KTRosenberg OP

1w

Unfortunately, the example doesn't show how to integrate scene understanding for realistic lighting. It just seems to show how to add passthrough in the background. Is there an example that is more advanced, showing how to do occlusion, use the environment texture, do lighting with the scene reconstructed mesh, etc.?

If not, that's super needed. It's not so straightforward.

0