SceneKit

RSS for tag

Create 3D games and add 3D content to apps using high-level scene descriptions using SceneKit.

Posts under SceneKit tag

22 Posts

Post

Replies

Boosts

Views

Activity

Best approach for animating a speaking avatar in a macOS/iOS SwiftUI application
I am developing a macOS application using SwiftUI (with an iOS version as well). One feature we are exploring is displaying an avatar that reads or speaks dynamically generated text produced by an AI service. The basic flow would be: Text generated by an AI service Text converted to speech using a TTS engine An avatar (2D or 3D) rendered in the app that animates lip movement synchronized with the speech Ideally the avatar would render locally on the device. Questions: What Apple frameworks would be most appropriate for implementing a speaking avatar? SceneKit RealityKit SpriteKit (for 2D avatars) Is there any recommended way to drive lip-sync animation from speech audio using Apple frameworks? Does AVSpeechSynthesizer expose phoneme or viseme timing information that could be used for avatar animation? If such timing information is not available, what is the recommended approach for synchronizing character mouth animation with speech audio on macOS/iOS? Are there examples of real-time character animation synchronized with speech on macOS/iOS? Any architectural guidance or references would be greatly appreciated.
0
0
802
Mar ’26
SceneKit shader compile and cache issues? :(
I'm making an apple watch game and it's running so smooth and nice while developing but I ran into a showstopping problem that I wouldn't even have noticed until i deleted the app and reinstalled... Every first-run on a freshly installed copy of the game has major frame hitches as it loads assets. Every run after that has ZERO hitches and runs like butter. Even if i install a new version on top of the old one (which is why i never noticed this problem in my almost finished game) it runs smooth. Framerate issues come back if I delete and reinstall. I think it's compiling shaders or caching geometry but I haven't been able to trick it into doing the hard work during non-critical moments in the game. I've tried preloading assets in the main menu, loading at the start of the level, loading and rendering in a tiny part of the screen, rendering behind dialog boxes... it seems no matter what i can only get rid of the frame hitches after the game has been played through once... and I'm going to lose all customers on that first terrible impression even though the game is actually really good and fun when it's running properly... Any suggestions how i can prewarm/precompile the assets to avoid framerate hitches and freezes during gameplay? I thought I was so close to release but I feel this is un-shippable! I can send a link to my test flight build to anyone that wants to see the problem first hand. Also please note that I am artist-forward with a technical lean, but certainly not a real programmer, unfortunately. Jiovanie
1
0
216
Apr ’26
Best approach for animating a speaking avatar in a macOS/iOS SwiftUI application
I am developing a macOS application using SwiftUI (with an iOS version as well). One feature we are exploring is displaying an avatar that reads or speaks dynamically generated text produced by an AI service. The basic flow would be: Text generated by an AI service Text converted to speech using a TTS engine An avatar (2D or 3D) rendered in the app that animates lip movement synchronized with the speech Ideally the avatar would render locally on the device. Questions: What Apple frameworks would be most appropriate for implementing a speaking avatar? SceneKit RealityKit SpriteKit (for 2D avatars) Is there any recommended way to drive lip-sync animation from speech audio using Apple frameworks? Does AVSpeechSynthesizer expose phoneme or viseme timing information that could be used for avatar animation? If such timing information is not available, what is the recommended approach for synchronizing character mouth animation with speech audio on macOS/iOS? Are there examples of real-time character animation synchronized with speech on macOS/iOS? Any architectural guidance or references would be greatly appreciated.
Replies
0
Boosts
0
Views
802
Activity
Mar ’26
SceneKit shader compile and cache issues? :(
I'm making an apple watch game and it's running so smooth and nice while developing but I ran into a showstopping problem that I wouldn't even have noticed until i deleted the app and reinstalled... Every first-run on a freshly installed copy of the game has major frame hitches as it loads assets. Every run after that has ZERO hitches and runs like butter. Even if i install a new version on top of the old one (which is why i never noticed this problem in my almost finished game) it runs smooth. Framerate issues come back if I delete and reinstall. I think it's compiling shaders or caching geometry but I haven't been able to trick it into doing the hard work during non-critical moments in the game. I've tried preloading assets in the main menu, loading at the start of the level, loading and rendering in a tiny part of the screen, rendering behind dialog boxes... it seems no matter what i can only get rid of the frame hitches after the game has been played through once... and I'm going to lose all customers on that first terrible impression even though the game is actually really good and fun when it's running properly... Any suggestions how i can prewarm/precompile the assets to avoid framerate hitches and freezes during gameplay? I thought I was so close to release but I feel this is un-shippable! I can send a link to my test flight build to anyone that wants to see the problem first hand. Also please note that I am artist-forward with a technical lean, but certainly not a real programmer, unfortunately. Jiovanie
Replies
1
Boosts
0
Views
216
Activity
Apr ’26