Loading USDZ files with SwiftUI

Question

Hello! I've been trying to create a custom USDZ viewer using the Vision Pro. Basically, I want to be able to load in a file and have a custom control system I can use to transform, playback animations, etc.

I'm getting stuck right at the starting line however. As far as I can tell, the only way to access the file system through SwiftUI is to use the DocumentGroup struct to bring up the view. This requires implementing a file type through the FileDocument protocol.

All of the resources I'm finding use text files as their example, so I'm unsure of how to implement USDZ files. Here is the FileDocument I've built so far:

import SwiftUI
import UniformTypeIdentifiers
import RealityKit

struct CoreUsdzFile: FileDocument {
    // we only support .usdz files
    static var readableContentTypes = [UTType.usdz]
    
    // make empty by default
    var content: ModelEntity = .init()
    
    // initializer to create new, empty usdz files
    init(initialContent: ModelEntity = .init()){
        content = initialContent
    }
    
    // import or read file
    init(configuration: ReadConfiguration) throws {
        if let data = configuration.file.regularFileContents {
            // convert file content to ModelEntity?
            content = ModelEntity.init(mesh: data)
        } else {
            throw CocoaError(.fileReadCorruptFile)
        }
    }
    
    // save file wrapper
    func fileWrapper(configuration: WriteConfiguration) throws -> FileWrapper {
        let data = Data(content)
        return FileWrapper(regularFileWithContents: data)
    }
}

My errors are on conversion of the file data into a ModelEntity and the reverse. I'm not sure if ModelEntity is the correct typing here, but as far as I can tell .usdz files are imported as ModelEntities.

Any help is much appreciated!

Dylan

580

Posted by

dganim8s

Reply

Load usdz files with Entity.load(contentsOf: URL) Reference

—
edj

Add a Comment

Answer 1

For example, this demonstrates how to load a USDz file from a file picker, as demonstrated here: https://vimeo.com/922469524

When you tap the button, it loads the system file picker and allows the user to load a USDz file from their downloads or locally saved files.

import RealityKit
import SwiftUI

@main
struct Application: App {
    @State var entity: Entity? = nil
    @State var showFilePicker: Bool = false

    var body: some SwiftUI.Scene {
        WindowGroup {
            VStack {
                // A view for displaying the loaded asset.
                //
                RealityView(
                    make: { content in
                        // Add a placeholder entity to parent the entity to.
                        //
                        let placeholderEntity = Entity()
                        placeholderEntity.name = "$__placeholder"
                        
                        if let loadedEntity = self.entity {
                            placeholderEntity.addChild(loadedEntity)
                        }
                        
                        content.add(placeholderEntity)
                    },
                    update: { content in
                        guard let placeholderEntity = content.entities.first(where: {
                            $0.name == "$__placeholder"
                        }) else {
                            preconditionFailure("Unable to find placeholder entity")
                        }
                        
                        // If there is a loaded entity, remove the old child,
                        // and add the new one.
                        //
                        if let loadedEntity = self.entity {
                            placeholderEntity.children.removeAll()
                            placeholderEntity.addChild(loadedEntity)
                        }
                    }
                )
                
                // A button that displays a file picker for loading a USDZ.
                //
                Button(
                    action: {
                        showFilePicker.toggle()
                    },
                    label: {
                        Text("Load USDZ")
                    }
                )
                    .padding()
            }
                .fileImporter(isPresented: $showFilePicker, allowedContentTypes: [.usdz]) { result in
                    // Get the URL of the USDZ picked by the user.
                    //
                    guard let url = try? result.get() else {
                        print("Unable to get URL")
                        return
                    }

                    Task {
                        // As the app is sandboxed, permission needs to be
                        // requested to access the file, as it's outside of
                        // the sandbox.
                        //
                        if url.startAccessingSecurityScopedResource() {
                            defer {
                                url.stopAccessingSecurityScopedResource()
                            }
                            
                            // Load the USDZ asynchronously.
                            //
                            self.entity = try await Entity(contentsOf: url)
                        }
                    }
                }
        }
    }
}

Posted by

Matt Cox

This is what I needed, thank you!

—
dganim8s

Add a Comment

Answer 2

I'm curious what you want the flow of the app to be like in an ideal world? FileDocument isn't the only way to get the USDZ loaded in, so I wouldn't rely on it unless you have to.

Posted by

Matt Cox

Add a Comment

Answer 3

For example, this demonstrates how to load a USDz file from a file picker, as demonstrated here: https://vimeo.com/922469524

When you tap the button, it loads the system file picker and allows the user to load a USDz file from their downloads or locally saved files.

import RealityKit
import SwiftUI

@main
struct Application: App {
    @State var entity: Entity? = nil
    @State var showFilePicker: Bool = false

    var body: some SwiftUI.Scene {
        WindowGroup {
            VStack {
                // A view for displaying the loaded asset.
                //
                RealityView(
                    make: { content in
                        // Add a placeholder entity to parent the entity to.
                        //
                        let placeholderEntity = Entity()
                        placeholderEntity.name = "$__placeholder"
                        
                        if let loadedEntity = self.entity {
                            placeholderEntity.addChild(loadedEntity)
                        }
                        
                        content.add(placeholderEntity)
                    },
                    update: { content in
                        guard let placeholderEntity = content.entities.first(where: {
                            $0.name == "$__placeholder"
                        }) else {
                            preconditionFailure("Unable to find placeholder entity")
                        }
                        
                        // If there is a loaded entity, remove the old child,
                        // and add the new one.
                        //
                        if let loadedEntity = self.entity {
                            placeholderEntity.children.removeAll()
                            placeholderEntity.addChild(loadedEntity)
                        }
                    }
                )
                
                // A button that displays a file picker for loading a USDZ.
                //
                Button(
                    action: {
                        showFilePicker.toggle()
                    },
                    label: {
                        Text("Load USDZ")
                    }
                )
                    .padding()
            }
                .fileImporter(isPresented: $showFilePicker, allowedContentTypes: [.usdz]) { result in
                    // Get the URL of the USDZ picked by the user.
                    //
                    guard let url = try? result.get() else {
                        print("Unable to get URL")
                        return
                    }

                    Task {
                        // As the app is sandboxed, permission needs to be
                        // requested to access the file, as it's outside of
                        // the sandbox.
                        //
                        if url.startAccessingSecurityScopedResource() {
                            defer {
                                url.stopAccessingSecurityScopedResource()
                            }
                            
                            // Load the USDZ asynchronously.
                            //
                            self.entity = try await Entity(contentsOf: url)
                        }
                    }
                }
        }
    }
}

Posted by

Matt Cox

This is what I needed, thank you!

—
dganim8s

Add a Comment

Answer 4

Of course if you do want to go down the document based app route, it's possible but it has some weirdness about it (mostly due to limitations with the FileDocument API). Because of the bizarre ways the document based API works, I'd still suggest using the file picker approach, despite this taking care of the sandbox access for you.

You're on the right lines with your code, but because the FileDocument API doesn't give you the URL, and there's no mechanism for loading a USDZ as an Entity from Data, you have to write the USDZ contents to a temporary directory that you can access from your app sandbox, and then load it using the standard mechanism.

This is demonstrated here: https://vimeo.com/922479510.

import RealityKit
import SwiftUI
import UniformTypeIdentifiers

struct USDDocumentLoaderDocument: FileDocument {
    // The entity will hold the contents of the USDZ file. 
    //
    var entity: Entity

    static var readableContentTypes: [UTType] { [.usdz] }

    init(configuration: ReadConfiguration) throws {
        // Load the contents of the USDZ file.
        //
        guard let data = configuration.file.regularFileContents else {
            throw CocoaError(.fileReadCorruptFile)
        }
        
        // Write the data to a temporary location.
        //
        let temporaryFile = FileManager.default.temporaryDirectory.appendingPathComponent(UUID().uuidString, conformingTo: .usdz)
        try data.write(to: temporaryFile)
        
        // Load the USDZ from the temporary file. This blocks, which isn't ideal.
        //
        self.entity = try Entity.load(contentsOf: temporaryFile)
    }
    
    func fileWrapper(configuration: WriteConfiguration) throws -> FileWrapper {
        throw CocoaError(.featureUnsupported)
    }
}

Posted by

Matt Cox

Add a Comment

Loading USDZ files with SwiftUI

Accepted Reply

Replies