Releasing a TextureObject from memory

Hi there!

I´m trying to make a 360 image carousel in RealityView/SwiftUI with very large textures. I´ve managed to load one 12K 360 image and showing it on a inverted sphere with a ShaderMaterialGraph made in Reality Composer Pro. When I try to load the next image I get an out of memory error. The carousel works fine with smaller textures.

My question is. How do I release the memory from the current texture before loading the next?

In theory the garbagecollector should erase it eventually?

Hope someone can help =)

Thanks in advance!

Best regards, Kim

Answered by Vision Pro Engineer in 822780022

Hello @Kito7777

The Swift programming language does not use a garbage collector. Instead, Swift uses automatic reference counting to manage memory. If you are coming from a language that does use a garbage collector, like C#, Java, or Go, you should take some time to understand the benefits and tradeoffs a language like Swift provides. Automatic reference counting means memory that is no longer referenced by anything (its "reference count" is 0) is automatically released.

The correct way to deallocate your texture is to remove all references to it. For example, you may need to set all references to your texture to nil. This might not be straight forward, especially if you are referencing the texture somewhere else in your program. If you share more about how you are storing your texture in memory, I'd be happy to troubleshoot it for you!

With all that said, 12K does seem very large. Are you sure this size is necessary? The sky dome in Presenting an artist's scene is only 8K and 7.6 MB, for comparison, although to be fair this texture is only for a dome shape and not a 360 sphere.

Accepted Answer

Hello @Kito7777

The Swift programming language does not use a garbage collector. Instead, Swift uses automatic reference counting to manage memory. If you are coming from a language that does use a garbage collector, like C#, Java, or Go, you should take some time to understand the benefits and tradeoffs a language like Swift provides. Automatic reference counting means memory that is no longer referenced by anything (its "reference count" is 0) is automatically released.

The correct way to deallocate your texture is to remove all references to it. For example, you may need to set all references to your texture to nil. This might not be straight forward, especially if you are referencing the texture somewhere else in your program. If you share more about how you are storing your texture in memory, I'd be happy to troubleshoot it for you!

With all that said, 12K does seem very large. Are you sure this size is necessary? The sky dome in Presenting an artist's scene is only 8K and 7.6 MB, for comparison, although to be fair this texture is only for a dome shape and not a 360 sphere.

Hi @Kito7777,

Without more context or code it's hard to know what's happening. Some thoughts though that might help you track it down.

Not sure if your reference to a 'garbage collector' was colloquial or intentional. There is no garbage collector in Swift it instead uses automatic reference counting. So for example this code:

let descriptor = MTLTextureDescriptor(...)
var texture: (any MTLTexture)? = myDevice.makeTexture(descriptor: descriptor)
texture = nil

will create a texture then immediately free it, because the reference count has gone to zero.

however if I pass that instance of (any MTLTexture)? around to lots of other things and they all maintain a strong reference then the reference count won't go to zero and thus it'll never be freed.

So my guess is that more than one object in your code has a reference to the texture and one or more of them is not setting their references to nil.

Thank you very much for both answers. Im learning new stuff every day =)

Ill try some more!

Best regards, Kim

Sorry guys, I still got problems. I tried to set the texture to nil, but the material still has a reference to the texture apparently?

The code is here:

import SwiftUI
import RealityKit
import RealityKitContent

struct GlobalData {
    var currNr: Int = 0
    var images: [String] = ["view0","view1","view2","view3","view4"]
    var matX: ShaderGraphMaterial!
    var currImage: TextureResource!
}

class ContentViewModel: ObservableObject {
    @Published var globalData: GlobalData

    init() {
        self.globalData = GlobalData(
            currNr: 0
        )
    }
}

struct ImmersiveView: View {
    @EnvironmentObject var vm: ContentViewModel
    @State var updating: Bool = false

    var body: some View {
        HStack {
            Button(action: {
                if !updating && vm.globalData.currNr < vm.globalData.images.count-1 {
                    vm.globalData.currNr += 1
                    updating = true
                } else {
                    vm.globalData.currNr = 0
                }
            }) {
                Text(String(vm.globalData.currNr))
            }
        }
        .offset(x: 0, y: -300)
        .scaleEffect(x: 2, y: 2)
        
        RealityView { content in
            do {
                vm.globalData.matX = try await ShaderGraphMaterial(named: "/Root/Mat_Stereo360",
                                                     from: "360Stereo.usda",
                                                     in: realityKitContentBundle)
            } catch {
                print("Error loading material: \(error)")
            }
                
            do {
                vm.globalData.currImage = try await TextureResource(named: vm.globalData.images[vm.globalData.currNr])
                try vm.globalData.matX.setParameter(name: "imageFile", value: .textureResource(vm.globalData.currImage))

            } catch {
                print("Error loading texture: \(error)")
            }
                
            let sphere = MeshResource.generateSphere(radius:1000)
            let entity = ModelEntity()
            entity.components.set(ModelComponent(mesh: sphere, materials: [vm.globalData.matX]))
            content.add(entity)
            
        } update: { content in
            Task {
                if updating { do {
                    vm.globalData.currImage = nil
                    vm.globalData.currImage =  try await TextureResource(named: vm.globalData.images[vm.globalData.currNr])
                    try vm.globalData.matX.setParameter(name: "imageFile", value: .textureResource(vm.globalData.currImage))
                    updating = false;
                } catch {
                    print("Error loading texture: \(error)")
                    updating = false;
                }
                }
            }
            
        }
    }
}

Thanks, this is my first RealityView/Swift project, so I really appreciate your help! =)

Best regards, Kim

Hey @Kito7777.

The ShaderGraphMaterial object is a structure which means that it is copied when you pass it around. For more background, you might consider reading Choosing Between Structures and Classes.

Rather than keeping a reference to this structure, I'd advise that you restructure your application so that you only keep a reference to the ModelEntity that contains the ShaderGraphMaterial you need to update the parameter with.

Using the following code in my update method I was able to change the skydome image without seeing any uptick in memory usage:

let resourceNamed = appModel.images[appModel.currentImage]

guard let sphereEntity = appModel.sphereEntity else { print("No sphereEntity"); return }
guard let modelComponent = sphereEntity.components[ModelComponent.self] else { print("No Model Component"); return }
guard var shaderGraphMaterial = modelComponent.materials.first as? ShaderGraphMaterial else { print("No ShaderGraphMaterial"); return }

Task {
    do {
        let textureResource = try await TextureResource(named: resourceNamed)
        try shaderGraphMaterial.setParameter(name: "imageFile", value: .textureResource(textureResource))
    } catch {
        print("Error loading texture resource: \(error)")
    }

    sphereEntity.model?.materials = [shaderGraphMaterial]
}

Let me know if this helps,
Michael

YES! Just what I needed.

Thanks a lot!

--Kim

Releasing a TextureObject from memory
 
 
Q