RealityKit MeshResource Memory Leak

Hello,

There isn't quite enough information here to determine if you should file a bug report for this issue or not. It is possible that your code is the cause of the memory growth, my recommendation is that you put together a small sample project that demonstrates the issue and post it back here.

Also, I did not observe any significant memory growth with the following snippet:

class GameViewController: NSViewController {
    
    @IBOutlet var arView: ARView!
    
    let modelEntity = ModelEntity()
    
    override func awakeFromNib() {
        
        let anchor = AnchorEntity()
        
        anchor.position = .init(0, 0, -30)
        anchor.addChild(modelEntity)
        
        arView.scene.addAnchor(anchor)
        
        Timer.scheduledTimer(withTimeInterval: 0.4, repeats: true) { _ in
            
            let mesh = MeshResource.generateSphere(radius: .random(in: 5...10))
            
            let model = ModelComponent(mesh: mesh, materials: [SimpleMaterial(color: .red, isMetallic: true)])
            
            self.modelEntity.model = model
            
        }
    }
}

So it seems that this is at least not an issue with MeshResource in every case, though maybe there is something more specific about your case.

Hi,

I've noticed similar behaviour on the iOS 16 Beta and was able to isolate it down into a really simple project. This is happening on iOS 16.0 (20A5312j) and Xcode Version 14.0 beta 3 (14A5270f). On iOS 15 things appear to be working fine.

Apparently this only happens with dynamically generated meshes, not for example when loading a model from a USDZ. Nevertheless it seems like a pretty critical bug and causes constant memory ramp up for me.

FB Number with sample project: FB10806403

import UIKit
import RealityKit

class ViewController: UIViewController {

    // if I change this to let arView = ARView() there is no memory leak
    lazy var arView = ARView()

    override func loadView() {
        view = arView
    }

    override func viewDidLoad() {
        super.viewDidLoad()

        let boxSize: SIMD3<Float> = .init(repeating: 0.3)

        // ↓ just allocating this MeshResource creates a leak on iOS 16
        let _: MeshResource = .generateBox(
            width: boxSize.x,
            height: boxSize.y,
            depth: boxSize.z
        )
    }
}

Setup:

Leak:

@arthurfromberlin

Continue to follow-up with your bug report, but also note that in the example code I provided above (where I create a new MeshResource every 0.4 seconds), I am not observing a build-up of MeshResources in Allocations. (iOS 16 beta 3, Xcode 14 beta 3)

If the MeshResources were leaking, I would expect the "# Persistent" here to be 219, and the "# Transient" to be 0.

Also, overall, I did not observe unbounded memory growth (there were fluctuations as the camera moved around, which is to be expected because ARKit's memory usage will fluctuate for features like scene reconstruction when the camera moves around, but these fluctuations were not unbounded in my observations).

Some new observations from today: In my project with the steady memory increase on iOS 16 I assign environmentTexturing = .automatic on my ARWorldTrackingConfiguration and subscribe to SceneEvents.DidAddEntity and SceneEvents.WillRemoveEntity. Even if I leave the subscription handler empty, I get that memory increase which quickly goes into the range of 1GB. As soon as I either remove the subscriptions (the handler is empty) or set environmentTexturing = .none memory consumption improves drastically and stays in manageable ranges. I understand that enabling environment texturing is expected to require more memory, but here I'm seeing constant growth.

I'm experiencing the same issues on Beta 4. Disabling environment texturing prevents the memory from constantly growing.

Memory debugger still reports the same leaks as in the screenshots above. Wether they are false positives or not; it's at least irritating during the development process. Additionally I found that if I run the default ARKit+RealityKit Xcode template project and enable sceneReconstruction = .mesh there are lot's of CFData leaks reported. No critical issue, but again I find it irritating during development and debugging. Screenshot attached.

Here's a self-contained example that leaks memory for me. ( Xcode Version 14.2 (14C18) Running on an iPhone 12 Pro, iOS Version 16.2)

Should I be doing something more to the old ModelEntity instances to clean them up than .removeFromParent()?

When the frame counter at the top of the display gets to 5000, memory usage is > 1Gb and climbing linearly.

Comparing exported memgraph files from short and long runs shows that the long run has accumulated 10s of thousands of NSArray instances.

Examining one shows that it's a __NSSingleObjectArrayI referenced by one REPrivateBufferMeshPayload and it references a REMeshPartDescriptor.

Not sure if this suggests any approach to avoid this memory leak to anyone? I'm new here.

Thanks.

import SwiftUI
import RealityKit
import simd

struct ContentView : View {
    @State var t: Double = 0.05
    @State var r: Double = 0.05
    @State var n: Int = 0
    let timer = Timer.publish(every: 1.0/30.0, on: .main, in: .common).autoconnect()
    var body: some View {
        ZStack {
            ARViewContainer(t: $r).edgesIgnoringSafeArea(.all).onReceive(timer) { _ in
                t = t + 0.05
                r = 0.15 + 0.1*sin(t)
                n += 1
            }
            VStack{
                Text("\(n)")
                Spacer()
            }
        }
    }
}

struct ARViewContainer: UIViewRepresentable {
    @Binding var t: Double
    func makeUIView(context: Context) -> ARView {
        let arView = ARView(frame: .zero)
        let anchor = AnchorEntity(
               .plane([.horizontal],
                    classification: [.table, .floor],
                    minimumBounds: [0.05, 0.05]
            )
       )
       arView.scene.anchors.append(anchor)
        return arView
    }

    func updateUIView(_ uiView: ARView, context: Context) {

        let anchor = uiView.scene.anchors[ uiView.scene.anchors.endIndex - 1 ]
        var to_remove: [Entity] = []
        for c in anchor.children {
            to_remove.append(c)
        }
        let _ = to_remove.map {c in c.removeFromParent()}

        var descr = MeshDescriptor(name: "ring")
        descr.positions = MeshBuffers.Positions( vertices_simd(t)  )
        descr.primitives = .triangles(  triangles() )
        let ring_entity = ModelEntity(
            mesh: try! .generate(from: [descr]),
            materials: [SimpleMaterial(color: .orange, isMetallic: true)]
        )
        ring_entity.setParent(anchor)
    }
}

func triangles()-> [UInt32] {
    let nh = 20
    let nr = 40
    var ind : [UInt32] = []
    let n = ( nr )
    for i in 0..<(nh-1) {
        for j in 0..<(nr-1) {
            ind = ind + [  (i*n + j) , (i+1 )*n + j    , ((i+1)*n + j+1) ].map{UInt32($0)}
            ind = ind + [  (i*n + j) , ((i+1)*n + j+1) , ((i)*n   + j+1) ].map{UInt32($0)}
        }
    }
    return ind
}

func vertices_simd(_ r: Double)-> [simd_float3] {
    let nh = 20
    let nr = 40
    let rt:Double = 0.01
    let y:Double = 0.01
    var vertices:[simd_float3] = []
    for i in 0..<nh {
        for j in 0..<nr {
            let theta = (Double.pi * 2.0 * Double(j)/(Double(nr)-1.0))
            let phi  =  (Double.pi * 2.0 * Double(i)/(Double(nh)-1.0))
            let x = Float((r + rt*cos(phi) )*cos(theta))
            let y = Float(y + rt*sin(phi))
            let z = Float((r + rt*cos(phi) )*sin(theta))
            let v = simd_float3(x,y,z)
            vertices.append( v )
        }
    }
    return(vertices)
}

@0xN

Please file a bug report if you haven't already using Feedback Assistant. I can reproduce with your example, though I'd recommend you include an even simpler example in your bug report, for example:

func generate() {
        var descr = MeshDescriptor(name: "triangle")
        
        descr.positions = .init([.zero, .one, .init(0.5, 0.5, 0.5)])
        descr.primitives = .triangles([0, 1, 2])
        
        try! MeshResource.generate(from: [descr])
}

.task {
    for _ in 0...1000000 {
        generate()
    }
}