While trying to control the following two scenes in 1 ImmersiveSpace, we found the following memory leak when we background the app while a stereoscopic video is playing.
ImmersiveView's two scenes:
- Scene 1 has 1 toggle button
- Scene 2 has same toggle button with a 180 degree skysphere playing a stereoscopic video
Attached are the files and images of the memory leak as captured in Xcode.
To replicate this memory leak, follow these steps:
- Create a new visionOS app using Xcode template as illustrated below.
- Configure the project to launch directly into an immersive space (set
Preferred Default Scene Session RoletoImmersive Space Application Session Rolein Info.plist. - Replace all swift files with those you will find in the attached texts.
- In
ImmersiveView, replace the stereoscopic video to play with a large 3d 180 degree video of your own bundled in your project. - Launch the app in debug mode via Xcode and onto the AVP device or simulator
- Display the memory use by pressing on keys
command+7and selectingMemoryin order to view the live memory graph - Press on the first immersive space's button "Open ImmersiveView"
- Press on the second immersive space's button "Show Immersive Video"
- Background the app
- When the app tray appears, foreground the app by selecting it
- The first immersive space should appear
- Repeat steps 7, 8, 9, and 10 multiple times
- Observe the memory use going up, the graph should look similar to the below illustration.
In ImmersiveView, upon backgrounding the app, I do:
- a reset method to clear the video's memory
- dismiss of the Immersive Space containing the video (even though upon execution, visionOS raises the purple warning "Unable to dismiss an Immersive Space since none is opened". It appears visionOS dismisses any ImmersiveSpace upon backgrounding, which makes sense..)
Am I not releasing the memory correctly?
Or, is there really a memory leak issue in either SwiftUI's ImmersiveSpace or in AVFoundation's AVPlayer upon background of an app?
//
// TestVideoLeakOneImmersiveViewApp.swift
// TestVideoLeakOneImmersiveView
//
import OSLog
import SwiftUI
let SUBSYSTEM = Bundle.main.bundleIdentifier!
let scaleForAttachements = SIMD3(2.5, 2.5, 2.5)
@main
struct TestVideoLeakOneImmersiveViewApp: App {
let logger = Logger(subsystem: SUBSYSTEM, category: "TestVideoLeakOneImmersiveViewApp")
@Environment(\.openImmersiveSpace) var openImmersiveSpace
@Environment(\.dismissImmersiveSpace) var dismissImmersiveSpace
@Environment(\.scenePhase) var scenePhase
@State private var appModel = AppModel()
var body: some Scene {
Group {
ImmersiveSpace(id: "InitialImmersiveView") {
InitialImmersiveView()
.environment(appModel)
.onAppear {
appModel.immersiveSpaceState = .open
// When app backgrounds, `immersiveSpaceID` will remain no matter which scene was showing prior to background.
// Since `InitialImmersiveView` is the first scene of the app, visionOS will display
// `InitialImmersiveView` when foregrounded.
// Setting immersiveSpaceID to `InitialImmersiveView` ensures that when it is changed
// in `InitialImmersiveView`, the listener here below will execute upon changing of `immersiveSpaceID`
appModel.immersiveSpaceID = "InitialImmersiveView"
}
.onDisappear {
appModel.immersiveSpaceState = .closed
}
}
.immersionStyle(selection: .constant(.mixed), in: .mixed)
ImmersiveSpace(id: "ImmersiveView") {
ImmersiveView()
.environment(appModel)
.onAppear {
appModel.immersiveSpaceState = .open
}
.onDisappear {
appModel.immersiveSpaceState = .closed
}
}
.immersionStyle(selection: .constant(.mixed), in: .mixed)
}
.onChange(of: appModel.immersiveSpaceID) { oldValue, newValue in
logger.info("\(#function) \(#line) `.onChange(of: appModel.immersiveSpaceID` oldValue \(String(describing: oldValue)) newValue \(String(describing: newValue))")
Task {
guard scenePhase == .active else {
logger.info("\(#function) \(#line) escaping since scene is not active")
return
}
logger.info("\(#function) \(#line) opening appModel.immersiveSpaceID: \(appModel.immersiveSpaceID)")
await dismissImmersiveSpace()
let result = await openImmersiveSpace(id: appModel.immersiveSpaceID)
switch result {
case .opened:
logger.info("\(#function) \(#line) should be presenting appModel.immersiveSpaceID: \(appModel.immersiveSpaceID)")
case .error:
logger.error("\(#function) \(#line) could not open scene appModel.immersiveSpaceID: \(appModel.immersiveSpaceID)")
assertionFailure("could not open scene")
case .userCancelled:
logger.info("\(#function) \(#line) userCancelled the openning of the scene appModel.immersiveSpaceID: \(appModel.immersiveSpaceID)")
fallthrough
@unknown default:
logger.error("\(#function) \(#line) unhandled result \(String(describing: result))")
assertionFailure("could not open scene")
}
}
}
.onChange(of: scenePhase, { oldValue, newValue in
logger.info("\(#function) \(#line) `onChange(of: scenePhase` scenePhase oldValue \(String(describing: oldValue)) scenePhase newValue \(String(describing: newValue))")
switch newValue {
case .active:
logger.info("\(#function) \(#line) scenePhase newValue \(String(describing: newValue))")
case .background:
logger.info("\(#function) \(#line) scenePhase newValue \(String(describing: newValue))")
// reset appmodel for initial immersive space
case .inactive:
logger.info("\(#function) \(#line) scenePhase newValue \(String(describing: newValue))")
@unknown default:
logger.warning("\(#function) \(#line) scenePhase newValue \(String(describing: newValue))")
}
})
}
}
//
// InitialImmersiveView.swift
// TestVideoLeakOneImmersiveView
//
import AVKit
import OSLog
import SwiftUI
import RealityKit
import RealityKitContent
struct InitialImmersiveView: View {
let logger = Logger(subsystem: SUBSYSTEM, category: "InitialImmersiveView")
@Environment(\.scenePhase) var scenePhase
@Environment(AppModel.self) var appModel
private let rootEntity = Entity()
var body: some View {
RealityView { content, attachments in
logger.info("\(#function) \(#line) `content, attachments in`")
updateScene(content: content, attachments: attachments)
} update: { content, attachments in
logger.info("\(#function) \(#line) `update: { content, attachments in`")
updateScene(content: content, attachments: attachments)
} placeholder: {
let _ = logger.info("\(#function) \(#line) `placeholder`")
ProgressView()
} attachments: {
let _ = logger.info("\(#function) \(#line) `attachments`")
Attachment(id: "OpenImmersiveView") {
Button {
// dismissCurrnt and open next immersive space
logger.info("\(#function) \(#line) Button `OpenImmersiveView` pressed, appModel.immersiveSpaceID \(appModel.immersiveSpaceID)")
appModel.immersiveSpaceID = "ImmersiveView"
} label: {
Text("Open ImmersiveView")
}
}
}
.onChange(of: scenePhase, { oldValue, newValue in
logger.info("\(#function) \(#line) `onChange(of: scenePhase` scenePhase oldValue \(String(describing: oldValue)) scenePhase newValue \(String(describing: newValue))")
switch newValue {
case .active:
logger.info("\(#function) \(#line) scenePhase newValue \(String(describing: newValue))")
case .background:
logger.info("\(#function) \(#line) scenePhase newValue \(String(describing: newValue))")
unloadScenes()
case .inactive:
logger.info("\(#function) \(#line) scenePhase newValue \(String(describing: newValue))")
@unknown default:
logger.warning("\(#function) \(#line) scenePhase newValue \(String(describing: newValue))")
}
})
}
func updateScene(content: RealityViewContent, attachments: RealityViewAttachments) {
logger.info("\(#function) \(#line)")
guard scenePhase == .active else {
logger.info("\(#function) \(#line) escaping since app backgrounded")
return
}
if rootEntity.children.isEmpty {
Task{
await loadScenes(content: content, attachments: attachments)
}
}
}
func loadScenes(content: RealityViewContent, attachments: RealityViewAttachments) async {
logger.info("\(#function) \(#line)")
guard let closeImmersiveSpaceButton = attachments.entity(for: "OpenImmersiveView") else {
logger.error("\(#function) \(#line) missing attachment")
assertionFailure("missing attachment")
return
}
closeImmersiveSpaceButton.position = [0.0, 0.1, -2.5]
closeImmersiveSpaceButton.scale = scaleForAttachements
rootEntity.addChild(closeImmersiveSpaceButton)
content.add(rootEntity)
}
func unloadScenes() {
logger.info("\(#function) \(#line)")
rootEntity.children.removeAll()
}
}
#Preview(immersionStyle: .mixed) {
ImmersiveView()
.environment(AppModel())
}
//
// ImmersiveView.swift
// TestVideoLeakOneImmersiveView
//
import AVKit
import OSLog
import SwiftUI
import RealityKit
import RealityKitContent
struct ImmersiveView: View {
let logger = Logger(subsystem: SUBSYSTEM, category: "ImmersiveView")
@Environment(\.scenePhase) var scenePhase
@Environment(\.dismissImmersiveSpace) var dismissImmersiveSpace
var videoViewModel: Immersive180VideoViewModel? = {
guard let url = Bundle.main.url(forResource: "a_large_650mb_video_file", withExtension: "mov") else {
assertionFailure("missing video")
return nil
}
return Immersive180VideoViewModel(url: url)
}()
private let rootEntity = Entity()
private let immersiveS3180VideoEntity = Entity()
@State private var isShowImmsersiveVideo = false
var body: some View {
RealityView { content, attachments in
logger.info("\(#function) \(#line) `content, attachments in`")
updateScene(content: content, attachments: attachments)
} update: { content, attachments in
logger.info("\(#function) \(#line) `update: { content, attachments in`")
updateScene(content: content, attachments: attachments)
} placeholder: {
let _ = logger.info("\(#function) \(#line) `placeholder`")
ProgressView()
} attachments: {
let _ = logger.info("\(#function) \(#line) `attachments`")
Attachment(id: "closeImmersiveSpaceButton") {
Button {
// show hide immsersive video
isShowImmsersiveVideo.toggle()
} label: {
Text(isShowImmsersiveVideo ? "Hide Immersive Video" : "Show Immersive Video")
}
}
}
.onChange(of: scenePhase, { oldValue, newValue in
logger.info("\(#function) \(#line) `onChange(of: scenePhase` scenePhase oldValue \(String(describing: oldValue)) scenePhase newValue \(String(describing: newValue))")
switch newValue {
case .active:
logger.info("\(#function) \(#line) scenePhase newValue \(String(describing: newValue))")
case .background:
logger.info("\(#function) \(#line) scenePhase newValue \(String(describing: newValue))")
unloadScenes()
case .inactive:
logger.info("\(#function) \(#line) scenePhase newValue \(String(describing: newValue))")
@unknown default:
logger.warning("\(#function) \(#line) scenePhase newValue \(String(describing: newValue))")
}
})
}
func updateScene(content: RealityViewContent, attachments: RealityViewAttachments) {
logger.info("\(#function) \(#line)")
guard scenePhase == .active else {
logger.info("\(#function) \(#line) escaping since app backgrounded")
return
}
if rootEntity.children.isEmpty {
Task{
await loadScenes(content: content, attachments: attachments)
}
} else {
controlScene(content: content, attachments: attachments)
}
}
func loadScenes(content: RealityViewContent, attachments: RealityViewAttachments) async {
logger.info("\(#function) \(#line)")
guard let closeImmersiveSpaceButton = attachments.entity(for: "closeImmersiveSpaceButton") else {
logger.error("\(#function) \(#line) missing attachment")
assertionFailure("missing attachment")
return
}
closeImmersiveSpaceButton.position = [0.0, 0.1, -2.5]
closeImmersiveSpaceButton.scale = scaleForAttachements
rootEntity.addChild(closeImmersiveSpaceButton)
content.add(rootEntity)
loadS3180Video(content: content)
}
func unloadScenes() {
logger.info("\(#function) \(#line)")
if let videoViewModel {
videoViewModel.reset()
}
immersiveS3180VideoEntity.children.removeAll()
rootEntity.children.removeAll()
Task {
await dismissImmersiveSpace()
}
}
func loadS3180Video(content: RealityViewContent) {
logger.info("\(#function) \(#line)")
guard let videoViewModel else {
logger.error("\(#function) \(#line) missing model")
assertionFailure("missing view model")
return
}
let entity = videoViewModel.setupContentEntity()
immersiveS3180VideoEntity.addChild(entity)
immersiveS3180VideoEntity.isEnabled = false
rootEntity.addChild(immersiveS3180VideoEntity)
}
func hideVideo() {
logger.info("\(#function) \(#line)")
guard let videoViewModel else {
logger.error("\(#function) \(#line) missing model")
assertionFailure("missing model")
return
}
immersiveS3180VideoEntity.isEnabled = false
videoViewModel.stop()
}
func showVideo() {
logger.info("\(#function) \(#line)")
guard let videoViewModel else {
logger.error("\(#function) \(#line) missing model")
assertionFailure("missing model")
return
}
immersiveS3180VideoEntity.isEnabled = true
videoViewModel.play()
}
func controlScene(content: RealityViewContent, attachments: RealityViewAttachments) {
logger.info("\(#function) \(#line)")
if isShowImmsersiveVideo {
showVideo()
} else {
hideVideo()
}
}
}
#Preview(immersionStyle: .mixed) {
ImmersiveView()
.environment(AppModel())
}
//
// Immersive180VideoViewModel.swift
// TestVideoLeakOneImmersiveView
//
import SwiftUI
import AVFoundation
import AVKit
import OSLog
import RealityKitContent
import RealityKit
class Immersive180VideoViewModel {
private let logger = Logger(subsystem: SUBSYSTEM, category: "Immersive180VideoView")
private var url: URL
private let avPlayer = AVPlayer()
init(url: URL) {
logger.info("\(#function) \(#line)")
self.url = url
}
func setupContentEntity() -> Entity {
logger.info("\(#function) \(#line)")
let playerItem = AVPlayerItem(url: url)
avPlayer.replaceCurrentItem(with: playerItem)
let material = VideoMaterial(avPlayer: avPlayer)
let hemisphereMesh = generateHemisphere(radius: 20.0)
let sphere = ModelEntity(mesh: hemisphereMesh)
sphere.model?.materials = [material]
let blackMaterial = SimpleMaterial(color: .black, roughness: .float(1), isMetallic: false)
let outerSphereMesh = MeshResource.generateSphere(radius: 15)
let outerSphere = ModelEntity(mesh: outerSphereMesh)
outerSphere.model?.materials = [blackMaterial]
outerSphere.position = [0, 0, 0]
outerSphere.scale = .init(x: 1E3, y: 1E3, z: 1E3)
let contentEntity = Entity()
contentEntity.scale *= .init(x: -1, y: 1, z: 1)
contentEntity.addChild(outerSphere)
contentEntity.addChild(sphere)
// Display the video in front of the user by rotating the entity 180 degrees around the Y-axis
let rotation = simd_quatf(angle: .pi, axis: SIMD3(x: 0, y: 1, z: 0))
contentEntity.transform.rotation = rotation
NotificationCenter.default.addObserver(
forName: .AVPlayerItemDidPlayToEndTime,
object: avPlayer.currentItem,
queue: .main) { [self] _ in
Task {
avPlayer.seek(to: .zero)
avPlayer.play()
}
}
return contentEntity
}
func generateHemisphere(radius: Float) -> MeshResource {
logger.info("\(#function) \(#line)")
var vertices: [SIMD3] = []
var indices: [UInt32] = []
var uvs: [SIMD2] = []
let segments = 36
let halfSegments = segments / 2
for latitude in 0...halfSegments {
let theta = Float(latitude) * Float.pi / Float(halfSegments)
let sinTheta = sin(theta)
let cosTheta = cos(theta)
for longitude in 0...segments / 2 { // Only cover 180 degrees
let phi = Float(longitude) * Float.pi / Float(segments / 2)
let sinPhi = sin(phi)
let cosPhi = cos(phi)
let x = radius * sinTheta * cosPhi
let y = radius * cosTheta
let z = radius * sinTheta * sinPhi
vertices.append(SIMD3(x, y, z))
uvs.append(SIMD2(1.0 - Float(longitude) / Float(segments / 2), 1.0 - Float(latitude) / Float(halfSegments))) // Flip UVs horizontally
}
}
for latitude in 0.. Bool {
logger.info("\(#function) \(#line)")
let type = "application/vnd.apple.mpegurl"
var request = URLRequest(url: url)
request.httpMethod = "HEAD"
let (_, response) = try await URLSession.shared.data(for: request)
if let httpResponse = response as? HTTPURLResponse {
if let mimeType = httpResponse.allHeaderFields["Content-Type"] as? String {
if mimeType.contains(type) {
logger.info("\(#function) \(#line) mime type \(mimeType) found in url \(url.absoluteString)")
return true
} else {
logger.error("\(#function) \(#line) mime type \(type) not found from url \(url.absoluteString)")
return false
}
} else {
logger.error("\(#function) \(#line) failed cannotParseResponse")
throw URLError(.cannotParseResponse)
}
} else {
logger.error("\(#function) \(#line) failed badServerResponse")
throw URLError(.badServerResponse)
}
}
func reset() {
logger.info("\(#function) \(#line)")
stop()
removeReplayObserver()
avPlayer.replaceCurrentItem(with: nil)
}
func removeReplayObserver() {
logger.info("\(#function) \(#line)")
NotificationCenter.default.removeObserver(self)
}
}
//
// AppModel.swift
// TestVideoLeakOneImmersiveView
//
import SwiftUI
/// Maintains app-wide state
@MainActor
@Observable
class AppModel {
var immersiveSpaceID = "InitialImmersiveView"
enum ImmersiveSpaceState {
case closed
case inTransition
case open
}
var immersiveSpaceState = ImmersiveSpaceState.closed
}
Hey @VaiStardom,
Upon further investigation this behaves correctly. There is a memory leak in your view model where your completion block in addObserver(forName:object:queue:using:) is capturing the view model preventing it from being destroyed. To avoid a retain cycle, use a weak reference to self inside the block when self contains the observer as a strong reference.
In your case this would look like the following:
NotificationCenter.default.addObserver(
forName: .AVPlayerItemDidPlayToEndTime,
object: avPlayer.currentItem,
queue: .main) { [weak self] _ in
guard let self else { return }
Task {
self.avPlayer.seek(to: .zero)
self.avPlayer.play()
}
}
Let me know if you have additional questions,
Michael
Replies
3
Boosts
0
Views
600
Participants
2
