GameKit Access Point causes camera background image in ARKit to be black in iOS 18 beta only

Question

Created Aug ’24

Replies 3

Boosts 1

Participants 3

I have an AR game using ARKit with SceneKit that works just fine in iOS 17. In the iOS 18 betas, the AR background image shows black instead of showing the real world. As a result there's no tracking and obviously the whole game is useless.
I narrowed down the issue to showing the Game Center Access Point.

My app has ViewController 1 (VC1) showing the main menu and that's where I want to show the GC Access Point. From there you open VC2 which shows a list of levels. Selecting any level will open VC3 which has the ARScene.

Following is the code I use to start Game Center in VC1:

GKLocalPlayer.local.authenticateHandler = { gcAuthVC, error in
            let isGameCenterReady = (gcAuthVC == nil) && (error == nil)
            
            if let viewController = gcAuthVC {
                self.present (viewController, animated: true, completion: nil)
            }
            
            if error != nil {
                print(error?.localizedDescription ?? "")
            }
            
            if isGameCenterReady {
                GKAccessPoint.shared.location = .topLeading
                GKAccessPoint.shared.showHighlights = true
                GKAccessPoint.shared.isActive = true
            }
        }

When switching to VC2 I run GKAccessPoint.shared.isActive = false so that the Access Point will no longer show in any of the following VCs. I tried running it in VC1, VC2, and again in VC3 - it doesn't change anything. Once I reach VC3, the background is black.

If in VC1 I don't run GKAccessPoint.shared.isActive = true, so I don't activate the access point, the behavior is as follows:

If I wait until after the Game Center login animation completes and closes on its own and then I proceed to VC2 and VC3, the camera image will show correctly
If I quickly move to VC2 before the Game Center login animation has completed, so my code will close it by setting active = false, and then I continue to VC3, I will see the black background problem.

So it does look like activating the access point and then de-activating it causes the issue. BTW, if I activate the access point and leave it on in all VCs, the same black background issue persists.

Other than that, when I'm in VC3 with the black background and I switch to another app (so my game moves to the background), when it returns to the foreground, the camera suddenly shows the real world correctly!
I tried to manually reset the AR session by pausing and restarting it, but that didn't change anything. Also, when I check with the debugger, it looks like when the app comes back to the foreground it also doesn't run the session start code.
But something does seem to reset itself so I wonder what that is. Maybe I could trigger the same manually in my cdoe???

I repeat that everything works just fine in iOS 17 and below. This problem only started with the iOS 18 beta (currently on beta 5, but it started in some of the previous betas as well).
So could this be a bug in iOS 18?

As a workaround I could check the iOS version and if it's iOS18 not activate the access point, hoping that the user will not jump to VC2 too quickly, and show my own button which will open Game Center. But I'd rather give the users the full experience with their own avatar and the highlights showing up. Plus, certainly some users will move quickly to VC2 and that will be an awful experience.

Any help would be greatly appreciated. Thanks!

Boost

Answer 1

Frameworks Engineer OP

Apple

Aug ’24

Please reproduce the issue, then file an issue on feedbackassistant.apple.com with an attached sysdiagnose. That will help us investigate the issue. If you provide the FB number here, we can be sure to follow-up.

0

Answer 2

rafisaar OP

Aug ’24

Thanks! I reproduced the issue and filed FB14697134

In the meantime I realized that having app going to background then back to the foreground causes Game Center to re-login on its own so probably the ARSCNView gets to work properly and then Game Center kicks in and that's why it's working in this case.
To test this out, I moved the whole GameKit user authentication code from VC1 to VC3 and that way it also works fine (but I obviously don't want the Game Center graphics in the middle of my gameplay).

So this issue only happens if the access point is activated before the ARSCNView is activated.

0

Answer 3

dylanabsil OP

2w

Hello,

I am experiencing the same issue. I have a game that uses the camera, and when the user plays quickly while the Game Center "Hello" animation is active, the camera won't start or will freeze. It only resumes functioning once the Game Center animation has completed.

Below is a straightforward sample code to reproduce and test the issue:

//
//  ViewController.swift
//  ShowCameraIssue
//
//  Created by Dylan Absil on 02/12/2024.
//

import UIKit
import AVFoundation
import GameKit

class ViewController: UIViewController {
    private var captureSession: AVCaptureSession!
    private var previewLayer: AVCaptureVideoPreviewLayer!
    
    override func viewDidLoad() {
        super.viewDidLoad()
        handleCameraAuthorization()
    }
    
    func beginTest() {
        self.setupCameraPreview()
        
        DispatchQueue.global(qos: .userInitiated).async {
            self.captureSession.startRunning()
        }
        DispatchQueue.main.async {
            self.gameCenterLogin()
        }
    }
    
    private func handleCameraAuthorization() {
        switch AVCaptureDevice.authorizationStatus(for: .video) {
        case .authorized:
            beginTest()
        case .notDetermined:
            AVCaptureDevice.requestAccess(for: .video) { granted in
                if granted {
                    self.beginTest()
                }
            }
        default:
            print("Camera access denied or restricted.")
        }
    }
    
    override func viewDidLayoutSubviews() {
        super.viewDidLayoutSubviews()
        previewLayer.frame = self.view.bounds
    }
}

extension ViewController { // CAMERA
    private func setupCameraPreview() {
        captureSession = AVCaptureSession()
        captureSession.sessionPreset = .high
        
        guard let frontCamera = AVCaptureDevice.default(.builtInWideAngleCamera,
                                                        for: .video,
                                                        position: .front) else {
            return
        }
        
        do {
            let input = try AVCaptureDeviceInput(device: frontCamera)
            if captureSession.canAddInput(input) {
                captureSession.addInput(input)
            }
        } catch {
            return
        }
        
        previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
        previewLayer.videoGravity = .resizeAspectFill
        previewLayer.frame = self.view.bounds
        
        self.view.layer.addSublayer(previewLayer)
        
    }
}

extension ViewController { // GAME CENTER
    func gameCenterLogin() {
        GKLocalPlayer.local.authenticateHandler = { viewController, error in
            if let err = error {
                print(err)
            }
            else if let controller = viewController {
                self.present(controller, animated: true, completion: nil)
            }
        }
    }
}

0