Our application, https://apps.apple.com/us/app/gamecontroller-mapper/id6737088417 which maps game controller inputs to keyboard/mouse events system-wide, has stopped functioning properly after the macOS 15.4 update. Specifically, the app can no longer capture game controller inputs when running in the background, severely impacting its core functionality. Environment macOS version: 15.4 Previous working versions: All versions prior to 15.4 App type: Background utility with accessibility permissions Hardware: All game controller brands compatible with macOS Detailed Description Before macOS 15.4 Our application correctly captured game controller inputs from any brand connected to Mac and successfully translated them to keyboard/mouse events system-wide. Users could control any application (e.g., scrolling through documents in Preview using controller buttons) while our app ran in the background with the accessibility permissions granted. After macOS 15.4 The application only works when it has active focus (is in the foreground). When any other application gains focus, our app completely stops receiving or detecting any input events from the game controller while running in the background. For instance, pressing the 'down' button on the controller while another app is active results in no event being registered within our application. We've tried updating the app to work in accessory mode (in the menubar), but the issue persists. Steps to Reproduce Install our application on macOS 15.3 or earlier Grant accessibility permissions when prompted Connect a compatible game controller (e.g., Xbox or other controller) Open another application (e.g., Preview with a PDF document) Press buttons on the controller to navigate the document without touching the keyboard Expected result on 15.3: Controller inputs are translated to keyboard events, even when our app is in the background Upgrade to macOS 15.4 Repeat steps 2-5 Actual result on 15.4: Controller inputs are only translated to keyboard events when our application has focus Technical Implementation Our app uses: CGEvent.tapCreate() to create a global event tap CGEvent for simulating keyboard and mouse events GCController.extendedGamepad?.valueChangedHandler for detecting controller inputs Proper NSAccessibilityUsageDescription and appropriate entitlements GCController.shouldMonitorBackgroundEvents = true to ensure controller events continue when the app is inactive Possible Relation to Recent Changes We noticed in the macOS 15.4 Release Notes: Game Controller - Resolved Issues: Fixed: Game controllers might stop responding when accessibility features, such as Voice Over, are enabled. (141497799) We suspect this fix might have introduced a regression or intentional limitation affecting applications like ours that rely on background event simulation with game controller input. Impact This change severely impacts: Applications designed to use game controllers as assistive input devices for users who may have difficulty using traditional keyboard and mouse inputs Applications for media control, presentation navigation, and other similar use cases Users who rely on our application for accessibility purposes Questions Is this an intentional security change or an unintended side effect of the controller fix mentioned in the release notes? Are there any new APIs or alternative approaches we should implement to restore functionality? If this is a system bug, when can we expect a fix? We would greatly appreciate any guidance on how to restore our application's functionality. Thank you for your assistance.

I am developing a vision os app for controlling an underwater ROV. I have ornaments with telemetry and buttons around a central video view feed. I have custom buttons mappings, such as "A" for locking the depth of the drone. However, when I look at buttons or certain ornaments, my custom gamepad logic is kept from running. This means that when a SwiftUI Button gains focus on visionOS, pressing the controller’s A button triggers the system’s default “click” on that Button rather than my custom buttonA handler. Essentially, focus interception by the system is stealing my A-press events and preventing my custom gamepad logic from running. Is there a way to disable the built in gamepad interaction and only allow my custom gamepad mappings?

I have an app that uses GameController to read the inputs of a connected JoyCon. However, the controller also interacts with the OS. For example, when I press the Home button on the controller, it brings me to the home of my Vision Pro. Is it possible to disable this interaction while still being able to read the controller inputs inside my app?

Hello everyone, I want send haptics to ps4 controller. CHHapticPatternPlayer and CHHapticAdvancedPatternPlayer good work with iPhone. On PS4 controller If I use CHHapticPatternPlayer all work good, but if I use CHHapticAdvancedPatternPlayer I get error. I want use CHHapticAdvancedPatternPlayer to use additional settings. I don't found any information how to fix it - CHHapticEngine.mm:624 -[CHHapticEngine finishInit:]_block_invoke: ERROR: Server connection broke with error 'Не удалось завершить операцию. (com.apple.CoreHaptics, ошибка -4811)' The engine stopped because a system error occurred. AVHapticClient.mm:1228 -[AVHapticClient getSyncDelegateForMethod:errorHandler:]_block_invoke: ERROR: Sync XPC call for 'loadAndPrepareHapticSequenceFromEvents:reply:' (client ID 0x21) failed: Не удалось установить связь с приложением-помощником. Не удалось создать или воспроизвести паттерн: Error Domain=NSCocoaErrorDomain Code=4097 "connection to service with pid 5087 named com.apple.GameController.gamecontrollerd.haptics" UserInfo={NSDebugDescription=connection to service with pid 5087 named com.apple.GameController.gamecontrollerd.haptics} My Haptic class - import Foundation import CoreHaptics import GameController protocol HapticsControllerDelegate: AnyObject { func didConnectController() func didDisconnectController() func enginePlayerStart(value: Bool) } final class HapticsControllerManager { static let shared = HapticsControllerManager() private var isSetup = false private var hapticEngine: CHHapticEngine? private var hapticPlayer: CHHapticAdvancedPatternPlayer? weak var delegate: HapticsControllerDelegate? { didSet { if delegate != nil { startObserving() } } } deinit { NotificationCenter.default.removeObserver(self) } private func startObserving() { guard !isSetup else { return } NotificationCenter.default.addObserver( self, selector: #selector(controllerDidConnect), name: .GCControllerDidConnect, object: nil ) NotificationCenter.default.addObserver( self, selector: #selector(controllerDidDisconnect), name: .GCControllerDidDisconnect, object: nil ) isSetup = true } @objc private func controllerDidConnect(notification: Notification) { delegate?.didConnectController() self.createAndStartHapticEngine() } @objc private func controllerDidDisconnect(notification: Notification) { delegate?.didDisconnectController() hapticEngine = nil hapticPlayer = nil } private func createAndStartHapticEngine() { guard let controller = GCController.controllers().first else { print("No controller connected") return } guard controller.haptics != nil else { print("Haptics not supported on this controller") return } hapticEngine = createEngine(for: controller, locality: .default) hapticEngine?.playsHapticsOnly = true do { try hapticEngine?.start() } catch { print("Не удалось запустить движок тактильной обратной связи: \(error)") } } private func createEngine(for controller: GCController, locality: GCHapticsLocality) -> CHHapticEngine? { guard let engine = controller.haptics?.createEngine(withLocality: locality) else { print("Failed to create engine.") return nil } print("Successfully created engine.") engine.stoppedHandler = { reason in print("The engine stopped because \(reason.message)") } engine.resetHandler = { print("The engine reset --> Restarting now!") do { try engine.start() } catch { print("Failed to restart the engine: \(error)") } } return engine } func startHapticFeedback(haptics: [CHHapticEvent]) { do { let pattern = try CHHapticPattern(events: haptics, parameters: []) hapticPlayer = try hapticEngine?.makeAdvancedPlayer(with: pattern) hapticPlayer?.loopEnabled = true try hapticPlayer?.start(atTime: 0) self.delegate?.enginePlayerStart(value: true) } catch { self.delegate?.enginePlayerStart(value: false) print("Не удалось создать или воспроизвести паттерн: \(error)") } } func stopHapticFeedback() { do { try hapticPlayer?.stop(atTime: 0) self.delegate?.enginePlayerStart(value: false) } catch { self.delegate?.enginePlayerStart(value: true) print("Не удалось остановить воспроизведение вибрации: \(error)") } } } extension CHHapticEngine.StoppedReason { var message: String { switch self { case .audioSessionInterrupt: return "the audio session was interrupted." case .applicationSuspended: return "the application was suspended." case .idleTimeout: return "an idle timeout occurred." case .systemError: return "a system error occurred." case .notifyWhenFinished: return "playback finished." case .engineDestroyed: return "the engine was destroyed." case .gameControllerDisconnect: return "the game controller disconnected." @unknown default: return "an unknown error occurred." } } } custom haptic events - static func changeVibrationPower(power: HapricPower) -> [CHHapticEvent] { let continuousEvent = CHHapticEvent(eventType: .hapticContinuous, parameters: [ CHHapticEventParameter(parameterID: .hapticSharpness, value: 1.0), CHHapticEventParameter(parameterID: .hapticIntensity, value: power.value) ], relativeTime: 0, duration: 0.5) return [continuousEvent] }

After many former OS and Xcode updates, my Game Controller Swift code generates a "DIS-CONNECTED" MESSAGE. Mac Sequoia 15.2 Xcode 16.2 Tried to update PlayStation controller firmware on my Mac. Still no luck with Xcode and its use of a game controller with tvOS.

I'm trying to add support to PS5 DualSense controller. when I try to use the API from here: https://developer.apple.com/documentation/gamecontroller/gcdualsenseadaptivetrigger?language=objc None of the API works, am I missed anything? The code is like this: if ( [ controller.extendedGamepad isKindOfClass:[ GCDualSenseGamepad class ] ] ) { GCDualSenseGamepad * dualSenseGamePad = ( GCDualSenseGamepad * )controller.extendedGamepad; auto funcSetEffectTrigger = []( TriggerEffectParams& params, GCDualSenseAdaptiveTrigger *trigger ) { if ( params.m_mode == TriggerEffectMode::Off ) { [ trigger setModeOff ]; NSLog(@"setModeOff trigger.mode:%d", trigger.mode ); } else if ( params.m_mode == TriggerEffectMode::Feedback ) { [ trigger setModeFeedbackWithStartPosition: 0.2f resistiveStrength: 0.5f ]; } else if ( params.m_mode == TriggerEffectMode::Weapon ) { [ trigger setModeWeaponWithStartPosition: 0.2f endPosition: 0.4f resistiveStrength: 0.5f ]; } else if ( params.m_mode == TriggerEffectMode::Vibration ) { [ trigger setModeVibrationWithStartPosition: position amplitude: amplitude frequency: frequency ]; } }; if ( L2 ) { funcSetEffectTrigger( params, dualSenseGamePad.leftTrigger ); } if ( R2 ) { funcSetEffectTrigger( params, dualSenseGamePad.rightTrigger ); } } I've also tested to add "Game Controllers" capability to Target, still not working. Can't find anything else from the document or forums. I've no idea what need to do.

When turning VoiceOver ON, GCController does not send button press events for "Button A" and "Button Center". This happens when using Siri 2nd generation remote (with dedicated arrow buttons on the circle around center button) and also when using iOS remote. I didn't test it on old Siri 1st generation with touchpad without arrow buttons. Example: gameController.microGamepad?.allButtons.forEach { button in button.valueChangedHandler = { [weak self] _, _, _ in self?.buttonHandler(gameController: gameController, button: button) } private func buttonHandler(gameController: GCController, button: GCControllerButtonInput) { print("BUTTON: Pressed \(button.description) isPressed=\(button.isPressed) isTouched=\(button.isTouched)") } #endif VoiceOver ON (incorrect behavior): BUTTON: Pressed Direction Pad Left (value: 0.030, pressed: 1) isPressed=true isTouched=true BUTTON: Pressed Direction Pad Down (value: 0.079, pressed: 1) isPressed=true isTouched=true BUTTON: Pressed Direction Pad Left (value: 0.000, pressed: 0) isPressed=false isTouched=false BUTTON: Pressed Direction Pad Down (value: 0.000, pressed: 0) isPressed=false isTouched=false VoiceOver OFF (correct behavior): BUTTON: Pressed Direction Pad Left (value: 0.137, pressed: 1) isPressed=true isTouched=true BUTTON: Pressed Direction Pad Up (value: 0.078, pressed: 1) isPressed=true isTouched=true BUTTON: Pressed Button A (value: 1.000, pressed: 1) isPressed=true isTouched=true BUTTON: Pressed Button Center (value: 1.000, pressed: 1) isPressed=true isTouched=true BUTTON: Pressed Button A (value: 0.000, pressed: 0) isPressed=false isTouched=false BUTTON: Pressed Button Center (value: 0.000, pressed: 0) isPressed=false isTouched=false BUTTON: Pressed Direction Pad Left (value: 0.000, pressed: 0) isPressed=false isTouched=false BUTTON: Pressed Direction Pad Up (value: 0.000, pressed: 0) isPressed=false isTouched=false I could use for detection Direction Pad Left/Right/Up/Down and detect position between -0.7 and +0.7 and handle it as center button press, because I use that on old Siri remote where I need to distinguish center button and arrows (for switching TV channels by Up/Down and Skip forward/back by Left/Right arrows), but for new Siri remote it would be unnecessary workaround. Does anybody know why the center/select button is not detected when VoiceOver is ON. Is there another way of detecting it using GCController? I don't want to use SwiftUI onTapGesture for this one particular case. Is it an unexpected bug in tvOS APIs or is there some specific reason why center button is not handled by GCController when VoiceOver is ON? Thanks.

I am unable to get VisionOS 2.0 (simulator) to receive the GCControllerDidConnect notification and thus am unable to setup support for a gamepad. However, it works in VisionOS 1.2. For VisionOS 2.0 I've tried adding: .handlesGameControllerEvents(matching: .gamepad) attribute to the view Supports Controller User Interaction to Info.plist Supported game controller types -> Extended Gamepad to Info.plist ...but the notification still doesn't fire. It does when the code is run from VisionOS 1.2 simulator, both of which have the Send Game Controller To Device option enabled. Here is the example code. It's based on the Xcode project template. The only files updated were ImmersiveView.swift and Info.plist, as detailed above: import SwiftUI import GameController import RealityKit import RealityKitContent struct ImmersiveView: View { var body: some View { RealityView { content in // Add the initial RealityKit content if let immersiveContentEntity = try? await Entity(named: "Immersive", in: realityKitContentBundle) { content.add(immersiveContentEntity) } NotificationCenter.default.addObserver( forName: NSNotification.Name.GCControllerDidConnect, object: nil, queue: nil) { _ in print("Handling GCControllerDidConnect notification") } } .modify { if #available(visionOS 2.0, *) { $0.handlesGameControllerEvents(matching: .gamepad) } else { $0 } } } } extension View { func modify<T: View>(@ViewBuilder _ modifier: (Self) -> T) -> some View { return modifier(self) } }

Hi, I’m creating a game and I’m just wondering if I can integrate GCVirtualController in my SwiftUI app.

Hi community. I am trying to adopt my first person shooter iOS game for running on MacOS environment. I need to lock the pointer when I enter battle mode, and unlock in lobby. On iOS all works fine (with mouse and keyboard) - pointer locks and unlocks according to my commands. However, on MacOS I faced the following behavior: after switching the pointer lock state and setNeedsUpdateOfPrefersPointerLocked invocation, the pointer does not locked immediately. To enable pointer lock, the user must click in the window. I checked the criteria listed in documentation: I do have fullscreen mode, I monitor UISceneActivationState and can confirm it is UISceneActivationStateForegroundActive, I do not use MacCatalyst (it is disabled in app's capabilities). However pointer locks only after click on window, which is weird. Can someone confirm that this is the exact behaviour as designed by Apple developers, or am I doing anything wrong. I have read the note: "Bringing an app built with Mac Catalyst to the foreground doesn’t immediately enable pointer lock. To enable pointer lock, the user must click in the window. To exit pointer lock, users can use Command-tab to switch to another app, or using Command-tilde.", but again, I don't use MacCatalyst. Any hints are highly appreciated! Best regards. refs: https://developer.apple.com/documentation/apple-silicon/running-your-ios-apps-in-macos https://developer.apple.com/documentation/uikit/uiviewcontroller/3601235-preferspointerlocked?language=objc

Hello! Currently watching the Envision the Future: Build great apps for visionOS" webinar, and lots of questions coming up. Thx for offering this online! For those of us with "VR legs", how can we go about setting up custom hand/finger gestures that would enable us to add the functionality for teleporting and navigating within our fully Immersive environments? Both smooth, and snap turn/teleport options would be great, thx! This is adjacent to my previous question on how to setup a PS5 controller to do something similar. Think Half-Life: ALYX as the gold standard for VR navigation.

I recently completed a freelance project where I was tasked with creating room-scale environments that could be used as AR elements. As a bonus, I suggested that these could be done to scale, and repurposed for eventual viewing in Vision Pro. To illustrate, I was able to quickly create a simple Immersive project in Xcode, add the USDZ file (authored in Maya, with baked lighting from Arnold) to Reality Composer Pro, and compile for quick sending to headset. I then would do screen recordings inside the immersive space, which the client loved to see. However, I am unable to walk around due to the boundary limitations. My next obvious thought is, how can I setup the “player” camera so that I can control with a PS5 controller inside AVP? In addition to Maya, I’m an Unreal Engine artist, and have been waiting patiently to get any projects compiled for AVP. With 5.5 release, I was able to get a VR Template test over to AVP, where I have rudimentary navigation control via the PS5 controller. Ideally, I’d also love to learn how to set this up natively, so I can take simple USDZ scenes created in Maya, import to RCP, setup a simple camera controller, and then be able to use this to navigate my VR Immersive spaces on Vision Pro. How can we go about doing this? Part two of this question/suggestion is, how would I go about controlling a rigged, animated character in AR/passthrough mode in a similar fashion? Thx!

Hi, I'm building a windowed game in visionOS which requires a gamepad. And I'm using a PS5 controller for this case. However, I found a few problems: When looking at the window close button, and press X in the gamepad, the window will be closed. This can happen accidentally when the user is playing. When press the PS button(the home button), I want my app to handle it, e.g. take the user to the title screen. But visionOS will always capture this input and opens the home screen(App Library). How to avoid these 2 issues? Or in general, how to make the gamepad input only available to my app but not the visionOS system?

Phone glitchs a lot more my flashlight won’t turn on half the time even when it says it’s on its not on settings page likes to freeze randomly will restart my phone as well and controller has more issues connecting on certain games

The specification states that the timestamp property is a DOMHighResTimeStamp which always represents a value in milliseconds. However, in my testing (on iOS as I don't have a mac) the timestamp is given as seconds. This could break compatibility if an app/library is not tested in Safari. Worse is that the deviation seems to be undocumented as every compatibility chart states full compatibility and defines the property as milliseconds. Any workaround would also be up to one frame incorrect. So a stutter can affect input accuracy. I know this might seem small but it could mean that an app or game is unplayable in Safari if it relies on this for input.

I have a simple example of a motion matching (MxM for Unity) character controller that uses Unity's input system and gamepad support. In editor the scene and inputs work as expected. When I build to headset the app stops at an initialization step where my game controller should kick in. The app doesn't crash but my character is frozen in A-Pose and doesn't respond to input. I'm wondering if this error I'm seeing in the logs is what's causing it? And if so how do I fix it? error 15:56:11.724200-0700 PolySpatialProjectTemplate NSBundle file:///System/Library/Frameworks/GameController.framework/ principal class is nil because all fallbacks have failed I'm using Xcode 16 beta 6 Unity 6000.0.17f1 VisionOS 2.0 beta 9

The question is: We did not add "GameController framework", but we do not know why the "GameController framework" is loaded at startup. I am checking the launch time of our app, using Instruments->App launch I was confused to find the GameConroller framework loaded I check the project, in the plist file, no configuration GCSupportedGameControllers, GCSupportsControllerUserInteraction related key. What else causes the GameController framework to load?

Now I am using iOS 16 beta 6, I can get buttonA/buttonB is pressed event. but cannot get accelerator motion data info.      //work     buttonA?.valueChangedHandler = {(_ button: GCControllerButtonInput, _ value: Float, _ pressed: Bool) -> Void in       print(">>> ButtonA tapped")     }     //work     buttonB?.valueChangedHandler = {(_ button: GCControllerButtonInput, _ value: Float, _ pressed: Bool) -> Void in       print(">>> ButtonB tapped")     }           // this is not works     gameController.motion?.valueChangedHandler = { (motion: GCMotion)->() in       print(">>>> motion data \(motion.acceleration.x) \(motion.acceleration.y) \(motion.acceleration.z)")       if let delegate = self.motionDelegate {         delegate.motionUpdate(motion: motion)       }     } Is there any plan to support this?

struct GameSystem: System { static let rootQuery = EntityQuery(where: .has(GameMoveComponent.self) ) init(scene: RealityKit.Scene) { } func update(context: SceneUpdateContext) { let root = context.scene.performQuery(Self.rootQuery) for entity in root{ let game = entity.components[GameMoveComponent.self]! if let xMove = game.game.gc?.extendedGamepad?.dpad.xAxis.value , let yMove = game.game.gc?.extendedGamepad?.dpad.yAxis.value { print("x:\(xMove),y:\(yMove)") let x = entity.transform.translation.x + xMove * 0.01 let y = entity.transform.translation.z - yMove * 0.01 entity.transform.translation = [x , entity.transform.translation.y , y] } } } } I want to use the game controller's direction keys to control the continuous movement of Entity in visionOS. When I added a query for handle button presses in the ECS System, I found that the update interface was not called at a frequency of 30 frames per second. Instead, it executes once when I press or release the key. Is this what is the reason? I want to keep moving by holding down the controller button, is there a better solution? I hope this moving process will be smooth and not stuck.

Hello, I have an iPad app that users are running on their M1 / M2 MacBooks thanks to the "Designed for iPad" feature. Some of them reported to me that the pressed keyboard keys were not recognized. According to my source code, it seems that my custom views (that are set as firstResponder) do not get pressesBegan events. While I could not reproduce this specific problem on my M1 Macbook, I found a strange problem that may be related. My view also supports interaction with game controllers. I found that the emulated controller (which is using the keyboard, when the controller emulation feature of the Designed for iPad app is set to On) has some problems. The valueChangedHandler of the GCExtendedGamepad from the GCGameController is only fired if the app is compiled with the "Requires full screen" option set to On. If Requires full screen is unchecked in XCode, the Emulated Controller is still present in the list of game controllers, but no button callbacks are triggered. Note that a real game controller connected via USB will work correctly no matter how Requires full screen is set. Could there be a focus bug of the keyboard not sending events when the app is not a full screen app (resizable on mac, and multitask on iPad)? Or is there something I can change to avoid this problem? I'm lost.