External Display Support in IOS App

Body:

Hello,

I am facing a challenging issue with my SwiftUI iOS application, which is designed to work on an iPad and connect to an external display. The app, called "EasyJoin," is intended to provide a single-touch interface for joining conference meetings. It pulls events from a calendar and provides a "Join" button to connect to the meeting.

The Goal:

  • Mirror the app on an external display in its native aspect ratio.
  • Also mirror any other client applications launched from EasyJoin (such as Google Meet, Teams, WebEx, Zoom) to the external display in its native aspect ratio.

The Issue:

While I have been successful in displaying the app on the external display, the iPad screen goes black as soon as the external display is connected. I need both the iPad and the external display to show the app simultaneously, each in their native aspect ratios.

What I've Tried:

  • Created separate UIWindow objects for the internal and external displays.
  • Used NotificationCenter to listen for UIScreen.didConnectNotification and UIScreen.didDisconnectNotification.
  • Tried managing windows through both AppDelegate and SceneDelegate.
  • Explicitly set windowLevel for both internal and external windows.

Despite these efforts, the issue persists. The external display works as expected, but the iPad screen remains black.

Here is a snippet of my SceneDelegate.swift:

// ... (Code for setting up UIWindow and listening for screen connect/disconnect)
@objc func screenDidConnect(notification: Notification) {
    // ... (Code for setting up external UIWindow)
    externalWindow?.isHidden = false
}
@objc func screenDidDisconnect(notification: Notification) {
    externalWindow?.isHidden = true
    externalWindow = nil
}

I would appreciate any guidance or suggestions to resolve this issue. Thank you!


Feel free to copy and paste this into a new post on the Apple Developer Forums. Hopefully, you'll get some specialized assistance that can help resolve the issue.

Replies

First of all, UIScreen.didConnectNotification and UIScreen.didDisconnectNotification are deprecated. They should not be used.

https://developer.apple.com/documentation/uikit/app_and_environment/building_a_desktop-class_ipad_app

Use scenes, instead of UIScreen, to determine which hardware display shows your UI. For information, see Scenes.

Consider the best use of additional space that an external display provides. Support interactive resizable windows through scenes with a windowApplication role, or present noninteractive content with a windowExternalDisplayNonInteractive scene. For more information, see Presenting content on a connected display.

https://developer.apple.com/documentation/uikit/windows_and_screens/presenting_content_on_a_connected_display

So, as mentioned, you should look at connecting scenes. If you are on the embedded display, you will see a windowExternalDisplayNonInteractive offered to you. By default, this scene will mirror the embedded display. If you put content there, it will kick the system out of mirroring. But as indicated by the name, it is not interactive.

The user could also launch your application on an external display with Stage Manager, If they do that, then you will receive a separate, interactive, windowApplication on the external display

Also mirror any other client applications launched from EasyJoin (such as Google Meet, Teams, WebEx, Zoom) to the external display in its native aspect ratio.

You cannot control what other applications do with the shared resource of the external display. This is up to their application.

While I have been successful in displaying the app on the external display, the iPad screen goes black as soon as the external display is connected

It would be helpful to see more of your code and how you are initializing the UIWindow instances.

Thank you for your detailed response. I've tried implementing scenes as you suggested, but I'm still facing issues. Below are the details:

Current Behavior:

  1. The app works fine on the iPad.
  2. When connected to an external display, the external display does not maintain a 16:9 aspect ratio.
  3. The iPad screen sometimes goes black when the external display is connected.

Code Samples:

AppDelegate.swift

@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {
    func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
        return true
    }
}

SceneDelegate.swift

class SceneDelegate: UIResponder, UIWindowSceneDelegate {
    var window: UIWindow?
    func scene(_ scene: UIScene, willConnectTo session: UISceneSession, options connectionOptions: UIScene.ConnectionOptions) {
        guard let windowScene = (scene as? UIWindowScene) else { return }
        let window = UIWindow(windowScene: windowScene)
        window.rootViewController = ViewController()
        self.window = window
        window.makeKeyAndVisible()
        if windowScene.activationState == .foregroundActive {
            windowScene.sizeRestrictions?.maximumSize = CGSize(width: 1920, height: 1080)
            windowScene.sizeRestrictions?.minimumSize = CGSize(width: 1920, height: 1080)
        }
    }
}

ViewController.swift

class ViewController: UIViewController {
    override func viewDidLoad() {
        super.viewDidLoad()
        let label = UILabel()
        label.text = "Hello, EasyJoin!"
        label.textColor = UIColor.black
        label.translatesAutoresizingMaskIntoConstraints = false
        self.view.addSubview(label)
        NSLayoutConstraint.activate([
            label.centerXAnchor.constraint(equalTo: self.view.centerXAnchor),
            label.centerYAnchor.constraint(equalTo: self.view.centerYAnchor)
        ])
    }
}

Answers to Your Questions:

  1. Use of Deprecated Methods: I've removed the use of UIScreen.didConnectNotification and UIScreen.didDisconnectNotification as you suggested.
  2. Initialization of UIWindow Instances: As shown in the code samples above, I'm initializing UIWindow instances in the SceneDelegate.

My Questions:

  1. How can I ensure that the external display shows the app in a 16:9 aspect ratio?
  2. How can I prevent the iPad screen from going black when the external display is connected?
  3. Is there a better approach to handle multiple displays in UIKit?

I would appreciate any further guidance on this issue.

At this point, I'm just really confused as to why IOS doesn't seem to allow me to use different aspect ratio/resolutions for each display. No matter what I do, if the iPad screen is active then the external display is always mirroring the aspect ratio of the iPad, rather than the native aspect ratio of the external disoplay. This shouldn't be difficult and yet it's beginning to seem impossible. Are you able to provide sample code of how to have my iPad display itself on the iPad in the Ipad's native aspect ratio/resolution, while also displaying itself in the native resolution/aspect ratio of the external display at the same time?

I've tried implementing the suggestions, but I'm still facing the same issue: the external display is not showing content in its native 16:9 aspect ratio. The iPad display looks correct, but the external display is inheriting the iPad's aspect ratio.

Here's the updated code based on the suggestions:

SceneDelegate.swift

import UIKit
import SwiftUI

class SceneDelegate: UIResponder, UIWindowSceneDelegate {

    var window: UIWindow?
    var externalWindow: UIWindow?

    func scene(_ scene: UIScene, willConnectTo session: UISceneSession, options connectionOptions: UIScene.ConnectionOptions) {
        
        if let windowScene = scene as? UIWindowScene {
            if windowScene.screen === UIScreen.main {
                let contentView = ContentView()
                let window = UIWindow(windowScene: windowScene)
                window.rootViewController = UIHostingController(rootView: contentView)
                self.window = window
                window.makeKeyAndVisible()
            } else {
                let contentView = ContentView()
                let externalWindow = UIWindow(windowScene: windowScene)
                externalWindow.rootViewController = UIHostingController(rootView: contentView)
                self.externalWindow = externalWindow
                externalWindow.makeKeyAndVisible()
            }
        }
    }

    func scene(_ scene: UIScene, didDisconnectFrom session: UISceneSession, options connectionOptions: UIScene.ConnectionOptions) {
        if let windowScene = scene as? UIWindowScene {
            if windowScene.screen !== UIScreen.main {
                self.externalWindow = nil
            }
        }
    }
}

Info.plist

I've also updated the Info.plist to include configurations for UIWindowSceneSessionRoleExternalDisplayNonInteractive.

I've set the external display to be non-interactive using the windowExternalDisplayNonInteractive role, as suggested. However, the aspect ratio remains incorrect.

Any further guidance would be greatly appreciated.