Input with Metal-CPP, overriding MTK::View

Hi robinleman,

There are two approaches you can take to accomplish this.

1. In a typical UI application, you would have a view controller or a swift UI view that would be able to capture the keyboard and mouse input via subclassing, and then forward that to the rest of your application components.

2. Another approach is to use the GameController framework, which allows you to install a callback for mouse, keyboard, and even gamepad events.

Given these two options, the option more in line with the "Learn Metal with C++" sample code I would recommend is to use the GameController framework. This will prevent you from needing to subclass MTK::View.

As you mention, you will have to leverage Objective-C in this case. If you prefer to keep your C++ and Objective-C code separate and modular, you can create a separate Objective-C file that installs the callbacks for the input events, and have these call into a C++ lambda. You could even have the Objective-C code expose a C or C++ interface to the rest of your program. This approach would allow you to seamlessly register for and handle events from the C++ side.

Although this doesn't answer the question since it asks "Hopefully without having an Objective-C bridge", I would like to put here as a reference answer to those which only cares about things getting done:

For getting the input to work while working with a MTKView and using the Metal/Game template from Xcode you can:

1. Create a new Objective-C file, (I named it InputView.m)
2. Make it extend UIView or NSView
3. On GameViewController, start it with [[InputView alloc] initWithFrame:mtkView.frame]
4. Call [mtkView addSubView:myInputView]
5. Call [inputView becomeFirstResponder]
6. In your InputView, implement - (BOOL) acceptsFirstResponder{ return YES;}
7. Now you can check on the input related methods, such as - (void)mouseDown:(NSEvent*)event.
8. If you need to check the mouse movement, you'll need to add a NSTrackingArea by calling [self addTrackingArea:myTrackingArea]

Hey Hipreme (or anyone else), on the chance that you see this. I'm trying to get this working in the default metal kit 3D rendering sample. My GameViewController looks like this:

#import "GameViewController.h"
#import "Renderer.h"

@interface InputView : NSView

@end

@implementation InputView

- (BOOL)acceptsFirstResponder{ return YES; }

- (void)mouseDown:(NSEvent*)event
{
    printf("mouse down");
}

- (void)keyDown:(NSEvent *)event
{
    printf("key down");
}

@end

@implementation GameViewController
{
    MTKView *_view;
    
    InputView *_input_view;

    Renderer *_renderer;
}

- (void)viewDidLoad
{
    [super viewDidLoad];

    _view = (MTKView *)self.view;

    _view.device = MTLCreateSystemDefaultDevice();

    if(!_view.device)
    {
        NSLog(@"Metal is not supported on this device");
        self.view = [[NSView alloc] initWithFrame:self.view.frame];
        return;
    }

    _renderer = [[Renderer alloc] initWithMetalKitView:_view];

    [_renderer mtkView:_view drawableSizeWillChange:_view.bounds.size];

    _view.delegate = _renderer;
    
    _input_view = [[InputView alloc] initWithFrame:_view.frame];
    
    BOOL fr = [_input_view becomeFirstResponder];
    printf("fr %d\n", fr);
}

@end

becomeFirstResponder is returning true, but I'm not getting and logging from mouseDown or keyDown. I've tried a few other configurations of this same approach with similar results. I'm brand new to obj-c and Apple dev in general. Any suggestions?

@Hipreme @Graphics and Games Engineer @m0zrat

I have some actual working code -- and wanted to share. I extended MTK::View with some Objective-c code (objcpp), and wrote an adapter.

@Graphics and Games Engineer -- I first tried to use NSNotificationCenter to get a Notification of a GCKeyboard connection, and by assigning an objective-c block to keyChangedHandler; for whatever the reason, I could not get the notifcation center to propagate the NSNotification properly, so I gave up on this.

@m0zrat Then, I read some documentation on basic event handling that I would recommend anyone who comes across this thread to read:

basic event handling in objective-c

key events in objective-c

Finally, working code here -->

First define a .hpp header, call it something like ViewAdapter.hpp

namespace Explorer {
class ViewAdapter {
public:
  virtual MTK::View* get(CGRect frame);
  virtual void printDebug();
};
}; // namespace Explorer

Then, declare an objective-c interface for your objcpp code.

/**
 * 1) ViewAdapter -> ViewExtender (C++ -> Obj-C)
 * 2) ViewAdapter -> returns extension of MTK::View. (Obj-C -> C++)
 **/

@interface ViewExtender : MTKView {
}
+ (void)load:(CGRect)frame;
+ (ViewExtender *)get;
- (void)printDebug;
@end

Now it's time to implement both in the same file. You can mix objective-c code with c++ code by using the .mm extension. So your file should be called something like ViewExtender.mm

ViewExtender *adapter;

MTK::View *Explorer::ViewAdapter::get(CGRect frame) {
  [ViewExtender load: frame];
  return (__bridge MTK::View *)[ViewExtender get];
}

void Explorer::ViewAdapter::printDebug() {
  ViewExtender *ref = [ViewExtender get];
  [ref printDebug];
}

@implementation ViewExtender

+ (void)load:(CGRect)frame {
  NSLog(@"Loading Objective-c ViewAdapter ...");
  NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
  adapter = [[self alloc] initWithFrame:frame];
  [adapter init];
  [pool release];
}

+ (ViewExtender *)get {
  return adapter;
}

- (id)init {
  BOOL isFirstResponder = [self becomeFirstResponder];
  NSLog(@"Is first responder: %@", isFirstResponder ? @"Yes" : @"No");
  return self;
}

// Needs to be overwritten to accept keyboard events.
// Learn more here:
// https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/EventOverview/EventHandlingBasics/EventHandlingBasics.html#//apple_ref/doc/uid/10000060i-CH5
- (BOOL)acceptsFirstResponder {
  return YES;
}

- (void)keyDown:(NSEvent *)event {
  NSLog(@"Detected a key down event.");
  [event type];
}

- (void)printDebug {
  NSLog(@"ViewAdapter debug info ...");
}


@end

This on its own will return to you a MTK::View* object, you still need to set the device by using MTK::View::setDevice(...). But now once your view is open, you should see once you type, a logging message should appear: 'Detected a key down event.' Let me know if this works for you!