In OpenGL there are several ways to determine which primitive the user has clicked on or selected, see e.g. https://www.opengl.org/archives/resources/faq/technical/selection.htm
What is a simple, idiomatic, efficient way to do this in Metal? Looking for "metal selection" on Google gives results about steel and stuff that aren't very helpful.
I don't believe there is such thing as "Metal selection mechanism". Metal is too low-level for that. You can easily implement one, though. What I'd do is to create dedicated "identification" texture for use as additional render target, say MTLPixelFormatR8Uint for 8 bit ID space or maybe MTLPixelFormatR16Uint for 16 bits. Then I'd modify MTLRenderPassDescriptor used for scene rendering so that "id texture" would be additional color attachment for it. Sensible setup will include something like MTLLoadActionClear, with MTLClearColor being something that is interpreted as "no object", and MTLStoreActionStore. And finally, all fragment shaders that are used to draw object identifying themselves would have to be modified so that they put object's ID value into "id texture" render target.
That way you'd have id's of objects in "id texture". Now for identification of particular object(s) at provided coordinates. DO NOT read "id texture" directly via MTLTexture getBytes, as doing this requires heavy synchronization (you'll basically have to wait for rendering command buffer to finish before reading). This is acceptable for testing and debugging only, unless you don't care for performance.
Proper way to do that is (I think) to make a compute shader with coordinates as input and writing id's as output, preferably to buffer.
What I'd do is this:
1) Preferred solution would be when I don't need object id on CPU at all. Say, you just want to highlight (draw again but using different color perhaps) selected object, or something like this. Then your compute shader could read coordinates and set some flags in buffers (or perhaps generate indirect compute/draw specs), so that subsequent drawing calls would draw selected object(s) differently. This is very nice as it does not require GPU-CPU roundtrip AT ALL.
2) Of course, in most cases you have to get object id to CPU. So your compute shader has to write object id(s) (definitely batch 'em if there are to be more than one per frame, round-trips take time!) to some scratch buffer. To commandBuffer that dispatches compute shader in question you should attach completion handler (MTLCommandBuffer addCompletedHandler). Completion handler should immediately read scratch buffer contents (these are now ready) and then call whatever CPU code that needs to know object ids.
Be careful to use buffer storage model that works best for fast GPU/CPU transfer on your device. AFAIK "shared" would be best on iOS right now, and "managed" works better on iOS. Be also aware of the fact that certain storage modes on certain iOSes require manual synchronization before access can be made, even if you had waited for operations on GPU to finish.
Hope that helps
Michal
Thanks, I'll give it a try, hoping that I understand enough, still rather new to Metal.
Just as a side note, you shouldn't be using GL_SELECTION mode in GL either. It comes from the dark ages of OpenGL, doesn't scale with the hardware and was removed in modern core OpenGL. The scissor + object id rasterization is easy to implement and is fast. Ray-casting without any rasterization might be even faster if you use use appropriate data structures for organizing your geometry.
