User Profile

I must recant my last comment. I was successful in retrieving custom properties on both Device and Stream using the following code: const CMIOExtensionProperty CMIOExtensionPropertyCustomPropertyData_just = @"4cc_just_glob_0000"; const CMIOExtensionProperty CMIOExtensionPropertyCustomPropertyData_dust = @"4cc_dust_glob_0000"; const CMIOExtensionProperty CMIOExtensionPropertyCustomPropertyData_rust = @"4cc_rust_glob_0000"; + (NSMutableDictionary*)sampleCustomPropertiesWithPropertySet:(NSSet<CMIOExtensionProperty>*)properties { NSMutableDictionary* dictionary = [NSMutableDictionary dictionary]; if ([properties containsObject:CMIOExtensionPropertyCustomPropertyData_just]) { CMIOExtensionPropertyState* propertyState = [CMIOExtensionPropertyState propertyStateWithValue:@"Property value for 'just'"]; [dictionary setValue:propertyState forKey:CMIOExtensionPropertyCustomPropertyData_just]; } if ([properties containsObject:CMIOExtensionPropertyCustomPropertyData_dust]) { const size_t sData_length = 12; static const unsigned char sData[sData_length] = { 0xFE,0xED,0xFE,0xED, 0xFE,0xED,0xFE,0xED, 0xFE,0xED,0xFE,0xED }; CMIOExtensionPropertyState* propertyState = [CMIOExtensionPropertyState propertyStateWithValue:[NSData dataWithBytes:sData length:sData_length]]; [dictionary setValue:propertyState forKey:CMIOExtensionPropertyCustomPropertyData_dust]; } if ([properties containsObject:CMIOExtensionPropertyCustomPropertyData_rust]) { NSString* propertyValue = [NSString stringWithFormat:@"Custom property value for property '%@'.", CMIOExtensionPropertyCustomPropertyData_rust ]; CMIOExtensionPropertyState* propertyState = [CMIOExtensionPropertyState propertyStateWithValue:propertyValue]; [dictionary setValue:propertyState forKey:CMIOExtensionPropertyCustomPropertyData_rust]; } return dictionary; } Then for CMIOExtensionDeviceSource, I passed the custom property values as a dictionary to the initializer: - (NSSet<CMIOExtensionProperty> *)availableProperties {     return [NSSet setWithObjects: CMIOExtensionPropertyDeviceTransportType, CMIOExtensionPropertyDeviceModel, CMIOExtensionPropertyCustomPropertyData_just, CMIOExtensionPropertyCustomPropertyData_dust, CMIOExtensionPropertyCustomPropertyData_rust, nil]; } - (nullable CMIOExtensionDeviceProperties *)devicePropertiesForProperties:(NSSet<CMIOExtensionProperty> *)properties error:(NSError * _Nullable *)outError { NSMutableDictionary* dictionary = [self sampleCustomPropertiesWithPropertySet:properties]; CMIOExtensionDeviceProperties *deviceProperties = [CMIOExtensionDeviceProperties devicePropertiesWithDictionary:dictionary]; And for CMIOExtensionStreamSource: - (NSSet<CMIOExtensionProperty> *)availableProperties {     return [NSSet setWithObjects: CMIOExtensionPropertyStreamActiveFormatIndex, CMIOExtensionPropertyStreamFrameDuration, CMIOExtensionPropertyCustomPropertyData_just, CMIOExtensionPropertyCustomPropertyData_dust, CMIOExtensionPropertyCustomPropertyData_rust, nil]; } - (nullable CMIOExtensionStreamProperties *)streamPropertiesForProperties:(NSSet<CMIOExtensionProperty> *)properties error:(NSError * _Nullable *)outError { NSMutableDictionary* dictionary = [self sampleCustomPropertiesWithPropertySet:properties]; CMIOExtensionStreamProperties* streamProperties = [CMIOExtensionStreamProperties streamPropertiesWithDictionary:dictionary]; This error message below is not related to the actual format of the key, the error is regarding the format of the propertyState (ie. the data associated with the key). CMIO_DAL_CMIOExtension_Stream.mm:1165:GetPropertyData 50 wrong 4cc format for key 4cc_cust_glob_0000 I haven't tried on the Ventura beta but on 12.4 I am in the practice of rebooting after every update to my extension. If you're having problems it might be due to the fact that an old instance of your extension is being used by the system.

I have determined experimentally that under Objective-C (at least), the format of the key for a custom property is as follows: CMIOExtensionProperty myCustomPropertyKey = @"cust_glob_0000"; The fcc_ prefix is either Swift-specific or superfluous. I'm on 12.3. Still no data returned, the value I get is always 0.

When I attempt to access my custom property (on a stream), I get this error: CMIO_DAL_CMIOExtension_Stream.mm:1165:GetPropertyData 50 wrong 4cc format for key 4cc_cust_glob_0000 CMIO_DAL_CMIOExtension_Stream.mm:1171:GetPropertyData unknown property error cust CMIOHardware.cpp:328:CMIOObjectGetPropertyData Error: 2003332927, failed This message is only triggered if I attempt to access the property from the client side. However, the os_log output shows the correct value for the property. Error code 2003332927 is FOUR_CHAR_CODE('who?') which maps to kCMIOHardwareUnknownPropertyError

ObjC is what I haz. I believe you must first expose the key via the availableProperties method on your CMIOExtensionStreamSource. From the header: typedef NSString *CMIOExtensionProperty NS_TYPED_ENUM API_AVAILABLE(macos(12.3)); From the sample/template code: - (NSSet<CMIOExtensionProperty> *)availableProperties { //    return [NSSet setWithObjects:, CMIOExtensionPropertyStreamFrameDuration, nil]; CMIOExtensionProperty myCustomPropertyKey = @"4cc_cust_glob_0000";     return [NSSet setWithObjects: CMIOExtensionPropertyStreamActiveFormatIndex, myCustomPropertyKey, nil]; } In the client application you use the CoreMedia C API to set/get the property value: CMIOObjectPropertyAddress myCustomPropertyAddress = { .Selector = FOUR_CHAR_CODE('cust') .Scope = kCMI0ObjectPropertyScopeGlobal, .mElement = kCMI0ObjectPropertyElementMain }; // CMIOObjectHasProperty(object,&myCustomPropertyAddress); // CMIOObjectGetPropertyData(object, &myCustomPropertyAddress, 0, NULL, *propertyDataSize, propertyDataSize, propertyValue)); To see how to query the device list, open streams and retrieve property values, this sample may be useful: https://github.com/bangnoise/cmiotest

I just wanted to note that the CoreAudio team has done a superb job of dealing with these same basic problems. I develop both coreaudiod drivers and CoreMediaIO DAL drivers and note the following: Under Apple Silicon (Big Sur, Monterrey) CoreAudio loads code signed Intel binary audio output drivers just fine, it appears to run them in a special Intel service helper that runs under Rosetta2. Even in an app that has not adopted the Hardened runtime AVFoundation only vends CMIO DAL drivers if they have the appropriate fat binary code. The CoreAudio support is seamless and invisible to the user and ideally we'd have the same situation for CMIO sometime in the near future. So QuickTime Player is unable to access even Apple Silicon signed CMIO DAL drivers even while exposing Intel versions of analogous audio drivers. Currently AVFoundation is loading CMIO DAL drivers into the process using the driver. I believe this code should be run in a service process. QTKit used this technique and it was very fast so it's not a performance issue AFAIK. I believe that Apple, by virtue of the fact that Apple code signed libraries are approved by the Hardened runtime, could create a CMIO DAL driver that would seamlessly "remote" all the 3rd party (code signed) CMIO DAL drivers into an XPC process. I believe this technique was used to allow QTKit to vend 32-bit VDIG drivers to 64-bit applications. I think it's a lot to expect that end users understand these types of issues: if they install an app that includes a virtual camera driver (or in the case of Blackmagic, a hardware device), I think it's clear their intent is to make use of those features.

I have found that setting the contents property of a CALayer is an effective drawing technique but when used with IOSurfaceRef or CVPixelBufferRef it is necessary to double buffer the incoming surfaces because if you set the CALayer.contents twice in a row with the same CVPixelBufferRef it does not display the updated contents. On earlier versions of Mac OS it seems like the color matrix attachments of the CVPixelBuffer are not interpreted or ignored so colors may be off but on recent versions of MacOS the color matrix attachments of the CVPixelBuffer will be applied when rendering the CALayer.contents making this a very powerful technique and frankly eliminating huge amounts of drawing code in Metal or OpenGL.

I believe the solution is to include both an iOS-10 extension with com.apple.broadcast-services (generated by Xcode 8) and also an iOS-11 extension with "com.apple.broadcast-services-upload" (generated by Xcode 9 and later). The two components can share the same source code during build. In any event, the UI model is slightly different.

I believe there is a problem in the texturing that stems from a misunderstanding of who owns the GL textures created by CVOpenGLTextureCacheCreateTextureFromImage. I believe the TextureCache is the owner of the GL texture object. Therefore, when you call CVOpenGLTextureRelease shortly after taking the textureName, the GL texture is in fact returned to the pool of textures, so CoreVideo can immediately reuse it. You must hold on the to the reference count of CVOpenGLTextureRef until you are done with the GL texture owned by the CVOpenGLTextureRef.Also, I believe the call to CVPixelBufferLockBaseAddress is unnecessary because for an OpenGL compatible CVPixelBuffer, the backing will be an IOSurfaceRef which means the pixel data is already on the GPU and the GL texture will be created via a GPU copy (blit). The call to CVPixelBufferLockBaseAddress will cause the IOSurfaceRef to be mapped into system memory which is a very expensive operation on some GPU models. If CVOpenGLTextureCacheCreateTextureFromImage needs to lock the pixel buffer I'm sure it will do so. In my implementations which use XXXTextureCacheCreateFromImage (iOS,Mac,Metal) I simply pass the CVPixelBufferRef without locking the base address and it's working fine.There is a similar misuse of the CoreVideo APIs in this metal sample code which uses the Metal equivalent API CVMetalTextureCacheCreate. This code sorta works on iOS but is totally broken on Mac because the id&lt;MTLTexture&gt; returned from the call to CVMetalTextureGetTexture(textureRef) is really only valid until the CVMetalTextureCache decides to recycle the underlying storage which can be very soon since on the Mac it looks like the pool is only two textures.https://developer.apple.com/library/content/samplecode/MetalVideoCapture/Listings/MetalVideoCapture_AAPLRenderer_mm.htmlI ported AVBasicVideoOutput to Metal on both Mac &amp; iOS but am having some initial problems YCrCb texturing under OpenGL on the desktop (which is how * found this thread). I believe Apple is preferring to use Metal internally and if you can switch to Metal I'm sure you'll be pleased with the increased performance and simplified macOS/iOS unified rendering code.https://developer.apple.com/library/content/samplecode/AVBasicVideoOutput/Listings/AVBasicVideoOutput_APLViewController_m.html

Regarding my recent adventures to port AVBasicVideoOutput to Metal and Desktop GL, I believe the fastest texturing path on OSX is to use kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange and use IOSurface backed CVPixelBuffers. Then, create two textures, one for each of the two planes in the IOSurfaceRef and use a GLSL shader to combine them.NSDictionary* pixelBufferAttributesBiPlanarYCrCb = [NSDictionary dictionaryWithObjectsAndKeys: @(kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange),(id)kCVPixelBufferPixelFormatTypeKey, @(YES),(id)kCVPixelBufferOpenGLCompatibilityKey, @(YES),(id)kCVPixelBufferIOSurfaceOpenGLTextureCompatibilityKey, [NSDictionary dictionary],(id)kCVPixelBufferIOSurfacePropertiesKey, nil];I'm fairly new to CoreProfile GL, but it appears that CGLTexImageIOSurface2D only works with rectangle textures:// luma/luminance texture, full resolutionCGLTexImageIOSurface2D(glContext,GL_TEXTURE_RECTANGLE,GL_R8,(GLsizei)textureSize.width,(GLsizei)textureSize.height,GL_RED,GL_UNSIGNED_BYTE,surfaceRef, 0);// chroma texture, subsampledCGLTexImageIOSurface2D(glContext,GL_TEXTURE_RECTANGLE, GL_RG8, (GLsizei)planeSize.width, (GLsizei)planeSize.height, GL_RG, GL_UNSIGNED_BYTE, surfaceRef, 1);This is much closer conceptually to the working code on EAGL that apple provides in AVBasicVideoOutput.