CoreML custom layer implemented by `object-C` doesn't work in `swift` testing project

I convert a pytorch model to mlmodel with a custom layer, and create a test app in swift to test my model. When i implement the custom layer by swift, it works well. However when i implement the custom layer by object-C, the code return

2022-01-14 17:58:49.964377+0800 CustomLayers[2547:968723] [coreml] Error in adding network -1.
2022-01-14 17:58:49.965023+0800 CustomLayers[2547:968723] [coreml] MLModelAsset: load failed with error Error Domain=com.apple.CoreML Code=0 "Error in declaring network." UserInfo={NSLocalizedDescription=Error in declaring network.}
2022-01-14 17:58:49.965085+0800 CustomLayers[2547:968723] [coreml] MLModelAsset: modelWithError: load failed with error Error Domain=com.apple.CoreML Code=0 "Error in declaring network." UserInfo={NSLocalizedDescription=Error in declaring network.}
Fatal error: 'try!' expression unexpectedly raised an error: Error Domain=com.apple.CoreML Code=0 "Error in declaring network." UserInfo={NSLocalizedDescription=Error in declaring network.}: file CustomLayers/model_2.swift, line 114
2022-01-14 17:58:49.966267+0800 CustomLayers[2547:968723] Fatal error: 'try!' expression unexpectedly raised an error: Error Domain=com.apple.CoreML Code=0 "Error in declaring network." UserInfo={NSLocalizedDescription=Error in declaring network.}: file CustomLayers/model_2.swift, line 114
(lldb)

It seems the model load failed with object-C custom layer. So i wonder does object-C custom layer implementation can't work with swift project? Although i try to set the CustomLayers-Bridging-Header.h. It still doesn't work.

System Information

  • mac OS: 11.6.1 Big Sur
  • xcode: 12.5.1
  • coremltools: 5.1.0
  • test device: iphone 11
Up vote post of stx-000 Down vote post of stx-000
1.1k views
Posted by
stx-000
Reply
Add a Comment

Replies

It may be helpful to double check that the name of your objective-c class conforming to the MLCustomLayer protocol matches the custom layer name described in the model. You can check the name the model expects by looking at the Dependencies tab in Xcodes view of the .mlmodel (or .mlpackage) file.

Note: Another thing to keep in mind is that Swift class names are prefixed with your module/bundle, e.g. YourApp.YourCustomLayerClass. It is best to encode an objective-C compatible class name in your model, e.g. YourCustomLayerClass, if you want the flexibility to implement it in both objective-C and Swift. In Swift, you would use the @objc(YourCustomClass) to expose it to the Core ML runtime without the module prefix.

Posted by
Frameworks Engineer
Up vote reply of Frameworks Engineer Down vote reply of Frameworks Engineer

  • Yeah, i have checked the name of objective-c class conforming to the MLCustomLayer protocol matches the custom layer name described in the model. It's GridSampleLayer. And i implement the objective-c as shown below,

    GridSampleLayer.h #import <CoreML/MLCustomLayer.h> #import <Foundation/Foundation.h> #import <CoreML/CoreML.h> @interface GridSampleLayer : NSObject<MLCustomLayer> @end GridSampleLayer.m #import <Foundation/Foundation.h> #import <CoreML/CoreML.h> #import <CoreML/MLMultiArray.h> #import <Vision/Vision.h> #import "GridSampleLayer.h" #import <Accelerate/Accelerate.h> @implementation GridSampleLayer {     } - (BOOL) evaluateOnCPUWithInputs:(NSArray<MLMultiArray *> *)inputs outputs:(NSArray<MLMultiArray *> *)outputs error:(NSError *__autoreleasing _Nullable *)error{ NSLog(@"get in cpu func"); ...... ...... } - (instancetype) initWithParameterDictionary:(NSDictionary<NSString *,id> *)parameters error:(NSError *__autoreleasing _Nullable *)error{   self = [super init];   return self; } - (NSArray<NSArray<NSNumber *> * > *) outputShapesForInputShapes:(NSArray<NSArray<NSNumber *> *> *)inputShapes error:(NSError *__autoreleasing _Nullable *) error{ return inputShapes; } - (BOOL) setWeightData:(NSArray<NSData *> *)weights error:(NSError *__autoreleasing _Nullable *) error{   return true; } @end

    Are there some problems？

    —
    stx-000
Add a Comment

This has been solved. I found it's caused by the return value of outputShapesForInputShapes not matching MLCustomLayer protocol. Now i have resolved it. Thanks!

Posted by
stx-000
Up vote reply of stx-000 Down vote reply of stx-000
Add a Comment