Using Metal to create metal library from precompiled metallib file on iPhone6 is unexpected slow

Problem: 

We are using Xcode tools to precompile metal shader source into metallib binary files and using [MTLDevice newLibraryWithFile] to create MTLLibrary from the metallib at runtime so that we can avoid compilation from source according to Metal Best Practice. But we found that on iPhone6 with iOS 12.5, the create time comes to 23ms, which takes longer than expected because we are using the most simple code from Apple's sample project "HelloTriangle". Then we test another two ways to create a metal library. newLibraryWithSource takes 134ms, and newLibraryWithData takes 0.7ms to read the file and 20ms to create library from data. All the time costs are shown by the uploaded files. And the results show that it does take 20ms to create metal library from metallib after reading the static file into system memory, without any I/O cost.

When shader source are precompiled into metallib, what takes so long when creating metal library with those binary at runtime? Is it only happens on iPhone6 or any special GPU family? We want to know what are done during the creation and how to reduce the time, or just exclude the special devices from usage.


Reproducing: 

We are using the Metal sample demo: HelloTriangle from developer.apple.com to reproduce the problem. 

  1. using metal tool to compile metal source into air file.
xcrun -sdk iphoneos metal -c -target air64-apple-ios12.0 AAPLShaders.metal
  1. using metallib tool to assemble air files into metallib file.
xcrun -sdk iphoneos metallib AAPLShaders.air -o AAPLShaders.metallib
  1. loading library from metallib and compute the time cost.

NSString *libraryFile = [[NSBundle mainBundle] pathForResource:@"AAPLShaders" ofType:@"metallib"];

double startTime = [[NSDate date] timeIntervalSince1970] * 1000;

id<MTLLibrary> defaultLibrary = [_device newLibraryWithFile:libraryFile error:&libraryError];

double endTime = [[NSDate date] timeIntervalSince1970] * 1000;

NSLog(@"New Library from file: %f ms ", (endTime-startTime));


What are expected:

When we precompile the metal source into metallib, we hope the time it takes to create metal library at runtime can shrink to under 1ms, or at least not notable as 23ms. We test the same code on iPhone12 Pro, newLibraryWithFile only costs 0.2ms.


What we saw:

newLibraryWithFile takes 23ms

newLibraryWithData takes 20ms

newLibraryWithSource takes 134ms

Using Metal to create metal library from precompiled metallib file on iPhone6 is unexpected slow
 
 
Q