I'm trying to implement the Neural Style Transfer model by https://arxiv.org/pdf/1610.07629.pdf%C2%A0in an iOS app. The paper uses this conditional instance norm layer to allow a single model to learn multiple styles and also combine the styles.
Here is my code for it in Python:
class ConditionalInstanceNorm(tf.keras.layers.Layer):
def __init__(self, scope_bn, y1, y2, alpha):
super(ConditionalInstanceNorm, self).__init__()
self.scope_bn = scope_bn
self.y1 = y1
self.y2 = y2
self.alpha = alpha
def build(self, input_shape):
self.beta = self.add_weight(name="beta"+self.scope_bn, shape=(self.y1.shape[-1], input_shape[-1]), initializer=betaInitializer, trainable=True)
self.gamma = self.add_weight(name="gamma"+self.scope_bn, shape=(self.y1.shape[-1], input_shape[-1]), initializer=gammaInitializer, trainable=True)
def call(self, inputs):
mean, var = tf.nn.moments(x=inputs, axes=[1,2], keepdims=True)
beta1 = tf.matmul(self.y1, self.beta)
gamma1 = tf.matmul(self.y1, self.gamma)
beta2 = tf.matmul(self.y2, self.beta)
gamma2 = tf.matmul(self.y2, self.gamma)
beta = self.alpha*beta1 + (1. - self.alpha)*beta2
gamma = self.alpha*gamma1 + (1. - self.alpha)*gamma2
x = tf.nn.batch_normalization(x=inputs, mean=mean, variance=var, offset=beta, scale=gamma, variance_epsilon=1e-10)
return x
It requires me to pass in 3 hyper-parameters: y1, y2, and alpha. Each style has a unique y value (y1 and y2 are the different styles to allow combining of styles). Alpha determines the strength of influence of each style.
In python, the way I switch between different combinations of styles is by accessing each layer, identify the conditional instance norm layers then change the y1, y2, and alpha values stored:
for layer in filter(lambda x: "conditional_instance_norm" in x.name, model.layers):
layer.y1 = y1
layer.y1 = y2
layer.alpha = alpha
How can I do this with a .tflite model in swift? Right now I can run the model but it is stuck with the y1, y2, and alpha values it was initialised with.