tensorflow: Error converting NMT sequence to sequence example to .tflite model
I am following this example: https://www.tensorflow.org/tutorials/text/nmt_with_attention
It is working as it should be and saving checkpoints. I want to now convert this to a TF Lite model following this example: https://www.tensorflow.org/lite/convert/python_api#converting_a_savedmodel_ or https://www.tensorflow.org/lite/convert/python_api#converting_a_concrete_function_
Here is what I am running to save and them convert:
tflite_input_tensor = tf.constant(1., shape=[64, 39])
tflite_target_tensor = tf.constant(1., shape=[64, 7])
tflite_enc_hidden_tensor = tf.constant(1., shape=[64, 1024])
export_dir = "saved_models"
checkpoint.f = train_step
to_save = checkpoint.f.get_concrete_function(tflite_input_tensor, tflite_target_tensor, tflite_enc_hidden_tensor)
tf.saved_model.save(checkpoint, export_dir, to_save)
converter = tf.lite.TFLiteConverter.from_concrete_functions([to_save])
tflite_model = converter.convert()
But I am getting this error:
~/anaconda3/envs/main/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py in __call__(self, inputs, *args, **kwargs)
845 outputs = base_layer_utils.mark_as_return(outputs, acd)
846 else:
--> 847 outputs = call_fn(cast_inputs, *args, **kwargs)
848
849 except errors.OperatorNotAllowedInGraphError as e:
~/anaconda3/envs/main/lib/python3.6/site-packages/tensorflow_core/python/autograph/impl/api.py in wrapper(*args, **kwargs)
290 def wrapper(*args, **kwargs):
291 with ag_ctx.ControlStatusCtx(status=ag_ctx.Status.DISABLED):
--> 292 return func(*args, **kwargs)
293
294 if inspect.isfunction(func) or inspect.ismethod(func):
TypeError: call() missing 2 required positional arguments: 'hidden' and 'enc_output'
Trained with:
@tf.function
def train_step(inp, targ, enc_hidden):
loss = 0
with tf.GradientTape() as tape:
enc_output, enc_hidden = encoder(inp, enc_hidden)
dec_hidden = enc_hidden
dec_input = tf.expand_dims([targ_lang.word_index['<start>']] * BATCH_SIZE, 1)
# Teacher forcing - feeding the target as the next input
for t in range(1, targ.shape[1]):
# passing enc_output to the decoder
predictions, dec_hidden, _ = decoder(dec_input, dec_hidden, enc_output)
loss += loss_function(targ[:, t], predictions)
# using teacher forcing
dec_input = tf.expand_dims(targ[:, t], 1)
batch_loss = (loss / int(targ.shape[1]))
variables = encoder.trainable_variables + decoder.trainable_variables
gradients = tape.gradient(loss, variables)
optimizer.apply_gradients(zip(gradients, variables))
return batch_loss
EPOCHS = 3
for epoch in range(EPOCHS):
start = time.time()
enc_hidden = encoder.initialize_hidden_state()
total_loss = 0
for (batch, (inp, targ)) in enumerate(dataset.take(steps_per_epoch)):
batch_loss = train_step(inp, targ, enc_hidden)
total_loss += batch_loss
if batch % 100 == 0:
print('Epoch {} Batch {} Loss {:.4f}'.format(epoch + 1,
batch,
batch_loss.numpy()))
# saving (checkpoint) the model every 2 epochs
if (epoch + 1) % 1 == 0:
checkpoint.save(file_prefix = checkpoint_prefix)
print('Epoch {} Loss {:.4f}'.format(epoch + 1, total_loss / steps_per_epoch))
print('Time taken for 1 epoch {} sec\n'.format(time.time() - start))
Somehow the parameters for the Decoder call is not being passed in?
class Decoder(tf.keras.Model):
def __init__(self, vocab_size, embedding_dim, dec_units, batch_sz):
super(Decoder, self).__init__()
self.batch_sz = batch_sz
self.dec_units = dec_units
self.embedding = tf.keras.layers.Embedding(vocab_size, embedding_dim)
self.rnn = tf.keras.layers.GRU(self.dec_units,
return_sequences=True,
return_state=True,
recurrent_initializer='glorot_uniform',
unroll=True)
self.fc = tf.keras.layers.Dense(vocab_size)
# used for attention
self.attention = BahdanauAttention(self.dec_units)
def call(self, x, hidden, enc_output):
# enc_output shape == (batch_size, max_length, hidden_size)
context_vector, attention_weights = self.attention(hidden, enc_output)
# x shape after passing through embedding == (batch_size, 1, embedding_dim)
x = self.embedding(x)
# x shape after concatenation == (batch_size, 1, embedding_dim + hidden_size)
x = tf.concat([tf.expand_dims(context_vector, 1), x], axis=-1)
# passing the concatenated vector to the GRU
output, state = self.rnn(x)
# output shape == (batch_size * 1, hidden_size)
output = tf.reshape(output, (-1, output.shape[2]))
# output shape == (batch_size, vocab)
x = self.fc(output)
return x, state, attention_weights
I understand there may be some trouble converting the GRU layers, but I will tackle that next. This seems to blow up before it even can check if GRU is able to be converted.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 19 (5 by maintainers)
Glad to hear that it’s working for you.
I think with the new converter coming out soon, it’s probably easier to combine the encoder/decoder into the single graph and also wrap loop inside of the model. The new converter will be great at handling the control flow ops. Please be patient about it and stay tuned. Thanks!
For anyone following, I found a way to get this working by splitting up the encoder and decoder into two tflite models for the time being and then managing the loop that decodes the sequence externally.
I am guessing this is due to the fact certain control flow operations are not supported in TFLite that would allow a concrete function with a loop to convert properly. This will do for now. I am getting great accuracy and have this running in a native Android app on sample data.
Thank you for the help and looking forward to the new updated as we round out the year!