tensorflow: Cannot find the placeholder op that is an input to ReadVariableOp in tf lite conversion.

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes.
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
• TensorFlow installed from (source or binary): binary
• TensorFlow version (use command below): 2 beta
• Python version: 3.6.7
• Bazel version (if compiling from source):
• GCC/Compiler version (if compiling from source):
• CUDA/cuDNN version: Cuda 10
• GPU model and memory: Geforce GTX 1060

Describe the current behavior I have a pretty complicated model with three different inputs and I can save and load it with custom objects as a keras model with model.save() and model.load() methods. I want to convert it to tf lite and I get this error: ValueError: Cannot find the Placeholder op that is an input to the ReadVariableOp

Code to reproduce the issue

loss_function = get_loss_function()
    model = tf.keras.models.load_model(save_checkpoint_address, custom_objects={
        'customlayer': CustomLayer,
        'loss_function': loss_function
    })

converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)

Other info / logs

2019-06-11 19:55:42.378746: I tensorflow/core/grappler/devices.cc:55] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0
2019-06-11 19:55:42.378868: I tensorflow/core/grappler/clusters/single_machine.cc:359] Starting new session
2019-06-11 19:55:42.434841: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:716] Optimization results for grappler item: graph_to_optimize
2019-06-11 19:55:42.434867: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   function_optimizer: Graph size after: 2213 nodes (393), 3499 edges (663), time = 18.406ms.
2019-06-11 19:55:42.434874: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   function_optimizer: function_optimizer did nothing. time = 0.785ms.
Traceback (most recent call last):
  File "/home/siavash/programming/ximpa/carim_tensor/convert_to_tflite.py", line 32, in <module>
    convert_saved_model()
  File "/home/siavash/programming/ximpa/carim_tensor/convert_to_tflite.py", line 27, in convert_saved_model
    tflite_model = converter.convert()
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/lite/python/lite.py", line 348, in convert
    self._funcs[0])
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/convert_to_constants.py", line 166, in convert_variables_to_constants_v2
    raise ValueError("Cannot find the Placeholder op that is an input "
ValueError: Cannot find the Placeholder op that is an input to the ReadVariableOp.