tensorflow: ValueError: Cannot convert a Tensor of dtype resource to a NumPy array

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): N/A, as it can be reproduced in Google Colab
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
TensorFlow installed from (source or binary): - TensorFlow version (use command below): 2.1
Python version: - Bazel version (if compiling from source): N/A
GCC/Compiler version (if compiling from source): N/A
CUDA/cuDNN version: - GPU model and memory: N/A

Describe the current behavior: It is resulting in Error, InvalidArgumentError: Cannot convert a Tensor of dtype resource to a NumPy array., while running the First Code but is working fine when tf.keras.Input is replaced with tf.Variable in the Second Code.

Describe the expected behavior: Code should work fine with tf.keras.Input as well

Standalone code to reproduce the issue

  **Code with Error:**

import tensorflow as tf

num_uids = 50 #input_uid = tf.keras.layers.Input(shape=(1,), dtype=tf.int32, batch_size = 32) input_uid = tf.keras.layers.Input(shape=(1,), dtype=tf.int32) params = tf.Variable(tf.random.normal((num_uids, 9)), trainable=True)

param = tf.gather_nd(params, input_uid)

#input_shared_features = tf.keras.layers.Input(shape=(128,), dtype=tf.float32, batch_size = 32) input_shared_features = tf.keras.layers.Input(shape=(128,), dtype=tf.float32) combined = tf.concat([param, input_shared_features], axis=-1)

net = tf.keras.layers.Dense(128)(combined)

  **Working Code:**

import tensorflow as tf

num_uids = 50 input_uid = tf.Variable(tf.ones((32, 1), dtype=tf.int32)) params = tf.Variable(tf.random.normal((num_uids, 9)), trainable=True)

param = tf.gather_nd(params, input_uid)

input_shared_features = tf.Variable(tf.ones((32, 128), dtype=tf.float32)) combined = tf.concat([param, input_shared_features], axis=-1)

net = tf.keras.layers.Dense(128)(combined)

Please find the Github Gist.

There is a Stack Overflow Question also, associated with this issue.

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 19 (4 by maintainers)

Most upvoted comments

Same problem here. Tensorflow version

'2.2.0-rc3'

Code to reproduce the issue:

import tensorflow as tf
import tensorflow_hub as hub
import tensorflow_datasets as tfds

embedding = "https://tfhub.dev/google/tf2-preview/gnews-swivel-20dim/1"
hub_layer = hub.KerasLayer(embedding, input_shape=[], 
                           dtype=tf.string, trainable=True)

Dense = tf.keras.layers.Dense
fc_model = tf.keras.Sequential()
fc_model.add(hub_layer)
fc_model.add(Dense(4096, activation=tf.nn.swish))
fc_model.add(Dense(1))

ds_train, ds_val, ds_test = tfds.load(
    name="imdb_reviews", 
    split=('train[:60%]', 'train[60%:]', 'test'),
    as_supervised=True)

fc_model.compile(optimizer='adam',
              loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
              metrics=['accuracy'])

fc_model.fit(ds_train.shuffle(10000).batch(512), validation_data=ds_test.batch(512), epochs=5, verbose=1)

def representative_dataset_gen():
  for _ in range(num_calibration_steps):
    # Get sample input data as a numpy array in a method of your choosing.
    yield [input]


converter = tf.lite.TFLiteConverter.from_keras_model(fc_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen
tflite_quant_model = converter.convert()

The issue:


---------------------------------------------------------------------------

InvalidArgumentError                      Traceback (most recent call last)

<ipython-input-31-b07abdccac46> in <module>()
      8 converter.optimizations = [tf.lite.Optimize.DEFAULT]
      9 converter.representative_dataset = representative_dataset_gen
---> 10 tflite_quant_model = converter.convert()

6 frames

/usr/local/lib/python3.6/dist-packages/tensorflow/lite/python/lite.py in convert(self)
    457     frozen_func, graph_def = (
    458         _convert_to_constants.convert_variables_to_constants_v2_as_graph(
--> 459             self._funcs[0], lower_control_flow=False))
    460     input_tensors = [
    461         tensor for tensor in frozen_func.inputs

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/convert_to_constants.py in convert_variables_to_constants_v2_as_graph(func, lower_control_flow, aggressive_inlining)
    704   """
    705   graph_def, converted_inputs = _convert_variables_to_constants_v2_impl(
--> 706       func, lower_control_flow, aggressive_inlining)
    707   frozen_func = _construct_concrete_function(func, graph_def, converted_inputs)
    708   return frozen_func, graph_def

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/convert_to_constants.py in _convert_variables_to_constants_v2_impl(func, lower_control_flow, aggressive_inlining)
    455 
    456   # Get mapping from node name to variable value.
--> 457   tensor_data = _get_tensor_data(func)
    458 
    459   # Get mapping from function name to argument types.

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/convert_to_constants.py in _get_tensor_data(func)
    215       data = map_index_to_variable[idx].numpy()
    216     else:
--> 217       data = val_tensor.numpy()
    218     tensor_data[tensor_name] = {
    219         "data": data,

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in numpy(self)
    959     """
    960     # TODO(slebedev): Consider avoiding a copy for non-CPU or remote tensors.
--> 961     maybe_arr = self._numpy()  # pylint: disable=protected-access
    962     return maybe_arr.copy() if isinstance(maybe_arr, np.ndarray) else maybe_arr
    963 

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in _numpy(self)
    927       return self._numpy_internal()
    928     except core._NotOkStatusException as e:
--> 929       six.raise_from(core._status_to_exception(e.code, e.message), None)
    930 
    931   @property

/usr/local/lib/python3.6/dist-packages/six.py in raise_from(value, from_value)

InvalidArgumentError: Cannot convert a Tensor of dtype resource to a NumPy array.

+12

gaceladri on Apr 23, 2020

Same issue, any update?

ruiguo-bio on May 4, 2020

I had the same issue, and was able to work around it like this:
indices = Input(name='indices', shape=(), dtype='int32')
params = tf.Variable(params)

class GatherLayer(keras.layers.Layer):
    def call(self, indices, params):
        return tf.gather(params, indices)

output = GatherLayer()(indices, params)
Rather unintuitively, the critical part is actually that the function arguments (params and indices) are swapped in the custom layer. If you don’t do that, you still get the same error. I didn’t dig deeper into why this is the case.

As obtuse as this might be, this worked for me too!

anjany on Jun 25, 2020

I had the same issue, and was able to work around it like this:

indices = Input(name='indices', shape=(), dtype='int32')
params = tf.Variable(params)

class GatherLayer(keras.layers.Layer):
    def call(self, indices, params):
        return tf.gather(params, indices)

output = GatherLayer()(indices, params)

Rather unintuitively, the critical part is actually that the function arguments (params and indices) are swapped in the custom layer. If you don’t do that, you still get the same error. I didn’t dig deeper into why this is the case.

jscissr on Jun 17, 2020

I have posted a solution in the stackoverflow question associated with this issue - https://stackoverflow.com/a/62161525/2352424 Hope that helps

jithinjees on Jun 2, 2020

I also had the same issue. Are there any solutions?

ryogam8 on May 21, 2020