tensorflow: tf.keras.metrics.MeanIoU have some conflicts with sparse_categorical_crossentropy

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template

System information

TensorFlow installed from (source or binary): source
TensorFlow version (use command below): 2.0 RC
Python version: 3.6
CUDA/cuDNN version: 10.0
GPU model and memory: 12Gb

You can collect some of this information using our environment capture script You can also obtain the TensorFlow version with: 1. TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)" 2. TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"

Describe the current behavior

mIOU = tf.keras.metrics.MeanIoU(num_classes=20)
model.compile(optimizer='Adam',
              loss='sparse_categorical_crossentropy',
              metrics=["accuracy", mIOU])

Hi, sparse_categorical_crossentropy will calculate the probabilities of 20 classes, which have the shape (None, 64, 1024, 20). However, the label has the shape (None, 64, 1024). Thus, the mIOU got unequal shape inputs. It got errors

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 2
Comments: 18

Most upvoted comments

class UpdatedMeanIoU(tf.keras.metrics.MeanIoU):
  def __init__(self,
               y_true=None,
               y_pred=None,
               num_classes=None,
               name=None,
               dtype=None):
    super(UpdatedMeanIoU, self).__init__(num_classes = num_classes,name=name, dtype=dtype)

  def update_state(self, y_true, y_pred, sample_weight=None):
    y_pred = tf.math.argmax(y_pred, axis=-1)
    return super().update_state(y_true, y_pred, sample_weight)

+10

dhaneshr on Oct 12, 2020

I believe this is the code which reproduces the issue:

import tensorflow as tf

NUM_CLASSES = 10

X = tf.random.uniform(minval=-1, maxval=1, shape=(10, 10, 3), dtype=tf.float32)
label = tf.random.uniform(minval=0,
                          maxval=NUM_CLASSES-1,
                          shape=(10, 10),
                          dtype=tf.int32)
ds = tf.data.Dataset.from_tensors((X, label)).batch(1)

model = tf.keras.models.Sequential([
    tf.keras.layers.Conv2D(NUM_CLASSES, (1, 1), padding='same'),
    tf.keras.layers.Activation('softmax')
])

loss_obj = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False)

model.compile(optimizer='sgd',
              loss=loss_obj,
              metrics=[tf.keras.metrics.MeanIoU(NUM_CLASSES, name='mIoU')])

model.fit(ds, epochs=1, validation_data=ds)

Running this throws the following exception: InvalidArgumentError: Shapes of all inputs must match: values[0].shape = [100] != values[1].shape = [1000]

pawelkobojek on Oct 8, 2019