tensorflow: TypeError: true_fn and false_fn arguments to tf.cond must have the same number, type, and overall structure of return values

Issue type

Bug

Have you reproduced the bug with TensorFlow Nightly?

No

Source

binary

TensorFlow version

2.17.0

Custom code

Yes

OS platform and distribution

Linux CentOS 7.9

Mobile device

No response

Python version

3.12.4

Bazel version

No response

GCC/compiler version

No response

CUDA/cuDNN version

No response

GPU model and memory

No response

Current behavior?

We used tensorflow for 3 class classifications. The code worked well on existed environment: Python 3.8.3 + tensorflow 2.13.1. But when we tried on new enviroment: Python 3.12.4 + tensorflow 2.17.0, several errors blocked.

Standalone code to reproduce the issue

Here are the simple code to reproduce the issue.

import tensorflow as tf
from tensorflow.keras.layers import Dense
import numpy as np


def build_model():
    metrics = ['accuracy', tf.keras.metrics.AUC(multi_label=True)]
    loss = tf.keras.losses.sparse_categorical_crossentropy
    num_class = 3
    inputs = tf.keras.layers.Input(shape=(40, 36))
    lstm_out = tf.keras.layers.LSTM(32, return_sequences=False)(inputs)
    output = Dense(num_class, activation='softmax')(lstm_out)
    model = tf.keras.Model(inputs, output)
    model.compile(loss=loss, optimizer=tf.keras.optimizers.Adam(), metrics=metrics)
    model.summary()
    return model

def data_gen():
    while True:
        yield np.random.rand(128, 40, 36), np.random.randint(0, 3, (128, 1))

model = build_model()
cw = {0: 0.3, 1:2.5, 2:3.2}
model.fit(data_gen(), epochs=2, steps_per_epoch=10, class_weight=cw)

The first error we met:

Traceback (most recent call last):
  File "/data/release/infinity_stock2/tf_model.py", line 25, in <module>
    model.fit(data_gen(), epochs=2, steps_per_epoch=10, class_weight=cw)
  File "/root/.virtualenvs/new_stock/lib/python3.12/site-packages/keras/src/utils/traceback_utils.py", line 122, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/root/.virtualenvs/new_stock/lib/python3.12/site-packages/keras/src/trainers/data_adapters/__init__.py", line 108, in get_data_adapter
    raise ValueError(
ValueError: Argument `class_weight` is not supported for Python generator inputs. Received: class_weight={0: 0.3, 1: 2.5, 2: 3.2}

To solve the issue, we used tf dataset to wrap the generator.

import tensorflow as tf
from tensorflow.keras.layers import Dense
import numpy as np


def build_model():
    metrics = ['accuracy', tf.keras.metrics.AUC(multi_label=True)]
    loss = tf.keras.losses.sparse_categorical_crossentropy
    num_class = 3
    inputs = tf.keras.layers.Input(shape=(40, 36))
    lstm_out = tf.keras.layers.LSTM(32, return_sequences=False)(inputs)
    output = Dense(num_class, activation='softmax')(lstm_out)
    model = tf.keras.Model(inputs, output)
    model.compile(loss=loss, optimizer=tf.keras.optimizers.Adam(), metrics=metrics)
    model.summary()
    return model

def data_gen():
    while True:
        yield np.random.rand(128, 40, 36), np.random.randint(0, 3, (128, 1))

model = build_model()
cw = {0: 0.3, 1:2.5, 2:3.2}
data = tf.data.Dataset.from_generator(data_gen, output_types=(tf.float32, tf.float32),  output_shapes=((None, 40, 36), (None, None)))
model.fit(data, epochs=2, steps_per_epoch=10, class_weight=cw)

And another error happened:

Traceback (most recent call last):
  File "/data/release/infinity_stock2/tf_model.py", line 26, in <module>
    model.fit(data, epochs=2, steps_per_epoch=10, class_weight=cw)
  File "/root/.virtualenvs/new_stock/lib/python3.12/site-packages/keras/src/utils/traceback_utils.py", line 122, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/root/.virtualenvs/new_stock/lib/python3.12/site-packages/tensorflow/python/ops/cond_v2.py", line 876, in error
    raise TypeError(
TypeError: true_fn and false_fn arguments to tf.cond must have the same number, type, and overall structure of return values.

true_fn output: Tensor("cond/Identity:0", shape=(None,), dtype=int64)
false_fn output: Tensor("cond/Identity:0", shape=(None,), dtype=int32)

Error details:
Tensor("cond/Identity:0", shape=(None,), dtype=int64) and Tensor("cond/Identity:0", shape=(None,), dtype=int32) have different types

It looks like class weight might not work well.

Relevant log output

No response

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 19 (1 by maintainers)

Most upvoted comments

@henghamao Looking into the keras/src/backend/tensorflow/trainer.py and keras/src/trainers/data_adapters code, class_weight and sample_weight should behave in a similar way for most common data types (I checked the code for ArrayDataAdapter and tf.dataset types, but haven’t looked into other types). In fact, class_weights are converted into sample_weights internally in both implementations. For example: https://github.com/keras-team/keras/blob/master/keras/src/trainers/data_adapters/array_data_adapter.py#L66-L74.

To test this, I initialized two versions of the model in your code and set the initial weights for the models to be the same. Then I ran model.fit() on the first model with sample_weights, and on the second model with the class_weights (using the dictionary). Note that I set shuffle=False explicitly during the model.fit() call, so that each training step for both models uses the same data. The metrics (accuracy, auc and loss) are identical for each epoch for both models. Here is the gist.

A few caveats. I tested the code on TF 2.18.0. I’m not sure if an earlier version had a different implementation. Both the models in the gist pass the inputs and outputs to the model directly i.e. they do not use a generator. class_weights do not work with a generator and there was never support for it AFAIK (source).

I hope this clears up your question on whether sample_weight and class_weight work the same. Let me know if you have any other doubts.

Hi @henghamao , Apologies for the delay. I tried running your code on Colab using TensorFlow 2.13.0, and it worked fine. I also tested it with versions 2.17.0 and the nightly release, and I encountered the same issue. However, it works fine with an alternative approach. Please find the gist here for reference. You mentioned that the training speed is faster in this alternative way — this is because Keras 3 is working on JAX, which enhances performance compared to older versions. Thank you!

@henghamao I modified the data_gen function to output not only the data (x_batch, y_batch) but also sample_weights, which are calculated based on the class_weights.

Instead of class_weight, the model now uses sample_weights per batch in the generator, which handles class imbalance.

can you check this code out:

import tensorflow as tf
from tensorflow.keras.layers import Dense
import numpy as np


def build_model():
    metrics = ['accuracy', tf.keras.metrics.AUC(multi_label=True)]
    loss = tf.keras.losses.sparse_categorical_crossentropy
    num_class = 3
    inputs = tf.keras.layers.Input(shape=(40, 36))
    lstm_out = tf.keras.layers.LSTM(32, return_sequences=False)(inputs)
    output = Dense(num_class, activation='softmax')(lstm_out)
    model = tf.keras.Model(inputs, output)
    model.compile(loss=loss, optimizer=tf.keras.optimizers.Adam(), metrics=metrics)
    model.summary()
    return model

def data_gen(class_weights):
    while True:
        x_batch = np.random.rand(128, 40, 36)
        y_batch = np.random.randint(0, 3, (128, 1))
        # Apply class weights to the labels
        sample_weights = np.vectorize(lambda x: class_weights[x])(y_batch)
        yield x_batch, y_batch, sample_weights

model = build_model()
cw = {0: 0.3, 1: 2.5, 2: 3.2}
model.fit(data_gen(cw), epochs=2, steps_per_epoch=10)

this shows no error output for tis code image

what do you say about this??