transformers: ValueError: You have to specify either input_ids or inputs_embeds!

Details

I’m quite new to NLP task. However, I was trying to train the T5-large model and set things as follows. But unfortunately, I’ve got an error.

def build_model(transformer, max_len=512):
    input_word_ids = Input(shape=(max_len,), dtype=tf.int32, name="input_word_ids")
    sequence_output = transformer(input_word_ids)[0]
    cls_token = sequence_output[:, 0, :]
    out = Dense(1, activation='sigmoid')(cls_token)
    model = Model(inputs=input_word_ids, outputs=out)
    return model

model = build_model(transformer_layer, max_len=MAX_LEN)

It thorws

ValueError: in converted code:
ValueError                                Traceback (most recent call last)
<ipython-input-19-8ad6e68cd3f5> in <module>
----> 5     model = build_model(transformer_layer, max_len=MAX_LEN)
      6 
      7 model.summary()

<ipython-input-17-e001ed832ed6> in build_model(transformer, max_len)
     31     """
     32     input_word_ids = Input(shape=(max_len,), dtype=tf.int32, name="input_word_ids")
---> 33     sequence_output = transformer(input_word_ids)[0]
     34     cls_token = sequence_output[:, 0, :]
     35     out = Dense(1, activation='sigmoid')(cls_token)
ValueError: You have to specify either input_ids or inputs_embeds

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 4
Comments: 22 (17 by maintainers)

Links to this issue

bert language model - "You have to specify either input_ids or inputs_embeds", but I did specify the input_ids - Stack Overflow

Most upvoted comments

@prashant-kikani @HarrisDePerceptron

For decoder_input_ids , we just need to put a single BOS token so that the decoder will know that this is the beginning of the output sentence. (Even in GLUE task, T5 still looks at every output label as a complete sentence )

We can see a concrete example by looking at the function prepare_inputs_for_generation which is called by model.generate (generate function is here : https://github.com/huggingface/transformers/blob/master/src/transformers/generation_tf_utils.py )

See line 298 in the above link :

if self.config.is_encoder_decoder:
            if decoder_start_token_id is None:
                decoder_start_token_id = bos_token_id

and line 331:

# create empty decoder_input_ids
            input_ids = (
                tf.ones(
                    (effective_batch_size * num_beams, 1),
                    dtype=tf.int32,
                )
                * decoder_start_token_id
            )

and see T5’s prepare_inputs_for_generation which change the above input_ids into decoder_input_ids implementation at : https://github.com/huggingface/transformers/blob/08f534d2da47875a4b7eb1c125cfa7f0f3b79642/src/transformers/modeling_tf_t5.py#L1367

ratthachat on Oct 27, 2020

@ratthachat - thanks for you message! We definitely need to provide more TF examples for the T5 Model. I want to tackle this problem in ~2 weeks.

In TF we use the naming convention inputs, so the you should change to model.fit({"inputs": x_encoder}) . I very much agree that the error message is quite misleading and correct it in this PR: #4401.

patrickvonplaten on May 16, 2020

I’m not 100% sure what you want to do here exactly. T5 is always trained in a text-to-text format. We have a section here on how to train T5: https://huggingface.co/transformers/model_doc/t5.html#training

Otherwise I’d recommend taking a look at the official paper.

patrickvonplaten on Apr 5, 2020

Hi @innat,

T5 is an encoder-decoder model so you will have to provide both input_ids and decoder_input_ids to the model. Maybe taking a look at the T5 docs (especially the “Examples”) can help you 😃

patrickvonplaten on Apr 5, 2020

Good issue! really helps me.

sunyuhan19981208 on Jul 19, 2023

Hey @dxlong2000,

I’ll open a new issue for this to make it more visible as I think this error happens quite often. See: https://github.com/huggingface/transformers/issues/16234

patrickvonplaten on Mar 17, 2022

Hi @patrickvonplaten Please help me with this error.

I’m doing inference with a T5-base model which I finetuned on GLUE tasks.

It’s giving error like ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds

While doing inference, we just need to provide input_ids for the encoder right? Why do we need decoder_input_ids?

And as it’s inference, my labels will also be None. So, this part will not execute. decoder_input_ids = self._shift_right(labels)

Waiting for your reply. Thank you.

prashant-kikani on Sep 15, 2020

Hi @enzoampil,

A PR for a cleaner Error message would be nice if you feel like it 😃. It would be good if the error message could change between ValueError: You have to specify either input_ids or inputs_embeds if self.is_decoder == False and ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds if self.is_decoder == True. So adding a simple if statement to the error message is definitely a good idea!

patrickvonplaten on Apr 28, 2020