transformers: ValueError: The following `model_kwargs` are not used by the model: ['length']

System Info

4.22.2

Who can help?

@SaulLu

Information

The official example scripts
My own modified scripts

Tasks

An officially supported task in the examples folder (such as GLUE/SQuAD, …)
My own task or dataset (give details below)

Reproduction

            prompt = tokenizer(user_input, return_tensors='pt', return_length=True)
            prompt = {key: value.to(device) for key, value in prompt.items()}
            out = gpt.generate(**prompt, ...)

When using “return_length=True” with the tokenizer, the error is given. This is from a change in a recent version and did not happen in older versions.

ValueError: The following model_kwargs are not used by the model: ['length'] (note: typos in the generate arguments will also show up in this list)

Expected behavior

Model should not produce an error when “return_length” is set to True Downgrade to 4.21.0 fixes the problem and according to my googling this is what people are doing

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 4
Comments: 19 (3 by maintainers)

Most upvoted comments

@zzxslp

Change these line at https://github.com/salesforce/BLIP/blob/main/models/med.py#L932 as following:

from

    def prepare_inputs_for_generation(self, input_ids, past=None, attention_mask=None, **model_kwargs):
        input_shape = input_ids.shape
        # if model is used as a decoder in encoder-decoder model, the decoder attention mask is created on the fly
        if attention_mask is None:
            attention_mask = input_ids.new_ones(input_shape)

        # cut decoder_input_ids if past is used
        if past is not None:
            input_ids = input_ids[:, -1:]

        return {
            "input_ids": input_ids, 
            "attention_mask": attention_mask, 
            "past_key_values": past,
            "encoder_hidden_states": model_kwargs.get("encoder_hidden_states", None),
            "encoder_attention_mask": model_kwargs.get("encoder_attention_mask", None),
            "is_decoder": True,
        }

    def prepare_inputs_for_generation(self, input_ids, past=None, attention_mask=None, encoder_hidden_states=None, encoder_attention_mask=None, **model_kwargs):
        input_shape = input_ids.shape
        # if model is used as a decoder in encoder-decoder model, the decoder attention mask is created on the fly
        if attention_mask is None:
            attention_mask = input_ids.new_ones(input_shape)

        # cut decoder_input_ids if past is used
        if past is not None:
            input_ids = input_ids[:, -1:]

        return {
            "input_ids": input_ids, 
            "attention_mask": attention_mask, 
            "past_key_values": past,
            "encoder_hidden_states": encoder_hidden_states,
            "encoder_attention_mask": encoder_attention_mask,
            "is_decoder": True,
        }

Why: https://github.com/huggingface/transformers/blob/v4.23.1/src/transformers/generation_utils.py#L899

tjysdsg on Nov 20, 2022