transformers: Covid-19 - TPU V3-1024 - T5 11B: Tensorflow to Pytorch conversion failed

We are training a large scale T5-11B model using TPU V3-1024 for a Covid-19 project We tried to convert the TensorFlow checkpoint to the Pytorch version, but it did fail. Could you please help us to figure out the problem since this model is very important for Covid-19 research.

🐛 Bug

Information

Model I am using (Bert, XLNet …): T5

Language I am using the model on (English, Chinese …): Protein Sequences

The problem arises when using:

  • the official example scripts: (give details below)
  • my own modified scripts: (give details below)

The tasks I am working on is:

  • an official GLUE/SQUaD task: (give the name)
  • my own task or dataset: (give details below)

To reproduce

Steps to reproduce the behavior:

  1. The config file:
{
  "architectures": [
    "T5WithLMHeadModel"
  ],
  "d_ff": 65536,
  "d_kv": 128,
  "d_model": 1024,
  "decoder_start_token_id": 0,
  "dropout_rate": 0.1,
  "eos_token_id": 1,
  "initializer_factor": 1.0,
  "is_encoder_decoder": true,
  "layer_norm_epsilon": 1e-06,
  "model_type": "t5",
  "n_positions": 512,
  "num_heads": 128,
  "num_layers": 24,
  "output_past": true,
  "pad_token_id": 0,
  "relative_attention_num_buckets": 32,
  "vocab_size": 128
}
  1. conversion command:
python convert_t5_original_tf_checkpoint_to_pytorch.py \
  --tf_checkpoint_path xxx/tensorflow \
  --config_file xxx/t5-11b-config.json \
  --pytorch_dump_path xxx/pytorch
  1. Error:
Building PyTorch model from configuration: T5Config {
  "architectures": [
    "T5WithLMHeadModel"
  ],
  "d_ff": 65536,
  "d_kv": 128,
  "d_model": 1024,
  "decoder_start_token_id": 0,
  "dropout_rate": 0.1,
  "eos_token_id": 1,
  "initializer_factor": 1.0,
  "is_encoder_decoder": true,
  "layer_norm_epsilon": 1e-06,
  "model_type": "t5",
  "n_positions": 512,
  "num_heads": 128,
  "num_layers": 24,
  "output_past": true,
  "pad_token_id": 0,
  "relative_attention_num_buckets": 32,
  "vocab_size": 128
}

INFO:transformers.modeling_t5:Converting TensorFlow checkpoint from /mnt/lsf-nas-1/lsf/job/repo/elnaggar/prot-transformers/models/t5/tensorflow/bfd100
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/relative_attention_bias with shape [128, 32]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/relative_attention_bias_slot_v with shape [128, 32]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_000/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_001/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_002/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_003/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_004/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_005/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_006/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_007/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_008/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_009/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_010/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_011/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_012/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_013/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_014/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_015/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_016/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_017/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_018/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_019/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_020/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_021/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_022/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/EncDecAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_002/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_002/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_002/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_002/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_002/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_002/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_002/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/block_023/layer_002/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight decoder/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/relative_attention_bias with shape [128, 32]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/relative_attention_bias_slot_v with shape [128, 32]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_000/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_001/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_002/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_003/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_004/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_005/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_006/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_007/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_008/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_009/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_010/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_011/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_012/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_013/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_014/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_015/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_016/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_017/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_018/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_019/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_020/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_021/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_022/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/k with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/k_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/k_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/o with shape [16384, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/o_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/o_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/q with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/q_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/q_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/v with shape [1024, 16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/v_slot_vc with shape [16384]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/SelfAttention/v_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_000/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_001/DenseReluDense/wi/kernel with shape [1024, 65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_001/DenseReluDense/wi/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_001/DenseReluDense/wi/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_001/DenseReluDense/wo/kernel with shape [65536, 1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_001/DenseReluDense/wo/kernel_slot_vc with shape [65536]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_001/DenseReluDense/wo/kernel_slot_vr with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_001/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/block_023/layer_001/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/rms_norm/scale with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight encoder/rms_norm/scale_slot_v with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight global_step with shape []
INFO:transformers.modeling_t5:Loading TF weight shared/embedding with shape [128, 1024]
INFO:transformers.modeling_t5:Loading TF weight shared/embedding_slot_vc with shape [1024]
INFO:transformers.modeling_t5:Loading TF weight shared/embedding_slot_vr with shape [128]
INFO:transformers.modeling_t5:Transposing numpy weight of shape (1024, 16384) for ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'k']
INFO:transformers.modeling_t5:Initialize PyTorch weight ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'k']
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/k_slot_vc
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/k_slot_vr
INFO:transformers.modeling_t5:Transposing numpy weight of shape (16384, 1024) for ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'o']
INFO:transformers.modeling_t5:Initialize PyTorch weight ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'o']
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/o_slot_vc
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/o_slot_vr
INFO:transformers.modeling_t5:Transposing numpy weight of shape (1024, 16384) for ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'q']
INFO:transformers.modeling_t5:Initialize PyTorch weight ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'q']
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/q_slot_vc
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/q_slot_vr
INFO:transformers.modeling_t5:Transposing numpy weight of shape (128, 32) for ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'relative_attention_bias']
INFO:transformers.modeling_t5:Initialize PyTorch weight ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'relative_attention_bias']
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/relative_attention_bias_slot_v
INFO:transformers.modeling_t5:Transposing numpy weight of shape (1024, 16384) for ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'v']
INFO:transformers.modeling_t5:Initialize PyTorch weight ['decoder', 'block_000', 'layer_000', 'SelfAttention', 'v']
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/v_slot_vc
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/SelfAttention/v_slot_vr
INFO:transformers.modeling_t5:Skipping decoder/block_000/layer_000/rms_norm/scale
Traceback (most recent call last):
  File "xxx/convert_t5_original_tf_checkpoint_to_pytorch.py", line 61, in <module>
    convert_tf_checkpoint_to_pytorch(args.tf_checkpoint_path, args.config_file, args.pytorch_dump_path)
  File "xxx/convert_t5_original_tf_checkpoint_to_pytorch.py", line 36, in convert_tf_checkpoint_to_$ytorch
    load_tf_weights_in_t5(model, config, tf_checkpoint_path)
  File "xxx/modeling_t5.py", line 102, in load_tf_weights_in_t5
    pointer = getattr(pointer, "weight")
  File "xxx/anaconda3/envs/transformers/lib/python3.7/site-packages/torch/nn/modules/module.py", line 594, in __getattr__
    type(self).__name__, name))
AttributeError: 'T5LayerSelfAttention' object has no attribute 'weight'

Expected behavior

T5 tensorflow model should be converted to pytorch model.

Environment info

  • transformers version: 2.11.0
  • Platform: Linux-4.15.0-101-generic-x86_64-with-debian-buster-sid
  • Python version: 3.7.7
  • PyTorch version (GPU?): 1.5.0 (False)
  • Tensorflow version (GPU?): 2.2.0 (False)
  • Using GPU in script?: No
  • Using distributed or parallel set-up in script?: No

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 35 (35 by maintainers)

Most upvoted comments

Thanks @patrickvonplaten for the clarification. Using T5ForConditionalGeneration did solve my issue, and now my models work as excepted.

Thanks a lot for optimizing the shared encoder/decoder weights. This will be really helpful to allow large models 3B/11B to fit on a single GPU for inference. This will be the last point to cover before closing this issue.

+2
agemagician on Aug 17, 2020

I will take a deeper look next week on Monday when I’m back regarding the conversion from tensorflow to PyTorch.

Regarding, the decoder_input_ids (“For some reason, I have to send also the ids as decoder_input_ids.”) -> yes for T5 you have to input both input_ids and decoder_input_ids, for the first forward call this is usually:

from transformers import T5Model, T5Tokenizer
import torch

model = T5Model.from_pretrained("t5-small")  # your model of choice here
tokenizer = T5Tokenizer.from_pretrained("t5-small")

input_ids = tokenizer("This sentence is encoded and only has to be passed once.", return_tensors="pt").input_ids
decoder_input_ids = torch.tensor([[model.config.decoder_start_token_id]])

# first call uses the start token id for T5 decoder
outputs = model(input_ids, decoder_input_ids=decoder_input_ids)

# now next decoder token can be sampled from decoder outputs
next_decoder_input_ids = torch.argmax(outputs.last_hidden_state, dim=-1)
decoder_input_ids = torch.cat([decoder_input_ids, next_decoder_input_ids], dim=-1)

# second call can reuse encoder outputs and continues auto-regressive language generation
# model.generate(...) does that automatically

outputs = model(encoder_outputs=outputs.decoder_past_key_values[0], decoder_input_ids=decoder_input_ids)

# The same holds true for TF

from transformers import TFT5Model, T5Tokenizer
import tensorflow as tf

model = TFT5Model.from_pretrained("t5-small")  # your model of choice here
tokenizer = T5Tokenizer.from_pretrained("t5-small")

input_ids = tokenizer("This sentence is encoded and only has to be passed once.", return_tensors="tf").input_ids
decoder_input_ids = tf.convert_to_tensor([[model.config.decoder_start_token_id]], dtype=tf.dtypes.int32)

# first call uses the start token id for T5 decoder
outputs = model(input_ids, decoder_input_ids=decoder_input_ids)

# now next decoder token can be sampled from decoder outputs
next_decoder_input_ids = tf.cast(tf.argmax(outputs[0], axis=-1), tf.dtypes.int32)
decoder_input_ids = tf.concat([decoder_input_ids, next_decoder_input_ids], axis=-1)

# second call can reuse encoder outputs and continues auto-regressive language generation
# model.generate(...) does that automatically

# inputs (encoder inputs) has to be defined because of keras convention => it's not needed though after first 
# pass, so set to None
model(None, encoder_outputs=outputs[1][0], decoder_input_ids=decoder_input_ids)

So this means that the decoder_input_ids should not be the same as the input_ids (encoder inputs) but should correspond to the auto-regressively generated text by the model, that starts with the decoder_start_token_id. Hope this makes it a bit clearer.

+1
patrickvonplaten on Jul 29, 2020

Ok, my apologies for not reading your issue carefully enough. For the inference API, that’s probably the reason though.

+1
julien-c on Jul 23, 2020

Please see https://github.com/huggingface/transformers/issues/5986#issuecomment-663090043

+1
julien-c on Jul 23, 2020

@misrasaurabh1 No we didn’t use the pytorch version.

+1
agemagician on Jul 18, 2020
Read more comments on GitHub
← transformers: Corrupted Relative Attention in T5 Decoder
transformers: Custom train/validation file not supported in run_qa.py →