pyreft: [P1] [Error] can not use bfloat16 and TypeError: Object of type type is not JSON serializable
Thanks for your wonderful model, but I have got some problems.
- can not use bfloat16.
File "/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 1211, in forward
outputs = self.model(
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 992, in forward
causal_mask = self._update_causal_mask(attention_mask, inputs_embeds, cache_position, past_seen_tokens)
File "/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 1095, in _update_causal_mask
causal_mask = torch.triu(causal_mask, diagonal=1)
RuntimeError: "triu_tril_cuda_template" not implemented for 'BFloat16'
- I run the
main_demo.ipynb, but got the error:
Traceback (most recent call last):
File "/mnt/geogpt-gpfs/pyreft/inference.py", line 61, in <module>
_ = trainer.train()
File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 1859, in train
return inner_training_loop(
File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 2116, in _inner_training_loop
self.control = self.callback_handler.on_train_begin(args, self.state, self.control)
File "/opt/conda/lib/python3.10/site-packages/transformers/trainer_callback.py", line 371, in on_train_begin
return self.call_event("on_train_begin", args, state, control)
File "/opt/conda/lib/python3.10/site-packages/transformers/trainer_callback.py", line 415, in call_event
result = getattr(callback, event)(
File "/opt/conda/lib/python3.10/site-packages/transformers/integrations/integration_utils.py", line 636, in on_train_begin
model_config_json = model.config.to_json_string()
File "/opt/conda/lib/python3.10/site-packages/transformers/configuration_utils.py", line 938, in to_json_string
return json.dumps(config_dict, indent=2, sort_keys=True) + "\n"
File "/opt/conda/lib/python3.10/json/__init__.py", line 238, in dumps
**kw).encode(obj)
File "/opt/conda/lib/python3.10/json/encoder.py", line 201, in encode
chunks = list(chunks)
File "/opt/conda/lib/python3.10/json/encoder.py", line 431, in _iterencode
yield from _iterencode_dict(o, _current_indent_level)
File "/opt/conda/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict
yield from chunks
File "/opt/conda/lib/python3.10/json/encoder.py", line 325, in _iterencode_list
yield from chunks
File "/opt/conda/lib/python3.10/json/encoder.py", line 438, in _iterencode
o = _default(o)
File "/opt/conda/lib/python3.10/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type type is not JSON serializable
I find the issues 69, but I use the main_demo.ipynb, so it does not work for me.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 22
Thank you for your reply. Now I understand how to set Gradient_cccumulation_steps and batch size.
Sorry about the confusion - but I think there is nothing being changed here, since for hyperparameter, what matters is the effective batch size, which is batch size (bounded by the GPU MEM) times the gradient accumulation step (you can set whatever you want to match the effective batch size). I think for the script you sent earlier and my settings, they all have an effective batch size of 32.
Note in the paper, we only report effective batch size, not per device batch size.
Hope these help. Thanks.