LyCORIS: extract_lycon error

hi, I had been using the extract_lycon scripts for a while with no problem. Recently, I started to getting the following error:

loading u-net: <All keys matched successfully>
loading vae: <All keys matched successfully>
loading text encoder: <All keys matched successfully>
loading u-net: <All keys matched successfully>
loading vae: <All keys matched successfully>
Traceback (most recent call last):
  File "D:\AI Arts\LyCORIS\tools\extract_locon.py", line 129, in <module>
    main()
  File "D:\AI Arts\LyCORIS\tools\extract_locon.py", line 98, in main
    db = load_models_from_stable_diffusion_checkpoint(args.is_v2, args.db_model)
  File "D:\AI Arts\LyCORIS\lycoris\kohya_model_utils.py", line 901, in load_models_from_stable_diffusion_checkpoint
    info = text_model.load_state_dict(converted_text_encoder_checkpoint)
  File "C:\Users\rugia\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for CLIPTextModel:
        Missing key(s) in state_dict: "text_model.embeddings.position_ids".

the cmd:

python ./tools/extract_locon.py --safetensors --device=cuda --mode="fixed" --linear_dim=64 --conv_dim=32 "xxx/v1-5-pruned-emaonly.ckpt" "input.ckpt" "output.safetensors"

I am not familiar with python, and I have already spent a whole day trying figure out what went wrong to no avail… 😦 Does anyone know what could cause this? Thanks

About this issue

Original URL
State: closed
Created a year ago
Comments: 29 (15 by maintainers)

Most upvoted comments

@rugia813 I can change my code to accept missing key and use a warning instead error Will update this recently

KohakuBlueleaf on Jul 20, 2023

well, it is kinda funny. now that it does not require position_ids, so the custom model should load, yes. but it is throwing the error Unexpected key(s) in state_dict: "text_model.embeddings.position_ids" when loading the base sd 1.5 model… so it is still not working, for the opposite reason. maybe this issue will be resolved in future version of transformers?

just curious, do you know what does the position_ids do, and does it play a role in model fine-tuning or inference?

position_ids just a torch.arange Originally implement as buffer so it appeared in states_dict

KohakuBlueleaf on Aug 1, 2023

@wshaf97 thx! I will try to make a utils to add position ids Just found that kohya have that for sd2 but not sd1

KohakuBlueleaf on Jul 28, 2023