ggml: gpt2 error

Hi I have fine tuned gpt2 with huggingface and pytorch. Ran the gpt-2/convert-h5-to-ggml.py but when I try to run the model I get this

main: seed = 1689063614
gpt2_model_load: loading model from 'models/gpt2-train/ggml-model.bin'
gpt2_model_load: n_vocab = 50259
gpt2_model_load: n_ctx   = 1024
gpt2_model_load: n_embd  = 768
gpt2_model_load: n_head  = 12
gpt2_model_load: n_layer = 12
gpt2_model_load: ftype   = 1
gpt2_model_load: qntvr   = 0
gpt2_model_load: ggml tensor size = 240 bytes
gpt2_model_load: ggml ctx size = 384.78 MB
gpt2_model_load: memory size =    72.00 MB, n_mem = 12288
gpt2_model_load: tensor 'model/wpe' has wrong size in model file: got 3145728, expected 1572864
main: failed to load model from 'models/gpt2-train/ggml-model.bin'

About this issue

Original URL
State: closed
Created a year ago
Comments: 21 (6 by maintainers)

Commits related to this issue

specify build type for ctest on windows (#371) — committed to CCLDArjun/ggml by Green-Sky a year ago

Most upvoted comments

I had to do this

tokenizer = GPT2Tokenizer.from_pretrained(dir)

for i in range(hparams["vocab_size"]):
    text = tokenizer.decode([i]).encode('utf-8')
    fout.write(struct.pack("i", len(text)))
    fout.write(text)

instead of

for key in encoder:
    text = bytearray([byte_decoder[c] for c in key])
    fout.write(struct.pack("i", len(text)))
    fout.write(text)

for key in encoder_added:
    text = bytearray([byte_decoder[c] for c in key])
    fout.write(struct.pack("i", len(text)))
    fout.write(text)

stellanhaglund on Jul 17, 2023