Nuitka: File "importlib.py", line 126, in import_module ModuleNotFoundError: No module named 'huggingface_hub.repocard'

Upon running the main.exe executable

File "importlib.py", line 126, in import_module
ModuleNotFoundError: No module named 'huggingface_hub.repocard'

Python 3.10.3 Nuitka version 1.7.9

Using Pip3

from transformers import LlamaTokenizer, TextGenerationPipeline
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
import logging

logging.basicConfig(
    format="%(asctime)s %(levelname)s [%(name)s] %(message)s", level=logging.INFO, datefmt="%Y-%m-%d %H:%M:%S"
)
# if need files lmk I will provide links.
quantized_model_dir = r"C:\Users\Tensor\Desktop\TestSentencePiece\Pygmalion-7b-4bit-GPTQ-Safetensors"
model_dir = r"C:\Users\Tensor\Desktop\TestSentencePiece\Pygmalion-7b-4bit-GPTQ-Safetensors"
vocab_file = r"C:\Users\Tensor\Desktop\TestSentencePiece\Pygmalion-7b-4bit-GPTQ-Safetensors\tokenizer.model"
tokenizer_file = r"C:\Users\Tensor\Desktop\TestSentencePiece\Pygmalion-7b-4bit-GPTQ-Safetensors\tokenizer.json"

tokenizer = LlamaTokenizer(vocab_file=vocab_file,tokenizer_file=tokenizer_file,use_fast=True)

examples = [
    tokenizer(
        "auto-gptq is an easy-to-use model quantization library with user-friendly apis, based on GPTQ algorithm."
    )
]
print(tokenizer("auto_gptq is", return_tensors="pt"))

quantize_config = BaseQuantizeConfig(
    bits=4,  # quantize model to 4-bit
    group_size=-1,  # it is recommended to set the value to 128
    desc_act=True,  # set to False can significantly speed up inference but the perplexity may slightly bad
)

model = AutoGPTQForCausalLM.from_quantized(model_name_or_path=model_dir, device="cuda:0",quantize_config=quantize_config,use_safetensors=True,model_basename="Pygmalion-7B-GPTQ-4bit-act-order")

#inference with model.generate
print(tokenizer.decode(model.generate(max_new_tokens=100,**tokenizer("auto_gptq is", return_tensors="pt").to(model.device))[0]))

python -m nuitka .\Main.py --follow-imports --show-progress --show-scons --show-modules --include-package=setuptools --noinclude-dask-mode=nofollow --noinclude-numba-mode=nofollow --lto=no --standalone --report="gptqfail.txt"

About this issue

Original URL
State: closed
Created a year ago
Reactions: 1
Comments: 24 (13 by maintainers)

Most upvoted comments

So, this is on current factory and staging (commercial variant of factory). For datasets specifically, a bit of configuration was added, so it works now as well. The compilation of it is terribly long unfortunately due to the large amounts of modules being pulled in.

For huggingface_hub itself that is now working in that context. Let me know if any of the reproducers given here still fail and in what way, potentially in new issues though, since I want to close this for that issue alone.

kayhayen on Sep 27, 2023

Ok, got that figured out, need to try that on the example and in my CI, but it should be good. For a moment, when changing the attribute lookup of the hard imported module to a hard imported name lookup through a factory function, it wasn’t annotating the exception exit, and a try/except as is done for imports, to release temps, that then are eliminated later, was noticing that. So there is hope here. I want to also add it to Nuitka-Watch, however this is clearly 1.9 material since it depends all on the lazy loader changes and import enhancements needed to get there. But I will want to release 1.9 soon anyway.

kayhayen on Sep 22, 2023

I have been adding support the the .pyi based attach stub recently, and it’s not yet finished, but once it is, I will turn to do something for huggingface_hub as well, which seems to force inline the result of the .pyi file. I do not have a workaround yet, but this is what I am working on, give it a few days more, then this will work as well.

kayhayen on Sep 3, 2023

This does it: from huggingface_hub import DatasetCard compiles and only imports a single module, which is a bit on the low side, but very nice for reproducing it. So far I had multiple thousand test cases only.

I am trying to distill and idea out of things, how a plugin could support this:

# Lazy loader vendored from https://github.com/scientific-python/lazy_loader

We either read _SUBMOD_ATTRS and re-implement their __getattr__ mapping to importlib.import_module which seems kind of doable. I think I have encountered these in skimage and sklearn, but there the .pyi apprach was taken, and manually written out. But if we manage to support this in one plugin, it would be super nice.

For Nuitka however, it’s a probably that the import name nodes, and import nodes for that matter, would have to not change the imported name (although they could, but with multiple named imports, it could get messy), but the module they cause a follow to needs to be hooked by plugins.

kayhayen on Aug 31, 2023

Part of the 1.9 release I just made.

kayhayen on Nov 23, 2023

This is now on develop, about to announce 1.8 and then closing down on 1.9 soon after I hope, not sure how much of 3.12 work I will do before though, lets see.

kayhayen on Sep 30, 2023

Trying the new example right now, finally catching up on this issue.

kayhayen on Aug 30, 2023