alpa: Distributed Generation Script - benchmark_text_gen.py - CUDA related error

HI,

While running the ray and benchmark_text_gen.py script, I see an error related to CUDA (snapshot below). I have verified the CUDA, CUDNN versions and GPU info and everything looks fine. Any possible hints on this.

cuda: 11.3 cudnn: 8.2 gcc: 11.3.0

image

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 16 (8 by maintainers)

Most upvoted comments

@BalajiAJ This might be a simple namespace importing conflict. we (unsafely) use the examples as a Python package name. There are a lot of from examples.opt_serving.xxx import yyy. I guess your Python has some trouble resolving these import.

You can refer to this thread and try some similar fix?

In the recent release (v0.1.4), @merrymercy fixed this issue (see his reply above).

@zhisbug Thanks a lot for all the quick guidance and issue resolution steps. I am able to run the model without any errors. Thanks again. Appreciate your timely support!