alpa: Distributed Generation Script - benchmark_text_gen.py - CUDA related error

HI,

While running the ray and benchmark_text_gen.py script, I see an error related to CUDA (snapshot below). I have verified the CUDA, CUDNN versions and GPU info and everything looks fine. Any possible hints on this.

cuda: 11.3 cudnn: 8.2 gcc: 11.3.0

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 16 (8 by maintainers)

Most upvoted comments

@BalajiAJ This might be a simple namespace importing conflict. we (unsafely) use the examples as a Python package name. There are a lot of from examples.opt_serving.xxx import yyy. I guess your Python has some trouble resolving these import.

You can refer to this thread and try some similar fix?

In the recent release (v0.1.4), @merrymercy fixed this issue (see his reply above).

@zhisbug Thanks a lot for all the quick guidance and issue resolution steps. I am able to run the model without any errors. Thanks again. Appreciate your timely support!

BalajiAJ on Jul 6, 2022

@BalajiAJ See this PR for more information: https://github.com/alpa-projects/alpa/pull/589#event-6929171772

zhisbug on Jul 5, 2022