Your current environment
Using the latest main branch self-compiled LLM engine (v0.7.4.dev418+gd47807ba) to run the Gemma-3 model, only the Gemma-3-1b-it model outputs text content normally, while the 4/12/27B models output empty text.
### Running Environment
Ubuntu 20.04
Python 3.12
CUDA 12.0
CPU - 16 cores
Memory - 64GB
### Command executed:
vllm serve google/gemma-3-4b-it --tensor-parallel-size 2 --dtype=half --gpu-memory-utilization 0.9 --max-model-len=17370
🐛 Describe the bug
curl http://localhost:8000/v1/completions \
> -H "Content-Type: application/json" \
> -d '{
> "model": "google/gemma-3-4b-it",
> "prompt": "who are you?",
> "stream": true
> }'
data: {"id":"cmpl-39c393fdbc284c8fbc31de7db7c50770","object":"text_completion","created":1741949943,"model":"google/gemma-3-4b-it","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":null}
data: {"id":"cmpl-39c393fdbc284c8fbc31de7db7c50770","object":"text_completion","created":1741949943,"model":"google/gemma-3-4b-it","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":null}
data: {"id":"cmpl-39c393fdbc284c8fbc31de7db7c50770","object":"text_completion","created":1741949943,"model":"google/gemma-3-4b-it","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":null}
data: {"id":"cmpl-39c393fdbc284c8fbc31de7db7c50770","object":"text_completion","created":1741949943,"model":"google/gemma-3-4b-it","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":null}
data: {"id":"cmpl-39c393fdbc284c8fbc31de7db7c50770","object":"text_completion","created":1741949943,"model":"google/gemma-3-4b-it","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":null}
data: {"id":"cmpl-39c393fdbc284c8fbc31de7db7c50770","object":"text_completion","created":1741949943,"model":"google/gemma-3-4b-it","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":null}
data: {"id":"cmpl-39c393fdbc284c8fbc31de7db7c50770","object":"text_completion","created":1741949943,"model":"google/gemma-3-4b-it","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":null}
data: {"id":"cmpl-39c393fdbc284c8fbc31de7db7c50770","object":"text_completion","created":1741949943,"model":"google/gemma-3-4b-it","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":null}
data: {"id":"cmpl-39c393fdbc284c8fbc31de7db7c50770","object":"text_completion","created":1741949943,"model":"google/gemma-3-4b-it","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":null}
data: {"id":"cmpl-39c393fdbc284c8fbc31de7db7c50770","object":"text_completion","created":1741949943,"model":"google/gemma-3-4b-it","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":null}
data: {"id":"cmpl-39c393fdbc284c8fbc31de7db7c50770","object":"text_completion","created":1741949943,"model":"google/gemma-3-4b-it","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":null}
data: {"id":"cmpl-39c393fdbc284c8fbc31de7db7c50770","object":"text_completion","created":1741949943,"model":"google/gemma-3-4b-it","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":null}
data: {"id":"cmpl-39c393fdbc284c8fbc31de7db7c50770","object":"text_completion","created":1741949943,"model":"google/gemma-3-4b-it","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":null}
data: {"id":"cmpl-39c393fdbc284c8fbc31de7db7c50770","object":"text_completion","created":1741949943,"model":"google/gemma-3-4b-it","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":null}
data: {"id":"cmpl-39c393fdbc284c8fbc31de7db7c50770","object":"text_completion","created":1741949943,"model":"google/gemma-3-4b-it","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":null}
data: {"id":"cmpl-39c393fdbc284c8fbc31de7db7c50770","object":"text_completion","created":1741949943,"model":"google/gemma-3-4b-it","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":"length","stop_reason":null}],"usage":null}
Before submitting a new issue…
Looks like your GPU is too old. As a last resort you can try using float32 but it will increase memory usage
systemrole, notdeveloperYou need to also fix the dtype problem as I mentioned