vllm: IP Bind Error on v0.5.0.post1
Using the same forum.As I am encountering the same error on 0.5.0.post1 version.
I am launching openai.api server using the following command and consistently seeing the IP-Bind error. It works on 0.4.2 version which is my current image. I am looking to upgrade to newer vllm version to access features/fixes that were failing on earlier version.
CMD="python -u -m vllm.entrypoints.openai.api_server \
-- host 0.0.0.0 \
--port $VLLM_PORT \
--model $launch_model \
--tensor-parallel-size $NUM_GPUS \
--download-dir /data"
Error:
ERROR: [Errno 98] error while attempting to bind on address ('0.0.0.0', 9000): address already in use
Appreciate any help with this.
_Originally posted by @pchunduru10 in https://github.com/vllm-project/vllm/issues/1141#issuecomment-2192256236_
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 15 (8 by maintainers)
In case someone else runs into this on Kubernetes:
Don’t name the service targeting your vLLM pods
vllmas this is cause variables like VLLM_PORT to be set by the Kubelet https://kubernetes.io/docs/concepts/services-networking/service/#environment-variablesand this variable apparently is used by VLLM itself.
Thank you @youkaichao. This saves my day.
@frittentheke thanks for the report, updated the doc at https://github.com/vllm-project/vllm/pull/5916
That’s too much maintenance burden. I prefer to warn users that they should use another prefix to avoid conflict.
oh,
VLLM_PORTis actually not the port the server will listen on, it is another environment variable that determines the port vllm uses for internal communication. see https://docs.vllm.ai/en/latest/serving/env_vars.html for details.You need to change the env name.