vllm: Missing prometheus metrics in `0.3.0`
First of all, thanks for the great open source library!
The docs promise a few more additional metrics that I’m not seeing in vLLM 0.3.0, have these been removed? I.e. if I hit /metrics of the OpenAI API server for a deployed model… you’ll see no vllm:time_to_first_token_seconds or vllm:time_per_output_token_seconds or vllm:e2e_request_latency_seconds
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 1
- Comments: 17 (8 by maintainers)
I think a new release will be pushed soon -> https://github.com/vllm-project/vllm/issues/2859