vllm: openai.BadRequestError: Error code: 400 - {'object': 'error', 'message': 'max_tokens must be at least 1, got -186.', 'type': 'BadRequestError', 'param': None, 'code': 400}
Your current environment
The output of `python collect_env.py`
🐛 Describe the bug
ERROR: Exception in ASGI application Traceback (most recent call last): File “/usr/local/lib/python3.8/dist-packages/uvicorn/protocols/http/httptools_impl.py”, line 426, in run_asgi result = await app( # type: ignore[func-returns-value] File “/usr/local/lib/python3.8/dist-packages/uvicorn/middleware/proxy_headers.py”, line 84, in call return await self.app(scope, receive, send) File “/usr/local/lib/python3.8/dist-packages/fastapi/applications.py”, line 1106, in call await super().call(scope, receive, send) File “/usr/local/lib/python3.8/dist-packages/starlette/applications.py”, line 122, in call await self.middleware_stack(scope, receive, send) File “/usr/local/lib/python3.8/dist-packages/starlette/middleware/errors.py”, line 184, in call raise exc File “/usr/local/lib/python3.8/dist-packages/starlette/middleware/errors.py”, line 162, in call await self.app(scope, receive, _send) File “/usr/local/lib/python3.8/dist-packages/starlette/middleware/cors.py”, line 91, in call await self.simple_response(scope, receive, send, request_headers=headers) File “/usr/local/lib/python3.8/dist-packages/starlette/middleware/cors.py”, line 146, in simple_response await self.app(scope, receive, send) File “/usr/local/lib/python3.8/dist-packages/starlette/middleware/exceptions.py”, line 79, in call raise exc File “/usr/local/lib/python3.8/dist-packages/starlette/middleware/exceptions.py”, line 68, in call await self.app(scope, receive, sender) File “/usr/local/lib/python3.8/dist-packages/fastapi/middleware/asyncexitstack.py”, line 20, in call raise e File “/usr/local/lib/python3.8/dist-packages/fastapi/middleware/asyncexitstack.py”, line 17, in call await self.app(scope, receive, send) File “/usr/local/lib/python3.8/dist-packages/starlette/routing.py”, line 718, in call await route.handle(scope, receive, send) File “/usr/local/lib/python3.8/dist-packages/starlette/routing.py”, line 276, in handle await self.app(scope, receive, send) File “/usr/local/lib/python3.8/dist-packages/starlette/routing.py”, line 69, in app await response(scope, receive, send) File “/usr/local/lib/python3.8/dist-packages/starlette/responses.py”, line 277, in call await wrap(partial(self.listen_for_disconnect, receive)) File “/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py”, line 597, in aexit raise exceptions[0] File “/usr/local/lib/python3.8/dist-packages/starlette/responses.py”, line 273, in wrap await func() File “/usr/local/lib/python3.8/dist-packages/starlette/responses.py”, line 262, in stream_response async for chunk in self.body_iterator: File “/usr/local/lib/python3.8/dist-packages/starlette/concurrency.py”, line 63, in iterate_in_threadpool yield await anyio.to_thread.run_sync(_next, iterator) File “/usr/local/lib/python3.8/dist-packages/anyio/to_thread.py”, line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File “/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py”, line 877, in run_sync_in_worker_thread return await future File “/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py”, line 807, in run result = context.run(func, *args) File “/usr/local/lib/python3.8/dist-packages/starlette/concurrency.py”, line 53, in _next return next(iterator) File “api_server.py”, line 99, in stream chat_response = client.chat.completions.create( File “/usr/local/lib/python3.8/dist-packages/openai/_utils/_utils.py”, line 275, in wrapper return func(*args, **kwargs) File “/usr/local/lib/python3.8/dist-packages/openai/resources/chat/completions.py”, line 667, in create return self._post( File “/usr/local/lib/python3.8/dist-packages/openai/_base_client.py”, line 1213, in post return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)) File “/usr/local/lib/python3.8/dist-packages/openai/_base_client.py”, line 902, in request return self._request( File “/usr/local/lib/python3.8/dist-packages/openai/_base_client.py”, line 993, in _request raise self._make_status_error_from_response(err.response) from None openai.BadRequestError: Error code: 400 - {‘object’: ‘error’, ‘message’: ‘max_tokens must be at least 1, got -186.’, ‘type’: ‘BadRequestError’, ‘param’: None, ‘code’: 400}
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 1
- Comments: 17 (10 by maintainers)
We’ll re-open this issue if more information becomes available.
Please open a new issue and provide more detailed information.