langchain: Timeout Error OpenAI

I am facing a Warning similar to the one described here #3005

WARNING:langchain.embeddings.openai:Retrying langchain.embeddings.openai.embed_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised Timeout: Request timed out: HTTPSConnectionPool(host='api.openai.com', port=443): Read timed out. (read timeout=600). It just keeps retrying. How do I get around this?

About this issue

Original URL
State: open
Created a year ago
Reactions: 8
Comments: 30 (3 by maintainers)

Commits related to this issue

Increase `request_timeout` on ChatOpenAI (#3910) With longer context and completions, gpt-3.5-turbo and, especially, gpt-4, will more times than not take > 60seconds to respond. Based on some oth... — committed to langchain-ai/langchain by timothyasp a year ago

Most upvoted comments

@dnrico1 @La1c @gabacode When are y’all getting the error? For instance, I am getting it through my websocket app deployed on Azure (it’s a chatbot application). Weirdly enough, I don’t face it when I run the application locally.

+11

shreyabhadwal on Apr 28, 2023

Hi HaochenQ

May be deploying your solution in a virtual machine might solve your problem. When I moved from AWS Lambda to EC2, the problem got resolved

Thank you@jpsmartbots, I tried to deploy my container with an Azure VM, but the issue persists.

For those of you who are facing 504 gateway timeout issues Retrying langchain.embeddings.openai.embed_with_retry.<locals>._embed_with_retry in 4.0 seconds as it raised Timeout: Request timed out: HTTPSConnectionPool(host='api.openai.com', port=443): Read timed out. (read timeout=600). with Azure App Services, the issue is because the default HTTP timeout of Azure App Service 230/240 seconds while the default timeout of OpenAI APIs is 600 seconds. Before langchian hear back from OpenAI and do a retry, Azure returns an error and our app appears down. You can use request_timeout - OpenAIEmbeddings(request_timeout=30) to avoid time timeout from Azure side and somehow the retry call to OpenAI from langchain can always work.

Not sure why the langchian call to the OpenAI after a period of inactivity will fail and cause a timeout.

HaochenQ on Aug 31, 2023

Hey all, I believe this being fixed in the openai-python client should also help with this issue, and with generations:

https://github.com/openai/openai-python/pull/387

The async and sync request_timeouts are NOT identical.

ShantanuNair on Sep 20, 2023

Same issue here. Works for a bit and then starts timing out. I just can’t nail down when it happens and why. There doesn’t seem to be a rhyme or reason. Seems to happen a lot more on production (gcp) than locally. Although it happens on both. Seems to happen with short sentences more than long ones. Although not exclusively. It happens a LOT though. Like 1 out of 4 requests.

bigrig2212 on Jun 14, 2023