bolt.diy: problem with ollama models
Describe the bug
ollama is corrctly configured, running, and no chat probleme were reported using the console prompt ollama.api wroking correctly with packet sender ollama is selected in Bolt ai chat web pages one of my ollama (qwen2.5-coder:0.5b (494.03M)) model is selected (!!! i have same probleme with other models) but i have the following error on my terminal
APICallError [AI_APICallError]: Not Found
at C:\Users\HP\source\repos\LLMs\Bolt\node_modules\.pnpm\@ai-sdk+provider-utils@1.0.20_zod@3.23.8\node_modules\@ai-sdk\provider-utils\src\response-handler.ts:72:16
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at postToApi (C:\Users\HP\source\repos\LLMs\Bolt\node_modules\.pnpm\@ai-sdk+provider-utils@1.0.20_zod@3.23.8\node_modules\@ai-sdk\provider-utils\src\post-to-api.ts:81:28)
at OllamaChatLanguageModel.doStream (C:\Users\HP\source\repos\LLMs\Bolt\node_modules\.pnpm\ollama-ai-provider@0.15.2_zod@3.23.8\node_modules\ollama-ai-provider\src\ollama-chat-language-model.ts:230:50)
at fn (file:///C:/Users/HP/source/repos/LLMs/Bolt/node_modules/.pnpm/ai@3.4.33_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.7.2__zod@3.23.8/node_modules/ai/core/generate-text/stream-text.ts:378:23)
at file:///C:/Users/HP/source/repos/LLMs/Bolt/node_modules/.pnpm/ai@3.4.33_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.7.2__zod@3.23.8/node_modules/ai/core/telemetry/record-span.ts:18:22
at _retryWithExponentialBackoff (file:///C:/Users/HP/source/repos/LLMs/Bolt/node_modules/.pnpm/ai@3.4.33_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.7.2__zod@3.23.8/node_modules/ai/util/retry-with-exponential-backoff.ts:37:12)
at startStep (file:///C:/Users/HP/source/repos/LLMs/Bolt/node_modules/.pnpm/ai@3.4.33_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.7.2__zod@3.23.8/node_modules/ai/core/generate-text/stream-text.ts:333:13)
at fn (file:///C:/Users/HP/source/repos/LLMs/Bolt/node_modules/.pnpm/ai@3.4.33_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.7.2__zod@3.23.8/node_modules/ai/core/generate-text/stream-text.ts:414:11)
at file:///C:/Users/HP/source/repos/LLMs/Bolt/node_modules/.pnpm/ai@3.4.33_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.7.2__zod@3.23.8/node_modules/ai/core/telemetry/record-span.ts:18:22
at chatAction (C:/Users/HP/source/repos/LLMs/Bolt/app/routes/api.chat.ts:65:20)
at Object.callRouteAction (C:\Users\HP\source\repos\LLMs\Bolt\node_modules\.pnpm\@remix-run+server-runtime@2.15.0_typescript@5.7.2\node_modules\@remix-run\server-runtime\dist\data.js:36:16)
at C:\Users\HP\source\repos\LLMs\Bolt\node_modules\.pnpm\@remix-run+router@1.21.0\node_modules\@remix-run\router\router.ts:4899:19
at callLoaderOrAction (C:\Users\HP\source\repos\LLMs\Bolt\node_modules\.pnpm\@remix-run+router@1.21.0\node_modules\@remix-run\router\router.ts:4963:16)
at async Promise.all (index 0)
at defaultDataStrategy (C:\Users\HP\source\repos\LLMs\Bolt\node_modules\.pnpm\@remix-run+router@1.21.0\node_modules\@remix-run\router\router.ts:4772:17)
at callDataStrategyImpl (C:\Users\HP\source\repos\LLMs\Bolt\node_modules\.pnpm\@remix-run+router@1.21.0\node_modules\@remix-run\router\router.ts:4835:17)
at callDataStrategy (C:\Users\HP\source\repos\LLMs\Bolt\node_modules\.pnpm\@remix-run+router@1.21.0\node_modules\@remix-run\router\router.ts:3992:19)
at submit (C:\Users\HP\source\repos\LLMs\Bolt\node_modules\.pnpm\@remix-run+router@1.21.0\node_modules\@remix-run\router\router.ts:3755:21)
at queryImpl (C:\Users\HP\source\repos\LLMs\Bolt\node_modules\.pnpm\@remix-run+router@1.21.0\node_modules\@remix-run\router\router.ts:3684:22)
at Object.queryRoute (C:\Users\HP\source\repos\LLMs\Bolt\node_modules\.pnpm\@remix-run+router@1.21.0\node_modules\@remix-run\router\router.ts:3629:18)
at handleResourceRequest (C:\Users\HP\source\repos\LLMs\Bolt\node_modules\.pnpm\@remix-run+server-runtime@2.15.0_typescript@5.7.2\node_modules\@remix-run\server-runtime\dist\server.js:402:20)
at requestHandler (C:\Users\HP\source\repos\LLMs\Bolt\node_modules\.pnpm\@remix-run+server-runtime@2.15.0_typescript@5.7.2\node_modules\@remix-run\server-runtime\dist\server.js:156:18)
at C:\Users\HP\source\repos\LLMs\Bolt\node_modules\.pnpm\@remix-run+dev@2.15.0_@remix-run+react@2.15.0_react-dom@18.3.1_react@18.3.1__react@18.3.1_typ_zyxju6yjkqxopc2lqyhhptpywm\node_modules\@remix-run\dev\dist\vite\cloudflare-proxy-plugin.js:70:25 {
cause: undefined,
url: 'http://127.0.0.1:11434/api/chat',
requestBodyValues: {
format: undefined,
model: 'gpt-4o-mini',
options: { num_ctx: 32768, num_predict: 8000, temperature: 0 },
messages: [ [Object], [Object] ],
tools: undefined
},
statusCode: 404,
responseHeaders: {
'content-length': '65',
'content-type': 'application/json; charset=utf-8',
date: 'Thu, 05 Dec 2024 21:01:25 GMT'
},
responseBody: '{"error":"model \\"gpt-4o-mini\\" not found, try pulling it first"}',
isRetryable: false,
data: undefined,
[Symbol(vercel.ai.error)]: true,
[Symbol(vercel.ai.error.AI_APICallError)]: true
}
Link to the Bolt URL that caused the error
local
Steps to reproduce
- download bolt from this repo
- extract it to your location
$> pnpm install- download ollama
- $>ollama --version
$> ollama run qwen2.5-coder:0.5b- go to bolt file .env.local and set
OLLAMA_API_BASE_URL=http://127.0.0.1:11434
Expected behavior
no error
Screen Recording / Screenshot
Platform
- OS:W10
- Browser: Chrome
- npm version : 10.5.2
- pnpm version : 9.14.4
- ollama version : 0.4.7
Provider Used
ollama
Model Used
qwen2.5-coder:0.5b (494.03M)
Additional context
No response
About this issue
- Original URL
- State: open
- Created 2 years ago
- Reactions: 2
- Comments: 22
Ollama supports CORS for 127.0.0.1 and 0.0.0.0 by default. Additional origins can be configured using the
OLLAMA_ORIGINSenvironment variable. For details, refer to How to Handle CORS Settings in OLLAMA: A Comprehensive Guide.I´ve you dont use the latest main branch and build your docker-container again, than this is the problem, cause there was a bug which was fixed last week for the .env within docker 😉
(at least I guess this is also the problem then here)
See my docker-compose file what is working for me: https://github.com/leex279/bolt-diy-full-stack/blob/main/docker-compose-nvidia.yml
In my case was a memory problem.
Steps to debug:
If you see this, you are getting an error 500 for overloading the memory:
Once I have used a lower model, Bolt.diy have started to receive responses from Ollama
Hope it helps
Are you running bolt in docker? You can try changing
localhosttohost.docker.internal, or any IP in task manager, and make sure ollama listens on0.0.0.0I had a version ( already ) cloned and configured 2 months ago on my windows 11 computer. Today I updated this version ( git pull origin main ), I did run “npm install” but now I have the same issue ( like everybody here ), I don’t see my list of Ollama models anymore.
I noticed that you needed to set the API Ollama url in my .env.local : OLLAMA_API_BASE_URL=http://localhost:11434
It restarted the server, but it didn’t make any difference… I also stopped my docker container, removed it and relaunched the docker command, still same issue
I’m using my local LLM models with this project, but it seems not to work anymore… 😦
I tested my Ollama url with curl and it works perfectly And still NodeJS 18.20.3 used in bolt.diy and on my computer is installed NodeJS 22.11.0
As long as a request from the BOLT server to the OLLAMA server fails (including the case when the OLLAMA server is not running during the BOLT server startup, as well as for all other model providers), the server will ignore the model settings in the browser and directly use the default model
claude-3-5-sonnet-latestfor the conversation.My PATCH fixed this issue, ensuring that the server always follows the model settings from the browser when making requests. However, if the browser also cannot access the OLLAMA server, the OLLAMA model cannot be selected. This is an exceptional case, and the server should ideally throw an exception. But for compatibility, I chose to retain the error behavior where the server defaults to selecting
claude-3-5-sonnet-latest.Since the main branch is focused on making changes to unimportant UI elements, I didn’t have time to follow up with further commits, so the PR has been closed.
I can confirm that today this is still happening even with the latest commits.