bolt.diy: Models not loaded if Ollama is not on locakhost

Describe the bug

I have Ollama running on a different server (http://192.168.1.202:11434). The models do not show in the dropdown.

Link to the Bolt URL that caused the error

http://localhost:5173/

Steps to reproduce

  1. In env.local set the OLLAMA_API_BASE_URL to a server, not localhost (eg. http://192.168.1.202:11434)
  2. Start Bolt, the Ollama models do not show.

Expected behavior

The Ollama models should show in the dropdown.

Screen Recording / Screenshot

No response

Platform

  • OS: Linux (Ubuntu 24.04)
  • Browser: Firefox

Additional context

In constants.ts, the Ollama url ‘localhost’ is hard coded, this url should come from OLLAMA_API_BASE_URL.

async function getOllamaModels(): Promise<ModelInfo[]> {
  try {
    const response = await fetch(`http://localhost:11434/api/tags`);
    const data = await response.json();
    return data.models.map((model: any) => ({
      name: model.name,
      label: `${model.name} (${model.details.parameter_size})`,
      provider: 'Ollama',
    }));
  } catch (e) {
    return [];
  }
}

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15

Most upvoted comments

@NeoMatrixJR I think I figured out the same problem and for me it is still not working with https.

Access to fetch at 'https://192.168.1.106:8443/api/tags' from origin 'https://192.168.1.106' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

Tried so add the control-headers by myself, but didnt work 😦

I think most people here dont test with a server as well, instead just using it locally all.

This is easy to overcome.

In app/lib/.server/llm/model.ts Modify the import for the ollama-ai-provider import { createOllama } from 'ollama-ai-provider';

And then modify the ollama block

export function getOllamaModel(model: string) {
  const ollama = createOllama({
    baseURL: 'http://{ip here}:11434/api',
  });

  return ollama(model);
}

I have it running by hardcoding my Ollama url into node_modules/ollama-ai-provider/dist/index.js line 692

function createOllama(options = {}) {
  var _a;
  const baseURL = (_a = (0, import_provider_utils8.withoutTrailingSlash)(options.baseURL)) != null ? _a : "http://192.168.1.202:11434/api";
  const getHeaders = () => ({
    ...options.headers
  });

This change has Bolt working, sort of… It generates code in the main chat window but no code window or terminal window, but that’s another problem… Could be because I’m on Linux.

@fcools What did you edit? I tried editing .env.local with

OLLAMA_API_BASE_URL=http://server-ip:port

and constants.ts

const response = await fetch(`http://server-ip:port/api/tags`);

is there anything else that’s needed? Since this didnt work for me

What he means is that if you go to getOllamaModels() method and hard code your Ollama base URL in it, it only populates the Ollama models in it. Now when you pick a model, it attempts to use your localhost with that model name and fails with model not found error.