Models keep unlading in OLLAMA

[hysel]

Hi

I am using multiple models in my local OLLAMA setup.

When I use LocalPilot, the models load with a very short TTL, causing interactions to be slow while the server unloads and reloads the models.

Is there a way to prevent this from happening?

thanks

Itamar

[JalpeshVadgama]

@hysel can you please provide the configuration of your laptop based on that I will provide you best models that will work on the same.

[hysel]

@hysel can you please provide the configuration of your laptop based on that I will provide you best models that will work on the same.

I am currently using a home server equipped with 2 Nvidia V100 32GB GPUs and 1 RTX 5000 16GB GPU.

[skyfox69]

Hi,

I'm observing the same behavior on my computer. Ollama is running in a Docker container on an RTX A5000 (24GB) and the model freehuntx/qwen3-coder:14b:

> docker exec -it ollama ollama ps
NAME                      ID             SIZE     PROCESSOR    CONTEXT    UNTIL
freehuntx/qwen3-coder:14b c4df7399b89e   14 GB    100% GPU     32768      47 minutes from now

One of the reasons for the restarts is that the context size is dynamically calculated by LocalPilot. With every change to the context size, Ollama stops the running model and restarts it with the new context size. This wastes a valuable 10 minutes on my system, during which I can't work properly.

I rebuilt the LocalPilot plugin locally with modified code and, in addition to setting the fixed context size to 32768 bytes, I also increased the keep_alive value from 5 to 50 minutes. This keeps the model loaded for longer, as my AI requests are often more than 5 minutes apart.

Perhaps there's a way to adjust these values via configuration or force fixed values – I just haven't found the setting?

[joshgivenpi]

Hi, I am the same issue. It would be nice to have some configuration in LocalPilot for this. I am running Ollama on a separate server that has plenty of VRAM with 128GB of unified memory so I have Ollama set to leave the models running forever and I have the models set to what context I want to use, but LocalPilot kills all that. I will do what @skyfox69 did in the meantime.