Skip to main content

LLM Locally

To put an LLM locally (which works similar to ChatGPT) we need two components - Ollama and Open WebUI. In this process we need to use both Ollama and Open WebUI, that is why I made a separate file for detailing the process of making LLM work locally.

Ollama manages the downloading, loading/unloading, deleting, making new models and more of LLMs gracefully.

Open WebUI is the user interface similar to ChatGPT but with much more features hidden inside.


Change Ollama Models Directory in Ubuntu Linux

1.1. Copy installed ollama models from one disk to another

cp -r <curret/path/to/models> <destination/to/store/model/files>
  • Example command: cp -r /mnt/INTERNAL/Ollama/.ollama /home/appuser/Ollama/.ollama

1.2. Get it from any other system (Linux) in your network

See Secure Copy (SCP)


sudo systemctl stop ollama

3. Get user and group owner names of your destination folder

Go to the folder where you're going to keep ollama files

cd /home/appuser/Ollama/.ollama (Example)

Check details of all files

ls -lk -a

4. Create and Override Ollama service using config file

sudo mkdir -p /etc/systemd/system/ollama.service.d; sudo nano /etc/systemd/system/ollama.service.d/override.conf

Add the path, user, and group of new location in the file

Example

[Service]
Environment="OLLAMA_MODELS=/home/appuser/ollama/.ollama/models" # change this path to your path
User=appuser # change this path to your User
Group=appuser # change this path to your Group

5. Restart System Daemon (this is not a system reboot) and Ollama

sudo systemctl daemon-reload

sudo systemctl restart ollama

6. Check the working

ollama list

7. Try downloading a model and check if it is saving to the new location

ollama run <model_name>
  • Get new model name from ollama.com/models


Optionals

Check details of all directories

ls -ld -a

Status of Ollama

To see status of ollama (whether it is running, stopped, exited, error)

sudo systemctl status ollama

Error logs of Ollama

To see the error logs of ollama (limiting to the latest 50 lines)

journalctl -u ollama -n 50

Ollama and Open WebUI from separate computers

hidden