Ok, I have an old server, and old
As you can see, this is well below what most people use. For real work, I'd want at least one GPU, and I guess I'd want my disks set up as a RAID-0.
Why Devuan and not Debian? SysV instead of System-D. Reliability. Personal preference, but it means when a system (like Ollama) is set up for system-d, I have to do (a bit more) work.
First, make sure curl and cat-certificates installed, then install Ollama, test.
sudo apt update sudo apt install curl ca-certificates # next step takes a while curl -fsSL https://ollama.com/install.sh | sh # test it. Don't worry if it fails, but this should work ollama --version ollama serve # In another terminal, try to connect curl http://127.0.01:11434 # in the other terminal still, install llama3.2 ollama pull llama3.2:3b ollama run llama3.2:3b # /bye will exit # in main terminal, ^c will exit the ollama service
At this point, you have a working server. YAY. For why I chose Ollama and llama3.3:3B, read the bottom
I'm assuming the Ollama install would set up a service under System-D (from what I read), but I'm using Sys-V, so I need to build an init script and configuration file manually. Ignore this whole section if you are not using Devuan.
Create the file ollama in /etc/init.d. So
# use your favorite editor instead of 'edit' edit /etc/init.d/ollama
Now, copy/paste the following in there.
#!/bin/sh ### BEGIN INIT INFO # Provides: ollama # Required-Start: $remote_fs $network # Required-Stop: $remote_fs $network # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Ollama local LLM server ### END INIT INFO PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin DAEMON=/usr/local/bin/ollama NAME=ollama USER=ollama GROUP=ollama RUNDIR=/run/$NAME PIDFILE=$RUNDIR/$NAME.pid . /lib/lsb/init-functions [ -f /etc/default/ollama ] && . /etc/default/ollama : "${OLLAMA_HOST:=127.0.0.1:11434}" : "${OLLAMA_MODELS:=/var/lib/ollama/models}" : "${OLLAMA_LOG:=/var/log/ollama.log}" ensure_dirs() { install -d -o "$USER" -g "$GROUP" -m 0755 "$RUNDIR" install -d -o "$USER" -g "$GROUP" -m 0755 /var/lib/ollama install -d -o "$USER" -g "$GROUP" -m 0755 "$OLLAMA_MODELS" install -d -m 0755 /var/log touch "$OLLAMA_LOG" chown "$USER:$GROUP" "$OLLAMA_LOG" } start() { log_daemon_msg "Starting $NAME" ensure_dirs start-stop-daemon --start --background --make-pidfile \ --pidfile "$PIDFILE" \ --chuid "$USER:$GROUP" \ --startas /bin/sh -- -c \ "exec env OLLAMA_HOST=\"$OLLAMA_HOST\" OLLAMA_MODELS=\"$OLLAMA_MODELS\" \"$DAEMON\" serve >>\"$OLLAMA_LOG\" 2>&1" RET=$? if [ $RET -eq 0 ]; then sleep 2 if kill -0 "$(cat "$PIDFILE")" 2>/dev/null; then log_end_msg 0 else log_end_msg 1 fi else log_end_msg $RET fi } stop() { log_daemon_msg "Stopping $NAME" start-stop-daemon --stop --pidfile "$PIDFILE" --retry 5 RET=$? rm -f "$PIDFILE" log_end_msg $RET } status() { if [ -f "$PIDFILE" ] && kill -0 "$(cat "$PIDFILE")" 2>/dev/null; then echo "$NAME is running" exit 0 else echo "$NAME is not running" exit 3 fi } case "$1" in start) start ;; stop) stop ;; restart) stop; start ;; status) status ;; *) echo "Usage: $0 {start|stop|restart|status}" exit 1 ;; esac exit 0
Add a user to your system, then set the init levels. Do not attempt to start the service yet.
sudo adduser --system --group --home /var/lib/ollama ollama sudo chmod 755 /etc/init.d/ollama sudo update-rc.d ollama defaults
Finally, on Devuan, some base configurations are done in /etc/default/servicename, so let's create the file /etc/default/ollama
Note: The default is for the service to be available only on localhost. However, I am building this in a LAN, and want my other machines to be able to access it, so I chose to make it available to the entire LAN. You'll see the OLLAMA_HOST=10.10.10.10:11434 entry in the sample config. That assumes my server is running on IP 10.10.10.10 and making itself available to anyone on the subnet. If you want it only accessible from localhost, change it to 127.0.0.1, ie OLLAMA_HOST=127.0.0.1:11434
edit /etc/default/ollama
OLLAMA_HOST=10.10.10.10:11434 OLLAMA_MODELS=/var/lib/ollama/models OLLAMA_LOG=/var/log/ollama.log
Finally, it is all set up, so start it up
sudo service ollama start
If you set OLLAMA_HOST to anything but 127.0.0.1, you will need to send each cli command by setting an environmental variable first. So, for example, to do a simple ollama list, you would need.
# works for localhost ollama list # works in my setup (ie, tied to an external IP OLLAMA_HOST=10.10.10.10:11434 ollama list
I'm lazy, so I set it up permenantly for the root shell
echo 'export OLLAMA_HOST=10.10.10.10:11434' >> /root/.bashrc source /root/.bashrc ollama list
The first line above creates an environmental variable in /root/.bashrc, so the OLLAM_HOST is set each time you log in. The second line simply runs it so you don't have to log out and log back in. The last line tests.
ollama list # should show you the model you have installed
ollama run llama3.2:3b
The second command connects you to llama3.2:3b's shell. You can muck around in there for a while. When you are through, type /bye to exit.
I was, and remain, a newbie here. I have used AI for software development and systems administration, but never did anything with the actual workings of it.
My first decision was, what do I install. That seems a deeper question, so I'm going to answer it in the separate article Choosing AI Software.