Run an LLM on a VPS: Ollama, RAM and Speed

What a VPS needs to run an LLM

Memory decides everything. The model has to fit in RAM, with room left for the operating system, the context window and whatever else the server runs. Cores decide how fast the words come out; disk only decides how long the download takes.

RAM — the size of the model plus about 1.5 GB for the system and the context.
vCPU — four cores is the practical floor for CPU inference. Speed scales roughly with cores up to eight.
Disk — 20 GB of NVMe is enough for two 4-bit models. Mechanical disk makes the first load painfully slow.
Swap — set 2 GB so a model that overshoots does not take the SSH session down with it.

Model size against RAM and VRAM

Quantisation is the whole trick. A 4-bit model stores each weight in about half a byte, so a 7B model drops from roughly 14 GB to about 4.5 GB. The table below is the arithmetic to plan a VPS around, not a benchmark.

Parameters	4-bit weights	RAM to plan	GPU VRAM
3B	~2 GB	4 GB	4–6 GB
7–8B	~4.5 GB	8 GB	8 GB
13–14B	~8 GB	12–16 GB	12–16 GB
30–34B	~19 GB	32 GB	24 GB
70B	~40 GB	48–64 GB	2 × 24 GB

Above 8-bit precision the numbers double: a 65B model at 8-bit is about 74 GB of weights. That is a dedicated machine, not a VPS you rent for a week.

Install Ollama on Ubuntu 24.04

Ollama ships as a single binary with a systemd unit, so the install is one command. Run it as root or with sudo on a fresh server.

Update the box first: sudo apt update && sudo apt upgrade -y
Install: curl -fsSL https://ollama.com/install.sh | sh
Check the service came up: systemctl status ollama should show active (running).
Confirm the listener is local only: ss -lntp | grep 11434 shows 127.0.0.1:11434, not 0.0.0.0.

You know it worked when ollama --version prints a version and the status line has no restart count climbing.

Pull a model and test it

Start small. A 3B model proves the whole path in two minutes, and you can move up once the API answers.

ollama pull llama3.2:3b downloads about 2 GB. The default tag is the 4-bit build.
ollama run llama3.2:3b "Summarise this in one line: our office moves to Pune on Monday."
Warm it for later requests with curl -s http://127.0.0.1:11434/api/generate -d '{"model":"llama3.2:3b","prompt":"hello","stream":false}' and check for a response field in the JSON.
Watch memory while it answers: free -h and ollama ps. If RAM is pinned and swap is filling, drop to a smaller model.

Add a chat interface with Open WebUI

A browser interface makes the model usable by people who will not touch a terminal. Open WebUI talks to Ollama on the same host and stores its data in a Docker volume.

Install Docker: curl -fsSL https://get.docker.com | sh
Run the container: docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
Open http://SERVER-IP:3000. The first account you create becomes the administrator, so create it yourself before sharing the URL.
Pull more models from the admin panel, or with ollama pull on the host.

Expose the API without exposing the port

Ollama has no authentication. Anyone who can reach port 11434 can run any model on your server, and the bill is your CPU. Never open it to the internet, even briefly.

Keep OLLAMA_HOST=127.0.0.1 and let nginx proxy /api/ to it over HTTPS on 443.
Put a password in front of the proxy: htpasswd -c /etc/nginx/.htpasswd apiuser, then auth_basic in the server block.
Close everything except 22, 80 and 443: sudo ufw allow OpenSSH && sudo ufw allow 'Nginx Full' && sudo ufw enable
Set a request limit in nginx. A long prompt on a 70B model can hold a worker for minutes.

How fast CPU inference actually is

Expect 5–15 tokens a second for a 7B model on four to eight modern cores, and 2–6 tokens a second for a 13B. That is slower than reading speed for long answers, and fine for summaries, classification and background jobs.

Three settings move the number more than a bigger plan. Lower the context window (num_ctx) to what you actually use. Keep the model loaded with OLLAMA_KEEP_ALIVE=30m so the first token does not wait for a reload. Batch work overnight instead of expecting interactive speed from a CPU box.

Self-hosting against a paid API

Self-hosting wins on privacy, on predictable cost and on rate limits you control. It loses on speed and on idle hours. A model on your own server costs the same whether it answers a thousand requests or none.

Use a paid API while you are building and your volume is spiky. Move to your own VPS when one of three things is true: the data cannot leave your infrastructure, the monthly API spend is larger than the server, or you need the model available when a third-party service is down.

A WebJi® VPS for a self-hosted model

A WebJi® VPS gives you root access, so the Ollama service, the Docker container and the nginx proxy are yours to configure. Entry plans start at ₹499 a month, before 18% GST, and the RAM and core count for a larger model are quoted by email at support@webji.services before you pay.

Ask for the size that matches the table above rather than the cheapest plan, and say which model you intend to run. If you also want a database on the same box, the Supabase self-hosting guide covers that stack.

Frequently Asked Questions

Can I run an LLM on a VPS without a GPU?

Yes, on CPU. A quantised 7B model runs at roughly 5–15 tokens a second on four to eight cores, which suits summaries, classification and background jobs. Interactive chat feels slow, and a 70B model needs far more RAM than a small VPS has.

How much RAM do I need for a 7B model?

Plan for 8 GB. The 4-bit weights are about 4.5 GB, and the rest covers the operating system, the context window and the model runtime. A 13B model needs 12–16 GB, and a 70B model needs 48 GB or more.

How do I install Ollama on a server?

Run curl -fsSL https://ollama.com/install.sh | sh on Ubuntu 24.04 as root. The script installs the binary and a systemd unit. Check systemctl status ollama, then pull a model with ollama pull llama3.2:3b.

Is my Ollama API exposed to the internet?

Not by default: it listens on 127.0.0.1:11434. Do not change that to 0.0.0.0. Put nginx in front with HTTPS and a password, and allow only ports 22, 80 and 443 through the firewall, because Ollama itself has no authentication.

Which models run well on CPU only?

The 3B and 7–8B families at 4-bit are the practical range: Llama 3.2 3B, Qwen 2.5 7B and Mistral 7B. They answer in a few seconds and fit in 8 GB of RAM. Larger models run, but at speeds that only suit batch work.

Is self-hosting an LLM cheaper than paying for API calls?

Only above a certain volume. A VPS costs the same when idle, so low or spiky usage is cheaper on a paid API. Self-host when the monthly API bill exceeds the server cost, when data cannot leave your infrastructure, or when you need the model to stay up independently.

Run an LLM on a VPS