Representative image showing a configuration for installing Ollama and running a local LLM directly on a Linux server

Overview

Ollama is a local AI execution tool that lets you run an LLM directly on your own server or PC.

Since you download the model you need and run it in your own environment, you can build chatbot, document summarization, and code assistance features without sending data to an external AI service.

This post covers how to install Ollama on a Linux server and allow external access.

It also covers criteria for choosing a model suitable for an Oracle Cloud A1 ARM environment, along with HTTP API call examples.

Points to note

Ollama is mainly used via an HTTP API.

If you allow external access, model calls may become possible from outside the same network, so access control is required.

When exposing it directly to the internet, it is recommended to set up the protective measures below together.

  1. Allow the 11434 port only for trusted IP ranges via a firewall.
  2. Bypass direct access using a VPN. Personally, I recommend Tailscale.
  3. Put it behind a reverse proxy (such as nginx) and add authentication.
  4. Explicitly restrict the allowed IPs.

Installation

Script installation

1curl -fsSL https://ollama.com/install.sh | sh

Verify execution

1ollama --version

Check the service

1systemctl status ollama

Enable auto-start for the service

1sudo systemctl enable ollama

Check the logs

1journalctl -u ollama -f

Allowing external access

Check the bind IP

  • Based on the example below, it is bound to 127.0.0.1
  • This means it can only be accessed locally
1ss -tlnp | grep 11434
2
3# Result
4# LISTEN 0      4096                     127.0.0.1:11434      0.0.0.0:*

Create the configuration directory

1sudo mkdir -p /etc/systemd/system/ollama.service.d

Edit the environment configuration file

1sudo vim /etc/systemd/system/ollama.service.d/override.conf

Add the following content

1[Service]
2Environment="OLLAMA_HOST=0.0.0.0:11434"

Apply the service

1sudo systemctl daemon-reload
2sudo systemctl restart ollama

Check the binding again

  • *:11434 allows connections from all IPs
1ubuntu@a1-free:~$ ss -tlnp | grep 11434
2
3# Result
4# LISTEN 0      4096                             *:11434            *:*

Based on Oracle Cloud A1 ARM 2OCPU / 12 RAM

**PurposeModelRecommendationSpeed****Tool CallingKoreanMemoryNotes**
🥇 Chat + Tool combined useQwen3:4B⭐⭐⭐⭐⭐★★★★☆★★★★★★★★★★4~5GBMost recommended
Chat onlyGemma3:4B⭐⭐⭐⭐☆★★★★★★★★☆☆★★★★☆4GBFast response
LightweightLlama3.2:3B⭐⭐⭐⭐☆★★★★★★★★☆☆★★★★☆3GBLightest
High quality (slow)Qwen3:8B⭐⭐⭐☆☆★★☆☆☆★★★★★★★★★★8~10GBSlow on CPU
Development/coding specializedQwen3-Coder⭐⭐⭐⭐☆★★☆☆☆★★★★★★★★★★LargeClose to coding-only use
  • No GPU, an always-free instance
  • On the spec above I used Qwen3:4B, but it was too slow to be usable…
    • Without a GPU, the limitations are clear
    • It’s about only usable for batch scheduling

Installing an LLM Model

1# chat + tool chain
2ollama pull qwen3:4b
3
4# fast chat
5ollama pull gemma3:4b

Using the LLM

Warm up

 1curl -s \
 2  -w "\n\nHTTP Status: %{http_code}\nTotal Time : %{time_total}s\n" \
 3  http://localhost:11434/api/generate \
 4  -d '{
 5    "model":"qwen3:4b",
 6    "prompt":"hi",
 7    "stream":false,
 8    "keep_alive":"5m",
 9    "options": {
10      "num_predict": 32
11    }
12  }'

Running a prompt

 1curl -s \
 2  -w "\n\nHTTP Status: %{http_code}\nTotal Time : %{time_total}s\n" \
 3  http://localhost:11434/api/chat \
 4  -d '{
 5    "model":"qwen3:4b",
 6    "messages":[
 7      {
 8        "role":"user",
 9        "content":"한국어로 대답해줘. 1+1?"
10      }
11    ],
12    "think": false,    
13    "stream":false
14  }'

Additional Configuration

Edit the environment configuration

1sudo systemctl edit ollama

Add the required environment variable

1# default keep alive time
2# -1: unlimited
3# example: 10m, 1h, ... 
4Environment="OLLAMA_KEEP_ALIVE=-1"

Apply the service

1sudo systemctl daemon-reload && sudo systemctl restart ollama