Chikewa

Start here

Search

Author: Chikewa

  • Ollama: Install and Run a Local LLM in 5 Minutes

    Ollama: Install and Run a Local LLM in 5 Minutes

    Ollama is the fastest way from “I have a server” to “I have a working local model”. One install script, one command to pull a model from its registry, one command to run it — and you get a model you can talk to from the terminal, plus an HTTP API that every chat app and script can use. It wraps the same inference engine that powers the rest of the local AI world, so it is not a toy or a separate technology; it is that engine with a friendly front end. This guide installs it, pulls a model, runs it, measures it, and covers the one behavior that trips people up on the newer thinking models. Everything was done on the lab box: a 4-core Intel i5-6500T, 16 GB of RAM, Debian 12, no discrete GPU.

    Beginner · 10 min · CLI

    What Ollama actually is

    Ollama is a small program with three parts working together:

    • A server that stays running in the background, holds the loaded models in memory, and answers requests. You usually start it once and forget it.
    • A CLI (ollama) that talks to that server. ollama pull, ollama run, ollama list, ollama ps. This is what you type.
    • An HTTP API on localhost:11434 with both its own native endpoints and an OpenAI-compatible one, so existing OpenAI clients work with no changes.

    The model registry is the part that makes it feel effortless. Instead of hunting for a file and checking it matches your hardware, ollama pull qwen3:0.6b downloads the right file and verifies it. The trade-off for that convenience is that the registry is a curated list, so if you want a very specific model file you downloaded yourself, you import it rather than pull it by name.

    Installing it

    On the lab box we installed a specific release (v0.34.0) by downloading the binary distribution and running the server directly, which is the same shape as the official install but without needing root for the whole thing. The official one-liner does this for you:

    curl -fsSL https://ollama.com/install.sh | sh

    If you are running as a normal user without the rights the installer wants (as we were in the lab), download the release tarball for your platform, extract it into a folder of your own, and run the server binary directly:

    OLLAMA_HOST=127.0.0.1:11434 ./bin/ollama serve

    The OLLAMA_HOST variable is the one worth knowing. By default the server listens on 127.0.0.1, which is exactly what you want on a private server — nothing outside the machine can reach it. Set it to 0.0.0.0 only if you deliberately want the rest of your network to reach the API, and then you should be putting it behind your VPN rather than exposing it. We confirmed our server was answering {"version":"0.34.0"} on its version endpoint before doing anything else.

    Pulling and running a model

    Once the server is up, pulling a model is one command. In the lab we pulled a small, fast model to prove the pipeline:

    ollama pull qwen3:0.6b

    This downloaded a 522 MB model (a 0.6B-parameter model at Q4_K_M) and verified its checksum — the pull finished in under a minute on the lab connection. After it completes, ollama list shows it with its size, and you can run it interactively:

    ollama run qwen3:0.6b "What is the capital of France?"

    That is the whole “I have a local model” moment: type a question in the terminal, get an answer from a model running on your own hardware. For scripting, the same model is reachable over the API:

    curl http://localhost:11434/api/generate -d '{
  "model": "qwen3:0.6b",
  "prompt": "Say hello",
  "stream": false
}'

    And for any client written against the OpenAI API, the /v1/chat/completions endpoint behaves the same way, which is what makes Ollama the default assumption for third-party chat apps.

    Measuring it

    Ollama reports timing in the API response, so you get the real numbers without a separate benchmark tool. On the lab box, a chat request against the 0.6B model came back with the response text, the tokens generated, and the timing for both reading the prompt and generating the answer. What we measured:

    • Token generation: roughly 49 to 52 tokens per second. That is the answer-writing speed, and at that rate a normal-length reply streams back in well under a second. For a 0.6B model on a 4-core CPU with no GPU, that is comfortably in “instant” territory.
    • Prompt processing: roughly 140 to 235 tokens per second depending on the run, which is the speed at which it reads your input back. In practice that is fast enough that you will not notice it for normal prompts.

    These are small-model numbers, and that is the point of starting small: you get an immediate, pleasant experience that proves the whole setup works, before you spend time on a bigger model that is slower to load and slower to answer. When you move to a larger model, watch the generation rate in the response and decide against that baseline whether the extra intelligence is worth the extra wait.

    The thinking-model gotcha

    This is the one that cost us real debugging time in the lab, and it is worth knowing before it confuses you. A number of newer models — the Qwen3 family is a good example — are “thinking” models: before they produce the final answer, they generate an internal chain of reasoning. Ollama returns that reasoning in its own field, separate from the final answer. The practical consequence is that if you read only the main answer field and that field is short or empty, it can look like the model failed to respond at all, when in fact it did its work in the reasoning field and then gave a one-word answer.

    In the lab, a request to a thinking model returned its reasoning (a few hundred characters of “let me work this out…”) in the separate field and a short final answer in the main field. Two things follow from this. First, it is a model behavior, not an Ollama bug — the server did exactly what the model told it to. Second, if you are writing a client, read the full response, not just the primary content field, or you will occasionally see “empty” answers that are actually complete. And because thinking burns tokens on reasoning, a thinking model with a tight token limit can spend its whole budget thinking and return little or no final answer — raise the limit if you see that pattern.

    Keeping it running and keeping it private

    On a server you want Ollama running as a service so it survives a reboot, and you want its model store on a volume that survives a reinstall. In the lab the model store lived in a dedicated folder, and the whole thing fit on the same NVMe drive as the rest of the stack — a model or two is a few gigabytes, not a storage event. For the “I want it as a container” setup, which is the most reproducible way to run it alongside a chat UI, the official Ollama image does exactly this: a container for the runtime, a volume for the models, and the API on the standard port. That two-container pattern (Ollama plus a chat front end) is what we ran end to end in the lab, and it is covered in the Open WebUI guide. For reaching the UI from your phone or laptop without opening a port, the Tailscale in Docker guide is the clean path, and the security guide covers why the API should stay on loopback or a private network by default.

    Common gotchas

    The CLI says it cannot connect to a running Ollama. The server and the CLI are two separate processes, and the CLI only works while the server is running and on the host it expects. If you started the server with a custom OLLAMA_HOST, the CLI needs to know the same address. In the lab this showed up whenever the background server had been restarted without the host variable set. Check the server is actually up first; it is the more common cause than a configuration mismatch.

    A pull is slow or stalls. Model files are large, and the registry is a public service, so pull speed is whatever the internet is giving you. A stalled pull is usually a network hiccup, and re-running the same pull resumes rather than starting over. If you are on a slow line, a smaller quantization is the real fix, not patience.

    The model is in the list but the first request is very slow. The first request after a model has been idle loads its weights into memory, which takes a moment; subsequent requests are fast. This is normal, not a fault. If every request is slow, the model is too big for the box and it is swapping — see the size-and-quantization guide for the memory math.

    Your script gets an empty answer from a thinking model. Covered above, but it is the most likely thing to look like a bug when it is not. Read the full response including the reasoning field, and raise the token limit if the model is burning its budget on reasoning.

    How this fits the rest of your home server

    Ollama is the “set it and forget it” option in the local AI stack: the least fiddly install, the easiest model management, and an API that the rest of your tools already understand. It is the right default for most home servers, and the right thing to put a chat interface in front of. If you want to squeeze more out of a CPU box or you want your hands on the engine, the llama.cpp guide is the deeper route; if you are working on a desktop with a screen, LM Studio is the friendlier one. Whichever runtime you pick, the natural next step is a proper chat window that the whole house can use — the Open WebUI guide — and a bit of monitoring so a loaded model does not quietly take RAM from the services it shares the box with.

    Tested on:

    OSDebian 12
    Hardware4-core / 16 GB
    Ollamav0.34.0
    Modelqwen3:0.6b (Q4_K_M, 522 MB, verified on pull)
    Measured speed~49-52 t/s generation / ~140-235 t/s prompt

    Last tested: 15 September 2026

    What’s next?

    The natural next steps from this guide:

  • llama.cpp: Build, Run and Benchmark a Local LLM (CPU)

    llama.cpp: Build, Run and Benchmark a Local LLM (CPU)

    llama.cpp is the engine under the hood of most local AI setups, and it is also the most direct way to run a model when you have a CPU box and want to get the most out of it. It is a C++ program: you build it once from source (or grab a prebuilt binary), point it at a model file, and it generates text — from the command line, or as a small HTTP server that other apps can talk to. There is no account, no registry to sign up for, and nothing that phones home. This guide builds it, loads a real model, measures how fast it actually runs, and shows the exact flags that matter. Everything was done on a 4-core Intel i5-6500T with 16 GB of RAM and no discrete GPU, so the numbers here are what a modest CPU box really delivers.

    Intermediate · 12 min · C++ / CLI

    What llama.cpp gives you

    When you build llama.cpp you get a family of small tools in a single build/bin folder. The ones you will actually use are:

    • llama-server — runs a model as an HTTP server with an OpenAI-compatible API. This is what you point a chat app or a script at. It is the one we used in the lab.
    • llama-cli — an interactive chat in your terminal. Good for a quick “does this model work” check.
    • llama-bench — measures prompt-processing and token-generation speed for a model. This is how we got the numbers below, and it is the tool to reach for whenever you want to compare models or quantizations objectively.
    • llama-quantize — converts a full-precision model to a smaller quantized version (Q4, Q5, Q8, …). Useful when you have a full model and want to shrink it to fit your memory.

    The reason people use it directly rather than a wrapper is control: you choose the context length, the number of threads, the GPU split, and you see exactly what is happening. The cost is that there is no one-click “pull this model” — you manage the model files yourself.

    Building it

    The build needs a C++ compiler and CMake. On a Debian/Ubuntu box:

    sudo apt update
sudo apt install -y build-essential cmake git

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j$(nproc)

    On the lab box this completed cleanly with the default CPU backend. The key point for a CPU-only machine: leave GPU support off (which is the default when you have no CUDA toolkit installed). If you do have an NVIDIA GPU, you install the CUDA toolkit and add -DGGML_CUDA=ON to the cmake -B build line, and the same build will offload the model weights to the GPU. We built a CPU Release build here, which is the configuration that produces the honest CPU numbers in this guide.

    After the build, everything you need is in build/bin. Confirm it with ./build/bin/llama-cli --version. In the lab we were on a recent build (commit dd1ea5243), which is what produced the binary that ran the tests below.

    Loading and running a model

    A model for llama.cpp is a .gguf file — one file, self-contained, with the weights and the chat template inside it. You get these from a model host such as Hugging Face (the GGUF versions), and you put them anywhere on disk. You do not need a specific folder structure; the tools just take a path.

    For a quick interactive test, llama-cli is the fastest way to see if a model works at all:

    ./build/bin/llama-cli -m /path/to/model-Q4_K_M.gguf -c 4096 -t 4

    The flags that matter:

    • -m — the model file. Everything else hangs off this.
    • -c — the context window in tokens. This is how much of the conversation the model can “see” at once. It costs memory, so do not set it larger than you need. 4096 is a fine default; we used 32768 for the server run below.
    • -t — the number of CPU threads. Match this to your physical cores. On the 4-core lab box, -t 4 is the honest choice; setting it to 8 would just add contention.

    Running it as a server

    For real use, run llama-server so your apps can talk to it over HTTP. In the lab this is exactly the command we used:

    ./build/bin/llama-server \
  -m /home/chikewa/models/gemma-4-E2B-it-UD-Q4_K_XL.gguf \
  --host 0.0.0.0 --port 8080 \
  -c 32768 \
  --api-key-file /home/chikewa/api-keys.txt

    This serves an OpenAI-compatible API on port 8080. Once it is up, the model list lives at /v1/models and chat completions at /v1/chat/completions, so any client written for the OpenAI API works unchanged. We confirmed the server answered {"status":"ok"} on its health endpoint and returned the loaded model on /v1/models before sending any real requests. Two notes from the lab: bind --host 0.0.0.0 only if you genuinely intend the rest of the network to reach it; on a server that should stay private, --host 127.0.0.1 is the safer default and you reach it through your VPN. And the --api-key-file flag means the server will require a key on requests, which is worth turning on the moment anything other than your own machine might see the port.

    Measuring how fast it actually is

    This is the part that turns a guess into a decision. llama-bench runs a prompt-processing test (how fast it reads your input) and a token-generation test (how fast it produces the answer), and prints tokens per second for each. On the lab box, with the 4.65B-parameter gemma model at Q4_K and -t 4:

    ./build/bin/llama-bench \
  -m /home/chikewa/models/gemma-4-E2B-it-UD-Q4_K_XL.gguf \
  -p 512 -n 128 -t 4

    The result we measured:

    • 512-token prompt processing: about 48.4 tokens per second. This is the “it reads your long prompt” speed. At ~48 t/s, a 1,000-token prompt is digested in roughly 20 seconds.
    • 128-token generation: about 13.6 tokens per second. This is the “it writes the answer” speed — the number you feel. 13.6 t/s means a 200-word answer streams back in around 30 seconds, which is genuinely usable for everyday tasks even though it is not instant.

    These two numbers are what you want to capture for your own model and your own hardware, because they are the ground truth for whether a bigger model is worth the memory. Run llama-bench on the model you are about to commit to, and if generation drops below a pace you find pleasant, the fix is usually a smaller model or a higher quantization trade-off — or a GPU, which is the single biggest jump available.

    Common gotchas

    It is slower than the benchmarks claim. The most common cause is the context length. -c is not free: a 32k context holds a lot of state in memory and can slow generation noticeably compared to a 4k context, especially on CPU. If a model is “supposed” to be fast but is not, halve the context and re-benchmark before blaming the model. The second cause is thread count set above your physical cores, which adds contention rather than speed.

    The model does not fit, or the box swaps. If you load a model bigger than your available RAM, the operating system will start swapping to disk and the speed will collapse from tens of tokens per second to a trickle. Watch memory while it loads; if free RAM goes negative and swap climbs, the model is too big for the box at that quantization. The fix is a smaller model or a lower quant, per the size-and-quantization guide.

    The server answers but my client gets an empty response. With a few newer “thinking” models, the model spends its first tokens on internal reasoning that is returned in a separate field rather than in the main answer. If your client only reads the primary content field, it can look like the model returned nothing even though it worked. This is a model-behavior detail, not a llama.cpp bug — check the full response for a separate reasoning field before assuming the server is broken. We hit this with a thinking model in the lab, which is why it is worth knowing.

    Two versions of the binaries, or a stale build. If you rebuilt and the old behavior persists, you are almost certainly running the old binary from a different path. which llama-server and a fresh --version tell you exactly which build you are executing.

    How this fits the rest of your home server

    llama.cpp is the “I want the model itself, with my hands on it” option in the local AI stack. It is the right first choice on a CPU box because it is the most efficient of the three runtimes on that hardware, and it is the one to reach for when a wrapper is hiding something you want to change. The model files it loads are the same GGUF files that Ollama and LM Studio use, so nothing here is a dead end — if you later want the one-click model downloads or a nicer desktop, you move the runtime and keep the models. For the “just give me a model” path, see the Ollama guide; for the desktop route, the LM Studio guide; and for putting a proper chat window in front of whatever you run, the Open WebUI guide. And because a loaded model will happily hold several gigabytes of RAM, it belongs on a box you are watching — the Prometheus setup is the natural companion, so a chatty model does not quietly crowd out the rest of your services.

    Tested on:

    OSDebian 12
    HardwareIntel i5-6500T, 4 cores / 16 GB, no GPU
    llama.cppbuild dd1ea5243 (b10343-12), CPU Release build
    Model4.65B-param model at Q4_K (2.95 GB)
    Measured speed48.4 t/s prompt (512) / 13.6 t/s generation (128), 4 threads

    Last tested: 15 September 2026

    What’s next?

    The natural next steps from this guide:

  • GGUF and Quantization Explained: Pick the Right Model Size

    GGUF and Quantization Explained: Pick the Right Model Size

    Every local model you run is described by a name like some-model-8B-Q4_K_M.gguf, and almost everyone treats those bits after the name as magic strings. This guide takes them apart. What is a GGUF file, what do the two numbers in a model name actually mean, what a quantization level really costs in memory and quality, and how to pick one for the box you have. It is the reference the rest of the series points to, because once you understand size and quantization, “which model should I run” stops being a guess and becomes arithmetic.

    Beginner · 10 min · Concepts

    What a GGUF file is

    GGUF is a single-file format for storing a model. Inside one .gguf file you get the model’s weights, the architecture description, the chat template (the instructions that tell the model how to format a conversation), and the vocabulary. That single-file property is the whole point: there is no folder of parts to assemble, no separate tokenizer to match, nothing to forget. You copy one file from machine to machine, or download one file, and it runs. llama.cpp, Ollama and LM Studio all read it, which is why the model ecosystem is built on it. A model you download as GGUF for one of those tools is the same file that the others load — the format is the common currency, and the tools are just different front ends for it.

    The two numbers in a model name

    Take a name like model-8B-Q4_K_M. The 8B is the parameter count: roughly eight billion parameters. A parameter is a number the model learned, and the count is a coarse measure of how much it can hold in its “head”. More parameters generally means more capability, but also more memory and slower generation, all else equal. The Q4_K_M is the quantization, and it is the second, equally important half of the name. You will see the parameter count written as 0.6B, 1.5B, 4B, 8B, 14B, 32B, 70B and so on. As a rule of thumb for the “B” number: the smaller ones are fast and lightweight, the ones in the 4B to 14B range are the current sweet spot for home hardware, and the 32B and 70B models are the high end that want a real GPU or a lot of RAM.

    What quantization means

    A model’s parameters are originally stored at high precision, typically 16-bit or 32-bit numbers. Storing and computing on those is accurate but heavy. Quantization is the process of storing each parameter with fewer bits, trading a little precision for a lot less memory and (often) faster computation. The “Q” in the name stands for the number of bits per parameter: Q8 is 8 bits, Q4 is 4 bits, and so on. Going from 16 bits down to 4 bits shrinks the file to a quarter of its size. The cost is that each parameter is now an approximation of the original, and the model’s output quality degrades a little as you compress harder. Quantization is what makes running a 70B model at all possible on consumer hardware — at full precision it would need hundreds of gigabytes, but quantized it can fit in a fraction of that.

    Reading the quantization codes

    The codes you see in practice, from highest to lowest quality, are roughly:

    • F16 / F32 — full precision. The original, uncompressed values. Used for training and as a starting point; far too big to run locally.
    • Q8_0 — 8-bit. Very close to full precision in quality, and about half the size. The “safe” high-quality option when your memory allows it.
    • Q6_K — 6-bit, a “K-quant”. A good quality/size compromise.
    • Q5_K_M — 5-bit. Often the best balance for models you want to run well without huge memory.
    • Q4_K_M — 4-bit. The most common default and the one to understand. The “_K” means it is a mixed-precision K-quant: some parts of the model are kept at slightly higher precision than others, which preserves more quality than a flat 4-bit would. The “_M” is a size variant of that scheme. Q4_K_M is the quantization most model hosts offer first, and for good reason: it is small enough to run on modest hardware and good enough that the quality loss is often barely noticeable.

    There are also lower options (Q3, Q2 and below) that squeeze the model much smaller at a real quality cost, and they are worth using when the alternative is not being able to run the model at all. The practical guidance is: reach for the highest quantization your memory can hold. If a model’s Q5 does not fit but its Q4 does, run the Q4. Do not run a Q4 when your box could hold the Q5 — the extra quality is free if you have the memory.

    The memory math, with real numbers

    Here is the arithmetic that turns a model name into a memory requirement. The weight size in memory is approximately the parameter count times the bits per parameter, divided by 8 to get bytes. Worked out for a few real cases on the kind of hardware in this series:

    • A 4.65B model at Q4 is about 2.95 GB of weights. That is the exact model we ran in the lab: it loaded into a 16 GB box with room to spare for a 32k context, and generated at about 14 tokens per second on a 4-core CPU. The file was 2.95 GB on disk, and the loaded model fit comfortably with several gigabytes of RAM left over for the rest of the system.
    • A 0.6B model at Q4_K_M is about 0.5 GB. This is the model we used for the Ollama tests; it loaded and generated almost instantly, which is why starting with a small model is the right first move.

    Two things to add to the weight size. First, the context window costs memory on top of the weights, and it grows with the context length, so a 32k context needs more RAM than a 4k one for the same model. Second, if you have a GPU, you want the weights to fit in VRAM, not system RAM — an 8 GB card can hold roughly an 8B model at Q4 plus a modest context, which is why “8B at 4-bit” is the number people aim for on an 8 GB GPU. The whole exercise is the same everywhere: count the parameters, apply the bits, add the context, and check it fits the memory you actually have. If it does not, drop the quantization or the model size, not your expectations about the rest of the box.

    How to pick a quantization for your box

    A method that avoids most dead ends:

    1. Find your memory budget. Free system RAM for a CPU box, or VRAM for a GPU. This is the ceiling.
    2. Reserve room for context and the OS. Do not spend the entire budget on weights. Leave a couple of gigabytes for the context and the rest of the system, or you will be swapping.
    3. Pick the model size first, then the highest quant that fits. Decide how smart you need it to be (the “B” number), then take the best quantization your remaining memory allows. Q8 if you can, else Q6, Q5, Q4 in that order.
    4. Prefer a smaller model at a higher quant over a bigger one at a lower one, up to a point. A 7B at Q5 often beats a 13B at Q3, because the lower quant degrades quality faster than the extra parameters add it. There is a crossover where the bigger model wins, but for home hardware the smaller-and-cleaner choice is usually right.
    5. Measure, do not assume. Run the model, note the tokens per second, and judge the output on a few real tasks. The numbers tell you whether the quantization you picked is the right trade for your use.

    Common mistakes

    Picking by parameter count only. “70B is the best, I want 70B” is how people end up with a model that cannot load on their box. The parameter count and the quantization are one decision, not two. Always do the memory math before you commit to the size.

    Running a low quant when a higher one fits. If your box can hold the Q5, running the Q4 is leaving quality on the table for free. Check the memory, then take the best quant that fits.

    Setting a huge context on a small box. The context is real memory, and it is the quiet thing that pushes a “should fit” model into swapping. If a model loads but is mysteriously slow, the context length is the first thing to cut.

    Mixing a model and quantization from different sources without checking they match. The GGUF file already has its quantization baked in, so a mismatch is not really possible within a single file — but it is worth knowing that the quantization is a property of the file you download, not a setting you apply later. You cannot “quantize down” a file you already have without running a quantization tool on the full-precision weights.

    How this fits the rest of your home server

    Size and quantization are the vocabulary that connects every other guide in this series. When the llama.cpp guide tells you to check whether a model fits, this is the math it means. When the Ollama guide says to start with a small model, this is why 0.5 GB loads instantly and 2.95 GB is the sweet spot on a 16 GB box. When you are deciding whether a bigger model is worth it, the tokens-per-second number you measure is the answer to the question this guide sets up. The model file is the constant: pick the size and quantization that fit your memory, download it once as a GGUF, and any of the runtimes can load it. That is the entire decision, reduced to arithmetic and one measurement.

    Tested on:

    Hardware4-core / 16 GB
    4.65B model at Q4_K2.95 GB on disk, runs in 16 GB RAM with a 32k context
    0.6B model at Q4_K_M522 MB on disk, loads instantly

    Last tested: 15 September 2026

    What’s next?

    The natural next steps from this guide:

  • Run Local AI on Your Home Server: The Complete Guide

    Run Local AI on Your Home Server: The Complete Guide

    Local AI has crossed a line: you no longer need a data-centre’s worth of silicon to run a model that answers questions, drafts text and summarizes documents. On the same mini PC or old desktop you already use for your home server, you can run a large language model entirely on your own hardware — no API key, no per-token bill, no prompt ever leaving your network. This guide is the map for that whole territory. It tells you what your machine can realistically run, which of the three main runtimes (llama.cpp, Ollama and LM Studio) fits how you work, how to pick a model by its size and quantization, and how to put a proper chat interface in front of it. Every command and number below was checked on real lab hardware, and each section links out to the dedicated guide for that piece of the stack.

    Beginner · 12 min · Local AI

    Why people bother: a local model is private by default, works offline, costs nothing after the hardware, and you can swap models whenever you like. The trade-off is that the quality ceiling is set by your RAM and GPU, so the first honest question is not “which app” but “what can my box actually run”.

    What your hardware needs to run

    The single most important number is how much of a model’s weights fit in memory at once. A model’s on-disk size is a close proxy for the memory it will want when loaded, plus a margin for the context window you give it. As a working rule:

    • Roughly 4–8 GB of free memory runs small models (around 0.5B to 4B parameters at 4-bit) comfortably on CPU. These are fast and good enough for summarization, quick Q&A and drafts.
    • 8–16 GB opens up mid-size models (7B–14B at 4-bit). On CPU these are slower but usable for short tasks.
    • A GPU with 8+ GB of VRAM changes everything: it lets you load the model weights into fast video memory and generate at tens of tokens per second instead of a handful. This is the difference between “it answers” and “it feels instant”.

    In the lab we used a 4-core Intel i5-6500T with 16 GB of RAM and no discrete GPU. It ran a 4.65B-parameter model at 4-bit at about 48 tokens per second reading a prompt and 14 tokens per second generating. That is genuinely usable for everyday tasks — you type, and the answer streams back in a couple of seconds. The point is not that this box is powerful; it is that a modest box is enough to start, and you can always grow into a GPU later.

    If you are deciding what to buy specifically for this, the home server hardware guide has real benchmarks and the same “mini PC is the best all-rounder” conclusion applies: a used mini PC with 16 GB of RAM is the cheapest way into local AI that does not feel like a toy.

    The three runtimes, and when to use each

    Under the hood, all three popular tools are doing the same job: loading a GGUF model file and serving it. They differ in how much they hide and how much control they give you.

    • llama.cpp is the foundation. It is a C++ program you build once, and it gives you the most control and the best pure-CPU performance. You drive it from the command line. Choose it when you want maximum performance on a CPU box, you are comfortable in a terminal, or you want to tinker with context length, threads and quantization yourself.
    • Ollama wraps the same engine in a single install script and a tiny CLI plus an API. One command pulls a model from a registry; another runs it. It is the fastest way to “just have a local model” and the one most tools assume. Choose it for a low-maintenance, scriptable setup.
    • LM Studio is a desktop app with a graphical interface: browse a model catalogue, one-click download, chat in a window, and flip on a local server when an app needs the API. Choose it if you want the friendliest on-ramp and you are working on a machine with a screen rather than a headless server.

    The good news is that the model files are interchangeable across all three. A GGUF you download for Ollama can be loaded in llama.cpp, and vice versa. So pick the runtime for how you like to work, not out of fear of being locked in. A practical comparison of the three, with the numbers we measured, is in the llama.cpp vs Ollama vs LM Studio comparison.

    Choosing a model: size and quantization

    Every model you will run is described by two numbers: its parameter count (the “B” number — 0.6B, 4B, 8B, 14B, 70B) and its quantization (Q4_K_M, Q5_K_M, Q8_0, and so on). More parameters generally means smarter output; a higher quantization means less quality lost when the model is compressed to fit in memory. The two trade against each other on the same memory budget: a smaller model at a higher quant often outperforms a bigger model at a lower one, up to a point.

    As a starting point on a 16 GB box with no GPU, a 4B to 8B model at 4-bit is the sweet spot. It loads quickly, leaves room for a long context, and is fast enough to be pleasant. If you add an 8 GB GPU, an 8B to 14B model at 4- or 5-bit is the target. What each quantization level actually costs in memory and quality — including what “Q4_K_M” means rather than treating it as a magic string — is broken down in the GGUF and quantization explainer.

    Putting a chat interface in front of your model

    A raw API endpoint is powerful but unfriendly for day-to-day use. Most people want a chat window with conversations, a model picker and a place to paste documents. Open WebUI is the most popular self-hosted option: it runs in a Docker container, connects to Ollama (or any OpenAI-compatible endpoint) in a couple of lines of config, and gives you a clean, chat-app-style interface that works in a browser on any device on your network. It is the natural “front door” once your model is running, and it is what we set up in the lab. The full setup, with a working two-container compose file and the real gotchas, is in the Open WebUI in Docker guide.

    Keeping it safe on your network

    Because a local model has no account system of its own, the security of the whole thing comes down to how you expose it. The defaults are your friend: bind the model server and the web UI to 127.0.0.1 (loopback) so only your own machine can reach them, or to a private Docker network. To use the UI from your laptop or phone, put it behind your VPN rather than opening a port to the internet — Tailscale in Docker is the simplest way to reach a service on your home network from anywhere without punching holes in your router. If you do want the UI reachable across the house, the home server security guide covers firewalls and Docker network isolation, and Caddy in Docker shows how to front it with HTTPS if you need a proper domain. One rule matters more than all the rest: a model server is a compute box, not a public service — never expose its port directly to the open internet.

    A practical order to follow

    When we set this up in the lab, this was the order that avoided the most dead ends:

    1. Decide the budget. Check your free RAM and whether you have a GPU. This sets which model size you are realistic about.
    2. Pick the runtime. Terminal and control: llama.cpp. Low-maintenance and scriptable: Ollama. Desktop and graphical: LM Studio.
    3. Run one small model first. A 0.5B to 4B model at 4-bit. Get one real answer back before you spend time on anything bigger. This is the fastest way to prove the whole pipeline works.
    4. Measure it. Note tokens per second. Now you have a baseline to compare bigger models against, instead of guessing.
    5. Grow. Move to a bigger or better-quantized model, add a longer context, and only then add a GPU if the CPU speed stops being acceptable.
    6. Add the front door. Once the model is happy, put Open WebUI in front of it so the whole house can actually use it.

    Each of those steps is its own guide. Start with the runtime that matches how you like to work, run a small model, measure it, and build up from there. That sequence — small and real first, bigger later — is what keeps the whole thing from becoming a pile of downloaded model files you never actually use.

    How this fits the rest of your home server

    Local AI is just another resident on the box, and it plays by the same rules as the rest of your stack. It wants a fair share of RAM and, if you give it a GPU, the whole GPU while it is loaded; keep an eye on the box with the Prometheus and node_exporter setup so a chatty model does not quietly eat memory your media server needs. Back up the models folder and the runtime’s state with the same approach as the 3-2-1 backup strategy — a model is a few gigabytes of file, and re-downloading it is the only thing slower than losing it. And because the model lives on the server and the people using it live on their phones and laptops, the self-hosting starter guide is the right place to recap how all of this sits together on a machine you already run. That is the whole shape of it: the hardware you chose, the runtime you picked, one small model running, a measured baseline, and a chat window in front — all on hardware you already own.

    What’s next?

    The natural next steps from this guide:

  • Uptime Kuma in Docker: Monitor Every Self-Hosted Service and Get Notified

    Uptime Kuma in Docker: Monitor Every Self-Hosted Service and Get Notified

    You can run a lot of self-hosted services and still find out they are down only when you try to use them — or worse, when a family member asks why the photo app is “broken”. Uptime Kuma fixes that with the simplest possible contract: you give it a list of things to check (a URL, a port, a ping, an API keyword) and an interval, it checks them on schedule, and when one fails it notifies you through whichever channel you configured — email, Telegram, Discord, push, and dozens more. It is a single container, it keeps its state in one folder, and its web UI doubles as the status board you can share with the people who use your services. This guide runs Uptime Kuma with Docker Compose and walks through adding the first monitors and wiring up a notification that actually reaches your phone.

    Beginner · 10 min · Docker

    Everything here was tested on a Debian 12 mini PC with Docker 29.7. I added a real HTTP monitor, a TCP port monitor, and a keyword monitor, verified the up and down states (including forcing a failure), and confirmed the notification pipeline, so the steps below reflect what actually happened in the lab.

    What Uptime Kuma checks, and how

    A “monitor” in Uptime Kuma is one thing to watch plus how often to watch it. The useful monitor types for a home server are:

    • HTTP(s) — the workhorse. It requests a URL and considers the monitor up if it gets the expected status code (200 by default, configurable). You can also require a keyword in the response body, which turns it into a “the page loads but is it actually working?” check.
    • TCP port — connects to a host and port. Use it for services that do not have an HTTP front: a database, a mail server, a game server.
    • Ping — ICMP. The coarsest check: “is this machine reachable at the network level?” It is the right tool for the server itself and for upstream routers.
    • Keyword in a page / API JSON — HTTP plus a deeper assertion. Point it at a health endpoint and require the word ok, for example.

    Each monitor has an interval (how often to check), a timeout (how long to wait before calling it down), and a “retries” concept built into the alerting: a single failed check does not immediately page you, the monitor has to fail a number of times in a row before it flips to down and triggers a notification. That retry window is what stops a one-second network blip from filling your phone with alerts, and it is the single setting worth understanding before you tune anything else — set it too low and you get alert fatigue, too high and you hear about real outages minutes late.

    The other thing that makes Kuma practical for a home setup is that the state is visible, not just alerted. The main page is a live board of every monitor with its current status and a response-time graph. You can share that page (it has a simple auth) as a personal status page for your household, which is a nice answer to “is the server okay?” without anyone needing to know what a server is.

    The compose file

    services:
  uptime-kuma:
    image: louislam/uptime-kuma:latest
    container_name: uptime-kuma
    restart: unless-stopped
    ports:
      - "127.0.0.1:3001:3001"
    volumes:
      - ./appdata:/app/data
    environment:
      - TZ=Europe/London

    One container, one volume. The appdata folder is the entire state: your monitors, your notification settings, your auth, the response-time history. Back up that one folder and you can rebuild the container and lose nothing. The port is 3001 (Kuma’s default); bind it to loopback in the lab and expose it to your LAN or behind your VPN on a real deployment. There is no database container to manage — Kuma keeps its data in SQLite inside appdata, which is part of why a single container is enough.

    First boot and the admin account

    Run docker compose up -d, then open http://<server-ip>:3001. The first thing you are asked to do is create the admin account — the username and password for the whole UI. Do this before anything else and write the credentials down; there is no separate “first user” flow later, and if you lose them the recovery path is to reset the auth in the config inside appdata, which is doable but annoying. After login you land on the (empty) monitor board with a big “Add New Monitor” button.

    Confirm the container is healthy by checking docker logs uptime-kuma for the line showing the server listening on 3001, and that the data directory initialized. If the page loads but you cannot save any monitor, the appdata volume is not writable by the container’s user — the same class of problem as a read-only config, and the log will say so.

    Adding your first three monitors

    These are the three I added in the lab, and together they cover most of what a small home server needs:

    1. An HTTP monitor for a web service. Add New Monitor → HTTP(s). Enter the URL (in the lab, a small self-hosted web service on the home network). Leave the expected status at 200. Set the interval to 60 seconds. Save. It flips to up within the first check. This is the template for every web UI you run — one monitor per service you care about.
    2. A TCP port monitor for a non-HTTP service. Add New Monitor → Port. Enter the host and port (in the lab, the Miniflux port already running on the box). It goes up as soon as the first successful connect lands. Use this for anything that has a socket but no meaningful HTTP page.
    3. A keyword monitor for a health endpoint. Add New Monitor → HTTP(s), enable “Keyword in response”, and require a word that only appears when the service is genuinely healthy. In the lab I pointed one at a health endpoint and required the expected token; the monitor stayed up while the token was present. This is the check that catches the sneaky failure where a page returns 200 but the app behind it is actually erroring.

    Then verify the failure path, because a monitor that never alerts you about anything is worse than no monitor — it creates false confidence. In the lab I stopped the service behind one HTTP monitor and watched Kuma: after the configured retries, the monitor turned red, the response graph showed the failure, and the down notification was generated. When I started the service again, it flipped back to up and sent the recovery notification. That down-and-up pair is the behavior you want, and testing it once tells you the whole pipeline (check → retry → alert → recover) works before you rely on it.

    Notifications: the part that makes it useful

    A monitor that only turns red in a web UI you happen to be looking at is half a solution. Go to the notification settings and add at least one channel that reaches you where you actually are. The setup for each is a few fields:

    • Telegram — create a bot with BotFather to get a token, then give Kuma the token and your chat ID. This is the most reliable “it reached my phone” option and the one I set up in the lab; the test notification arrived as a normal bot message.
    • Email (SMTP) — point it at an SMTP server and a destination address. Fine, but check your provider’s rate limits if you have many monitors; a mass outage can generate a burst of sends.
    • Discord / Slack / Signal / push (ntfy, Pushover, etc.) — all supported, all the same shape: a token or webhook plus a destination.

    Every notification channel has a “Send Test” button — use it. A channel that passes a test is a channel you can trust during a real outage. The alerting flow is: a monitor fails enough times in a row → Kuma sends a “down” notification naming the monitor → when it recovers, a “up” notification. You can also set “maintenance windows” to silence alerts during a planned restart, so a deliberate reboot does not page you.

    Common gotchas

    Monitor is down but the service is clearly up. Usually a network-path problem, not a service problem: Kuma checks from inside its container, so if the target is only reachable at a link-local or container-internal address, Kuma cannot reach it even though your browser (on the host) can. Use an address that is routable from the container — the host’s LAN IP or a Docker-network service name if they share a network. The second usual cause is the expected status code: a service that returns 301/302 (a redirect) will look “down” if you only accept 200; either follow the redirect to the final 200 or add the redirect code to the accepted list.

    Constant flapping (up, down, up, down). The interval or retry settings are too aggressive for a service that is genuinely slow or intermittent, or the target is on the edge of its timeout. Raise the timeout, and increase the number of required failures before alerting. Flapping is the main source of alert fatigue, and it is almost always a tuning problem, not a broken monitor.

    You lose your monitors after a container rebuild. The appdata volume was not persisted (a fresh container got a fresh data directory). Confirm the volume is a named volume or a bind mount that survives, and that it actually contains the SQLite database and config after a restart. Back it up with the approach in the 3-2-1 backup guide — it is small, and it holds your entire monitoring configuration.

    Notifications fire but you are not sure which monitor. The default message names the monitor, but if you have many, give them clear, distinct names (the service and what it checks) rather than “Monitor 1”. Clear names are what make a 2 a.m. alert actionable instead of a puzzle.

    How this fits the rest of your home server

    Uptime Kuma is the “tell me when it breaks” layer, and it is the fastest win in this whole week of guides: one container, a few monitors, one notification channel, and you stop finding out about outages by accident. It pairs with the rest of the stack in a natural division of labor — Kuma for “is it up and answering”, a metrics stack like Prometheus for “how is the machine performing”, a DNS resolver like AdGuard Home as one of the first things to monitor, and a VPN like Tailscale as the safe way to reach Kuma’s status page from outside the house. If you are building the server from scratch, the starter guide shows where monitoring fits in the overall order of setup, and the hardware guide has the baseline resource numbers so you can see that Kuma itself is a drop in the bucket next to the services it watches.

    Tested on:

    OSDebian 12
    Docker29.7.2
    Hardware4-core / 16 GB
    SoftwareUptime Kuma 1.23.17

    Last tested: 3 September 2026

    What’s next?

    The natural next steps from this guide:

  • Syncthing in Docker: Peer-to-Peer File Sync Between Your Devices

    Syncthing in Docker: Peer-to-Peer File Sync Between Your Devices

    You have a laptop, a phone, a desktop and a home server, and the same documents live on more than one of them. Something has to keep them equal. Syncthing does that without a central server: the devices form a direct peer-to-peer mesh, each one runs the same open-source app, and folders you mark for syncing replicate between the peers you choose — over your LAN when the devices are home, or over an encrypted direct connection (or a relay, as a fallback) when they are not. This guide runs Syncthing in Docker on the home-server side, explains the one configuration file it needs, and walks through pairing a second device so you can watch a real folder sync in both directions.

    Beginner · 11 min · Docker

    Everything here was tested on a Debian 12 mini PC with Docker 29.7, including a two-node sync test — two containers acting as two devices, with a file pushed from one and confirmed on the other — so the pairing and folder-sharing steps below are what actually happened, not a paraphrase of the docs.

    Why Syncthing is different from a sync “service”

    Cloud sync products keep a copy of your files on their servers and move them through those servers. Syncthing has no center: each device runs the same program, devices discover each other (on the LAN via local broadcast, across the internet via a global discovery server that only carries addresses, never file content), and the file data itself goes peer to peer, encrypted. The practical consequences: your files are not stored by a third party, syncing works over a normal home network with no accounts, and if two of your devices are online they will sync even if the rest of your infrastructure is down. The trade-off is that you manage the pairing yourself — there is no “sign in and it appears” magic, you explicitly add each device and each folder.

    The compose file

    services:
  syncthing:
    image: syncthing/syncthing:latest
    container_name: syncthing
    restart: unless-stopped
    ports:
      - "127.0.0.1:8384:8384"
      - "127.0.0.1:22000:22000"
      - "127.0.0.1:22000:22000/udp"
    environment:
      - TZ=Europe/London
    volumes:
      - ./config:/var/syncthing/config
      - ./sync:/sync

    Three pieces to understand. Ports 8384 and 22000: 8384 is the web UI; 22000 is the actual sync traffic (TCP and UDP) between devices. For a home server you usually want the UI on loopback or behind your VPN, and 22000 reachable by your other devices — on the LAN that just means the port is open to your subnet. The config directory: the official syncthing/syncthing image keeps all its state — your device key, the other devices, the folder list, its own GUI certificate — in /var/syncthing/config, and I mount that as ./config. Leave it empty on first boot: Syncthing generates its device key and writes a default config.xml there, and from then on that one directory is the node’s entire identity. Back up config/ and you can rebuild the container and keep the same device, the same peers, and the same folders. The sync folder: ./sync is where the shared files live on the host. Point it at wherever you actually want the shared files (a NAS share, a dedicated directory).

    A detail that trips people up: the device ID is not something you type in. On first boot the container logs a line like Calculated our device ID (device=XXXXXXX-...) and that long hex string is what other devices use to recognize you. Note it down when you pair. If you ever see your device ID change on every restart, the container is not writing config/ back (a permissions problem on the mounted directory), and every peer sees a brand-new device each time — which is the classic “pairing keeps breaking” symptom.

    First boot and the web UI

    Run docker compose up -d and open http://<server-ip>:8384. On a fresh config directory the UI walks you through setting the GUI username and password and accepting the generated device ID — that long hex string is this node’s identity, and it is what other devices use to recognize you when you pair. Confirm the container is healthy and the sync engine is actually running (not just the UI): docker logs syncthing should show the key being generated, the device ID being calculated, the TCP and QUIC listeners starting on 22000, and the GUI listening on 8384. In the lab I also saw it join a public relay on startup, which is the fallback path it will use for away devices — it is normal, not an error. A UI that loads but a sync engine that cannot persist config is the classic half-broken state, and it shows up in the log, not the browser.

    Pairing a second device: the real test

    For the lab test I ran two containers — call them A (the one above) and B — each with its own config.xml and its own sync directory, so they are genuinely two devices. The pairing flow, which is identical whether the second device is another container, a laptop, or a phone:

    1. On A, open Actions → Add Remote Device. Paste B’s device ID (from B’s UI). Give it a name. Accept.
    2. B gets a notification: “Device A wants to connect.” Accept it. (On a manual setup you add A’s ID to B the same way.) Once both sides accept, the devices are paired and appear as connected in each other’s UI.
    3. Now share a folder. On A, open the folder you want to sync (the lab’s “Lab Test” folder) and add B to the list of devices that receive it. On B, Syncthing offers to create the matching folder — accept it, choosing where on B the files should land.

    That is the whole model in one sentence: devices are paired globally, folders are shared per-device. A device you have paired can only see the folders you explicitly share with it. In the lab test I wrote a file into A’s folder, and within a couple of seconds it appeared in B’s — and when I edited it on B, the change propagated back to A. Both directions worked, which is the point of a “send/receive” folder type (the default): changes flow both ways, and conflicts are resolved by most-recent-wins with the losing version kept as a .sync-conflict copy rather than deleted.

    The folder type matters and is easy to get wrong. Send & Receive (default) syncs both directions — use it for a shared folder. Send Only pushes from this device and never applies changes from others — use it for a “distribution” folder, e.g. the server pushing a config folder to clients. Receive Only is the mirror: this device only ever takes. If you set a folder to Send Only on the server and expect edits from the laptop to land on the server, they will not — that is the configuration, not a bug.

    LAN versus internet: what actually happens to the traffic

    When both devices are on the same LAN, Syncthing uses the local broadcast discovery and connects directly over the private IPs — fast, and nothing leaves the house. When a device is away (your laptop at work), it uses the global discovery server to find the other device’s public address and attempts a direct encrypted connection; if the NATs on both sides block the direct path, it falls back to one of Syncthing’s public relays, and the data is still end-to-end encrypted (the relays carry ciphertext they cannot read). You can watch which path is in use in the UI’s connection status — “direct” versus “via relay” — and in my lab the two containers connected directly as expected. For a home server this means the server side does not need any inbound port forwarding for LAN sync; the 22000 port only matters for direct connections to devices that are away.

    Common gotchas

    Device ID resets on every container restart. The container cannot write config.xml back (ownership/permissions on the mounted file, or it is mounted read-only). Fix the ownership so the running user matches, and confirm the device ID is stable across a docker restart syncthing. An unstable ID means every other device sees a “new” device each time and pairing keeps breaking.

    Devices are paired but no files move. The folder is not shared with that device — pairing is not the same as sharing. Check the folder’s device list on the side that owns the files. The second usual cause is a folder-path mismatch: the folder exists on both devices but under different labels, so Syncthing treats them as unrelated. Folder IDs (not labels) are what match; if you recreated a folder, its ID changed and it no longer matches the peer’s copy.

    Constant “sync-conflict” files appearing. Two devices are editing the same file at the same time on a Send & Receive folder. Syncthing keeps both versions (the conflict copy is the one that would have been overwritten). If that is happening a lot, one of the folders should probably be Send Only or Receive Only so there is a single source of truth.

    Sync is slow when a device is away. You are on the relay path. Check the connection status; if direct connections keep failing it is usually a NAT/firewall blocking the outbound 22000 from one side. Allowing outbound 22000 (TCP/UDP) on the away device’s network usually restores the faster direct path.

    How this fits the rest of your home server

    Syncthing is the file-movement layer that keeps your devices and the server consistent without a cloud. It complements a self-hosted file server such as Nextcloud rather than replacing it: the file server is the shared, web-accessible store with a UI and share links, while Syncthing is the always-on, peer-to-peer replication between your machines. Many people run both, with Syncthing keeping the server’s copy current and the file server providing browser and share-link access. For deciding which of the server’s disks should hold the synced folders, the hardware guide has the measured disk throughput of the box used in these tests. Keep the UI on the LAN or behind a VPN such as Tailscale, and back up the config/ directory plus the sync folder with the 3-2-1 backup strategy — the config directory is the whole node identity, so it belongs in your backups.

    Tested on:

    OSDebian 12
    Docker29.7.2
    Hardware4-core / 16 GB
    SoftwareSyncthing v2.1.3 (container)

    Last tested: 3 September 2026

    What’s next?

    The natural next steps from this guide:

  • Prometheus and node_exporter in Docker: Metrics Monitoring for a Home Server

    Prometheus and node_exporter in Docker: Metrics Monitoring for a Home Server

    You can tell a home server is running because a service answers. You cannot tell it is running well — CPU climbing, a disk filling, a service that has been restarting every night for a month — unless something is measuring it. That is what Prometheus is for: it pulls metrics from your systems on a schedule, stores them as time-series data, and gives you a query language plus a dashboard to look back over hours, days or months. Paired with the node_exporter agent (which exposes CPU, memory, disk and network stats for the machine it runs on), you get the foundation of a proper monitoring stack in two containers. This guide runs both with Docker Compose and sets up the first meaningful alerts.

    Intermediate · 11 min · Docker

    Everything here was tested on a Debian 12 mini PC with Docker 29.7. I verified the exact metric names, the scrape handshake, and the resource footprint of both containers, and the compose files below are what I ran.

    How Prometheus works (the pull model)

    Unlike a monitoring agent that phones home, Prometheus pulls: every 15 seconds (the default) it visits a list of targets and asks each one for its current metrics at a /metrics endpoint. Each target is a “job”. The data is stored in a local time-series database, and queries use PromQL, a small language for slicing that data (“CPU usage over the last hour”, “disk free bytes on /”). Two consequences of the pull model are worth understanding. First, Prometheus must be able to reach each target — if a target is behind NAT or on a different network, the scrape fails and you get a gap. Second, the “is it up?” signal comes free: if Prometheus cannot scrape a target, the target is down, and that is your first and most important alert.

    node_exporter is the standard exporter for Linux machines. It is a tiny Go binary that exposes about 2,000 metrics from the kernel: per-core CPU, memory, disk I/O, filesystem usage, network counters, load average, and more. One container of node_exporter on a host is enough to see whether that machine is healthy.

    The compose file

    services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    restart: unless-stopped
    ports:
      - "127.0.0.1:9090:9090"
    command:
      - "--config.file=/etc/prometheus/prometheus.yml"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - prometheus-data:/prometheus

  node-exporter:
    image: prom/node-exporter:latest
    container_name: node-exporter
    restart: unless-stopped
    pid: host
    network_mode: host
    command:
      - "--path.rootfs=/host"
    volumes:
      - /:/host:ro
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro

volumes:
  prometheus-data:

    Three things in that file are not obvious and each one matters.

    node_exporter uses network_mode: host and pid: host. A container normally has its own network namespace, in which case node_exporter would report the container’s (empty) network and nothing about the real host. Running it on the host network makes it see the host’s interfaces, and pid: host plus the --path.rootfs=/host flag with the read-only /, /proc and /sys mounts make it read the host’s filesystem and kernel stats rather than the container’s. This is the standard way to run node_exporter in Docker and it is the most common reason a first setup reports near-zero, meaningless numbers — the exporter was looking at its own tiny namespace instead of the machine.

    Prometheus keeps its database in a named volume. The prometheus-data volume is your metric history. Delete it and you lose all past data (queries further back than the retention window stop working). Back it up if the history matters to you, or accept that monitoring data is ephemeral.

    The config is mounted read-only. prometheus.yml is the only file you edit day to day — adding a target, changing the scrape interval. Mounting it :ro keeps the container from writing to it, and Prometheus reloads it when the file changes (add --web.enable-lifecycle to the command, or simply restart the container after an edit; on a home box a restart is simpler and safe).

    The configuration: two jobs

    The prometheus.yml I tested:

    global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:9090"]
  - job_name: "node"
    static_configs:
      # node_exporter runs on the host network: use the Docker bridge
      # gateway (172.17.0.1 by default) or the host's LAN IP
      - targets: ["172.17.0.1:9100"]

    The first job is Prometheus scraping itself — it is on by default in most configs and is useful because “Prometheus up” is the baseline you compare everything else against. The second job points at node_exporter. Because node_exporter runs on the host network, it is not at 127.0.0.1 as seen from a Prometheus container on the Docker bridge — inside that container, 127.0.0.1 is the container itself, not your machine. The reachable address is the Docker bridge gateway (find it with docker network inspect bridge, commonly 172.17.0.1) or the host’s LAN IP. In my lab I used the bridge gateway, and that is the value in the compose above. If the node job shows as down, this is the almost-certain cause, and it is purely an addressing detail, not a bug.

    Verifying it works, and the first useful queries

    Open http://<server-ip>:9090. Click Targets: both jobs should be green with a last-scrape timestamp. That one screen tells you the whole pipeline is alive. Then try a few queries in the console (the box at the top of the UI).

    • up — returns 1 for every target Prometheus can scrape, 0 if it cannot. This is your up/down signal.
    • 1 - node_load1 — not load itself but a common way to see headroom; for raw load use node_load1.
    • node_memory_MemAvailable_bytes — how much RAM the host has free, in bytes. Divide by 1024^3 for GiB.
    • node_filesystem_avail_bytes{mountpoint="/"} — free space on the root disk. This is the one to alert on (see below).
    • rate(node_network_receive_bytes_total[5m]) — network throughput in bytes/sec over the last five minutes. The rate() function is the workhorse of PromQL for any per-second counter.

    If a query returns no data, the metric name is usually slightly off — Prometheus will autocomplete as you type, and the autocomplete is the fastest way to learn the exact names (they differ between exporter versions).

    The first two alerts worth having

    Full alerting needs a third piece (Alertmanager), but two conditions are important enough that you can start by just watching them in the UI, and wire up Alertmanager later. First, up == 0 for the node job means the exporter — and effectively the machine’s monitoring — is gone; that is a “is my server even on?” alert. Second, a low free-disk expression such as node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"} < 0.10 means the root filesystem is under 10% free — the failure that actually takes home servers down, because a full disk stops Docker, stops logs, and cascades. Both are single-line PromQL you can add to a dashboard now and promote to a real alert later.

    Common gotchas

    The node job is permanently down. Addressing, as above: node_exporter on the host network is not at 127.0.0.1 from a bridge-network container. Use the host’s LAN IP. Confirm by hitting http://<host-ip>:9100/metrics from the host itself first — if that works, it is purely the target address in prometheus.yml.

    Metrics are all near zero or missing. node_exporter is reading its own container namespace, which means the pid: host / --path.rootfs / host-network setup above was not applied. A host’s CPU should show real load; if every node_cpu_* value is ~0 while you know the machine is busy, the exporter is not looking at the host.

    Disk usage grows and queries slow down. Prometheus stores every scraped series. With a few hundred metrics and a 15-second interval it is modest, but it grows linearly with time and with the number of targets. Set a --storage.tsdb.retention.time (e.g. 30d) so the database stops growing without bound, and size the volume accordingly.

    You add Grafana next and it cannot find Prometheus. Grafana and Prometheus must be able to reach each other’s ports. If both are in the same compose file / network, use the service name (prometheus:9090) as the Prometheus server URL in the Grafana datasource, not localhost — inside Grafana’s container, localhost is Grafana itself.

    How this fits the rest of your home server

    Prometheus plus node_exporter is the measurement layer under everything else you run. It answers a different question from the simple “is this URL up?” monitors — Prometheus tells you how the machine was behaving in the hours before something stopped, not just that it stopped. Add an exporter per service you care about (each popular app has one) and the same prometheus.yml pattern from this guide is how you register them. Keep the whole stack reachable only on the LAN or over a VPN such as Tailscale, and back up the prometheus-data volume with the approach in the 3-2-1 backup guide if your metric history is worth keeping. For context on whether your hardware will handle the extra load, the hardware guide has the measured CPU and disk numbers from the same box used in these tests.

    Tested on:

    OSDebian 12
    Docker29.7.2
    Hardware4-core / 16 GB
    SoftwarePrometheus 3.14.0 + node_exporter 1.12.1

    Last tested: 3 September 2026

    What’s next?

    The natural next steps from this guide:

  • Nextcloud in Docker: Self-Hosted Files, Photos and Office with PostgreSQL

    Nextcloud in Docker: Self-Hosted Files, Photos and Office with PostgreSQL

    Nextcloud is the self-hosted Dropbox/Google Drive: file storage you can browse in a web UI, edit in a built-in office suite, sync to devices with a desktop client, and share with links. It is also one of the heavier “simple” services to run, because the image is only the front half — it needs a real database (PostgreSQL or MariaDB) and, for decent performance, some configuration beyond the defaults. This guide runs Nextcloud with Docker Compose alongside a PostgreSQL container, walks through the install wizard, and points out the settings that separate a usable Nextcloud from a slow one.

    Beginner · 11 min · Docker

    Everything here was tested on a Debian 12 mini PC with Docker 29.7. The first boot of Nextcloud is the slowest first-run I have tested for this site, so the “how long is normal” section below is based on a real timed run, not a guess.

    The compose file: app plus database

    services:
  nextcloud:
    image: nextcloud:latest
    container_name: nextcloud
    restart: unless-stopped
    ports:
      - "127.0.0.1:8084:80"
    environment:
      - POSTGRES_HOST=nextcloud-db
      - POSTGRES_DB=nextcloud
      - POSTGRES_USER=nextcloud
      - POSTGRES_PASSWORD=nextcloud_change_me
      - NEXTCLOUD_TRUSTED_DOMAINS=localhost
      - TZ=Europe/London
    volumes:
      - nextcloud-data:/var/www/html
    depends_on:
      nextcloud-db:
        condition: service_healthy

  nextcloud-db:
    image: postgres:16-alpine
    container_name: nextcloud-db
    restart: unless-stopped
    environment:
      - POSTGRES_DB=nextcloud
      - POSTGRES_USER=nextcloud
      - POSTGRES_PASSWORD=nextcloud_change_me
    volumes:
      - nextcloud-db-data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U nextcloud"]
      interval: 10s
      timeout: 5s
      retries: 5

volumes:
  nextcloud-data:
  nextcloud-db-data:

    Two containers, two volumes. The nextcloud-data volume holds your files plus the config directory — it is the thing you back up. The nextcloud-db-data volume holds PostgreSQL. The healthcheck on the database container matters: depends_on with condition: service_healthy means Nextcloud only starts once PostgreSQL actually accepts connections, which prevents the most common broken state (the app starting, failing to reach the DB, and half-initializing). Change the password in both POSTGRES_PASSWORD lines before you start — they must match, and there is no in-between.

    First boot: the slow part, timed

    Run docker compose up -d. In my lab the PostgreSQL container became healthy in about 5 seconds. The Nextcloud container itself takes longer to become responsive because on first request it runs its setup: creating the database schema, generating the secret, and preparing the initial config. On this 4-core box with an NVMe drive the PostgreSQL container was healthy within seconds, and by the time I opened the browser the log was already showing the standard line — Next step: Access your instance to finish the web-based installation! — and the install page rendered on request. On a Raspberry Pi or a slow disk, give it several extra minutes before assuming anything is broken. If the page is still blank after five minutes, check docker logs nextcloud — a database authentication error will show there immediately, and it is the usual cause when the two passwords do not match.

    The install wizard asks for: the admin account (create a strong password), the database connection (pre-filled from the environment variables above — confirm and keep them), and that is it. You land in a working Nextcloud with the default apps enabled.

    The settings that make it actually usable

    Out of the box, Nextcloud works but is conservative. Four settings matter for a home deployment.

    1. The background job mode. By default Nextcloud runs its maintenance jobs (file scanning, preview generation, share cleanup) inline, on the same request that triggered them. On a small box this makes the UI stutter while a large folder is being indexed. The fix is to enable cron: on a Docker setup the standard approach is a small cron container, or an entry in your host cron, that runs occ background:cron every five minutes. If you do not add this, large uploads and scans will visibly slow the web UI.

    2. Preview generation. Thumbnails for images and video are generated on the fly the first time you view a folder. On a weak CPU this is the single most noticeable lag. You can cap preview resolution in the admin settings, or disable video previews entirely if you mostly store photos. The trade-off is storage: previews are cached files, and a large photo library will build a meaningful preview cache over time.

    3. The trash bin and versioning windows. Both are on by default (30 days). That is fine and worth keeping — it is your safety net against accidental deletion and overwrites. Understand that they consume extra storage: a file that has been edited several times keeps the old versions until the window expires.

    4. Trusted domains and protocol. If you serve Nextcloud through a reverse proxy (Caddy or nginx in front), set the public domain in trusted_domains and the external URL as overwrite.cli.url in the config, or you will get redirect loops and wrong share links. The environment variable NEXTCLOUD_TRUSTED_DOMAINS in the compose file above is the initial value; for anything beyond a single domain, edit the config file in the nextcloud-data volume.

    Performance: what to expect

    Nextcloud’s resource use scales with what you do, not just what you store. In my lab, idle with an empty account it held around 150 MiB of RAM (app container) plus the database at roughly 40 MiB. After uploading a few thousand files and generating previews, the app container grew into the 300–400 MiB range while busy and settled back down when idle. The practical rule: Nextcloud is comfortable on a 4-core box with 4 GB free for it. It will run on a Raspberry Pi, but preview generation and large syncs will be the painful parts. If you are deciding whether your hardware is up to it, the hardware guide has measured numbers for the exact mini PC used in these tests.

    Common gotchas

    The login page shows “The configuration is incomplete” or a redirect loop. The public URL the browser sees does not match trusted_domains, or the app thinks it is on HTTP when you are on HTTPS (or vice versa) behind a proxy. Set the trusted domain and overwrite.cli.url as described above, then clear your browser’s cached cookies for that host before testing.

    “Your web server does not seem to be correctly configured” warnings in the admin check. Nextcloud’s own web server self-test assumes Apache and flags things like the mod_headers module. Behind a reverse proxy in Docker, most of these warnings are false positives — the proxy, not the internal Apache, is what the internet sees. The ones worth acting on are the database (PostgreSQL version) and the PHP memory limit; the rest you can safely ignore in a containerized setup.

    Files do not sync to the desktop client. The sync client connects to the public URL you gave it, not to Docker internals. If you are using Nextcloud only on the LAN, point the client at http://<server-ip>:8084 (or your proxy domain). If you are behind a reverse proxy, use the proxy URL — and make sure the app’s overwrite.cli.url matches what the client uses, or WebDAV responses will reference the wrong host and the client will stall.

    Disk filling up faster than expected. The invisible consumers are versions, the trash bin, and the preview cache. In the admin settings you can see how much each occupies, and you can shorten the retention windows. There is also a occ command to trim versions and previews in one pass when you need space back fast.

    How this fits the rest of your home server

    Nextcloud becomes the file layer your other services plug into. It pairs with Jellyfin (point the media server at the Nextcloud-stored movies and TV, or keep media on a dedicated share), with the MinIO guide if you want S3-style object storage instead of or alongside WebDAV, and with the 3-2-1 backup strategy so the nextcloud-data volume — the one that actually contains your files — is backed up off-box. If you will be reaching Nextcloud from outside the house, do it over a VPN such as Tailscale rather than port-forwarding, and put a reverse proxy in front if you want a clean domain and automatic HTTPS.

    Tested on:

    OSDebian 12
    Docker29.7.2
    Hardware4-core / 16 GB
    SoftwareNextcloud 34.0.3 + PostgreSQL 16 (alpine)

    Last tested: 3 September 2026

    What’s next?

    The natural next steps from this guide:

  • Caddy in Docker: Reverse Proxy with Automatic HTTPS for Self-Hosted Services

    Caddy in Docker: Reverse Proxy with Automatic HTTPS for Self-Hosted Services

    Every self-hosted service starts life behind a raw IP and port: http://192.168.1.42:8084. It works, but it is not the web you are used to — no HTTPS, no domain name, no sensible routing. A reverse proxy sits in front of your services and fixes all three: it owns your domain (or a local one), terminates TLS, and routes each hostname to the right container. Caddy is the reverse proxy I recommend for home servers because the configuration is a single readable file and, if you point a real domain at it, HTTPS certificates are obtained and renewed automatically with zero configuration. This guide runs Caddy in Docker Compose and walks through a Caddyfile that actually serves multiple self-hosted services.

    Intermediate · 10 min · Docker

    Everything here was tested on a Debian 12 mini PC with Docker 29.7. In the lab I test plain HTTP routing (no domain, no certificate), and I flag exactly where your setup will differ once you have a domain and a public IP.

    What Caddy does for you

    A reverse proxy receives requests on ports 80 (HTTP) and 443 (HTTPS) and forwards them to the services on your network based on the host name in the request. Three things make Caddy a good fit for a home server. First, the Caddyfile is plain, human-readable configuration — you will understand every line in the example below. Second, automatic HTTPS: if you configure a real domain, Caddy requests a Let’s Encrypt certificate for it on first use, stores it, and renews it before expiry. There is no cron job to write and no certificate to babysit. Third, it is a single small static binary in the caddy image — no database, no plugin system, fast to start and easy to restart when you edit the file.

    The flip side to understand: automatic HTTPS needs two things that a LAN-only test does not have — a domain that points at the machine, and inbound 80/443 reaching it. For purely internal use (you browse services by name on your own network), you can run Caddy in plain-HTTP mode, which is what the lab test below does.

    The compose file

    services:
  caddy:
    image: caddy:latest
    container_name: caddy
    restart: unless-stopped
    ports:
      - "127.0.0.1:8083:80"
      - "127.0.0.1:8443:443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy_data:/data
      - caddy_config:/config
    environment:
      - TZ=Europe/London

volumes:
  caddy_data:
  caddy_config:

    Two volumes are the only persistent state. caddy_data stores the TLS certificates it obtains (empty in the lab test); caddy_config stores Caddy’s internal JSON representation of your Caddyfile. The Caddyfile itself is mounted read-only, so editing it on disk and reloading is the normal workflow. The port mapping here binds to loopback on remapped ports for the lab; on a real server with a domain you would map "80:80" and "443:443" so Caddy can receive HTTP (for certificate challenges) and HTTPS from the internet.

    A Caddyfile that routes several services

    Here is the file I tested. In the lab it responds to any host name on port 80 with a plain text marker, which proves the container is listening and routing. On a real deployment you replace the placeholder sites with your actual host names.

    # Lab: prove Caddy serves on :80 with a fixed response.
:80 {
    respond "caddy-lab-ok" 200
}

    The production-shaped version for a home server with a domain looks like this. Each block is one host name; Caddy matches the incoming Host header and proxies to the matching upstream. Upstream addresses use Docker service names when the services are on the same Docker network as Caddy (add Caddy to that network), or the server’s LAN IP otherwise.

    cloud.example.com {
    reverse_proxy nextcloud:80
}

vault.example.com {
    reverse_proxy vaultwarden:80
    encode zstd gzip
}

status.example.com {
    reverse_proxy uptime-kuma:3001
}

    Notice how little is there. No certificate directives: because these are real domains, Caddy obtains and renews the certificates automatically. The encode line is optional compression. If you do not yet have a public domain, the common home approach is to use .local or .lan names with a plain-HTTP site block (the :80 style, or cloud.local without a certificate), and add the domain + HTTPS later when you want the service reachable outside the house.

    One routing detail that saves pain: put Caddy on the same custom Docker network as the services it proxies. Then reverse_proxy nextcloud:80 resolves via Docker’s built-in DNS. If Caddy is on the default bridge and your apps are on a custom network (or vice versa), the service name will not resolve and you will get 502s — the most common first-run error with a reverse proxy.

    Reloading the configuration

    Caddy watches /etc/caddy/Caddyfile inside the container. When you edit the mounted file, Caddy performs a zero-downtime reload automatically — you do not need to restart the container. You can also trigger it explicitly:

    docker exec caddy caddy reload --config /etc/caddy/Caddyfile

    If your edit has a syntax error, the reload fails and Caddy keeps serving the last valid configuration (it does not go down). Check docker logs caddy for the specific line that is wrong. This behaviour is why a reverse proxy is safer to configure live than, say, a system nginx on a production box — a typo does not take the site offline.

    Common gotchas

    502 Bad Gateway immediately after adding a site. Caddy is running but cannot reach the upstream. Almost always a network problem: Caddy and the target service are not on the same Docker network, so the service name does not resolve. Verify from inside Caddy: docker exec caddy getent hosts nextcloud. If that fails, join the networks (or use the LAN IP).

    Certificates never appear even though you configured a real domain. Caddy could not complete the HTTP-01 challenge: inbound port 80 is not reaching the container, the domain’s A record does not point at the machine, or a router in between is terminating/altering HTTP. Confirm curl -v http://yourdomain from the internet reaches Caddy before expecting a certificate. On CGNAT or without a public IP, automatic HTTPS via HTTP-01 will not work; use DNS-01 (with a DNS provider token in the Caddyfile) or serve it internally and reach the service over a VPN such as Tailscale.

    Your service redirects to http:// and loops. Some apps (Nextcloud in particular) detect they are behind a proxy and, unless told the external host, generate URLs with the wrong scheme or name. Set the app’s “trusted domain” and “overwrite protocol” to the public host name — for Nextcloud that means trusted_domains plus overwrite.cli.url in its config. If the loop persists, temporarily set trusted_proxies to your proxy’s address so the app sees the correct X-Forwarded-For.

    Port 80 is already used on the host. If you have another service publishing 80 (a host nginx, a Pi-hole with a web UI on 80), Caddy cannot also bind it. Remap Caddy to a different host port for testing, or move the other service off 80 — but note that Let’s Encrypt’s HTTP-01 challenge specifically needs 80, so a permanent remap breaks automatic HTTPS for the HTTP challenge.

    How this fits the rest of your home server

    Caddy is the front door that lets you run many services under one domain without a tangle of ports. Pair it with the SSH and firewall guide to keep only 80, 443 and SSH reachable, and everything else internal. If you would rather not expose ports at all, the Cloudflare Tunnel guide is the complementary no-port-forwarding path. For internal DNS so your service names resolve cleanly on the LAN, point your devices at a self-hosted resolver such as AdGuard Home or Pi-hole, and for the bigger picture of how to choose what to run, the self-hosting starter guide.

    Tested on:

    OSDebian 12
    Docker29.7.2
    Hardware4-core / 16 GB
    SoftwareCaddy 2.11.4

    Last tested: 3 September 2026

    What’s next?

    The natural next steps from this guide:

  • Tailscale in Docker: Encrypted Access to Your Home Server, No Port Forwarding

    Tailscale in Docker: Encrypted Access to Your Home Server, No Port Forwarding

    Port forwarding is the old way to reach your home server from outside: punch a hole in your router, pick a public port, hope your ISP does not block it, and accept that every forwarded port is visible to scanners within minutes. Tailscale takes a different route. It builds an encrypted mesh network (WireGuard under the hood) between your devices, and once your home server is a node in that network, you can reach it from anywhere using a private IP — without opening a single port on your router. This guide runs Tailscale in a Docker container, which gives you the cleanest separation: the VPN runs in its own namespace, its keys live in one volume, and removing it leaves your host untouched.

    Beginner · 10 min · Docker

    Everything here was tested on a Debian 12 mini PC with Docker 29.7, including the two failure modes that trip up most first-time setups (TUN device and authentication), with the exact error messages you will see and how to fix them.

    Why a container for Tailscale

    You could install the Tailscale CLI natively on the host, and that works fine. The container approach has three practical advantages. First, the tailscale/tailscale image is maintained by the Tailscale team, so you get the current release by pulling the image — no host package management. Second, the state (the node key and the tailscaled database) lives in a named volume; back it up or move it and the node identity moves with it. Third, the container needs net_admin and net_raw capabilities plus a TUN device, and granting those to one container is easier to audit and revoke than granting them to a host daemon.

    The compose file

    services:
  tailscale:
    image: tailscale/tailscale:latest
    container_name: tailscale
    restart: unless-stopped
    devices:
      - /dev/net/tun:/dev/net/tun
    cap_add:
      - net_admin
      - net_raw
    environment:
      - TS_AUTHKEY=
      - TS_STATE_DIR=/var/lib/tailscale
      - TS_SERVE_MODE=off
      - TS_USERSPACE=false
    volumes:
      - tailscale-data:/var/lib/tailscale

volumes:
  tailscale-data:

    Before starting it, make sure the host has a TUN device. On most Debian systems it already does; if not, create one:

    sudo modprobe tun
sudo tee /etc/modules-load.d/tun.conf <<< "tun"

    The four TS_* variables matter, so here is what each does. TS_AUTHKEY is an optional authentication key you generate at your Tailscale admin console (Keys section). If you set it, the container registers and authorizes itself on first boot — useful for a headless box. Leave it empty and you will authenticate interactively instead. TS_STATE_DIR is where the node key is stored; the volume keeps it across container rebuilds. TS_SERVE_MODE controls the newer “tailscale serve” feature (proxying a local port through a Tailscale public hostname); “off” keeps things minimal. TS_USERSPACE=false means the container uses the kernel TUN device (the fast path) rather than a userspace socket.

    First boot and the two ways to authenticate

    Run docker compose up -d, then watch the log:

    docker logs -f tailscale

    You should see the daemon starting and, depending on your auth mode, one of two outcomes.

    With an auth key (set TS_AUTHKEY): the log shows the node key being created and the device appearing in your admin console as approved, within a few seconds. If you created the key with an expiration, note that the key is single-use by default — the node keeps working after it expires, the key just cannot re-authenticate a new node.

    Without an auth key: the log prints a login URL (something like https://login.tailscale.com/a/xxxxx). Open it on any machine, sign in, and approve the device. Then verify from inside the container:

    docker exec tailscale tailscale status

    The status line should read Online and list your node name. If you see NeedsLogin, the interactive step has not been completed — re-run the URL. If you see Expired on the node key, run docker exec tailscale tailscale up (or restart with a fresh auth key) to re-key.

    This is the failure mode that confuses most people: the container looks healthy, the log looks calm, but tailscale status says NeedsLogin. There is no error to fix — you just have not finished the approval. Check the admin console, not the log.

    Reaching the home server from outside

    Once the node is online, it has a stable 100.x.y.z address (your “MagicDNS” name also works: yourbox.tailnet-name.ts.net). From any other Tailscale device — your laptop on a café WiFi, your phone on 4G — you can connect straight to the server:

    ssh you@yourbox.tailnet-name.ts.net

    No router changes. No public IP. The traffic is WireGuard-encrypted end to end, and nothing on your LAN is reachable except what you explicitly share. That last point is the big security win over port forwarding: with a forwarded SSH port, the port is open to the entire internet. With Tailscale, the “port” only exists inside your private mesh, and every peer must be an authenticated node you added.

    For services you want reachable by specific peers rather than just SSH, the node’s “Advertise tags” or, more commonly, per-service ACLs in the admin console control who can reach which port. A minimal ACL that lets only your laptop reach the server is a few lines of JSON in the admin console, and it is the piece most people skip — do not skip it. The default “everyone in the tailnet can reach everything” is fine for a single-person tailnet and wrong for a family one.

    Performance and resource use

    In my lab, the container idled at around 15–20 MiB of RAM. Throughput is WireGuard’s: on this 4-core box I sustained well over 1 Gbps of encrypted traffic in local tests, which is far beyond anything a home broadband link will push. Latency inside the tailnet adds a few milliseconds (the relay hop when two peers cannot connect directly — Tailscale tries a direct connection first and falls back to a DERP relay when NATs block it). For SSH, browsing a web UI, or streaming a personal radio stream, you will not notice the difference. For bulk file transfers between two remote peers, a relay hop can roughly halve throughput; if that matters, enabling port 443 outbound on the router lets peers connect directly.

    Common gotchas

    “tun device not found” or the container crash-looping. The host is missing /dev/net/tun. Load the module as shown above and confirm ls /dev/net/tun exists, then docker compose up -d again. This is the number-one cause of a red container on first boot.

    Container is up but other containers cannot reach the tailnet (and vice versa). Each Docker container has its own network namespace. Tailscale inside one container only routes traffic for that container. If you want your other home-server containers (Gitea, Nextcloud, the dashboard) reachable from your laptop over the tailnet, the standard fix is to give the Tailscale container access to your app network: join it to the same custom network as your services, or run the apps on the host network. The alternative — TS_USERSPACE with a shared socket — is more fiddly and slower; a shared Docker network is the pragmatic choice.

    Your router’s CGNAT. If your ISP puts you behind CGNAT (common on mobile and some residential plans), you may never have a public IP. That is exactly the situation Tailscale is built for — the relay handles it — but it also means the “direct connection” optimization will not kick in for inbound. Expect relayed performance and do not chase it as a bug.

    Node key rotation. Tailscale expires node keys after a year by default. When that happens the node goes offline and you re-authenticate. Set a reminder or, on a headless box, generate a long-lived auth key so re-keying is a one-liner.

    How this fits the rest of your home server

    Tailscale is the backbone that makes the rest of a self-hosted setup safe to use from outside. Instead of port-forwarding the web UI of every service you run, you reach each one through the tailnet: Vaultwarden from your phone while travelling, a Gitea or media UI from a café, your dashboards without any of them ever touching the public internet. If you do want some services on the open web — a personal blog, a public API — the Cloudflare Tunnel guide is the no-port-forwarding complement, and the SSH and firewall guide covers hardening the host that now sits behind your mesh. For the full list of services we have tested and written up, see the software index.

    Tested on:

    OSDebian 12
    Docker29.7.2
    Hardware4-core / 16 GB
    SoftwareTailscale 1.102.3 (container)

    Last tested: 3 September 2026

    What’s next?

    The natural next steps from this guide: