If you have grown weary of sending your source code to GitHub’s servers every time you want a completion suggestion, TabbyML is the answer you have been waiting for. It is a fully self-hosted, open-source coding assistant that speaks the same IDE extension language as GitHub Copilot but keeps every token on your own hardware. This guide walks you through standing up a TabbyML instance with Docker, wiring it into VS Code and JetBrains IDEs, and picking the right base model for your machine.

Why Self-Host a Coding Assistant?

Privacy is the obvious argument. Corporate codebases, client NDA work, and security-sensitive projects should not be shuttled through a third-party API you do not control. But self-hosting also means zero per-seat licensing costs, offline operation, and the freedom to swap underlying models whenever a better one ships.

TabbyML bundles an OpenAPI server, an optional admin dashboard, and first-class IDE plugins into a single project. It is actively maintained, has a growing community, and supports the two most capable open-weight coding models available today: DeepSeek Coder and StarCoder2.

Installing TabbyML with Docker

The fastest path to a running instance is Docker. The project publishes separate images for CPU-only and CUDA GPU targets.

GPU (NVIDIA CUDA) Setup

docker run -d \
  --name tabby \
  --gpus all \
  -p 8080:8080 \
  -v $HOME/.tabby:/data \
  tabbyml/tabby:latest \
  serve --model TabbyML/DeepSeek-Coder-6.7B --device cuda

Make sure you have the NVIDIA Container Toolkit installed (nvidia-container-toolkit) and that docker info | grep -i runtime shows nvidia as an available runtime.

CPU-Only Setup

docker run -d \
  --name tabby \
  -p 8080:8080 \
  -v $HOME/.tabby:/data \
  tabbyml/tabby:latest \
  serve --model TabbyML/StarCoder2-3B --device cpu

CPU inference is noticeably slower, so the 3B parameter StarCoder2 variant is the practical choice over the 6.7B or 15B weights. Expect roughly 8–15 tokens per second on a modern 8-core machine.

After a minute or two the model weights download automatically into ~/.tabby. Visit http://localhost:8080 to confirm the server is healthy. You will see a small dashboard with completion metrics and an API playground.

Connecting TabbyML to VS Code

Install the official Tabby extension from the VS Code Marketplace (search “Tabby” by TabbyML). After installation:

Open the Tabby extension settings (Ctrl+, → search “Tabby”).
Set Endpoint to http://localhost:8080.
Leave the token field blank if you have not enabled authentication in the server config.
Reload VS Code.

Inline ghost-text completions will appear as you type, using the same Tab to accept / Escape to dismiss UX that Copilot users already know.

Optional: Enable Authentication

In production, add --auth-token mysecrettoken to the Docker command. Then set the VS Code extension token field to match. This prevents other machines on your network from using your compute without permission.

Connecting TabbyML to JetBrains IDEs

Open Settings → Plugins → Marketplace, search for Tabby, and install the JetBrains plugin. Configuration is identical: point the endpoint to http://localhost:8080 and optionally supply a token.

The plugin works in IntelliJ IDEA, PyCharm, GoLand, WebStorm, Rider, and any other JetBrains product that supports third-party completion providers. Completions integrate with the native inline suggestion UI so there is no jarring context switch.

Choosing a Base Model

DeepSeek Coder

Model	Params	Languages	Best For
DeepSeek-Coder-1.3B	1.3B	87	CPU, low RAM
DeepSeek-Coder-6.7B	6.7B	87	GPU (8 GB VRAM)
DeepSeek-Coder-33B	33B	87	Multi-GPU workstations

DeepSeek Coder consistently tops fill-in-the-middle (FIM) benchmarks and has particularly strong Python, TypeScript, and Rust coverage. The 6.7B model is the sweet spot — it fits comfortably on a single RTX 3060 or 4060 and produces genuinely useful completions.

To switch models, stop the container and rerun the docker run command with a different --model flag:

docker run -d --name tabby --gpus all -p 8080:8080 \
  -v $HOME/.tabby:/data tabbyml/tabby:latest \
  serve --model TabbyML/DeepSeek-Coder-1.3B --device cuda

StarCoder2

StarCoder2 is the Hugging Face / ServiceNow collaborative model trained on The Stack v2. It supports 600+ programming languages, which makes it the better choice if your work involves niche languages like COBOL, Julia, or Zig. The 3B and 7B variants are well-optimized for inference and are slightly more permissively licensed than DeepSeek for commercial use.

serve --model TabbyML/StarCoder2-7B --device cuda

GPU vs CPU Inference: What to Expect

Setup	Model	Tokens/sec (approx)
RTX 4090 (24 GB)	DeepSeek-Coder-6.7B	80–120
RTX 3060 (12 GB)	DeepSeek-Coder-6.7B	30–50
Apple M3 Pro (via Docker)	StarCoder2-3B	15–25
8-core x86 CPU	StarCoder2-3B	8–15

Completion latency under 200 ms feels instant. Anything above 500 ms starts to break flow. If your CPU inference lands above 500 ms per suggestion, drop to the 1.3B DeepSeek model or invest in a used RTX 3060.

Advanced Configuration

TabbyML’s configuration file lives at ~/.tabby/config.toml. Two settings worth knowing:

[completion]
max_input_length = 1024   # context window fed to the model
timeout_secs = 5           # drop completions that take too long

[model]
http_api_timeout_secs = 30

You can also run TabbyML with a retrieval-augmented code completion feature that indexes your local repository and surfaces relevant snippets as additional context. Enable it by adding --chat-model TabbyML/Qwen2-1.5B-Instruct to the serve command, which unlocks a lightweight chat sidebar alongside completions.

Final Thoughts

TabbyML closes the gap between “good enough” open-weight models and a polished developer experience. A 6.7B DeepSeek Coder on an entry-level gaming GPU delivers completions that will genuinely surprise you, and the zero-egress-cost story is a legitimate advantage over any cloud-based alternative. Start with the Docker setup, spend an afternoon evaluating completions in your actual codebase, and adjust the model size from there.