active

Homelab AI infrastructure

An NVIDIA DGX Spark on my desk: local models, self-hosted services, and infrastructure as a learning practice.

NVIDIA DGX Spark · vLLM · llama.cpp · Qwen 3 · Gemma · Linux · self-hosting

The most valuable AI education I've found doesn't come from API calls — it comes from running the machines yourself.

My homelab is built around an NVIDIA DGX Spark: desk-sized, GPU-accelerated, and mine. It runs the local language models behind the AI writing pipeline and The Agentic Daily, plus whatever else I'm currently curious about.

What it actually is

NVIDIA DGX Spark  ·  GB10 Grace Blackwell  ·  121 GB unified memory  ·  aarch64

  serving      vLLM · llama.cpp
  models       Qwen 3 · Gemma 3-4 · DeepSeek
  constrained  GBNF grammars, JSON schema, structured decoding
  orchestrated Docker · cron · systemd

The unified memory is the part that changes what's possible: 121 GB shared between CPU and GPU means models that would need a multi-card rig fit on one desk-sized box, and the aarch64 architecture means roughly half the tooling needs a second look before it works.

Why local

  • You learn the whole stack. Serving a model yourself teaches you about memory, quantization, throughput, and failure modes that an API abstracts away.
  • Privacy is structural, not contractual. Drafts, experiments, and personal data never leave the room.
  • Cost is a one-time decision. Experimentation stops being metered, which changes what you're willing to try. This matters more than it sounds: when inference isn't billed per token, you can afford thirty replicates of an evaluation instead of one — and that is often the difference between a result and a coin flip.
  • You end up fixing the stack, not working around it. Running vLLM in anger surfaces real bugs. One of mine — a crash in the Gemma 4 tool parser — is now merged upstream.

What runs here

The centerpiece is The Agentic Daily, which publishes an edition every day without asking me first. Around it: the writing pipeline that feeds this blog, model experiments, evaluation harnesses, self-hosted services as they earn their keep, and — coming soon — this site's analytics on the same box.

This project is active — I write up pieces of it on the blog as they mature.