Michael Hospedales · Software engineer · Miami, FL
Web products, autonomous AI systems, and the ideas that come from making things. This is where I share what I'm building, learning, and wondering about.
An autonomous newsroom that turns a daily stream of sources into an edition, and a working experiment in whether AI can make reliable editorial decisions.
Readers following AI and technology, and builders investigating autonomous systems.
The product, decisions & lessons →
Also buildingAI writing pipeline →Homelab AI infrastructure →This site: the refresh →
I build to solve problems. I have creative interests outside of programming, like art and music. I used to make my own 3D models and assets for Quake and Unreal, back in the day…
The model card advertised a 4B multi-token-prediction head. No published GGUF contained it, and the architecture had no code path to run it. Both halves arrived within 36 hours from two different strangers, in incompatible forms — so I merged the head into the target file myself. It went from 27.75 to 43.30 tok/s, and three of my four mistakes along the way were about verification, not tensors.
An 87 GiB model got OOM-killed with nothing in its own log. The process that triggered it was a 27 MB dashboard service. The reason my inference server was the kernel first choice was not its size — it was a systemd user-manager default that scores every terminal-launched process to die before any system service. Then restarting it killed my editor.
A 125B model decoding at 26.6 tok/s looks unremarkable beside a 31B at 29.3 — until you notice the 31B is running a drafter, so its number is a forward-pass rate multiplied by an acceptance length. Per forward pass the big model was 3x faster. Here is how to decompose the number, and what the bandwidth arithmetic says is still on the table.
Four slots served my agentic pipeline at 1.2% prefix-cache reuse. One slot served the same pipeline at 86.7%, and saved 11.6 minutes of prefill in a 39-minute window. Slots schedule requests; they do not add compute — and on a single GPU they scatter the one thing a prefill-bound workload actually depends on.
Follow along at your own pace. Subscribe with RSS
Updated August 26, 2026
A problem, a half-formed idea, or a good conversation. No project or budget required.