Guides
Short guides on real developer problems. Each one has a worked example you can run and links to open a full breakdown with side chats.
Debug an LLM agent: find the first bad step in its trace
Debug an LLM agent run in Python. Read one recorded refund trace, label the first bad step, fix the tool wrapper and keep the run as a pytest test. No API key.
Turn a failed agent run into an eval that blocks the release
Save one failed LLM agent run as a pytest eval case, grade what changed and not what the agent said, and let CI block the release. Python, no API key.
Did the fix work, or was it luck? Compare two versions of an LLM agent
An LLM agent passed 7 of 10 runs before a fix and 9 of 10 after. Run an exact test in Python, plan how many runs you need and write a ship rule. No API key.
Retry a timed-out agent tool call without refunding twice
A refund call timed out and the agent retried. Did the customer get two refunds? Run it in Python, add an idempotency key and check the record. No API key.
One slow LLM call is blocking your queue. Measure it and fix it in Python
One 45 second LLM call can leave fast messages waiting minutes. Simulate a queue in Python, then fix it with workers that keep each chat in order. No API key.
Will 32 chats fit on one GPU? Size the KV cache in Python
A 7B model serves one chat fine. Will 32 chats fit on a 24 GiB GPU? Size the KV cache in Python from the model's config file, then set a chat cap.
Will 8-bit weights fit 32 chats on one GPU? Size 16, 8 and 4-bit in Python
A 7B model at 16 bits fits 16 chats on a 24 GiB GPU, not 32. Size 8-bit and 4-bit weights in Python, then see why 4-bit needs a scale per block.
Your RAG bot gave a wrong answer. Was it retrieval or generation? Find out in Python
A RAG bot says 30 days, the policy says 45. Count retrieval and generation misses in Python, fix the retriever one change at a time, then test it. No API key.
You fit 32 chats on one GPU. How fast will each one decode? Estimate it in Python
A 7B model fits 32 chats on a 24 GiB GPU. How fast is each one? Estimate tokens per second from memory bandwidth in Python, then check it. No GPU needed.
Your LLM labels each ticket for 2.2 cents. Can a small model take over? Shadow-run it in Python first
Run a small classifier beside your LLM, judge every disagreement, set a confidence threshold with a fallback and price it. Python, made-up tickets, no API key.
Did a hijacked PyPI package run on your machine? Audit Python startup hooks and pick a cooldown
Python runs .pth startup lines on every launch. Plant a harmless canary, scan site-packages for hooks, and pick a release cooldown from a real hijack timeline.
A long prompt makes your chat wait before the first word. Why? Size prefill and decode in Python
Size the wait before the first token and the speed after it from one formula, run it in Python, then check it against a measured llama.cpp table.
Will speculative decoding speed up your LLM chat? Size drafts, acceptance and load in Python
Size the speedup from acceptance rate, draft length and draft cost in Python, see why more chats shrink it, then check it against a paper and a measured table.
Your coding agent can read your API keys. Give it a ticket instead. Build the broker in Python
See three ways an agent leaks your key, then build a small broker in standard library Python that lets it call one route with a ticket that expires.
Studying another topic? Enter it on the home page to get a revision guide and side chats.