Apprenticing a small model
Articles - 2026-08-28 - by William Dunn
ai, software-development, architecture, agents, local-models
We run a small model as the lab's junior coding agent, on our own hardware.
Any model small enough to be cheap to host has these problems. For its first
weeks it chased the same problems down one at a time and got nowhere. We
stopped treating it as a tool to configure and started treating it as an
apprentice. An apprentice needs a shop. We built one. The shop is a policy
layer that refuses dangerous commands at the tool interface, a shared
procedure store holding every lesson the fleet has learned, a regression
suite that scores the model against fixed tasks with deterministic checks,
and a circuit breaker that stops it after two failures and escalates to a
frontier model. Each section below takes one mechanism, what it does, and
the reason it exists. The generic parts ship in
Rote (https://github.com/wdunn001/rote).
Building it that way inverts what a failure costs. A failure that ends in a
written procedure buys something permanent. It gets paid for exactly once.
The next agent inherits the fix without ever meeting the problem. Counted
that way every failure in this article was a success. The only wasted
failure is the one nobody wrote down.
The failures that make a coding agent expensive have little to do with
parameter count. It reinvents procedures settled months ago. It retries
commands that cannot succeed. It does something forbidden, gets corrected,
and starts the next session with no memory of the rule or the correction.
All of that traces to one property. A model holds no durable state across
the session boundary. Whatever it learned on Tuesday is gone on Wednesday.
The correction you typed is gone with it. The reflex is to reach for a
bigger model. A frontier model dropped into the same bare environment
fumbles in the same places at a higher price per token. The deficit sits in
the shop. That is the part you can actually change.
The policy layer
A working shop does not handle a dangerous tool by telling the apprentice to
be careful. It puts a guard on the saw and builds a jig so the wrong cut
cannot be made. Ours sits at the tool interface. A configuration file denies
the shell patterns that have caused damage in this lab. A pair of git hooks
rejects commits carrying the wrong identity, machine-prose punctuation,
attribution lines naming an AI, or credential-shaped strings in the diff.
Refusal happens before the request reaches a shell.
The failures that justify it are dull and repetitive. The model responds
verbosely on failed tasks. It keeps trying after repeated failures. It
starts dev servers in the foreground, where they hang until something kills
them. It invented a git identity for itself. Every one of those is cheap to
enforce against and expensive to argue with.
A told rule is a prompt whose compliance is probabilistic. An enforced rule
is a precondition that does not drift.
We told the model in plain language to run a forbidden command. That is a
rigged test and proves nothing on its own. Of course it tried. The tool
refused. Unprompted, it proposed an alternative that was also on the deny
list. Being corrected once had not put the rule into the apprentice. It
reached straight for a neighbouring way to do the same damage. That is the
argument for the jig. It is the only part of that test I would defend.
A deny list covers only harms somebody has already met. The list is not
written once and left alone. Widening it is a job for the frontier model. It
is better than the apprentice at enumerating the neighbouring ways to cause
the same damage. It does that work against the record of what the apprentice
has actually attempted.
A list drawn too broadly starts refusing legitimate work. That failure is
not silent. A blocked legitimate command is a blocked task. A blocked task
trips the breaker and files a ticket. The frontier model reads the ticket
and narrows the rule that was too broad. The guard on the saw gets adjusted
by the same loop that put it there.
The procedure store
A procedure the apprentice recalls beats one it derives. The derived version
is a fresh roll of the dice every time. The model re-derived work that was
already settled, scaffolding a project from first principles in a lab that
has had a standard way to do that for a year.
The procedure store holds small documents, each saying when to act and how,
each pointing at a library script for the steps that must run identically
every time. Vector indexing means lookup happens by meaning and not by exact
name. The catalog stays in the model's context. Loading a full procedure is
one call.
None of that is new. The store is sqlite-vec with a small off-the-shelf
embedding model, roughly twenty-two million parameters, the kind of thing
that has been sitting in a hundred tutorials for years. Semantic lookup over
a document set is decades old. The tools reach the model over the Model
Context Protocol. That is a wire format, and every tool here would work the
same against any other runtime that speaks it. There is no algorithm here
that somebody did not already have. The parts are ordinary. Assembling them
and then living with the result is the entire contribution.
A person reviews each document once. Every agent in the fleet recalls it
from then on. That is DRY, the rule that says do not repeat yourself,
applied at the level of the institution. It is the same shape as the
stenographic mediator,
where one canonical record lives in one place and every reader holds a
synced copy. The store is also where the shop stopped belonging to one
apprentice. The frontier model reads the same documents. Every agent we run
works off the same bench notes.
It settles an argument I made in an earlier
piece. A skill, in the
sense the word has taken on, is a document telling a model how to do
something. A document is a wish. The model may follow it. It may drift. You
find out afterwards. Recall against a reviewed procedure that hands the
deterministic steps to a script removes the drift. The uncertain step
shrinks to choosing which procedure to reach for.
The regression suite
A guild apprentice ended the term by making a test piece. A good enough one
earned the rank of master, which is where the word masterpiece comes from.
Ours is nine tasks, run with nobody watching, each with a check that returns
the same verdict every time. Add two numbers. Create a file with exact
contents. Read a supplied file and answer from it. Edit one setting without
disturbing the setting beside it. Load a named procedure from the store and
answer from it. Recall a fact from the memory index. Generate a new project
from a template without hanging. Look up a service through the lab's
directory tools. Commit a prepared change under the correct identity in a
repository where the hooks are armed.
Nearly half of those exist because the model failed exactly that task in
real work. Every run also scans output for raw tool markup, the machine
syntax a model is meant to execute and never print at you.
The first run scored 7 of 9. The scaffolding task produced nothing. The
commit task wandered into the wrong repository, where the model tried to
configure a fresh git identity for itself, invented on the spot, with an
email address at a domain nobody had given it. The hook rejected the commit.
One more measurement, taken while an earlier draft of this article was being
written. Asked to report its own word count, the model answered 2,497. The
file held 1,489. Self-report and measurement are different instruments. The
suite exists because only one of them is load-bearing.
The circuit breaker
An agent with no stopping rule grinds at an impossible task in silence until
somebody notices. That costs hours and teaches nobody anything. The failure
mode is not the wrong answer. It is the absence of any signal that the work
went wrong at all.
Circuit breaker is Michael Nygard's term for a component that stops calling
a failing dependency once failures cross a threshold. Ours applies that
shape to a model attempting a task. We already had a hand-off rail pushing
bulk work from the expensive model down to local hardware. The breaker
reverses the flow so the apprentice can hand work up to the master.
The model gets two honest tries. Then it downs tools. It files a ticket
recording what it wanted, what it tried, and what the errors actually said,
then moves on to work it can finish. Filing rings our phones through the
home automation system. Nothing in the shop penalises the apprentice for
stopping. Because a ticket is the one move guaranteed to produce a durable
lesson, asking for help is the most valuable thing it can do when stuck.
The queue lives beside the procedure store. A daily job wakes the frontier
model only when tickets are open and hands it a standing brief pointing at
the resolution procedure. An empty queue costs nothing. A full one gets
attention within a day without anyone remembering to call.
Both first-day failures went through this loop while the apprentice worked
on other things. The scaffolding failure came back as a procedure giving the
exact command with every prompt answered in advance. Nothing can stall
waiting for input. The identity failure came back as two rules and a
correction to the hook itself. We had written the hook stricter than the
shop it protected. The suite passed 9 of 9 by evening. The apprentice found
itself more capable on the next sync.
Handing the work back
A master who takes the work back has taught nothing by finishing it well.
This is the rule the loop exists to hold. It is also the one a person breaks
first. A service went down when an agent overwrote a host file holding the
only copy of two secrets. We recovered the secrets, rebuilt the
configuration, and verified the service inside an hour. It felt efficient.
The agent that caused the outage learned nothing from any of it. None of the
recovery passed through it.
The rule it violated is now a written procedure every agent loads before
touching a host file. That agent was resumed, handed its report card, and
sent to finish the work with the procedure in hand. Discipline that lives in
a person's willpower loses to any sufficiently urgent afternoon. It has to
live in the loop.
Where the knowledge lives
The nearest established technique is knowledge distillation. Distillation
trains a teacher model's behaviour into a student's weights. The difference
is where the knowledge ends up. Distilled knowledge compiles into a
checkpoint. It costs nothing at inference time. It is also invisible,
unauditable, and still probabilistic. It teaches one student per training
run. It dies with that checkpoint. It is apprenticeship by watching over a
shoulder, with nothing written down.
The store is also where a shop becomes yours. Most of what makes a team good
is specific and unglamorous. One client sends invoices in a format nobody
else uses. One machine has to be drained before it is serviced. One
environment never gets touched on a Friday. No model ships with that in its
weights. None ever will, since it is true only of you. Written into the
store it stays on your hardware, under your own backup, readable by a person
who can correct it when it changes. Every agent you run picks it up on the
next sync. That includes the model you swap in next year and the vendor's
model you rent for an afternoon. They visit. The shop is what remembers.
Knowledge in the procedure store costs a lookup. In exchange a person can
read it, correct it, and date it. Every agent receives it at once. It
outlives the model that produced it. When we swap the model for next year's
version, the shop stays.
Invisible inheritance runs in both directions. A defect propagates by the
same mechanism a lesson does. Nobody can read it, audit it, or trace where
it came from. That is the subject of an earlier piece on inherited
defects. A written procedure
can be wrong too. You can at least see that one and take it down.
What it would take at work
Rote (https://github.com/wdunn001/rote) is the base tool, the generic half
of this shop with the lab-specific parts stripped out. Four things stand
between it and something a company could run.
Vault configuration. Secrets are a local vault the operator wires by
hand. A company wants procedures that read credentials from whatever it
already runs, with every access recorded.
A real medium for the knowledge base. The store is sqlite-vec on one
disk. That is correct for a lab. A team needs a database somebody else backs
up, with the usual controls over who can read which procedure.
Document ingestion. Most of what a company knows sits in PDFs and scans.
Without OCR the store holds only what somebody typed into it by hand, which
is a small fraction of the real corpus.
Vision. A screenshot of a failing dashboard is the most common bug
report there is. A vision-capable model reads one fine. Nothing in the shop
accepts an image as an input, attaches it to a ticket, or files it against a
procedure. The screenshot never reaches the apprentice at all.
None of those is research. Each is a known kind of work with known tools.
That work is not hypothetical. Storyplane (https://storyplane.io) already
runs an enterprise version of this pattern. There the catalog has a product
around it. The review step becomes a governance and approval trail a company
can audit.
What the shop does not do
None of this makes the model smarter. The weights are the weights. What
changes is where its limited judgment gets spent. It does not deliberate
over forbidden commands it cannot reach. It does not re-derive procedures
available by lookup. Its failures no longer evaporate at the session
boundary, since the breaker converts them into tickets and the tickets into
procedures.
The shop also makes the experiment cheap. A lighter model at the bench
teaches it more. A capable model glides over the gaps and leaves them
undocumented. A weaker one walks into every gap there is.
┃ Every specific failure is an assumption you did not know you were making, now written down.
If it performs, the fleet just got cheaper to operate. If it fails, it fails
somewhere specific and leaves the procedures behind. The next apprentice
inherits them even though the one that produced them did not work out.
That is also what makes running your own
stack practical.
Sovereignty usually stalls on capability. The model you can host is not as
good as the one you rent. The work drifts back to somebody else's servers.
The shop closes most of that gap without buying better weights.
The bet is that useful work does not require a larger or smarter model. It
requires a better shop around the one you already have. A trade outlives
every apprentice who passes through it.