A 3,100 engine for a beginner

I play chess badly. My Chess.com games start in late September and the ratings beside my name are mostly three digits, some of them low ones. Two weeks after the first of those games I started writing a chess engine, and seventeen days later it plays at about 3,100 on Stockfish's strength scale. I'm not sure what that says about me.

It's called zchess, it's written in Zig, and as of today it's public. This is what it is, what I measured, the long list of things that didn't work, and why the half I'm proudest of is the part that plays down to my level.

One binary#

zchess is pure Zig with no dependencies. The engine, the self-play data generator, the neural-network trainer and the match runner are all subcommands of one binary, along with the game reviewer and a terminal mode for playing against it:

zchess              # UCI mode: point any chess GUI at it
zchess play         # play it in the terminal
zchess datagen      # self-play games, written out as training positions
zchess train        # train a network on them
zchess match        # new network against the current best, with an SPRT
zchess review       # find the mistakes in a player's games

About 9,500 lines of Zig. The board is bitboards plus a mailbox. Move generation is fully legal, from check masks and pin rays, and runs at about 400 million nodes a second in perft on an M2 Pro. Sliding pieces use magic bitboards rather than PEXT, because Apple Silicon doesn't have PEXT. The search is the standard modern stack: principal variation search, a transposition table, null-move and futility pruning, late-move reductions, quiescence with static exchange evaluation. None of that is new, and none of it was the interesting part.

Most of the code was written with Claude, commit by commit, and the history says so. What I brought was the questions and the habit of not believing a change until it had been measured, and that turned out to be most of the work.

A network that teaches itself#

The evaluation, the function that looks at a position and says who's better, is an NNUE: a small neural network with one trick that makes it cheap enough to call millions of times a second.

The input is 768 numbers: 12 kinds of piece times 64 squares, each one either there or not. The first layer turns those into 512 hidden values, and that's where nearly all the arithmetic is. But a move changes very little. A quiet move takes one piece off one square and puts it on another, so two of the 768 inputs change. A capture changes three. So instead of redoing the whole layer, you keep the 512 sums from the previous position and add one row of weights and subtract another. That kept sum is the accumulator, and keeping it is the whole idea: it makes the expensive layer almost free, and only the small layers after it are recomputed each time. Two accumulators, in fact: one from each side's point of view.

The network has to learn from something, and here it learns from itself. Generation 0 plays thousands of games with a hand-written evaluation (piece-square tables and mobility, the 1990s answer). Every quiet position from those games is kept with the search's score and the game's eventual result, 32 bytes each:

pub const Record = extern struct {
    occ: u64,
    /// One nibble per occupied square in ascending square order: color << 3 | type.
    pcs: [16]u8,
    /// Search score in centipawns from white's point of view.
    score: i16,
    /// Game outcome: 0 = black won, 1 = draw, 2 = white won.
    result: u8,
    stm: u8,
    ply: u16,
    _pad: [2]u8 = .{ 0, 0 },
};

The trainer, also Zig with hand-written forward and backward passes, fits a network to those positions. The new network plays the old evaluation. If it wins, it becomes the evaluation for the next generation's games, and the loop goes round again.

The first run took about two and a half hours on the laptop. Generation 2 beat the hand-written evaluation by 212 Elo. Generation 3 lost to generation 2, and the gains stopped there.

The labels mattered more than the loop#

When a self-training loop stalls, it's tempting to blame the training. Here the problem was what it was being trained towards.

Each position's target is a blend of two numbers: the search's score, and who eventually won the game. The game result sounds like the honest label: it's what actually happened. So I tried giving it more weight, 40% instead of 25%. That network lost to the old one by 117 Elo. Going the other way, down to 10%, won by 146.

The reason is how the games were played. Self-play at 5,000 nodes a move is fast and sloppy, and a game is decided by the last blunder in it, not by whether a position thirty moves earlier was good. A result tells you something about the whole game and almost nothing about any one position in it. The search score is about that position alone. It's a much better teacher, and the trainer's default has been 10% ever since.

Two more findings were about the data rather than the labels. Training on all the generations so far was worse than training on only the recent ones: the oldest games were played by weak networks, and they diluted the rest. And a network twice as wide, 512 hidden values instead of 256, won by 38 Elo even after paying for being slower to evaluate. I'd expected that one to be a wash.

Measuring is most of the work#

Four hundred games sounds like a lot. It carries about ±30 Elo of error, which is larger than nearly every improvement I'd go on to make. So every change after the first run was decided by an SPRT, a sequential probability ratio test. Games keep being played until the evidence crosses a bound for "this is at least as good" or for "this isn't better", and it stops at whichever comes first.

That makes the cost of a decision depend on the size of the effect. Singular extensions, worth about 37 Elo, decided themselves in 1,372 games. Four-way transposition table buckets ran for 15,160 games and finished at +1.2 ± 3.0: not nothing, probably, but not enough to keep.

Self-play only says whether a network beats its parent. To see how strong it actually is, I played gauntlets against Stockfish 19 with its strength limiter set to 2800, 3000 and its maximum, 3190, 200 games at each, on the same machine every time:

These are a yardstick between my own nets, not a rating anyone else would recognise. Stockfish's limiter is calibrated at slower time controls than these games, and full-strength Stockfish is in another league entirely; I only measured against the limiter. One pattern held throughout: a gain in self-play arrives at about half its size against Stockfish. Generation 7 was about 87 Elo better than generation 6 against its own family, and 45 better against Stockfish. A network is best at beating the opponent it learned from.

Six Raspberry Pis, a VM and a laptop#

Self-play data is embarrassingly parallel, so it went wherever there were cores: six Raspberry Pi 5s at home and a Hetzner VM, each running the generator as a systemd service that comes back after reboots, power cuts and crashes. The Pis dial out over WireGuard to the VM, so I can reach them from anywhere and nothing at home accepts an inbound connection. Each one runs a thermal guard that pauses at 75 °C and resumes at 68.

Two of the Pis dropped off the Wi-Fi a few minutes into full load and kept generating offline. One was fine on Ethernet, which points at power: a Pi 5 under sustained load wants a 5 A supply. Several more simply went unreachable at some point and stayed that way.

In the end the laptop did most of it. Of generation 9's 151 million positions, 106 million came from the Mac, about 17 million from each of the two Pis still answering, and 10 million from the VM. The Pis were good fun and a modest share of the data. The later networks train on a window of the last three generations, 470 million positions, in about six hours.

What didn't work#

Most of what I tried, I kept out. In rough order of how much it taught me:

• King buckets. Strong engines give each perspective a separate set of input weights depending on where its own king is. With eight buckets the network lost by 60. Its validation loss was the best of the lot, and that was the clue: the validation positions came from the same games as the training positions, so a network with eight times the weights could memorise its way to a good score. I moved validation to whole held-out games, so that no validation position shares a game with a training one. The retry, four buckets with a shared factoriser that every bucket learns through, came in at +2.1 ± 3.5 after nearly 10,000 games. The factoriser works. The buckets are worth a few Elo at most.
• Training only on quiet positions: −22. A guess I haven't tested: the search also asks the network about positions that aren't quiet, so it needs to have seen some.
• 45 epochs instead of 30: +0.7 ± 3.8, for an extra four hours of training.
• Correction history, a search heuristic that helps a lot of engines: −0.8 after 8,402 games.
• Smarter multithreading, helper threads skipping depths and voting on the move: −2.5.

What did work, in the search: continuation history (+26), singular extensions (+37), time management that spends more on unstable positions and less on settled ones (+22), capture history (+7). Storing the first layer's weights as 8-bit integers instead of 16 was worth +9. That's partly speed, and partly that it freed headroom elsewhere in the arithmetic and so made the quantisation more accurate overall.

The speed work needed no SPRT at all, which I liked. Two passes left the search unchanged node for node, the same moves at every depth, and cut the cycles per node by 1.42× and then a further 1.17×. Picking the next move to try was 21% of all the time spent, because the best score so far lived in memory and was reloaded on every comparison. Another 3.4% was zeroing a 1.5 KB struct at every node, arrays that are always written before they're read included.

The network gains are flattening: generation 8 was +30 over its parent, then +10.6, +8.7, +8.5. That's the shape of a project that's found its easy ground.

Then I played it#

An engine at 3,100 is no use to someone at a few hundred. So it needed a strength limit, and the obvious one is bad. Search less deeply, look at the top few moves, and pick one at random, weighted by score. It loses, so it is weaker, but it plays like an engine with a fault: king walks, a rook shuffled back and forth, a piece retreated for no reason. Nobody plays like that, and losing to it teaches you nothing.

So the pick is weighted by how natural a move looks too. Recaptures, captures, checks, rescuing an attacked piece, developing and castling go first; king walks, moves straight back and rook shuffles go last. The first few moves come from an opening book, weighted by how often strong players chose each one, for about as many moves as a player of that rating would know. Measured against real human moves, from my own games at 600 and from games between 2000+ players at 2000, it now plays the human's move 21% and 43% of the time, against 15% and 36% before. Its odd moves are about as rare as theirs.

Playing more like a person turned out to be worth about 100 Elo at the same setting, so the asked rating is lowered by that much. Afterwards the new version at 1400 scored −27 ± 40 against the old one, within error of even, and the old one had been calibrated against Stockfish's limiter set to the same numbers.

It also chooses its own level. After each game a script updates its estimate of my rating against it and sets the strength for the next game to where I'd score about 45%. That means close games that are slightly too hard, which are the ones where a decision mattered and the coach has something to say about it.

The coach#

zchess review searches every position of a player's games and flags the inaccuracies, mistakes and blunders, measured as lost winning chances. That's the method Lichess uses, which counts a blunder at +8 as smaller than one at +1. Then it says why in plain words, because "−3.2" teaches nothing:

9. Be2? Mistake: walked into a knight fork: Nxc2+ hits your king and rook.
6... Nxb5?? Blunder: allowed a discovered check: Nxc6+ uncovers check, and the knight wins your queen.

Each mistake gets a lesson (hung_piece, allowed_fork, missed_pin), and everything downstream groups by lesson. Mine is not subtle:

#1 leak: You left a piece hanging in 13 of your last 20 games (24 times, mostly your knights).
   habit: Before every move, check what your opponent's last move attacks, and that every
          piece you own is still defended after the move you want to play.

The same judgement runs live while I play. I use En Croissant as the board. While zchess thinks about its reply, it judges the move I just made on a thread of its own and sends the comment just before its move. The first comment of every game is my top leak and the habit that fixes it. When I make the same kind of mistake again, it says that too: "That's your #1 leak again."

The rest is training. The mistakes become puzzles, in two kinds: find the win (the tactic I missed) and spot the threat (my bad move is played, find the punishment). They come back on a spaced schedule. A blunder check drill shows a move I actually played and asks whether it was safe. Two in five were real mistakes. The rest were sound moves that still left a piece attacked, so I can't answer from the pattern and have to count attackers and defenders. That counting is the habit. And the report tracks whether the habit is taking hold: pieces hung per game, and the move of my first blunder, game by game.

The feature I'd keep if I could keep only one is the least clever thing in the repository. With hints on, after each engine move it lists my pieces that can now be taken for profit. There's no search, just attackers and defenders:

After 7. Nxc6+: Watch: your queen on d8 is attacked by a knight.

Training wheels, meant to come off once checking is a habit.

What it's for#

A 3,100 engine is, in one sense, a solved problem: Stockfish exists, it's free and it's far stronger. I knew that going in, and building one anyway taught me more about measurement than anything else I've written. Most ideas don't survive two thousand games, the ones that do are smaller than they felt, and the validation set can lie to you if you let it share games with the training set.

What I couldn't download was something that had read my games, knew which piece I'd leave hanging next, and would play me at exactly the level where that mattered. That turned out to be the better use of an engine.

The code, the trained networks and the training log with every SPRT are on GitHub, under the GPL.