A tiny language.
A model you can inspect.

Turning a wrong prediction into loss

Pokerese input ... <DECIDE>

What did the random model think might come next?

The model chooses among 1,609 possible Pokerese tokens. It does not simply pick one answer. Internally it scores all possible next tokens.

ACTION was outside the stored top-8, so this row comes from a compact read-only full-vocabulary softmax specimen for the same epoch-0 state.

The random model gave ACTION very little probability, so the loss for this token is large.

Cross-entropy uses the probability assigned to the target token, not simply whether the model's top guess was right or wrong.

This is the loss for one target token at one position. The real training loss combines scored target-token positions according to the target-only mask.

Simple examples

At random start, ACTION was not the model's preferred next token.

The random model preferred N1131 while the target token was ACTION.

Pokerese uses target-only causal loss. We only score the continuation the model is supposed to learn after the decision boundary; the long input describing the hand is context, not something the model is penalised for predicting.

We can now measure how wrong a prediction is.

Gradients: which way would reduce the loss?

We now have a loss number. The next question is which direction each learned number should move to make that loss smaller.

Optimizer: actually change the learned number

Gradient tells us which local direction would reduce loss. But the gradient does not change anything by itself.

The optimizer reads the gradients and decides how to adjust the learned parameters.

One learned number has now actually changed.

The real model does this across all trainable parameters involved in the training loss.

Next: put the whole thing together as one complete training step.

One complete real training step

The microscope showed one tiny piece. This is the real unit of training.

Real training does not train one token in isolation. Pokerese processes a small group together, averages cross-entropy over all masked target-token positions, computes gradients from that combined loss, and then Adam updates the trainable parameters that received gradients.

That was one training step: one batch.

The model now moves to the next batch and does the same thing again.

When it has worked through the whole training set once, that is one epoch.

Epoch: one complete pass through the training set

The previous section showed one real batch.

An epoch is what happens when Pokerese repeats that same training step across the whole training set once.

The stored epoch training loss is the simple mean of 1,299 per-batch target-only cross-entropy losses, not a token-weighted mean across the whole epoch.

One epoch is one complete trip through the training set.

Pokerese did not stop after one. It repeated this process for 12 epochs.

Next: watch what changed across those 12 passes.

12 epochs: watch the model learn

One epoch was not the end. Pokerese kept the parameters and Adam state, then repeated the same training process over the same training split with a new epoch-dependent batch order.

Training is gradual parameter adjustment, not one magical learning event.

The stored best epoch is 12, and in this run the latest checkpoint is also the best checkpoint. We'll unpack what "best" means next.

The model has now been trained.

But how do we know which epoch to trust?

Next: validation and choosing the best checkpoint.