What one recurrent step computes

The first series began at tensors and worked through learning rates, backprop, vanishing gradients, minibatches, normalisation and generalisation, and then went straight to attention in part eight. The space in between is empty. There were ways of handling order before attention, and without opening them up there is no saying what attention actually changed.

Start with the simplest one: the recurrent network.

What one step computes

Read the text left to right one character at a time, carrying everything read so far in a single vector. That vector is the hidden state. Reading one character is entirely this:

h = torch.zeros(683)                 # knowing nothing yet
for t in range(128):
    x = tok[idx[t]]                  # this character's embedding, (128,)
    h = torch.tanh(W_ih @ x + b_ih + W_hh @ h + b_hh)

Transform the new character once, transform the state you were carrying once, add them, squash with tanh. That is a step. Reading 128 characters runs that line 128 times.

One thing there is strange: neither W_ih nor W_hh carries a t. The same matrix is used all 128 times. A twenty-layer network has different weights at every layer; recurrence goes through the same ones 128 times over. That is why the vanishing gradient part four measured against depth happens against length here, and part eight comes back to it.

73% of the parameters are in one matrix

At width 683, the 637k splits like this:

tok.weight        (100, 128)     12,800   character -> embedding
rec.weight_ih_l0  (683, 128)     87,424   embedding -> state
rec.weight_hh_l0  (683, 683)    466,489   state -> state
four biases        (683) x 4      2,732
head.weight       (100, 683)     68,300   state -> next character
head.bias         (100)             100
                                637,845

W_hh alone is 466,489, or 73.1% of everything. A recurrent network spends almost all of its parameters moving state to state. Taking the input in costs 87,424, less than a fifth of that.

Double the width and W_hh quadruples. That is why widening a recurrent model is expensive.

Pulling out one unit

The hidden state is a bundle of 683 numbers. Take one of them, follow it across characters, and sometimes something readable comes out.

Training an RNN for 800 steps under the same protocol part seven uses, and correlating each unit’s activation with a few properties:

                      unit   correlation
is a space             546        +0.675
inside backticks       292        +0.226
characters since \n     594        -0.174
is uppercase           459        +0.169

546 stands out. Running it over a real sentence and plotting only that unit:

It climbs to about 0.9 at every space and falls back on letters. One number out of 683 is being used to mean “I have just read a space”.

Why that is useful is immediate. For predicting the next character it matters a great deal whether this is the start of a word, because the distribution over word-initial characters is not the distribution over word-internal ones.

The other three sit around 0.2, which is not readable in any real sense. One unit holding one concept is the exception; usually it is spread across many. This one happened to turn up.

Changing one character

How true is “carrying it along”? Make two copies of the same text differing in exactly one character at position 32, and measure how different the hidden state stays afterwards.

    after     RNN    LSTM     GRU
        1   0.760   0.824   0.820
        4   0.332   0.385   0.343
        8   0.122   0.161   0.124
       32   0.004   0.007   0.005
       95   0.000   0.000   0.000

Immediately after the change, 76~82% of the state differs. But half of it is gone within four characters, and less than 1% survives thirty-two.

The gated LSTM and GRU are almost identical. Which says the rate of forgetting is set by training rather than by architecture - and pulling the gates open to look is part two.

The thing to keep is this. A recurrent state remembers about four characters. And yet, as part seven shows, these models predict the next character rather well, because most characters are decided by the few just before them.

What is left

tanh was used without justifying it. relu lets the state diverge easily, which is why recurrent cells use a squashing function - unmeasured here.

The unit correlations are linear, so anything two units represent jointly is invisible to them. Far more will be missed by a one-unit probe than caught.

And these numbers come from a model trained 800 steps. Training longer raises the validation loss, as part seven shows, and what the units represent at that point was not looked at.

So

• One recurrent step is the single line h = tanh(W_ih·x + W_hh·h + b): the new character once, the carried state once, added and squashed
• It runs 128 times through the same matrix, unlike a deep network with different weights per layer
• Of 637,845 parameters, 466,489 - 73.1% - are W_hh, carrying state to state. Taking the input in is only 87,424
• Unit 546 of 683 correlates +0.675 with spaces, switching to about 0.9 at every space in real text
• Changing one character alters 76% of the state, but half is gone in four characters and under 1% survives thirty-two
• The gated LSTM and GRU forget at almost exactly the same rate