재귀와 합성곱 part 13 of 13

Shallow stacks use the whole bound

guide / / 10 sections

Part twelve left two things. Whether the 2.25 slope on the kernel axis is a formula - there were only three kernel values, so it was not offered as a law - and where the ceiling starts biting.

Both are measurable. Adding kernels 7 and 11 settles the first; cutting the depth to two or three layers settles the second.

Four predictions were written down before measuring.

config      bound   prediction   reasoning
k5 x2           8            8   the bound is under the ceiling of 12, so all of it
k5 x3          12           12   same, with the bound equal to the ceiling
k7 x4          24           16   2.25 x 7 = 15.75
k11 x4         40           25   2.25 x 11 = 24.75

The shallow end and the wide end

All at the same 637k budget.

config    kernel  layers   ch    params    bound
k3 x2         3       2   303   635,565       4
k3 x3         3       3   250   635,400       6
k3 x4         3       4   218   636,162       8
k5 x2         5       2   238   635,746       8
k5 x3         5       3   196   636,180      12
k7 x4         7       4   145   636,835      24
k11 x4       11       4   116   633,152      40

The task and the protocol are part twelve’s. The copy distance is swept for the break, and a distance counts as solved only when both seeds land under 2.0. For the small bounds, walking up one character at a time is cheaper than binary search, so that is what was done.

Results

receptive-field bound against the distance solved 0 8 16 24 using the whole bound kernel 3 kernel 5 kernel 7 kernel 9 kernel 11 0 8 16 24 32 40 receptive-field bound in characters at a matched depth of four layers 0 8 16 24 part twelve's line 6 12 17 21 25 3 5 7 9 11 kernel width
Above: the receptive-field bound and the distance actually solved, for thirteen configurations. The dashed diagonal is where the whole bound gets used; the two- and three-layer stacks sit on it, and deeper ones flatten at a height set by the kernel. Below: only the five configurations at a matched depth of four, against kernel width. Part twelve's line lands within half a character for kernels 5, 7, 9 and 11 and sits 1.5 above kernel 3.

With part twelve’s configurations that makes thirteen.

config    kernel  layers  bound   solved   fraction
k3 x2         3       2      4        4      1.00
k3 x3         3       3      6        6      1.00
k3 x4         3       4      8        6      0.75
k3 x8         3       8     16        7      0.44
k3 x12        3      12     24        8      0.33
k5 x2         5       2      8        8      1.00
k5 x3         5       3     12       11      0.92
k5 x4         5       4     16       12      0.75
k5 x6         5       6     24       12      0.50
k5 x8         5       8     32       12      0.38
k7 x4         7       4     24       17      0.71
k9 x4         9       4     32       21      0.66
k11 x4       11       4     40       25      0.62

Shallow means the bound is everything

k3 x2 solves 4 on a bound of 4, k3 x3 solves 6 on 6, and k5 x2 solves 8 on 8. Nothing is left over.

Part twelve ended on “raising the bound does not raise the usable distance”, and that was measured from a bound of 16 upward. Below it, the bound is the answer.

Only k5 x3 leaves one character, solving 11 on a bound of 12. That is the place where the bound and the ceiling coincide, so this single configuration cannot say which one it hit.

The elbow

Lined up per kernel, there is one shape.

kernel 5   layers  2    3    4    6    8
           bound   8   12   16   24   32
           solved  8   11   12   12   12

kernel 3   layers  2    3    4    8   12
           bound   4    6    8   16   24
           solved  4    6    6    7    8

It climbs along the bound and then flattens. Kernel 5 flattens at 12 and is already there at four layers - four layers and eight layers give the same thing. Kernel 3 flattens around 6, but not completely: from four layers to twelve it still gains two characters, 6, 7, 8.

Kernels 7 and 11

k7 x4 solves 17 and k11 x4 solves 25. Part twelve’s line 2.25k + 0.75 gives 16.5 and 25.5, so both land within one character of an integer measurement.

k11 was an extrapolation. Part twelve had kernels 3, 5 and 9, and 11 is outside that range. It held outside the range too.

The predictions written above said 16 and 25, which come from 2.25k with the intercept of 0.75 dropped. The line gives 16.5 and the measurement is 17. The line was right and my arithmetic was not.

Correcting part twelve’s line

Matching the depth at 4 and looking again, one point does not fit.

kernel (all four layers)   solved   part twelve's line
        3                       6                 7.50
        5                      12                12.00
        7                      17                16.50
        9                      21                21.00
       11                      25                25.50

Kernel 3 is off by 1.5 characters. The other four are within half.

The reason is that part twelve used 7 and 8 as kernel 3’s values, and those come from eight-layer and twelve-layer stacks. Kernel 5 reached 12 at four, six and eight layers, so it does not depend on depth; kernel 3 does, and putting its deep values on a kernel axis lifts them.

At a matched depth, kernel 3 is 6. Part twelve’s lower panel placed depth-mixed points on a kernel axis. The 2.25 slope survives for kernel 5 and above, but putting kernel 3 on that line was wrong.

How the predictions did

config      predicted   solved   result
k3 x2               4        4    right
k3 x3               6        6    right
k5 x2               8        8    right
k5 x3              12       11    one short
k11 x4             25       25    right
k7 x4              16       17    one off (dropped intercept)
k3 x4             7~8        6    two off

The three “the whole bound gets used” predictions were all right. The three that missed are all near the ceiling - where exactly the ceiling falls still cannot be called to within one character.

When you size one

One more line joins what parts eleven and twelve gave.

  • If the distance you need is small, size the stack by the formula. Bounds of 4, 6 and 8 deliver their bound
  • If the distance is past that kernel’s ceiling, more layers will not get you there. Kernel 5 gives 12 at four layers and 12 at eight
  • Getting past it means a wider kernel: 512, 717, 921, 1125. At a fixed budget that cuts the channels from 171 to 116

What is left

k5 x3 solving 11 on a bound of 12 was not resolved. There is no other way to build a bound of 12 with kernel 5 (three layers is the only one), so this single configuration cannot separate the bound from the ceiling.

Kernel 3’s ceiling was not reached. At 4, 8 and 12 layers it gives 6, 7 and 8, still climbing. Going to 20 or 30 layers would show where it stops, but at a fixed budget the channels keep shrinking, so at that point channels become a candidate cause.

There are still two seeds. Three of the thirteen had a distance where the seeds disagreed: k5 x8 at 13, k9 x4 at 22, and k11 x4 at 26 (0.0041 and 3.0002).

This is one task. The caveats from parts eleven and twelve carry over unchanged.

So

  • Configurations cut to 2 and 3 layers, plus kernels 7 and 11, all at the same 637k budget, extend part twelve’s table to thirteen
  • Shallow means the whole bound is used. k3 x2 is 4/4, k3 x3 is 6/6, k5 x2 is 8/8. Part twelve’s “the bound buys nothing” was measured from a bound of 16 upward
  • Kernel 5 climbs 8, 11, 12 along the bound and then flattens at 12. Four layers is already all of it, and eight gives the same
  • Kernel 3 goes 4, 6, 6, 7, 8 - it flattens more slowly, gaining two characters between four layers and twelve
  • k7 x4 solves 17 and k11 x4 solves 25; part twelve’s line gives 16.5 and 25.5, both within one character, and 11 was outside its fitted range
  • Kernel 3’s point on part twelve’s line is corrected. Part twelve put the eight- and twelve-layer values 7 and 8 on the kernel axis; at a matched depth of four it is 6, which is 1.5 below the line
  • Of seven predictions, four were right, two were off by one and one by two. All the misses are near the ceiling

Comments