fig-20260131-06: SN Representation

Sparse Network structure of trained RNN in Universal Model formalism

Event Space Structure

E_input
256 events
bytes 0x00-0xFF
→
→
E_output
256 events
predicted byte

Information Accounting

6.02
bpc (model)
8.00
bpc (random)
24.75%
explained
?
interpretable bpc

Discovered Clusters

Digits
0.9996 sim
0-9
Punctuation
0.999 sim
. , ! ? ; :
Vowels
0.988 sim
a e i o u
Consonants
no cluster
too diverse

Strongest Patterns

"in:SPACE"→"out:-"255
"in:SPACE"→"out:="235
"in:SPACE"→"out:#"234
"in:SPACE"→"out:("228
"in:SPACE"→"out:f"227

Patterns are Wikipedia formatting: space → special chars

Persistent Neurons

Neurons active across all tested sequences ("the ", "and ", "<page>"):

h[34]+ h[48]+ h[62]+ h[71]+ h[113]+

Reproducibility

Model: model.bin (Elman RNN, 256→128→256, 6.02 bpc)

Data: enwik9 first 10KB sample

SN file: model.sn

make && ./hutter stats model.bin