fig-20260131-07: SN Layered Graph
Sparse Network with Named Event Spaces and Pattern Flow
Layered Event Space Structure
Information Accounting (Compression Framing)
Framing: We describe compression in bits, then renormalize to remaining bits.
No percentages - each step describes a compression from X bpc to Y bpc.
Random baseline:
8.00 bpc
Bits captured by RNN:
2.31 bits
Bits captured by ESs:
1.37 bits
Result: 5 ESs (Digits, Punct, Vowels, Whitespace, Other) explain
59.3% of RNN's compression (1.37 of 2.31 bits/char).
Remaining: 0.94 bits/char unexplained by ES-level predictions.
This is what byte-specific patterns add beyond character class membership.
Named Events (E → N mapping)
Input Layer - Confirmed ESs
Digits (10)
Punctuation (6)
Vowels (5)
Whitespace (3)
XML <>/= (4)
Consonants (21)
Other (207)
Hidden Layer - Discovered
Persistent (5)
Digit-corr (10)
Punct-corr (6)
Vowel-corr (5)
Unassigned (102)
Output Layer - Predicted
Digits (10)
Punctuation (6)
Vowels (5)
Other (235)
Strongest Named Patterns
"SPACE" → "-", "=", "#", "(", "{" (Wikipedia formatting)
Multi-digit numbers: "1" → "9", "0" → "0", etc.
Vowel-consonant alternation in English words
Extraction Status
Completed:
- ES-based forward pass implemented (
./hutter es)
- Interpretable bpc computed: 6.63 bpc
- 5 ESs explain 59.3% of model's compression
Next (for tick-tock cycle):
- Feed ES membership as additional input features
- Train next round on augmented input
- Measure improvement in bpc
- Extract higher-level ESs from new hidden layer
Reproducibility
Figure: fig-20260131-07
Model: model.bin (Elman RNN, 256→128→256, 6.02 bpc)
ES confirmation: hypothesis-results.html (fig-20260131-04)
Data: enwik9
./hutter stats model.bin
./hutter probe enwik9 model.bin "prev=0" 5 # digit context