← Back to Hutter
Archive 2026-02-07/08
From SN Visibility through Pattern Chains to Skip-Patterns and Weight Construction
Papers
- sn-visibility-sat.pdf - Full SN visibility with W_hh patterns, UM comparison, strength calibration (7 pages)
(source)
- export-gap.pdf - The SN Export Gap: byte-weight mapping, Q=λ Bayesian view, BPTT-50 reframing, model taxonomy, data-terms (8 pages)
(source)
- pattern-chain.pdf - The Pattern-Chain UM: contiguous n-grams, backward trie, skip-k-grams, greedy offset selection (10 pages)
(source)
- pattern-prior.pdf - Pattern Priors and Skip-Patterns: backward trie, MI by offset, attention analogy, RNN weight construction from data
(source)
- hidden-quotient.pdf - The Hidden Quotient: forward/backward quotients, W_h as working memory, saturation as persistent storage, weight construction
(source)
- summary.pdf - Summary: definitions, model taxonomy, results, skeletal argument, open questions
(source)
Summary: Extends SN export to all three weight layers (2,999 RNN patterns).
Translates n-gram UM (order 2-11) into SN format (1,915 patterns).
Calibrates bottleneck: ratio drops from 0.73 (bigrams) to 0.31 (order 11).
Export Gap: Binary ES softmax matching condition: D = 2·pre/ln(2).
W_h quantization is the bottleneck (0.80 bpc alone). 8-bit mapping produces
chaotic results due to recurrent error amplification (0.09–2.2 bpc depending
on parameter). Stable reconstruction needs ~12 bits per weight.
Key Results
2,999 RNN patterns exported
522 Wx (in→hid), 1,006 Wh (hid→hid), 1,471 Wy (hid→out).
W_hh max strength = 3, W_hy max = 8.
Hub neurons in W_hh
h100 dominates with 72 connections (56% of neurons). h9 has strongest self-connection (w=-1.25, oscillator).
Bottleneck effect quantified
RNN chain strength / UM strength: 0.73 (bigrams) → 0.31 (order 11). Monotonic decrease. 99.3% of n-grams traceable.
Byte-weight mapping analysis
W_h is the bottleneck (0.80 bpc from quantization alone; W_y is free at 0.0003).
8-bit per-weight resolution is adequate, but recurrent amplification makes results
chaotic. Stable at ~12 bits (alpha=4096: 0.16 bpc).
Pattern-chain UM surpasses sat-rnn
Direct data-term patterns: 4.74 bpc (order 0) → 2.05 (bigram) → 0.56 (trigram)
→ 0.076 (order 10) → 0.067 (order 12). Surpasses sat-rnn (0.079) at order 10.
6,180 patterns, vanishing sparsity (~10-14 of I6×O).
Greedy skip-k-grams
4 non-contiguous bytes [1,8,20,3] reach 0.069 bpc (712 patterns), nearly matching
12 contiguous bytes (0.067 bpc, 6,180 patterns). Skip-8 reaches 0.043 bpc (834 patterns).
Offset 8 chosen before offset 2: complementary MI, not autocorrelation, drives selection.
RNN weight construction from data
Bigram construction: 2.10 bpc (matches counting baseline 2.05).
Shift-register 8 groups: 0.25 bpc train, 5.43 bpc test.
Sat-rnn comparison: 0.079 train, 8.22 test — construction generalizes better.
Interactive Visualizations
- viz-dashboard.html — Experiment dashboard: n-gram BPC by order, skip-k-gram comparison, SN pattern counts, quantization BPC, bottleneck ratio, weight construction vs training, hub neurons, greedy offset selection.
- sn-view-sat.html - Dual-mode SN viewer (RNN blue / UM green / Both). Shows all three layers with color-coded patterns.
Data
Source
- SN export & analysis:
sat_sn_full.c,
sat_um_sn.c,
sat_calibrate.c,
um_runner.c,
analyze_export.c
- Byte-weight mapping:
byte_weight_test.c,
byte_weight_test2.c,
byte_weight_test3.c,
byte_weight_test4.c,
byte_weight_test5.c
- Pattern chains:
pattern_chain.c
- Skip-pattern analysis:
backward_trie.c,
skip2gram.c,
skip3gram.c,
skip_kgram.c,
offset_viz.c
- RNN skip-pattern tracing:
skip2_rnn.c,
skip2_survival.c,
compare_wh.c
- Weight construction:
construct_rnn.c,
construct_skip.c,
construct_skip_greedy.c,
construct_skip_mlp.c
Navigation
← Previous: 20260206
Synthesis, SIMD optimization, saturation experiment, pattern chains.
Next: 20260208 →
Continuing from weight construction.