Back to Archives

Archive: 2026-01-31_3

Memory traces, time, integration, and explanatory sufficiency

Focus: Adding time to the Universal Model. Memory traces as sufficient statistics.
Key Questions: What is the sufficiency gap for ES-Markov vs full Markov?
Builds On: 2026-01-31_2 (critique, open questions)

Papers

Memory Traces, Integration, and Explanatory Sufficiency

Definitions: M_t, sufficient statistics h_t, integration g(h,x), explanatory sufficiency.

Product Space Embeddings

Construct embeddings via P(b|a). Vowel×CGroup example: (a,o) cluster, i is unique.

ω∞: The Sky-Hook Problem

Where does the prior come from? Human ingenuity as the source of predictive power.

Probabilistic Perfect Hashing via Primes

h(e₁,e₂) = p₁×p₂. Collision probability per bit. Optimal bit allocation.

Factor Maps as Patterns on U²

Factor maps, embeddings, and patterns are the same thing. They all live in U×U.

Pattern Injection: UM → RNN

Write patterns into weights. Skip training. The isomorphism goes both ways.

Figures

fig-01: Vowel Embeddings (7D)

Vowel→Consonant-Group distributions. Distance matrix shows (a,o) and (e,u) clusters.

fig-02: Two-Ring Pattern Hash

Outer ring = source ES, inner ring = target ES. Arcs show patterns, hash = p₁×p₂.

Empirical Results

Sufficiency Gaps (from 100M bytes of enwik9)

Representation          H(next|repr)
--------------------------------------
None (marginal)         5.0801 bits
ES(prev)                4.6523 bits
prev byte (1-Markov)    3.8859 bits

Sufficiency Analysis:
  ES-Markov gap:  0.7664 bits (vs 1-Markov)

Mutual Information:
  I(next; prev)    = 1.1942 bits
  I(next; ES_prev) = 0.4278 bits
  Ratio: ES captures 35.8% of 1-step MI

Source (.tex)

memory-traces.tex embeddings.tex omega-infinity.tex perfect-hash.tex factor-maps.tex pattern-injection.tex

Navigation

← Previous: 20260131_2

Tock methodology, ES extraction, critique of 53% claim.