Memory Trace Visualization

Tracking RNN hidden state h through text, analogous to arithmetic coding interval.

Analogy: AC interval [low, high) ↔ RNN hidden state h
Sample: 150 characters | Hidden dim: 128 | PCA variance: 38.9% + 28.2%

Hidden State Trajectory (PCA)

The RNN's 128D hidden state projected to 2D. Each point is one timestep. Color: blue→red over time.

Entropy Over Time

Output entropy (uncertainty) at each step. Lower = more confident prediction.

Entropy (bits)

Surprisal (-log₂ p)

Hidden State Components

First 8 dimensions of h over time. Shows how each component evolves.

Text with Entropy Coloring

Hover to see entropy. Red = high surprisal, Green = well predicted.

Key Insight: Like arithmetic coding's interval, the hidden state h accumulates context. High-entropy predictions "use up" precision. The trajectory shows how context flows through time.

← Back to Archive