P2: W_hh Spectral Radius

Testing if trained W_hh has eigenvalues near unit circle.

REFUTED
Spectral radius |λ_max| = 2.52
Prediction was: 0.9 < |λ_max| < 1.1

Prediction: For information preservation, W_hh should have eigenvalues near 1.
Rationale: |λ| < 1 causes decay, |λ| > 1 causes explosion.

Eigenvalue Distribution

Complex plane. Unit circle shown. Blue = inside, Red = outside.

Top 10 Eigenvalues

#	Eigenvalue	\|λ\|	Status
1	-2.35 - 0.91i	2.52	outside
2	-2.35 + 0.91i	2.52	outside
3	+1.23 - 1.78i	2.16	outside
4	+1.23 + 1.78i	2.16	outside
5	+0.58 - 1.96i	2.04	outside
6	+0.58 + 1.96i	2.04	outside
7	+1.38 - 0.80i	1.59	outside
8	+1.38 + 0.80i	1.59	outside
9	-0.89 - 1.21i	1.50	outside
10	-0.89 + 1.21i	1.50	outside

Statistics

Spectral radius	2.52
Mean \|λ\|	0.52
Std \|λ\|	0.51
Inside unit circle	115/128 (90%)
Near unit circle (0.9-1.1)	3/128 (2%)
Outside unit circle	13/128 (10%)

Insight: The RNN doesn't rely on eigenvalue tuning for stability.

Mechanism: W_hh is expansive (would explode), but tanh clamps to [-1,1].

Implication for Q8: Memory depth doesn't follow d_max = 24/H formula because:

Eigenvalues > 1 mean linear amplification, not decay
tanh saturation provides the "forgetting", not precision loss
Different memory mechanism than arithmetic coding analogy suggests

← Back to Archive