COACTIVATION CHAIN ANALYSIS
Testing how neurons track multi-character patterns
This relates to pattern length in the CMP paper

============================================================
COACTIVATION ANALYSIS: Common word 'the '
Sequence: 'the '
============================================================

Neuron activation by position (showing neurons >0.9 or <-0.9):
  't': +[] -[]
  'th': +[0,2,5,8,10,13,14,16...] -[1,3,4,6,7...]
  'the': +[0,1,2,3,4,5,8,9...] -[6,7,11,12,17...]
  'the ': +[0,1,3,4,7,9,10,13...] -[5,6,8,12,18...]

Persisting neurons (active at start and end): [0, 34, 48, 61, 62, 71, 107, 113]
Emerged neurons (inactive→active): [1, 3, 4, 7, 9, 10, 13, 14, 15, 16]...
Suppressed neurons (active→inactive): [33, 60, 65, 81, 84]

Hidden state evolution (cosine sim between consecutive steps):
  't' → 'th': 0.6778
  'th' → 'the': 0.5008
  'the' → 'the ': 0.4783

============================================================
COACTIVATION ANALYSIS: Common word 'and '
Sequence: 'and '
============================================================

Neuron activation by position (showing neurons >0.9 or <-0.9):
  'a': +[] -[]
  'an': +[0,2,5,8,10,12,13,16...] -[1,3,4,6,7...]
  'and': +[0,1,2,3,4,5,8,9...] -[6,7,11,12,15...]
  'and ': +[0,1,3,4,7,9,10,13...] -[5,6,8,12,17...]

Persisting neurons (active at start and end): [0, 34, 48, 62, 71, 107, 111, 113]
Emerged neurons (inactive→active): [1, 3, 4, 7, 9, 10, 13, 14, 15, 16]...
Suppressed neurons (active→inactive): [33, 60, 65, 81, 84, 99]

Hidden state evolution (cosine sim between consecutive steps):
  'a' → 'an': 0.6566
  'an' → 'and': 0.4615
  'and' → 'and ': 0.4732

============================================================
COACTIVATION ANALYSIS: XML tag '<page>'
Sequence: '<page>'
============================================================

Neuron activation by position (showing neurons >0.9 or <-0.9):
  '<': +[] -[]
  '<p': +[0,2,5,8,10,13,16,19...] -[1,4,6,7,15...]
  '<pa': +[0,1,2,3,4,5,8,9...] -[6,7,11,12,17...]
  '<pag': +[0,1,2,3,4,7,9,10...] -[5,6,8,12,18...]
  '<page': +[1,2,4,7,9,10,11,15...] -[3,5,6,8,13...]
  '<page>': +[0,2,7,9,10,11,12,15...] -[1,3,4,5,6...]

Persisting neurons (active at start and end): [0, 33, 34, 48, 60, 62, 71, 81, 84, 113]
Emerged neurons (inactive→active): [2, 7, 9, 10, 11, 12, 15, 17, 18, 19]...
Suppressed neurons (active→inactive): [65, 107]

Hidden state evolution (cosine sim between consecutive steps):
  '<' → '<p': 0.6868
  '<p' → '<pa': 0.4906
  '<pa' → '<pag': 0.4904
  '<pag' → '<page': 0.4494
  '<page' → '<page>': 0.6163

============================================================
COACTIVATION ANALYSIS: Bigram 'th'
Sequence: 'th'
============================================================

Neuron activation by position (showing neurons >0.9 or <-0.9):
  't': +[] -[]
  'th': +[0,2,5,8,10,13,14,16...] -[1,3,4,6,7...]

Persisting neurons (active at start and end): [0, 33, 34, 48, 60, 61, 62, 65, 71, 81]...
Emerged neurons (inactive→active): [2, 5, 8, 10, 12, 13, 14, 16, 19, 24]...
Suppressed neurons (active→inactive): []

Hidden state evolution (cosine sim between consecutive steps):
  't' → 'th': 0.6778

============================================================
COACTIVATION ANALYSIS: Bigram 'qu' (should predict 'u')
Sequence: 'qu'
============================================================

Neuron activation by position (showing neurons >0.9 or <-0.9):
  'q': +[] -[]
  'qu': +[0,2,5,8,10,13,14,16...] -[1,4,6,7,15...]

Persisting neurons (active at start and end): [0, 33, 34, 48, 60, 62, 65, 71, 81, 84]...
Emerged neurons (inactive→active): [2, 5, 8, 10, 12, 13, 14, 16, 19, 24]...
Suppressed neurons (active→inactive): []

Hidden state evolution (cosine sim between consecutive steps):
  'q' → 'qu': 0.6940

============================================================
INTERPRETATION
============================================================

Neurons that PERSIST through a sequence encode accumulated context.
Neurons that EMERGE encode pattern recognition (e.g., "th" detector).
Neurons that get SUPPRESSED encode exclusion (e.g., "not a number").

Longer coactivation chains = longer patterns being tracked.