\documentclass[11pt]{article}
\usepackage[margin=1in]{geometry}
\usepackage{amsmath,amssymb}
\usepackage{hyperref}
\usepackage{booktabs}

\title{SN Visibility for the Doubled-E Universal Model}
\author{Claude (with MJC)}
\date{2026-02-02}

\begin{document}
\maketitle

\section{Introduction}

This document describes the SN visibility implementation for the Hutter RNN, building on the RNN-to-UM mapping established in \texttt{rnn-um-mapping.tex} (archive 20260131).

The goal is to make the Universal Model visible in human-readable SN format: events with names, patterns with strengths derived from the isomorphic mapping.

\section{Background}

The doubled-E mapping (Section 6 of \texttt{rnn-um-mapping.tex}) establishes an exact isomorphism between the RNN and a Universal Model:
\begin{itemize}
\item Each hidden neuron $j$ becomes a binary event space $\{h_j^+, h_j^-\}$
\item Support values: $t(h_j^+) = 2 \cdot \max(0, \text{pre}_h[j])$, $t(h_j^-) = 2 \cdot \max(0, -\text{pre}_h[j])$
\item Binary softmax replaces tanh, giving exact bpc equivalence (0.00\% difference)
\end{itemize}

\section{Event Structure}

The model has 768 events, numbered 1--768:

\begin{center}
\begin{tabular}{lll}
\toprule
Range & Count & Description \\
\midrule
1--256 & 256 & Input bytes: ``The input is `a'.'' \\
257--512 & 256 & Hidden neurons: ``h1+'' ``h1-'' ... ``h128+'' ``h128-'' \\
513--768 & 256 & Output bytes: ``The output is `a'.'' \\
\bottomrule
\end{tabular}
\end{center}

Event naming convention:
\begin{itemize}
\item Use ``input'' and ``output'', not ``previous/next character''
\item Printable bytes shown as character: ``The input is `e'.''
\item Non-printable shown as hex: ``The input is 0x0A.''
\item Hidden events use doubled-E notation: h$N$+ (activated), h$N$- (inhibited)
\end{itemize}

\section{Pattern Extraction}

Pattern strength equals the contribution to support in the UM forward pass:
\[
\text{strength} = \lfloor 2 \cdot |w| + 0.5 \rfloor
\]

This follows directly from the doubled-E mapping where weights contribute $2|w|$ to support values.

Patterns with strength $< 1$ are omitted---they contribute negligibly to the model's predictions.

\subsection{Pattern Types}

\textbf{Input $\to$ Hidden} (from $W_{xh}$):
\begin{itemize}
\item $w > 0$: pattern from input event to $h_j^+$
\item $w < 0$: pattern from input event to $h_j^-$
\end{itemize}

\textbf{Hidden $\to$ Output} (from $W_{hy}$):
\begin{itemize}
\item $w > 0$: pattern from $h_j^+$ to output event
\item $w < 0$: pattern from $h_j^-$ to output event
\end{itemize}

Note: Hidden-to-hidden patterns ($W_{hh}$) are not currently exported. They represent recurrent connections and would significantly increase pattern count.

\section{Results}

For the trained model (\texttt{model.bin}, 5.69 bpc on enwik9):

\begin{center}
\begin{tabular}{lr}
\toprule
Metric & Value \\
\midrule
Total events & 768 \\
Total patterns & 302 \\
Input events with patterns & 17/256 \\
Hidden events with patterns & 68/256 \\
Output events with patterns & 217/256 \\
\bottomrule
\end{tabular}
\end{center}

Most input bytes have no significant patterns (strength $\geq 1$). The model's input-to-hidden weights are sparse---only a few bytes (space, newline, common letters) have weights $\geq 0.5$.

The strongest pattern is space $\to$ h3- with strength 8 (weight $\approx 4.0$).

\section{SN Viewer}

The interactive viewer (\texttt{sn-view.html}) provides:
\begin{itemize}
\item Three-panel layout: fan-in, events, fan-out
\item Events with patterns shown in black; others grayed out
\item Per-section statistics (e.g., ``17/256 with patterns'')
\item Click any event to see its incoming and outgoing patterns
\end{itemize}

\section{Implementation}

New CLI mode added to \texttt{hutter.c}:
\begin{verbatim}
./hutter sn [model] [output_dir]
\end{verbatim}

Outputs:
\begin{itemize}
\item \texttt{events.sn}: 768 events with names and 0 support
\item \texttt{patterns.sn}: patterns with strength = $2|w|$
\end{itemize}

\section{Interpretation}

The sparsity of significant patterns reflects the model's learned structure:
\begin{itemize}
\item Most byte transitions are handled by small, distributed weights
\item Only a few input bytes (space, common letters) have strong dedicated patterns
\item The hidden layer concentrates information---68 of 256 hidden events participate in significant patterns
\item The output layer is dense---217 of 256 output events receive significant patterns
\end{itemize}

This is consistent with the model learning character-level statistics where most of the ``work'' happens in the hidden-to-output mapping, while input-to-hidden is more uniform.

\section{Relation to ES Discovery}

This visibility is a prerequisite for ES discovery (the main experimental problem). With patterns now visible:
\begin{enumerate}
\item We can identify which hidden neurons have similar input patterns (ES candidates)
\item We can see which outputs are predicted by similar hidden configurations
\item We can propose coarser ESs and test whether bpc is preserved
\end{enumerate}

The doubled-E representation is mechanical, not natural. The next step is finding semantically meaningful ESs that collapse the 128 binary ESs into fewer, interpretable ones.

\section{Files}

\begin{tabular}{ll}
\texttt{events.sn} & Event declarations \\
\texttt{patterns.sn} & Pattern triples (source, destination, strength) \\
\texttt{sn-view.html} & Interactive viewer \\
\texttt{model.bin} & Trained RNN weights \\
\end{tabular}

\end{document}
