
\documentclass[11pt,a4paper]{article}
\usepackage[margin=1in]{geometry}
\usepackage{amsmath,amssymb,amsthm}

\newtheorem{conjecture}{Conjecture}
\newtheorem{definition}{Definition}
\newtheorem{proposition}{Proposition}
\newtheorem{example}{Example}
\newtheorem{remark}{Remark}

\title{Context Events and the Induction of Priors\\{\large v3: Generalized Oversupport}}
\author{Claude and MJC}
\date{18 February 2026}

\begin{document}
\maketitle

\begin{abstract}
We formalize the concept of a \emph{context event}: an atomic event in a causally prior event space that, through the UM forward pass, induces a prior distribution on downstream event spaces. We conjecture that every missing prior in P-programming corresponds to a missing context event. Central to this version: we generalize oversupport beyond predicted-vs-observed conflict to any ES with strong support for multiple events---which contradicts the ES epistemics that events are mutually exclusive. This can arise from sensory conflict, from multiple LPPs terminating at the same ES (``from the left''), or from internal model inconsistency. The organism's response is the same in all cases: immediate energetic expenditure to resolve the contradiction.
\end{abstract}

\section{Introduction}

In the UM framework, compression proceeds by learning joint event distributions across event spaces via Learned Probabilistic Patterns (LPPs). The KN-6 model achieves 1.784 bpc on enwik9 using a tower of byte-context event spaces connected by LPPs with KN smoothing.

Yet significant compression remains on the table. The gap between KN-6 (1.784 bpc) and the current best compressed size arises because KN smoothing uses a crude sliding window as context, discarding most sequential information.

We argue that this gap is precisely characterized by \emph{missing context events}: atomic events that, if present in the model, would condition the downstream LPPs to produce tighter output distributions.

\section{Definitions}

\begin{definition}[Event Space]
An event space $\mathrm{ES}_a$ is a named set of atomic events $\{e_1, \ldots, e_n\}$ with a support function $s: \mathrm{ES}_a \to [0, 255]$ assigning each event a current support value. The ES epistemics: events within an ES are \emph{mutually exclusive}. At most one event in the ES should have high support at any time.
\end{definition}

\begin{definition}[Context Event]
Given a target LPP from $\mathrm{ES}_a$ to $\mathrm{ES}_b$, a \emph{context event} $e_c \in \mathrm{ES}_c$ is an atomic event in a causally prior event space such that the LPP, conditioned on $e_c$ having nonzero support, produces a more accurate output distribution on $\mathrm{ES}_b$.
\end{definition}

\begin{definition}[Prior Induction]
When a context event $e_c$ has support $s(e_c) = k > 0$, and $e_c$ is connected to events in $\mathrm{ES}_a$ via atomic patterns with weights $w_{c,i}$, the UM forward pass induces a prior on $\mathrm{ES}_a$:
\[
s(e_i) \gets \max\bigl(s(e_i),\; \min(k, w_{c,i})\bigr)
\]
This is the standard max-min update. The pattern of weights $\{w_{c,i}\}$ characterizes which events in $\mathrm{ES}_a$ are ``expected'' under context $e_c$, and with what strength.
\end{definition}

\section{Generalized Oversupport}

\subsection{Beyond Predicted vs.\ Observed}

In v2, oversupport was defined as a conflict between the model's prediction and an external observation: the model supports $e$ but the world forces $x \neq e$. This is correct in the ``inner UM'' sense---the output ES under teacher forcing---but the more general statement is:

\begin{definition}[Oversupport (Generalized)]
An event space $\mathrm{ES}_a$ exhibits \emph{oversupport} when two or more events in $\mathrm{ES}_a$ have strong support simultaneously:
\[
|\{e \in \mathrm{ES}_a : s(e) > \tau\}| \geq 2
\]
This contradicts the ES epistemics: events in the same ES are mutually exclusive, so strong support for multiple events is a contradiction.
\end{definition}

\begin{definition}[Oversupport Magnitude]
The magnitude of oversupport at $\mathrm{ES}_a$ is the \emph{second-highest support} in the ES:
\[
\text{oversupport}(\mathrm{ES}_a) = \text{sort}(\{s(e) : e \in \mathrm{ES}_a\})_2
\]
interpreted relative to the total incoming energy flux---the total support for the whole ES, which should equal the support for some context event that projects onto this ES.
\end{definition}

If the second-highest support is zero, there is no oversupport: only one event is supported, consistent with the ES epistemics. If the second-highest support is high, the ES is in contradiction.

\subsection{Sources of Oversupport}

Oversupport can arise from multiple sources:

\begin{enumerate}
\item \textbf{Sensory conflict}: an external observation forces one event, but the model's forward pass supports another. This is the teacher-forcing case (output ES). But even sensory input isn't certain---the organism may expend energy (e.g., a double-take) to verify a percept. Typos in the data are an example even under teacher forcing.

\item \textbf{``From the left''}: multiple LPPs or pattern chains terminating at the same ES support different events. No observation is involved---the contradiction is entirely internal to the model. Different upstream contexts lead to conflicting conclusions at this ES.

\item \textbf{Model inconsistency}: the ES's events are not actually mutually exclusive for the data, meaning the ES factorization is wrong. The ``events'' are not really events in the UM sense.
\end{enumerate}

\subsection{What Oversupport Means}

We emphasize: oversupport at an ES does \emph{not} necessarily mean ``the model is confidently wrong.'' It has three possible interpretations:

\begin{enumerate}
\item The model \emph{is} wrong: the support pattern reflects an incorrect hypothesis. This is the ``model error'' case.
\item The ES \emph{isn't really an ES}: the events aren't truly mutually exclusive. The factorization of $E$ is wrong---what looks like one ES should be two, or the events need splitting.
\item Something \emph{genuinely rare} happened: it's raining but the grass isn't wet, because there's a zeppelin in the garden. The model is correct about the general case; this is just an unusual instance.
\end{enumerate}

The organism's response is the same in all three cases: immediate energetic expenditure to figure out which explanation applies. This is the startle response---``freeze and figure out what's going on.''

\subsection{The Binary Case: Argument by Contradiction}

In a binary event space $\mathrm{ES} = \{e_0, e_1\}$:

\begin{itemize}
\item \textbf{Undersupport = excluded middle}: neither event has support. The model has nothing to say. Violates the law of the excluded middle.
\item \textbf{Oversupport = contradiction}: both events have support. This is a logical contradiction---both True and False are asserted.
\end{itemize}

The generalization matters here: oversupport in the binary case can arise ``from the left'' without any observation. If one LPP supports $e_0$ and another supports $e_1$, that is a contradiction \emph{within the model}---an argument by contradiction. The model's own premises lead to incompatible conclusions.

This internal contradiction is a signal that something is wrong with the premises (the upstream patterns), not necessarily with the observation.

Both cases generalize naturally to non-binary ESs.

\subsection{Prediction Loss as a Special Case}

The identification of prediction loss with oversupport (from v2) is a special case of the generalized definition. At the output ES under teacher forcing:
\begin{itemize}
\item The model supports $e$ (via the forward pass).
\item The observation forces $x \neq e$ (via the input).
\item Both $e$ and $x$ have high support: oversupport.
\end{itemize}

The cross-entropy loss $\mathcal{L}_t = -\log P(x_t)$ measures this oversupport. Backpropagation computes $\nabla_\theta \mathcal{L}_t$---the gradient of the oversupport with respect to model parameters. But this is just one instance of oversupport (the sensory-conflict case at the output ES). The generalized definition captures internal contradictions that backpropagation cannot see, because they occur at ESs with no explicit observation.

\begin{remark}[The Ramification Principle]
Just as the special treatment of the output ES (softmax, loss, gradient) is a signal to ramify the mechanism throughout, the detection of oversupport should not be limited to ESs with external observations. \emph{Every} ES can exhibit oversupport (generalized), and the detection mechanism---measuring the second-highest support---requires no external observation at all.
\end{remark}

\section{The Conjecture}

\begin{conjecture}[Missing Prior = Missing Context Event]
In P-programming, whenever a model exhibits systematic oversupport at any ES---a pattern of contradiction that repeats---there exists a context event $e_c$ in some ES such that adding $e_c$ and its connections to the model eliminates the systematic component.
\end{conjecture}

This refines the v2 conjecture: it's not just systematic prediction errors (output-ES oversupport) but systematic contradictions at \emph{any} ES that indicate a missing context event.

\section{KN Smoothing as Crude Context}

In KN-6, the context for predicting byte $t$ is the preceding $k$ bytes $(b_{t-k}, \ldots, b_{t-1})$. The KN interpolation chain (order $6 \to 5 \to \cdots \to 1 \to$ unigram) is a tower of progressively coarser contexts.

\textbf{What KN gets right}: the sliding window captures local sequential dependencies.

\textbf{What KN misses}: everything outside the window. The context event for ``we are inside an HTML tag'' requires information from potentially hundreds of bytes earlier. KN-6 cannot represent this.

Our surprise analysis confirms this empirically: KN-6 achieves 0.38 bpc inside XML tags but 2.46 bpc outside them (86.8\% of mean surprise). The tag structure is highly predictable, but only if you know you're inside a tag---a context event that the 6-byte window almost never captures.

\section{Neural Reuse via Context Events}

A central problem in compression: the same local patterns (e.g., ``th'' $\to$ ``e'') occur in many different contexts (normal text, tags, attributes, URLs). A single LPP captures the average, losing the context-specific distribution.

Context events solve this: the LPP between byte context and byte output remains the same, but different context events in a prior ES activate different subsets of the LPP's patterns.

\begin{example}
Consider the bigram LPP from 1-byte context to output. In KN-6, $P(\text{`e'} \mid \text{`h'})$ is a single number. But with context events:
\begin{itemize}
\item $e_{\mathrm{word\_start}}$ active: ``h'' is likely ``he'', ``his'', ``had''
\item $e_{\mathrm{in\_tag}}$ active: ``h'' is likely ``href'', ``html''
\item $e_{\mathrm{after\_digit}}$ active: ``h'' is likely a hex digit
\end{itemize}
Each context event induces a different prior over the output ES, restricting the LPP to its relevant joint events. The oversupport that KN-6 experiences when predicting tag-interior ``h'' (expecting text patterns) is eliminated.
\end{example}

\section{The Boolean Interpretation}

Every context event has a Boolean reading. The max-min forward pass is:
\[
s(y) = \max_{\text{patterns}} \min(s(x_1), s(x_2), \ldots, s(x_n), w)
\]
which, when all supports are binary ($\{0, 255\}$), reduces to DNF:
\[
y = \bigvee_{\text{patterns}} \left(x_1 \wedge x_2 \wedge \cdots \wedge x_n\right)
\]

A context event adds one more term to each conjunction:
\[
y = \bigvee \left(c \wedge x_1 \wedge x_2 \wedge \cdots \wedge x_n\right)
\]
where $c$ is the context. This is the probabilistic syllogism.

\subsection{Finding Exceptions via Boolean Query}

In the Boolean interpretation, oversupport at an ES reveals \emph{exceptions to the currently represented rule}. Finding the corrective context event $c$ can be seen as a query over the dataset, looking for exceptional cases.

\begin{example}
If `h' appears after $e_{\mathrm{in\_tag}}$ and this combination is rare, we have a set of exceptions to the default rule for `h'. These exceptions are expressible in natural language as a probabilistic syllogism (when the events are labeled), and literally correspond (via $E \leftrightarrow \mathbb{N} \leftrightarrow \mathbb{Q}$) to a subset of the dataset that activates that Boolean circuit.
\end{example}

An LPP over the potential-context ES and the ``oversupport event'' (the second-most-supported event at the target ES) can be a way to find these exceptions. This is suggestive of an ``embedding into $P$'' implementation of surprise detection: the model itself, through its own LPPs, discovers which context events would resolve which oversupport events.

This remains a research direction. The ring pattern approach (see the surprise-mechanism paper) is one possible P-programmish implementation.

\section{The Methodology}

We propose a systematic procedure for identifying and implementing context events:

\begin{enumerate}
\item \textbf{Identify systematic oversupport.} Score the data with the current model. Find positions where oversupport repeats in a pattern---not isolated high-surprise events, but clusters sharing a common feature. Generalized oversupport (second-highest support at any ES, not just output) gives richer signal.

\item \textbf{Boolean interpretation.} Express the pattern: ``the model systematically oversupports $e$ when the actual situation involves $x$, whenever [some condition holds].''

\item \textbf{Natural language.} State the condition: ``we are inside an HTML tag'', ``the current word is a number'', ``we just saw a capital letter after a period.''

\item \textbf{Reify via E/N/Q.} Express the condition as an event in the integer representation (um-arithmetic-v4): identify the quotient space, the projection, the relevant equivalence class.

\item \textbf{Query the dataset.} Find concrete examples. Verify that the condition separates high-oversupport from low-oversupport positions. This query \emph{is} the Boolean circuit from step 2, applied to the data.

\item \textbf{P-program.} Implement the context event as an ES with events detected by a P-program. Connect it to the target LPP's input ES. The model can now condition on the context, eliminating the systematic oversupport.
\end{enumerate}

\section{Empirical Validation}

Our surprise analysis ({\tt umr surprise enwik9 1000000}) provides the first empirical test of the context events framework:

\begin{itemize}
\item \textbf{in\_tag}: 0.38 bpc inside tags vs.\ 2.46 bpc outside (86.8\% of mean). This is the single most powerful context event we have identified. It would eliminate the systematic oversupport that KN-6 experiences at tag boundaries.
\item \textbf{word\_len}: Boundaries (word\_len=0) have 3.98 bpc; mid-word (len 3--4) has 1.73 bpc. Word position is the second most powerful context event.
\item \textbf{Surprise distribution}: 40.7\% of positions have $< 0.25$ bpc (nearly free); 8\% have $> 7.75$ bpc (nearly random). The heavy tail is where context events are most needed.
\item \textbf{Top surprises}: 44--54 bits per byte at positions with rare markup, non-ASCII characters, and structural transitions. These are extreme oversupport events.
\end{itemize}

\section{Open Issues for P-Programming}

\subsection{Ramifying Oversupport Detection}

The key open problem: how to give every ES in the model oversupport detection. With the generalized definition, this is simpler than v2 proposed---we need only measure the second-highest support at each ES, which requires no external observation. The question is what the model \emph{does} with this signal.

\subsection{Discovery}

How do we systematically discover which context events to add? The Boolean query approach (find the dataset positions that activate the oversupport circuit, look for shared features) is promising. An LPP between potential-context ESs and oversupport signals could automate this---making the model self-improving.

\subsection{The Carrier Signal}

Memory traces can be stored ``inline'' via a carrier signal: an event representing a span of nonzero duration $\lambda$. The joint event $(e_\cdot, t_i, e_\cdot, t_{i+1})$ encodes temporal adjacency and can be recorded by an LPP. This converts sequential memory into associative memory. We defer detailed treatment to a future paper.

\section{Connection to the KN-Quotient Tower}

The kn-quotient papers (20260216 archive) established that KN smoothing has an algebraic structure: contexts are elements of $\mathbb{Z}/256^k\mathbb{Z}$, order reduction is a ring surjection, and the discount $D$ corresponds approximately to removing common evidence via GCD.

Context events extend this tower upward. The KN sliding window is \emph{already} a context event mechanism---just a crude one. What we propose is a generalization: richer context events, detected by P-programs rather than fixed sliding windows, with oversupport detection at every level of the tower rather than only at the output.

\section{Conclusion}

The context event concept, combined with generalized oversupport, unifies several threads:
\begin{itemize}
\item Oversupport is strong support for multiple events in the same ES---a contradiction of the ES epistemics. It can arise from sensory conflict, from internal disagreement (``from the left''), or from a wrong factorization of $E$. All three cases demand the same organism response: immediate energetic expenditure to resolve.
\item Cross-entropy loss at the output ES is a special case (sensory oversupport under teacher forcing). The general mechanism detects contradictions at any ES, including internal ones invisible to backpropagation.
\item In a binary ES, undersupport is the excluded middle, oversupport is contradiction. Internal oversupport from the left is argument by contradiction.
\item Finding corrective context events is a Boolean query over the dataset: which positions activate the oversupport circuit? An LPP between context ES and oversupport signal can automate this discovery.
\item The methodology (identify systematic oversupport $\to$ Boolean $\to$ NL $\to$ E/N/Q $\to$ query $\to$ P-program) provides a path from contradiction to model improvement that stays within the UM paradigm.
\end{itemize}

The next step is to implement generalized oversupport detection in the UM runner (second-highest support at each ES) and use it alongside the Boolean query mechanism to discover context events automatically.

\end{document}

