/* #pprog_p8v2_impl_goal_20260804 @variant_protocol @pprog_p8v2_questions_20260727 @pprog_p7_markov_goal_20260704 @pprog_pattern_query_goal_20260706 @pprog_explain_goal_20260722 @pprog_p4_memchain_trace_goal_20260630 @run_ofra_strip_goal_20260729 @handle_run_ofra @pp_main @pp_es @pp_pattern

NEXT IMPLEMENTATION PUSH: p8v2 step 1, as parallel variants

Fixture: tests/pprog/p8v2-words.md. Program block #wordsv2, settling in #f-p8, learning in
#omega-p8, token ES in #token. Q&A with the answers verbatim: #pprog_p8v2_questions_20260727 and the
"Still underspecified" run inside #wordsv2:post.

## Method: variants, not decisions

This push runs under #variant_protocol, which is the general form; read it first. In short: where
the design is genuinely open we do NOT rule. Each open point becomes an AXIS with two or three
reasonable alternatives, all of them get implemented, and the programmer picks the winner on each
axis by taste and by where the work is going. Measurement and visualization are evidence put in
front of that judgement, never the decision rule, and nothing is pruned automatically. The point of
the push is examples to look at, not a correct compressor: round-trip is not required for step 1.

Generation 1 is the baseline plus ONE variant per non-baseline alternative, each varying a single
axis: 8 runs for the 6 axes here, not the 72 of the cross product. All 8 are then compared side by
side. The winners compose into #wordsv2-v002 and generation 2 runs off that, with whatever axes are
interesting by then -- so the combinations that get tried in later generations are not the ones this
block lists.

A variant costs no CLI surface. An alternative is an OFRA block; a variant is a program block that
names alternatives in its E/T/P/f/Omega headers, on the tests/pprog/p7-hutter-order-1-markov-v002.md
precedent (a doc that just lists the ids it carries over). --run already resolves headers by id:

  cmpr --run '#wordsv2-v003' --ofra tests/pprog/p8v2-words.md --compress ...

Naming: alternatives suffix the block they replace (#f-p8-assign, #f-p8-add, #token-and-min,
#token-and-sum, #omega-p8-frozen, ...). Variants are #wordsv2-v001 upward, each with a one-line
summary of what it varies from the baseline.

## Step 0: fixture structure -- DONE 2026-08-04

The doc had a duplicated #f-p8/#omega-p8 pair and an unfenced #token; both were fixed by the
programmer. It is now nine blocks with no duplicate id: #f-p8, #omega-p8, #wordsv2, #token and their
prose runs. #wordsv2 is now headers plus a one-line summary, with the commentary in the prose around
it.

What remains at that level is the E: line, which still reads as if token were a single ES. Token is
a FAMILY indexed by position and by length k: token-1 does not exist (it would duplicate
mem_cell_1), and token-2 is the projection of memory cells 1 and 2 onto the input, i.e. the two
previous bytes onto the next.

## Frozen across every variant (so runs are comparable)

- Trace format exactly as p7 left it: surprising bytes direct, gaps unary after subtracting 1.
- k=2 only. k_max=64 is the architecture, not this push.
- Uncertainty/settling window W=128 bytes (4 * 32; 32 is the token length expected to catch most
  words). Max token length 64.
- LSA values are bytes in [0,255] read as log2 counts, increment gives w+1 with probability 1/2^w
  (LSA.md at the repo root).
- One model file, new format, whatever container the implementer picks; DSS is a runtime parameter.
- Abduction is settled and needs no block of its own: everything not most-likely is written
  directly, therefore everything most-likely is true, therefore the synthetic event goes to 255.
  This is the dual of sparsification, not an extra rule.
- Query layer per #pprog_pattern_query_goal_20260706: --patterns enumerates the (token, byte) ES
  pair, --explain gains the k=2 lines at a position where a joint rule fired. Token SN form is
  settled: "The 3-token at position 3 is "abc"." 0
- p1/p3/p4/p7 acceptance byte-for-byte green; zero-input P = 0.

A variant that cannot be unfolded in the query layer is dead on arrival regardless of its numbers.

## The axes

A. Settling update. #f-p8 rules assignment correct when the antecedent is absolute, and notes k=2 is
   a subset of k=1 (fully correlated) so assignment is correct there too; the general case is stated
   as addition against decay, to be found empirically.
   A1 assignment (baseline)          -- longest k that fires assigns; simplest, no rates at all.
   A2 LSA addition with decay        -- pattern application at rate f, decay at 2f per the
                                        constant-sum argument in #f-p8.
   A3 addition, no decay, renormalized per position to constant sum.
   Watch: whether A2/A3 converge inside W at all, and whether they differ from A1's argmax sequence
   anywhere worth seeing.

B. Backward direction of the M-1 pattern. Settling needs it leftward as well as rightward.
   B1 reconstruct the full LPP by normalizing the p7 forward table (baseline, per the answer).
   B2 store the full LPP both directions (65536 LSA bytes at k=2, cheap enough to be the control).
   Watch: where the reconstructed backward distribution disagrees with the stored one, and whether
   that disagreement reaches the trace.

C. AND gate arithmetic, at partial activation (the normal case in settling).
   C1 min(w_a, w_b) (baseline, "simple and sufficient for now" in #token).
   C2 sum with threshold (the traditional neuron shape #token_and describes).
   Watch: how often each fires at partial activation, and whether C2 needs per-sample tuning to fire
   at all.

D. Sparsification -- which bytes the trace records. #omega-p8's placeholder is greedy removal.
   D1 p7 rule frozen: surprising under the k=1 argmax (baseline; guarantees p7 parity, isolates the
      k=2 model as the only lever).
   D2 surprising under the settled argmax over W, single pass, no re-settle after a removal.
   D3 greedy: drop a byte if removing it does not change the most probable sequence within W,
      re-settling after each removal.
   Watch: how much the trace shrinks against D1, which bytes stop being recorded, and what D3 costs
   per removal.

E. Replay. Unanswered in the Q&A; the stated intent is replay from p7 into p8 after every change of
   pattern shape, with fusing the passes treated as an optimization.
   E1 single pass: k=1 table frozen from p7, k=2 learned over the same replayed trace (baseline).
   E2 two rounds: relearn k=1 after the k=2 patterns exist.
   E3 replay until the trace stops shrinking, cap 4 rounds.
   Watch: whether round 2 changes the k=1 table or the trace at all, and what a third round adds.

F. Token creation and pruning in omega.
   F1 collect everything, never prune (baseline; the DSS is small precisely so this works, and the
      score is deliberately unspecified until we have examples to look at).
   F2 fixed budget per k, kept by total LSA support.
   F3 fixed budget per k, kept by sharpness of the pattern's output distribution.
   F2/F3 exist to be ready when the DSS grows; neither is expected to matter at step-1 sizes.
   Watch: which tokens F2 and F3 discard that F1 keeps, and what those tokens were doing.

Baseline vector: A1 B1 C1 D1 E1 F1. That is #wordsv2-v001 and it is the closest thing to p7 that has
a k=2 model at all; if it does not beat p7 the encoding is wrong, not the design.

## Presenting the generation

Instrument every run identically, into one TSV appended by the acceptance script: variant id,
sample, U, S, P broken down (k=1 table + token table + gap section + trace bytes + header), r,
extrapolated k, round-trip yes/no, trace byte count, settling cost as pattern applications per input
byte, wall time.

The TSV is the weakest view of a generation and is not what the picks are made from. The generation
is presented as a rendering: the 8 variants over the SAME input positions, so where they diverge is
visible where it happens -- which bytes each one records, which patterns fired there, and what the
settled state looked like. tests/pprog/p8v2-viz-spec.md already specifies a 3-D rendering of E and P
over a fixed 128-byte prefix (k=8, N=128) with its placeholders explicitly fenced; extending it to
show variants side by side is the cheapest route and is part of this push.

Then stop, and put both in front of the programmer. Per #variant_protocol the picks are the
programmer's, by taste and by where the work is going; an alternative that looks worse now may be
the one with somewhere to go. Pruned alternatives keep their blocks in the doc with a PRUNED line
naming the generation and one line on why.

Samples: the first 64 bytes of enwik9 for eyeballing (the examples are already worked in the
fixture), then ~1 KB, then the p7 acceptance samples so the comparison against p7 is like for like.

## In scope

- The alternative blocks and the variant program blocks; the k=2 token model end to end under
  --compress with the reporting above; --patterns and --explain unfolding for the (token, byte)
  pair; acceptance-p8v2 running all 8 of generation 1 and writing the TSV; the side-by-side
  rendering; prior acceptance tests and tests/run-all green; warning-clean build.

## Out of scope

- k=3 and above; step 2 (strengths on k=1); pruning that matters (F2/F3 land but are not tuned);
  entropy-coding the gaps; the verb that finds interesting positions (kept separate per the answer
  to Q9); the interpreter cleanups #run_ofra_strip_goal_20260729 and #run_ofra_split_goal_20260729.
- Round-trip correctness. Expected to fail at first, and that is the cheapest way to pin down f,
  omega and LSA, which land together here and which the training-set prior gets wrong.

## Care

- The staleness key checksums the loaded doc bytes (#pprog_p7_markov_goal_20260704). All variants
  share one doc, so any edit regenerates every variant's blocks and binaries. Keep them in one doc
  anyway -- the shared prose is the point -- but expect the churn and do not read it as breakage.
  Verify the p7 way on the OTHER fixtures: no rewritepl and no cc on p1/p3/p4/p7 after the change.
- New kind string and a new pp_main NL branch, as at p7. pp_es/pp_pattern should stay byte-identical
  where they can, so cached PL is reused.

## Done when

Generation 1 is complete and presented: the baseline compresses a text sample with a k=2 token model
and reports the full breakdown, all 8 variants run and are in the TSV, --patterns and --explain show
the k=2 rules and the position stack for any of them, and the side-by-side rendering over a common
prefix exists. Then it goes to the programmer for the picks; #wordsv2-v002 and generation 2 are not
this push.

## IMPLEMENTED 2026-08-04 -- generation 1

Built and measured; the picks are the programmer's and are not made here.
Results, the eight points that needed a ruling, and what was deliberately left alone:
#pprog_p8v2_gen1_report_20260804 (carries the --needs line).

Where this push departed from the text above, each departure argued in that block:
- 11 runs, not 8: the protocol's formula over these six axes gives 1 + (2+1+1+2+2+2).
- D1 is read as p7's causal RULE SHAPE with the COMPOSITE argmax, not the k=1 argmax, because the
  literal reading cannot beat p7 once the token section is in P, which the same paragraph requires.
- the frozen message rule is min(w_p, w_s), not w_p - (255 - w_s), which annihilated every
  non-absolute message; both forms are recorded in #omega-p8-hyper.
- settling runs W sweeps (a message travels one position per sweep, so fewer leaves the far end of
  the window unsettled).
- the token section is class (b) of #hutter_metrics: sub-linear in the DSS and capped by the
  architecture at 65536 * 4 bytes, reported as bytes and extrapolated in log DSS. It is NOT part
  of a per-byte rate. (Generation 1 first reported it inside a single r over the whole model
  file, which at 1 KB ranked the keep rules backwards; that is what #hutter_metrics exists to
  stop.)

Artefacts: tests/pprog/p8v2-words.md (42 named blocks: the carried-over p7 blocks, the six axes'
alternatives, #omega-p8-hyper, and #wordsv2-v001..v011), #pp_wordsv2 (the whole interpreter side),
and, split the way #hutter_metrics requires, GENERATING: tests/pprog/acceptance-p8v2 and
gen1.tsv (measured facts only) with gen1-pos/; DISPLAYING: tests/pprog/gen1-report and
p8v2-diag.py (every rate, extrapolation and diagnostic, no compressions run); PUBLISHING:
../hutter's, so gen1.html and gen1-viz.py are no longer produced or referenced here, though
section 15 of tests/pprog/p8v2-viz-spec.md still specifies the panel.
*/