/* #pprog_p8v2_gen2_goal_20260806 @variant_protocol @pprog_p8v2_gen1_choices @pprog_p8v2_gen1_report_20260804 @pprog_p8v2_impl_goal_20260804 @pp_wordsv2 @hutter_metrics @hutter_publication_handoff @hutter_e64_viewer_prereq_20260805 P8v2 GENERATION 2: the axes made legible, and the first trial of the generation advance Generation 1 produced eleven runs and one pick. #pprog_p8v2_gen1_choices rules A1 out and A2 in, and then stops on every other axis -- not because the numbers were missing but because the alternatives could not be read: A3's renormalization has no formula anywhere, B1's reconstruction has no worked case, C2 has no example of the partial activation it exists to handle, and the sentence that reports settling ("v002 settles 27 of 64 and converges at 14") names three quantities the block that prints them never defines. So this generation's product is not a better rate. It is a generation the programmer can choose from. Generation 1 built the alternatives; generation 2 makes them legible and re-runs them off the new baseline, and it is the first time #variant_protocol's advance step (picks -> new baseline -> next generation) is exercised end to end. ## The new baseline: #wordsv2-v002, already built A2 B1 C1 D1 E1 F1 is exactly generation 1's v002, so the advance costs no new block. Generation 2's baseline is #wordsv2-v002 and every generation-2 variant differs from it on one axis. Variant ids are globally sequential and never reused: generation 1 spent v001..v011, so generation 2's new variants are #wordsv2-v012 upward. Each generation records its own baseline id rather than renumbering, so `cmpr --run '#wordsv2-v00N'` keeps meaning one thing forever. #f-p8-assign gets its PRUNED line: PRUNED generation 1 -- assignment discards every message but the longest that fires, so nothing accumulates and the axis has nowhere to go. ## The runs: baseline plus one variant per surviving non-baseline alternative v002 baseline A2 B1 C1 D1 E1 F1 (already built) v012 A3 f-p8-renorm v013 B2 mem_cell_markov_1-lpp v014 C2 token_and-sum v015 D2 omega-p8-settled v016 D3 omega-p8-greedy v017 E2 omega-p8-replay-2 v018 E3 omega-p8-replay-dry v019 F2 omega-p8-keep-support v020 F3 omega-p8-keep-sharp Ten runs, and none of them is a repeat of a generation-1 number. Generation 1 varied every axis off A1's assignment; these vary off A2's addition-with-decay, and the two axes generation 1 reported as degenerate are the ones most likely to move: B2 and C2 both NEVER CONVERGED under A1 (conv 64 at every position at e64), and under a rule that accumulates rather than overwrites there is no reason to expect that to repeat. D was inert under A1 for a structural reason -- Q4 of the generation-1 report: where a k=2 rule assigns, the settled argmax at a candidate is the causal prediction by construction, so no candidate is ever removable -- and that reason is gone under A2. Axis D becomes measurable here for the first time. Samples: the same ladder generation 1 used, so the comparison is like for like -- e64, e1k, e10k, then the opt-in rungs under P8V2_LADDER per #hutter_run_costs. ## The deliverable that this generation is actually for: the axes, on the page The programmer's instruction: the best description of the axes belongs on the visualization page, because that is what is in front of them when the picks are made. Where it is documented is ours to decide, and it is decided here: THE NORMATIVE DEFINITION STAYS IN p8v2-words.md. An alternative is an OFRA block; that block is what --run resolves and what the implementation follows. Nothing about that changes. THE PAGE GETS IT AS DATA, NOT AS PROSE SOMEONE RETYPED. ../hutter's docs/pprog/build-p8v2-gen1 currently carries a hand-written WHAT dict, one line per variant, copied out of the generation-1 goal block. That is the wrong source: it is a copy, it is already one generation stale the moment an axis moves, and it cannot carry a worked example. Generation 2 replaces it with tests/pprog/axes.json, emitted by a script on our side and linked by the builder the same way gen1-pos and gen1.tsv are: per AXIS letter, name, the question it decides, which alternative the baseline uses, and what to watch when comparing them per ALTERNATIVE id, digit, one-line label, the rule stated as a FORMULA where it has one, and one or more WORKED EXAMPLES per WORKED EXAMPLE the numbers, and the provenance that makes it checkable: variant, sample, position or byte, and the file it was read out of The prose fields are lifted from the alternative blocks in p8v2-words.md, so the page cannot drift from the design. The example fields are computed from the retained models under tests/pprog/models/p8v2/enwik9/ and the dumps in tests/pprog/gen1-pos/. Both are DISPLAYING work in the sense of #hutter_metrics: no compressions are run to produce the document. Publishing stays ../hutter's (#hutter_publication_handoff); we ship the data and the builder links it. WORKED EXAMPLE means pulled from the data, never an invented illustration of the text. Every one carries its provenance for that reason. ## The four things the choices block asked for, and what each takes 1. A3, RENORMALIZATION TO CONSTANT SUM 255. Owed as a formula AND a worked example. The formula is the LSA arithmetic #pp_settle actually runs (lsa_sum256 over the position's 256 entries, then the excess subtracted from every entry), written out; the example is one position of v003 at e64 stepped through a sweep -- entries before, the sum, the subtraction, entries after -- against the same position under v002, which decays by 2 instead. This is also the axis's own question: whether the constant-sum invariant is doing the work or the rate of approach to it is, and the two variants side by side at one position is the shortest statement of that. 2. B1, RECONSTRUCTION FROM THE ARGMAX TABLE. Owed for a rare byte and for common ones. From a retained model's 256-byte forward table: for "z", for "e" and for " ", the predecessor set {a : table[a] = b}, its size, ceil(log2 |set|), the message weight that survives, and the empty-set case that falls back to the background. Then the same three under B2's stored 65536-byte LPP, which is the whole point of the control: where the reconstruction and the stored column disagree, and whether the disagreement ever reaches the trace. 3. C2, THE AND GATE AT PARTIAL ACTIVATION. Owed as examples from the data. Positions where min fires and sum-with-threshold does not, and where both fire at different weights, taken from v002 against v014 at the same positions. Two absolute sources give 255 either way, so every example that says anything is a partial one; theta = 128 is a hyperparameter (#omega-p8-hyper) and the honest version of this example shows what the sources' weights actually are at these samples, since that is what decides whether the gate can fire at all. 4. THE SETTLING SENTENCE. settled_ok, mean_sweeps and conv are columns of gen1.tsv that nothing defines in words. Define them, on the page, next to the numbers: what counts as settled, what the sweep count is a mean over, that conv is PER WINDOW and not per position (#hutter_e64_viewer_prereq_20260805, item 4 of its six-things list), and that conv = W is the did-not-converge sentinel so it reads as 64 at e64 and not 128. The sentence the choices block quotes should be replaced on the page by the definitions plus the numbers, not by a better-worded version of the same sentence. ## Frozen across generation 2 Everything generation 1 froze: p7's trace format, k=2 only, W = 128 (or what is left of the sample), max token length 64, the message rule min(w_p, w_s), W sweeps per window, LSA as bytes in [0,255], one model file, and the query-layer requirement -- a variant that cannot be unfolded by --patterns and --explain is dead on arrival regardless of its numbers. ## Deliberately NOT in this generation THE k=0 REMOVAL. It is ruled -- the 256-byte background table is a defect and the correct form is a pattern from an always-present "true" event, recorded in #pp_wordsv2 and in #hutter_e64_viewer_prereq_20260805 -- and it is still not scheduled here. It changes the initial state of every unclamped position, so it moves settled_ok, mean_sweeps and conv for every variant. This generation exists to make exactly those three quantities legible; changing them in the same generation confounds the one thing being trialled. It moves class (a) only, so it can land later without invalidating any class (c) rate. Until it lands the viewer keeps reading the 256 bytes at model offset 48 as provisional, per the standing instruction. Also out: k=3 and above, strengths on k=1 (step 2), entropy-coding the gaps, and the interpreter cleanups #run_ofra_strip_goal_20260729 / #run_ofra_split_goal_20260729. ## Care - Every generation-2 variant is a new block in p8v2-words.md, and the staleness key checksums the whole doc, so adding them regenerates every variant's blocks and binaries -- generation 1's same warning. Verify the p7 way on the other fixtures: no rewritepl and no cc on p1/p3/p4/p7 after the change. - #pp_wordsv2 carries a BLOCKED note: #pp_dump settles every window of the whole sample, which caps the affordable prefix at 10^6. It wants a cap so it settles only the first few thousand positions. That is an NL change to #pp_dump and #pp_compress and it is a prerequisite for any rung above 10^6, not for this generation's ladder. - The empty-block cache wart from generation 1 is unfixed: a --rewritepl call that returns nothing gets cached under its staleness key and splices emptiness in forever after. Workaround is `rm .cmpr/pprog/blocks/-` and rerun. ## Decisions made here, on the programmer's behalf, and therefore the ones worth overruling - Generation 2's baseline is generation 1's v002 rather than a fresh block, and new variants continue the global v-numbering at v012 instead of restarting per generation. - All five open axes run again rather than only the ones with worked examples owed. The alternative was to document A, B and C and defer D, E and F; running them off A2 costs nine runs and is the only way to learn anything about D, which A1 made structurally inert. - The axis document is a data file we emit and ../hutter links, replacing their hand-copied WHAT dict, rather than prose in a block that someone re-types onto the page. - The k=0 removal is deferred, for the reason above. Just get rid of it; if there's anything that doesn't make sense without it, get rid of that too, or note whatever you think needs to be done. -MJC ## Done when The ten runs are in a generation-2 TSV alongside generation 1's, tests/pprog/axes.json exists and carries every axis, every surviving alternative and the four worked examples above with provenance, ../hutter's builder links it in place of WHAT, #f-p8-assign carries its PRUNED line, prior acceptance tests and tests/run-all are green, and the build is warning-clean. Then the generation-2 choices block is written with the needs line and it goes to the programmer, who picks. Generation 3 is not this push. ## IMPLEMENTED 2026-08-06 Done, and the numbers and the picks are in #pprog_p8v2_gen2_report_20260806. What was decided while doing it, and therefore what is worth overruling: THE k=0 REMOVAL LANDED. The "Deliberately NOT in this generation" section above is overruled by the programmer's own line in this block ("just get rid of it; if there's anything that doesn't make sense without it, get rid of that too"). Three places read the background and all three now say the honest thing: an unclamped position initialises to 256 zeros, #f-p8-assign assigns 256 zeros when nothing fires, and #mem_cell_markov_1-norm sends no backward message when a byte has no predecessor. Nothing replaced it (#pprog_p8v2_gen1_choices: the Markov model converges to the character frequency anyway). The model file lost its first 256-byte section, every offset past the header moved down 256, the magic did not change and stale files are caught by the size identity. Full account in #pp_wordsv2. BECAUSE IT ALSO MOVES THE ENTROPY STREAM -- learn_k0's draws are gone -- GENERATION 1 WAS RE-RUN rather than quoted, and the pre-removal gen1.tsv is not kept. THE ACCUMULATION NESTING IS NOW PINNED, and it had to be. v002's and v003's generated C added the messages entrywise with opposite loop nestings, and lsa_add draws from the shared stream, so axis A had never been a like-for-like comparison. #pp_wordsv2 now specifies message-outer, entry-inner and says why. Q1 of the report asks what to do about the class of bug this belongs to. NO v012. The plan reserved it for A3 off the new baseline, but on axis A the baseline is the thing that moved and A3-with-everything-else-at-baseline is the vector 311111, which is #wordsv2-v003 already. Generation 2 runs v003 and v012 stays unspent; every other id means what this block says. THE "CARE" WARNING ABOUT THE STALENESS KEY IS WRONG and is corrected in the report: a block's cache key is the checksum of its own expanded content and the binary's key is the checksum of the filled .c, so adding variants to p8v2-words.md costs nothing. Only an NL change re-rolls, which is what the k=0 removal was. THE AXIS DOCUMENT IS tests/pprog/axes.json, emitted by tests/pprog/p8v2-axes.py, gated by tests/pprog/acceptance-p8v2-axes (which fails if any prose is not its OFRA block byte for byte, or any worked example lacks provenance), and linked by ../hutter's build-p8v2-gen1 in place of its WHAT dict. The sweep-level examples exist because tests/pprog/p8v2-replay.py replays the run's every entropy draw in Python and checks itself against the retained model and the .pos dump first. STILL NOT DONE, and still not scheduled: the #pp_dump settling cap that #pp_wordsv2 records as BLOCKED, which is what caps the affordable prefix at 10^6. Axis D makes it worse -- v015 at 10^4 takes 186 s and 10^5 would be hours -- and Q3 of the report asks whether to pay for it. */