/* #pprog_p8v2_gen2_report_20260806 @pprog_p8v2_gen2_goal_20260806 @variant_protocol @pprog_p8v2_gen1_choices @hutter_metrics @pp_wordsv2 @hutter_publication_handoff GENERATION 2 IS BUILT AND AWAITS THE PICKS (run --needs) Ten runs off the new baseline, the k=0 background gone, and the four things #pprog_p8v2_gen1_choices asked to be able to READ now on the page as data. What follows is what the runs say; the picks are the programmer's (#variant_protocol). Data: tests/pprog/gen2.tsv (measured), tests/pprog/gen2-pos/ (per position), tests/pprog/axes.json (the axes and the worked examples), and `python3 tests/pprog/gen1-report tests/pprog/gen2.tsv` for the derived view. gen1.tsv WAS RE-RUN and is post-removal too, so the two generations are comparable with each other and with nothing published before 2026-08-06. ## What changed under the machine before any of these numbers k=0 IS GONE, on the instruction in the goal block ("just get rid of it"). Three readers, three answers: an unclamped position now initialises to 256 zeros, #f-p8-assign assigns 256 zeros when nothing fires, and #mem_cell_markov_1-norm sends NO backward message when a byte has no predecessor. Class (a) fixed drops 256 bytes for every variant, the model file lost a section, and every offset past the header moved down 256. Nothing replaced it, per your ruling: the Markov model converges to the character frequency, so if the base rate were needed it would be available. AND ONE THING WE FOUND WHILE DOING IT, which had to be fixed before axis A meant anything. The accumulating update says "add each message entrywise". v002's generated C did that entry-outer and v003's did it message-outer -- and lsa_add DRAWS FROM THE SHARED ENTROPY STREAM, so the two nestings consume it in different orders and give different runs. Generation 1's A2 and A3 were therefore not a like-for-like comparison and neither is any published number from before today. #pp_wordsv2 now pins the nesting as message-outer, entry-inner and says why. This is the class of thing the LLM will keep choosing for us wherever the NL leaves it open; see the question at the end. ## The runs Trace bytes are class (c), the only class that survives to 1e9. Baseline is v002. e10k trace settled_ok mean_sweeps roundtrip what moved v015 5599 5771 81.89 NO D2 -302 bytes v016 5633 5803 80.46 NO D3 -268 bytes v017 5882 4633 1.58 yes E2 -19 bytes v002 5901 5828 71.87 yes the baseline v003 5901 5942 37.68 yes A3 rate identical, settles more, converges sooner v013 5901 6043 126.54 yes B2 rate identical, +65536 bytes of class (a) v014 5901 5828 71.87 yes C2 BYTE-IDENTICAL TO THE BASELINE IN EVERY COLUMN v018 5928 4679 1.57 yes E3 +27 bytes v019 7834 5828->8468 42.42 yes F2 model 4296 -> 1024, trace +1933 v020 7878 5828->8443 41.33 yes F3 model 4296 -> 380, trace +1977 AXIS D IS MEASURABLE AT LAST, AND IT DOES NOT ROUND-TRIP. Under A1 the settled argmax at a candidate was the causal prediction by construction, so nothing was ever removable and D was inert. Under A2 it removes: 302 trace bytes at e10k for D2, 268 for D3, and 6 and 5 at e1k. But BOTH FAIL ROUND-TRIP at every size above 64 bytes. The removal is not sound as it stands -- the decoder settles the same window and does not recover the same byte -- so this is a real reduction that is not yet a real compression. That is the sharpest open thing in this generation. AXIS C IS NOW ANSWERED, NEGATIVELY, AND WE CAN SAY WHY. v014 matches v002 in every column including settle_apps. The worked example says exactly why, from the data: over all 3140 gate evaluations of v002's e64 run the two GATES differ 3140 times and the MESSAGES differ zero times, because the message is min(rule weight, gate) and the learned k=2 weights at this sample are 1 or 2. The gate never binds. C is not degenerate the way generation 1 made it look (it never converged under A1); it is invisible until the rule weights are large enough to be the smaller term. AXIS E CHANGES SETTLING MORE THAN ANYTHING ELSE HERE. v017/v018 converge at 1.58 and 1.57 sweeps against the baseline's 71.87 -- essentially at once -- while settling FEWER positions correctly (4633 against 5828). Relearning k=1 against the k=2 mask leaves a table that is worse on its own; the window reaches a fixed point immediately because there is less to propagate. E2 buys 19 trace bytes; E3 loses 27. AXIS A: A3 costs nothing and converges in half the sweeps (37.68 against 71.87) while settling 114 more positions correctly. The rate is identical because D1 does not let settling reach P. If D ever does round-trip, this is the axis that will matter. AXIS B: B2's only visible effect is on settling -- 215 more positions correct, but 126.54 mean sweeps against 71.87, i.e. it barely converges inside the window -- for 65536 bytes of class (a). The rate is identical. AXIS F: unchanged from generation 1 and unchanged by A2. Below 1e4 the budget does not bind at all. At 1e4 it binds and it is a bad trade in class (c): F3 gives back 380 bytes of model to spend 1977 more on trace. ## The four things you asked to be able to read, and where they are They are in tests/pprog/axes.json, which ../hutter's builder now links IN PLACE OF the hand-copied WHAT dict it used to carry. The emitter is tests/pprog/p8v2-axes.py and the gate is tests/pprog/acceptance-p8v2-axes, which fails if any alternative's prose is not the text of its OFRA block byte for byte, or if any worked example lacks provenance. 1. A3's RENORMALISATION, formula and worked example. Position 37, sweep 2, of v003 at e64, against v002 at the same position and sweep. The backward message from "." at 255 spreads over its two predecessors at cost 1, so 254 each; A3 accumulates, gets lsa_sum256 = 256, subtracts the excess of 1 from every entry and lands on 254; A2 subtracts 2 and lands on 252. Both vectors are given in full with their top entries and weight histograms. 2. B1's RECONSTRUCTION, for a rare byte and common ones, at e10k. "z" HAS AN EMPTY PREDECESSOR SET -- nothing has z as its argmax successor -- so under B1 no backward message fires at all, where the stored column under B2 has four entries. "e" has 9 predecessors, ceil(log2 9) = 4, so an absolute source delivers 251 to each of the nine; B2's column has 37 entries with weights 8 down to 1. 218 of the 256 bytes have an empty predecessor set. Whether the disagreement reaches the trace: under D1 it cannot, and the file says so with the two TSV rows side by side. 3. C2 AT PARTIAL ACTIVATION: the counts above, plus three concrete (position, sweep, w_a, w_b, gate, message) rows and the distribution of w_a + w_b and of the rule weights, which is what decides whether the gate can fire at all. 4. THE SETTLING SENTENCE IS REPLACED BY DEFINITIONS PLUS NUMBERS. settled_ok is out of D = min(M, 65536) and includes clamped positions, which are right by construction. mean_sweeps is a mean over WINDOWS, not positions. conv is PER WINDOW, repeated on every row of the dump, and conv = W is the did-not-converge sentinel, with W = 128 or what is left of the sample -- so at e64 it reads 64. ## Everything else that was asked for #f-p8-assign carries its PRUNED line. All prior acceptance tests are green with no rewritepl and no cc on p1/p3/p4/p7 (the p7 way, verified). tests/run-all 64/64 and tests/n_tcp/run-all 13/13. The build is warning-clean. Two departures from the goal block, both recorded where they happened: NO v012. The goal reserved it for "A3 off the new baseline", but on axis A the baseline is what moved, and A3-with-everything-else-at-baseline is the vector 311111, which IS #wordsv2-v003. Generation 2 runs v003 rather than minting a second id for the same vector, and v012 stays unspent. Every other id means what the goal says it means. THE STALENESS WARNING IN THE GOAL'S "CARE" LIST IS WRONG. Adding variants to p8v2-words.md does NOT regenerate every variant's blocks and binaries. The doc checksum names the generated .c FILE, but a block's cache key is the checksum of that block's own expanded content, and the binary's key is the checksum of the filled .c -- neither moves when the doc gains a block. Verified: the nine new variants cost zero LLM calls beyond the two new pp_settle vectors. The k=0 removal cost calls because it changed the NL, which is the real rule. ## The picks A A2 (baseline) | A3. A3 is free and converges in half the sweeps. Nothing on the rate. B B1 (baseline) | B2. B2 costs 65536 bytes of class (a) and moves only settling. C C1 (baseline) | C2. C2 is byte-identical to C1 at every sample we can run. Prune it, keep it as a control against the day rule weights are large, or re-ask it at a size where they are? D D1 (baseline) | D2 | D3. The only axis that moves the rate, and neither D2 nor D3 round-trips. Is a non-round-tripping trace reduction worth carrying into generation 3 as a thing to FIX, or is D1 the answer and settling stays a diagnostic? E E1 (baseline) | E2 | E3. E2 is 19 bytes better and collapses settling to 1.58 sweeps. Is "converges immediately because there is less to propagate" a win or a warning? F F1 (baseline) | F2 | F3. Nothing has changed; the budget does not bind below 1e4. ## Three questions Q1. THE NL-AMBIGUITY CLASS. The accumulation-nesting bug was invisible until a Python replay of the same run disagreed with the C. Anywhere the NL says "do X to each of these" and the operation draws entropy, the LLM picks an order and we inherit it. Do we (a) audit the remaining NL for stream-order ambiguity now, (b) add a rule that any entropy-consuming loop states its nesting, or (c) make the child's entropy stream per-site instead of global so order stops mattering? Why do we care about the order? The answer is (d) identify the places where we care about specific entropy bits being reproducible and identify them as bugs. The exception is for debugging and tracing, but generally we would resolve that by adding instrumentation rather than trying to debug one implementation by running a different one. -MJC Q2. D DOES NOT ROUND-TRIP. The removal pass settles with every OTHER recorded position clamped, but the decoder settles with the SAME clamped set and gets a different byte, because the decoder's unrecorded neighbours are not the sample's bytes. Should generation 3 make the removal test match what the decoder actually sees (settle the whole window from the decoder's own state), or is D dead? D is an AXIS, not a specific choice along that axis, and because you keep assuming that I know what the axes are everywhere, which I don't, I have NO IDEA wtf you are asking. -MJC Q3. THE LADDER STOPS AT 1e4 FOR D. v015 at e10k took 186 s and D at 1e5 is about three hours (#hutter_run_costs). Everything else runs to 1e6. Do we want D measured at 1e5 at that price, or is the round-trip failure enough to park it? Nothing should run for more than a few minutes while I'm waiting for visualization work. Once the basic runs are done and visualized, longer ones can be fired off to run in the background. There's already a protocol for this somewhere. Then the longer runs can added to existing visualizations whenever they finish. -MJC Written 2026-08-06. */