/* #hutter_metrics @pprogram_design_202606 @variant_protocol @pprog_p8v2_impl_goal_20260804 @pprog_p7_markov_goal_20260704 How the Hutter work reports its numbers One place that says what is measured, which class each measured quantity belongs to, how it extrapolates to the full DSS, and how an extrapolation has to be labelled. It governs the parent's report lines, the per-generation TSVs, and anything quoted into a goal or report block. The problem it exists to stop: our samples are tiny and the real DSS is 10^9, so almost every number we can measure is dominated by terms that vanish at 10^9. A single per-byte rate over the whole model file is therefore not a compression number, it is an artifact of the sample size, and it will rank things backwards. That has already happened once -- see the worked example. ## The three classes Every byte of S belongs to exactly one. (a) FIXED -- size does not depend on the DSS. the self-extracting binary; the model-file header; the k=0 background table (256 bytes); the k=1 byte model, which is 256 bytes when it records only the most common follower (p7 and p8v2 step 1), 512 with strengths (step 2), or at most 65536 for the full LPP (#mem_cell_markov_1-lpp). Extrapolation: none. Its contribution to S/U is fixed/DSS -- for a 26 KB binary at 10^9 that is 2.6e-5, and it is reported so that it is visibly negligible rather than silently dropped. Reported as BYTES. (b) SUB-LINEAR -- grows with the DSS but is capped by the architecture. the token section: kept k-rules times the entry size, capped at (possible contexts) times (entry size). At k=2 that cap is 65536 * 4 = 262144 bytes, i.e. 2.6e-4 of S/U at 10^9, whatever the sample says. Extrapolation: assume growth in log DSS. From a measurement at sample size M, model_b(DSS) = model_b(M) * log(DSS) / log(M), capped at the architectural bound. Reported as BYTES at the sample and BYTES extrapolated. NEVER as a per-byte rate on the sample: that is the error, because dividing a capped quantity by a tiny M and then multiplying by 10^9 charges it thousands of times over. (c) LINEAR -- scales with the DSS once the model is fixed. the memory trace: the gap section plus the recorded bytes. This is the number that matters. At 10^9, at anything like current densities, it is more than 99.99% of S, so it is the only quantity whose differences survive to the chart. Reported as a per-byte rate a = trace_bytes(M) / M. ## The composite S(DSS) = fixed + model_b(DSS) + a * DSS S/U = fixed/DSS + model_b(DSS)/DSS + a k = log(S/U) / log(0.99) ## Labelling Any number derived for a DSS other than the sample's own is an EXTRAPOLATION, says the word, and carries its basis: the sample size it was measured at and the target DSS. A measured number and an extrapolated number never appear in the same column without that distinction. The three classes are never summed into one per-byte rate. If a single headline is wanted it is a, the class (c) rate, with the class (a) and (b) byte counts beside it. ## The standing caveat on (c) "Linear once the model is fixed" is exact, and it is also why a at one prefix does not predict a at 10^9: a is measured under the model learned from THE SAME BYTES it is then measured on. At small M the model has simply memorised the sample -- most contexts were seen once, so the argmax is whatever followed that once -- and a is optimistic. As M grows the same contexts recur with different successors, the argmax stops being free, and a RISES toward its true value. That direction was worth measuring rather than assuming; the first version of this block asserted the opposite. On real enwik9 prefixes the baseline's a goes 0.219, 0.317, 0.584, 0.730, 0.770 at M = 64, 10^3, 10^4, 10^5, 10^6 -- still rising at 10^6, and decelerating. So a measured on a small prefix is a LOWER BOUND on the asymptote, and the smaller the prefix the more useless the bound. A single a is therefore a point on a curve and is read as one. Report a over a prefix ladder (10^3, 10^4, 10^5, ... as far as the run cost allows) and read the trend. enwik9 is at ../hutter/enwik9. Whether that trend is itself extrapolable is open and is not decided here. ## What is not a compression number At k=2 the entire model is bounded at 262144 bytes, three orders of magnitude below the differences between variants in a. A keep/prune rule therefore cannot be justified or refuted by compression at this k, and reporting one as if it could is the same mistake in a different place. What the keep rules are measured on instead: - trace density: recorded bytes / M - coverage: positions where a kept rule fires / M - hit rate: fires and is right / fires - value per rule: trace entries it eliminates, against its entry size in bytes - settling: convergence sweep, and pattern applications per input byte These say whether a rule is ready for k -> 64, where the table is not bounded and where the pruning question is real. ## Worked example: why the classes are not optional p8v2 generation 1 on the 1 KB sample, reported both ways. The r column is what #pprog_p8v2_gen1_report_20260804 printed; the a column is class (c) alone. variant what it varies model_b trace a r as printed v001 the baseline 1316 594 0.5801 1.8652 v008 E2, two replay rounds 1316 582 0.5684 1.8535 v010 F2, 256 by support 1024 641 0.6260 1.6260 v011 F3, 256 by sharpness 164 818 0.7988 0.9590 p7 -- 0 845 0.8252 0.8252 r ranks them v011 > v010 > v008 > v001. a ranks them v008 > v001 > v010 > v011. The rankings are reversed, and a is the one that survives to 10^9: the largest model_b in that table extrapolates to 3934 bytes at 10^9, which is 3.9e-6 of S/U, while the spread in a is 0.23. Read through r, the generation says pruning helps; read through a, it says every prune tried so far costs more prediction than it saves, and keeping everything is the best predictor we have. Extrapolated to DSS = 10^9 from M = 1024, by the composite above: v001 S/U = 0.580108 k = 54.2 v011 S/U = 0.798855 k = 22.3 v008 S/U = 0.568389 k = 56.2 p7 S/U = 0.825221 k = 19.1 v010 S/U = 0.626006 k = 46.6 Every one of those five numbers is an extrapolation from 1 KB and none of them is a chart point: the chart wants the full DSS, and it wants a p8v2 that has actually been through the pruning work this push is only the first step of. ## Who computes what The split this implies is worth stating as a rule, because getting it wrong is what made the first version expensive to correct: the extrapolation used to be computed inside the compressor's parent at run time and printed to stderr, so changing the policy meant re-running every compression. GENERATING emits facts about a (variant, sample) pair and nothing else: the byte counts of the three classes, the binary size, the round-trip, the settling cost. #pp_wordsv2's report lines, and tests/pprog/acceptance-p8v2, which writes them to tests/pprog/gen1.tsv. No rate, no extrapolation, no ranking. DISPLAYING derives everything: a, model_b(DSS), S/U, k, the rankings, the k=2 diagnostics (which are computable from sample and model, so they belong here too). tests/pprog/gen1-report and tests/pprog/p8v2-diag.py. Runs no compressions. PUBLISHING ../hutter's, which owns it anyway. A change of reporting policy therefore costs a re-render. Only a change of INPUTS costs a re-run. ## Status Written 2026-08-05, in response to generation 1 reporting the wrong thing. Implemented for the wordsv2 kind: #pp_wordsv2 prints the three classes as byte counts and refuses to print a derived quantity, acceptance-p8v2 writes measured facts only, and gen1-report applies everything above. The acceptance test greps for the absence of the old lines, so a derived quantity cannot quietly come back. Not implemented, deliberately, for the older kinds: p7's P is entirely class (c) already -- its table is class (a) and it has no class (b) at all -- so p7's r IS the class (c) rate and its reporting is correct under this scheme by construction. p8v1 (words) has the same defect p8v2 had, its lexicon being class (b), but it is a dead branch (#pprog_p8_words_20260706 is a declared false start), so its lines are left alone rather than churned. */