Why gap-blend beats ent-blend despite worse order selection. Per-position evidence from order-6 and order-8 UM n-gram models on enwik9.
36.2%Gap wins (freq)
1.053Gap mean adv (bpc)
63.3%Ent wins (freq)
0.456Ent mean adv (bpc)
+0.093Net (gap better)
Surprise Difference Histogram
Each bar shows how many positions had (ent_surprise - gap_surprise) in that range. Positive = gap-blend was better. The right tail (gap wins big) outweighs the left tail (ent wins small).
The asymmetry: Ent-blend wins 63% of positions by small amounts (mean 0.46 bpc). Gap-blend wins 36% of positions by large amounts (mean 1.05 bpc). Frequency × magnitude: gap's product is larger.
Dominant Order When Winning
Which n-gram order is dominant (highest gap / lowest entropy) at positions where each blend wins?
Gap loves order 0: At order 8, gap-blend wins while favoring the unigram for 46% of its winning positions. The unigram is never catastrophically wrong — it tracks base rates — so over-weighting it is safe. Meanwhile ent-blend distributes its wins across higher orders.
Gap Magnitude Distribution
How large is the maximum support gap at positions where each blend wins?