Can gzip be a language model?

The author demonstrates that gzip, the standard compression utility, can function as a language model by leveraging the mathematical equivalence between compression and prediction—symbols that compress well are those the model assigns high probability to. By scoring text continuations based on how much they compress when appended to a priming corpus, gzip can generate coherent-ish Shakespeare-like text through beam search, despite having no neural networks or learned parameters. While the output isn't perfectly coherent, it shows that compression algorithms inherently contain predictive models capable of capturing meaningful patterns from training data.