Can gzip be a language model?

(nathan.rs)

35 points | by networked 1 hour ago

2 comments

  • mg 3 minutes ago

        give it a normal text prompt, and it
        continues that prompt by searching
        for the byte sequences that compress
        best.
    
    One moment, how are we supposed to know how well that search was done? There is no way to search a meaningful part of the search space.

    So the result does only gives us some lower bound of how well gzip works as a "plausibility tester" of a continuation of a text. The space of possible sequences is gazillion orders of magnitude larger than what was searched. So there might be sequences in there that compress much better.

  • Culonavirus 5 minutes ago
    This tracks perfectly with Winrar being more profitable than OpenAI... coincidence? I think not!