1. 8
  1. 3

    Compression speed is similar to CMIX’ but a thousand times slower than xz -9.

    I would also add that xz’ -9 option is not necessarily an appropriate option: it’s exactly like -6 but with a larger dictionary.

    I spent a few CPU cycles: xz -9 gets a ratio of 0.213 and xz –lzma2=preset=6,dict=1G gets 0.200.

    And if I understand correctly, the model isn’t transmitted for decompression, therefore saving space but it has to be built again from scratch and that requires the exact same hardware and software versions. The base model (lower compression) contains 56M parameters and the large one 187M parameters. Everything is running with a Nvidia 3090 RTX GPU. That doesn’t sound practical at all to me. Sure it’s maybe a first but……

    1. 1

      I don’t think practicality was something that was optimized for here. At least, I didn’t see that stated as a goal (implied or otherwise).

      1. 2

        I’ve used “practical” because it’s used in https://bellard.org/nncp/nncp_v2.1.pdf :

        We presented the first practical Transformer implementation able to outperform the best text compression programs on the enwik9 benchmark [8]. Unlike most today’s state-of-the-art natural language processing results, it is achieved on a desktop PC with a single GPU

        Of course there are different levels of practicality but I would have rather said that it is compatible with desktop-class hardware. I mean… at 1KB/s, it takes close to two weeks to compress enwik9 and again two weeks to decompress it!

        1. 2

          Ah interesting. Yeah, I’d say that’s a pretty loose definition of “practical,” although he did kinda clarify what he meant by it.

    Stories with similar links:

    1. Lossless Data Compression with Neural Networks via tobym 3 years ago | 15 points | no comments