Jump to content
  • Advertisement

2 Screenshots

About This File

lolz is a compressor based on adaptive rANS, suitable for any data, but shows the best results on structured data. Made special models for dxt textures and raw graphics (in the distant future, perhaps, for raw audio will appear). It is possible to compress and decompress multi-threaded.
There are quite a few options in lolz, but some of them no longer work due to the lack of need for them, but they have not yet been removed from the codec. The default options are optimal in most cases.

Brief description of options:

Data detection options:

  • -dt [0..1] - enables / disables detection of pos_ctx / dxt / raw. no headers, everything is detected based on the analysis of data statistics. Default: -dt1;
  • -dtp [0..1] - enables / disables transmission of statistics from previous blocks to subsequent blocks in the detector. Default: -dtp1;
  • -dtb [0..1] - enables / disables enumeration of all options regardless of heuristics. Default: dtb0;
  • -dto [0..1] - enables / disables detection of the best positional o1 context. Default: -dto1;
  • -dtm [0..1] - enables / disables raw graphics multimedia detection. Default: -dtm1;
  • -dtw [0..1] - enables / disables width detection for raw graphics and dxt textures;
  • -dtd [0..1] - enables / disables detection of dxt textures;

Multi-threading options:

  • -mtt [0..1] - when multithreaded processing, specifies the used mode of operation. At 0, the dictionary size must be at least 2 times the block size. In this mode, data for each stream will be loaded alternately in block size. In this mode, in most cases, you can achieve better compression than in the second, however, decompression requires as many streams as in compression. When 1, each block is compressed separately, without dependencies on neighboring data, respectively, the compression here is usually worse than in the first mode, but you can specify any number of unpacking streams. It is for this mode that the options from cls.ini MaxThreadsUsage and MaxMemoryUsage are used. Default: -mtt0;
  • -mt [1..16] - sets the number of threads to process. With -mt1 and -mtt0, normal sequential lossless compression is obtained by dividing the stream into blocks. Default: -mt1;
  • -mtb [2..512] - sets the block size in MB. With -mt1 -mtt0, the value also plays, but minimal role. And more doesn't mean better. Usually, for -mtt0, the optimal value is about 32-64MB, so the size of the dictionary should be more than 2 times larger. For mtt1 mode, the dictionary size must be no more than the block size;

Main options:

  • -d [16..2032] - dictionary size in MB. Default: -d64;
  • -tt [1..256] - the number of paths to consider in the optimal parser. It greatly affects the speed and compression ratio, but not unpacking. You shouldn't ask more than 16, I assure you it's not worth it. Default: -tt4;
  • -oh [8..14] - sets the maximum number of bytes that the optimal parser will process at a time (2 ^ X). Default: -oh12;
  • -os [0 ..- oh] - sets the minimum number of bytes that the parser will process at a time (2 ^ X). Default: -os8;
  • -fba [0..4096] - sets the size of the minimum match, at which the parser will not bother much in calculations. Compression is decently accelerated (2 times) with a small loss of compression. At 0, these simplifications are disabled. Default: -fba256;
  • -fbb [0..4096] - THIS OPTION DOES NOT WORK AT THE MOMENT. Asked for even greater simplifications;
  • -al [0..1] - enables / disables calculation of literal price even if rep0 matches. Default: -al1;
  • -x [0..2] - turns on the slow modes of the parser with the calculation of (almost) all lengths of the found match, as well as the options match + lit + rep0match. Very slow and merciless. And the benefit is very small. It's easier to add -tt; Default: -x0;

Matchfinder options:

  • -rt [0..2] - THIS OPTION DOES NOT WORK AT THE MOMENT. I set the matchfinder type - lz, rolz or hybrid mode, but rolz did not meet expectations and I made all the changes without taking it into account, so it does not work now; Default: -rt0;
  • -mc [2..1023] - Specifies the maximum number of traversals of the binary tree of matches, after which matches for the given position are no longer searched; Default: -mc128;

Model options:

  • -cm [0..1] - enables / disables a simple context mixer in some critical places, which mixes a couple of models in each place. When enabled, it improves compression, but slows down decompression. Default: -cm1;
  • -bc [0..8] - sets the level of influence of the previous byte on the mixer; Default: -bc4;
  • -lm [0..4] - THIS OPTION DOES NOT WORK AT THE MOMENT. Specifies the type of an "elementary" literal. Complex high-order models with cm did not perform well, so I abandoned them, as did rolz. Default: -lm0;
  • -blo [0..8] - sets the degree of influence of the previous byte on the encoding of the upper part of the literal. Default: -blo8;
  • -bll [0..8] - sets the degree of influence of the previous byte on the encoding of the lower part of the literal. Default: -bll8;
  • -blr [0..8] - sets the degree of influence of the rep0lit byte on the encoding of the upper part of the literal. Default: -blr4;
  • -bm [0..8] - sets the degree of influence of the rep0lit byte on the encoding of the match type flag. Default: -bm4;
  • -pc [0..4] - sets the positional context for all encoding operations. It is automatically ignored when the detector is on. Default: -pc2;
  • -dmXY (X [0..3], Y [0..4]) - sets the model for encoding color pairs (X) and alpha channel (Y) pairs. At the maximum value of each parameter, adaptive switching between models is used, while the decompression speed is reduced, but the compression is better in most cases. Default: -dm34;
  • -gmXY (X [0..2], Y [0..1]) - X - sets the model for encoding raw graphics. at the maximum value, adaptive switching between models is enabled. However, in this case, it is rare to see a gain over the adaptive mode. Generally, mode 0 is in the lead, but its unpacking is 2 times slower than mode 1. Y - enables updating of model statistics when they were not used (for example, there was a long match). For X0 and X1, it usually gives a small gain in compression, but the speed drops by a factor of 2 (it all depends on the data). In general, the most optimal is -gm00, which is the default mode;

What's New in Version test22c4b

Released

  • Implemented ldmf (long distance match finder) which searches for matches outside the dictionary range of the main matchfinder. It was developed as an alternative to srep, with its own pros and cons. Disabled / enabled by the -ldmf [0..1] option. The ratio of the dependence of the compression ratio on the required memory for decompression is set by the -ldc [0..9] option at 0, the memory control for decompression is disabled, maximizing the compression ratio. The size of the minimum length for searching is specified by the -ldl [5..12] option. This option affects the compression (the less, the better the compression), the used memory for compression (the less, the more memory according to the formula ldmf_compr_mem = src_size / ((2 ^ [ldl]) / 16), i.e. when -ldl5 will add another half of the size of the input file to the memory already used by lolz), for the used memory for unpacking (the less, the more memory will be needed) and the compression speed (the less, the slower). Added the -lde [0..2] option to set the level of parsing ldmf matches. 2nd mode is the slowest, 0 is the fastest, but the difference in compression is minimal. The 1st mode is almost as fast as the 2nd, but the compression is usually worse than even the 0th mode. So for now I recommend using either -lde0 or -lde2. The default is -lde0;
  • Added console and cls.ini parameters for unpack -ldmfTempPath and -ldmfMaxMemoryUsage, which specify the path to ldmf swap files and memory size for ldmf, respectively. The ldmfMaxMemoryUsage parameter, when calculating memory, includes the size of the main dictionary and another penny on the model (but the latter is counted by eye, from the bulldozer), for example, with -ldmfMaxMemoryUsage = 64m, one thread and a dictionary size of 16MB will be allocated 16MB for memory under ldmf, the rest will be discarded in swap. The -ldmfDeleteTmp [0..1] parameter disables / enables deleting the swap file after unpacking. Disabling is only necessary for debugging purposes and to control the size of this file. Default = 1;
  • Remade the function of synchronizing the thread of the optimal parser and matchfinders. The efficiency of using cores in compression with a detector has increased, respectively, the overall compression rate has increased;
  • Cut down the use of large pages, still no difference;
  • Added the -ac [0..1] option, which enables the transfer of match coding price calculations to the matchfinder thread, which speeds up compression with -tt greater than 4, with a lower -tt the speed is either the same or worse. Compression may differ slightly from the -ac0 mode, both positively and negatively. Also, in this mode, you need a little more memory for compression, which depends on the -tt parameter (add_mem = 200k + 450k * tt);
  • Reduced the size of basic model statistics by limiting some -b* options that do not change in practice;
  • Changed coding models dxt color idx and alpha idx. Now better compression and faster unpacking of dxt textures;
  • Fixed small bugs and added additional contexts for the raw graphics compression model. The size of the statistics of the graphic model has grown from
  • 700kb to 2mb, but the compression has also improved (on a 24-bit picture from 5'544'705 to 5'406'975);
  • Fixed the -fbb option that did not work at some point, but there was a minimum of sense from it.
  • Thanks 1
 Share


User Feedback

You may only provide a review once you have downloaded the file.

There are no reviews to display.

×
×
  • Create New...

Important Information

By using this site, you agree to our Guidelines, Terms of Use, and Privacy Policy with it We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.