Class LZ77Compressor


  • public class LZ77Compressor
    extends java.lang.Object
    Helper class for compression algorithms that use the ideas of LZ77.

    Most LZ77 derived algorithms split input data into blocks of uncompressed data (called literal blocks) and back-references (pairs of offsets and lengths) that state "add length bytes that are the same as those already written starting offset bytes before the current position. The details of how those blocks and back-references are encoded are quite different between the algorithms and some algorithms perform additional steps (Huffman encoding in the case of DEFLATE for example).

    This class attempts to extract the core logic - finding back-references - so it can be re-used. It follows the algorithm explained in section 4 of RFC 1951 (DEFLATE) and currently doesn't implement the "lazy match" optimization. The three-byte hash function used in this class is the same as the one used by zlib and InfoZIP's ZIP implementation of DEFLATE. The whole class is strongly inspired by InfoZIP's implementation.

    LZ77 is used vaguely here (as well as many other places that talk about it :-), LZSS would likely be closer to the truth but LZ77 has become the synonym for a whole family of algorithms.

    The API consists of a compressor that is fed bytes and emits LZ77Compressor.Blocks to a registered callback where the blocks represent either literal blocks, back-references or end of data markers. In order to ensure the callback receives all information, the #finish method must be used once all data has been fed into the compressor.

    Several parameters influence the outcome of the "compression":

    windowSize
    the size of the sliding window, must be a power of two - this determines the maximum offset a back-reference can take. The compressor maintains a buffer of twice of windowSize - real world values are in the area of 32k.
    minBackReferenceLength
    Minimal length of a back-reference found. A true minimum of 3 is hard-coded inside of this implementation but bigger lengths can be configured.
    maxBackReferenceLength
    Maximal length of a back-reference found.
    maxOffset
    Maximal offset of a back-reference.
    maxLiteralLength
    Maximal length of a literal block.
    Since:
    1.14
    See Also:
    "https://tools.ietf.org/html/rfc1951#section-4"
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void compress​(byte[] data)
      Feeds bytes into the compressor which in turn may emit zero or more blocks to the callback during the execution of this method.
      void compress​(byte[] data, int off, int len)
      Feeds bytes into the compressor which in turn may emit zero or more blocks to the callback during the execution of this method.
      void finish()
      Tells the compressor to process all remaining data and signal end of data to the callback.
      void prefill​(byte[] data)
      Adds some initial data to fill the window with.
      • Methods inherited from class java.lang.Object

        equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Constructor Detail

      • LZ77Compressor

        public LZ77Compressor​(Parameters params,
                              LZ77Compressor.Callback callback)
        Initializes a compressor with parameters and a callback.
        Parameters:
        params - the parameters
        callback - the callback
        Throws:
        java.lang.NullPointerException - if either parameter is null
    • Method Detail

      • compress

        public void compress​(byte[] data)
                      throws java.io.IOException
        Feeds bytes into the compressor which in turn may emit zero or more blocks to the callback during the execution of this method.
        Parameters:
        data - the data to compress - must not be null
        Throws:
        java.io.IOException - if the callback throws an exception
      • compress

        public void compress​(byte[] data,
                             int off,
                             int len)
                      throws java.io.IOException
        Feeds bytes into the compressor which in turn may emit zero or more blocks to the callback during the execution of this method.
        Parameters:
        data - the data to compress - must not be null
        off - the start offset of the data
        len - the number of bytes to compress
        Throws:
        java.io.IOException - if the callback throws an exception
      • finish

        public void finish()
                    throws java.io.IOException
        Tells the compressor to process all remaining data and signal end of data to the callback.

        The compressor will in turn emit at least one block (LZ77Compressor.EOD) but potentially multiple blocks to the callback during the execution of this method.

        Throws:
        java.io.IOException - if the callback throws an exception
      • prefill

        public void prefill​(byte[] data)
        Adds some initial data to fill the window with.

        This is used if the stream has been cut into blocks and back-references of one block may refer to data of the previous block(s). One such example is the LZ4 frame format using block dependency.

        Parameters:
        data - the data to fill the window with.
        Throws:
        java.lang.IllegalStateException - if the compressor has already started to accept data