org.apache.commons.compress.compressors.lz77support

Class LZ77Compressor

java.lang.Object

org.apache.commons.compress.compressors.lz77support.LZ77Compressor

public class LZ77Compressor
extends java.lang.Object

Helper class for compression algorithms that use the ideas of LZ77.

Most LZ77 derived algorithms split input data into blocks of uncompressed data (called literal blocks) and back-references (pairs of offsets and lengths) that state "add length bytes that are the same as those already written starting offset bytes before the current position. The details of how those blocks and back-references are encoded are quite different between the algorithms and some algorithms perform additional steps (Huffman encoding in the case of DEFLATE for example).

This class attempts to extract the core logic - finding back-references - so it can be re-used. It follows the algorithm explained in section 4 of RFC 1951 (DEFLATE) and currently doesn't implement the "lazy match" optimization. The three-byte hash function used in this class is the same as the one used by zlib and InfoZIP's ZIP implementation of DEFLATE. The whole class is strongly inspired by InfoZIP's implementation.

LZ77 is used vaguely here (as well as many other places that talk about it :-), LZSS would likely be closer to the truth but LZ77 has become the synonym for a whole family of algorithms.

The API consists of a compressor that is fed bytes and emits LZ77Compressor.Blocks to a registered callback where the blocks represent either literal blocks, back-references or end of data markers. In order to ensure the callback receives all information, the #finish method must be used once all data has been fed into the compressor.

Several parameters influence the outcome of the "compression":

windowSize

the size of the sliding window, must be a power of two - this determines the maximum offset a back-reference can take. The compressor maintains a buffer of twice of windowSize - real world values are in the area of 32k.

minBackReferenceLength

Minimal length of a back-reference found. A true minimum of 3 is hard-coded inside of this implemention but bigger lengths can be configured.

maxBackReferenceLength

Maximal length of a back-reference found.

maxOffset

Maximal offset of a back-reference.

maxLiteralLength

Maximal length of a literal block.

Since:

1.14

See Also:

"https://tools.ietf.org/html/rfc1951#section-4"

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	LZ77Compressor.BackReference Represents a back-reference.
static class	LZ77Compressor.Block Base class representing blocks the compressor may emit.
static interface	LZ77Compressor.Callback Callback invoked while the compressor processes data.
static class	LZ77Compressor.EOD A simple "we are done" marker.
static class	LZ77Compressor.LiteralBlock Represents a literal block of data.

Constructor Summary

Constructors

Constructor and Description
LZ77Compressor(Parameters params, LZ77Compressor.Callback callback) Initializes a compressor with parameters and a callback.

Method Summary

Modifier and Type	Method and Description
void	compress(byte[] data) Feeds bytes into the compressor which in turn may emit zero or more blocks to the callback during the execution of this method.
void	compress(byte[] data, int off, int len) Feeds bytes into the compressor which in turn may emit zero or more blocks to the callback during the execution of this method.
void	finish() Tells the compressor to process all remaining data and signal end of data to the callback.
void	prefill(byte[] data) Adds some initial data to fill the window with.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LZ77Compressor

public LZ77Compressor(Parameters params,
                      LZ77Compressor.Callback callback)

Initializes a compressor with parameters and a callback.

Parameters:

params - the parameters

callback - the callback

Throws:

java.lang.NullPointerException - if either parameter is null

Method Detail

compress

public void compress(byte[] data)
              throws java.io.IOException

Feeds bytes into the compressor which in turn may emit zero or more blocks to the callback during the execution of this method.

Parameters:

data - the data to compress - must not be null

Throws:

java.io.IOException - if the callback throws an exception

compress

public void compress(byte[] data,
                     int off,
                     int len)
              throws java.io.IOException

Feeds bytes into the compressor which in turn may emit zero or more blocks to the callback during the execution of this method.

Parameters:

data - the data to compress - must not be null

off - the start offset of the data

len - the number of bytes to compress

Throws:

java.io.IOException - if the callback throws an exception

finish

public void finish()
            throws java.io.IOException

Tells the compressor to process all remaining data and signal end of data to the callback.

The compressor will in turn emit at least one block (LZ77Compressor.EOD) but potentially multiple blocks to the callback during the execution of this method.

Throws:

java.io.IOException - if the callback throws an exception

prefill

public void prefill(byte[] data)

Adds some initial data to fill the window with.

This is used if the stream has been cut into blocks and back-references of one block may refer to data of the previous block(s). One such example is the LZ4 frame format using block dependency.

Parameters:

data - the data to fill the window with.

Throws:

java.lang.IllegalStateException - if the compressor has already started to accept data