2.3.6. SNAPPY API - 5.2 English - 68552

Generic compression/decompression routines.

size_t Compress(Source *reader, Sink *writer)#

Compress the bytes read from “*source” and append to “*sink”. Return the number of bytes written.

Parameters	Direction	Description
reader	in,out	A Source is an interface that yields a sequence of bytes, you can initialize it by calling snappy::ByteArraySource(inBuf,inBufLen);where inBuf is the pointer to original data and inBufLen is the size of inBuf
writer	in,out	A Sink is an interface that consumes a sequence of bytes, you can initialize it by calling snappy::UncheckedByteArraySink(dest); where dest is the pointer to the destination buffer.

Returns:

Result	Description
Success	Return the number of bytes written.
Failure	Return 0 upon failure or NULL parameters are passed

size_t Compress(Source *reader, Sink *writer, CompressionOptions options)#

Same as Compress above but takes additional CompressionOptions.

Parameters	Direction	Description
reader	in,out	A Source is an interface that yields a sequence of bytes, you can initialize it by calling snappy::ByteArraySource(inBuf,inBufLen);where inBuf is the pointer to original data and inBufLen is the size of inBuf
writer	in,out	A Sink is an interface that consumes a sequence of bytes, you can initialize it by calling snappy::UncheckedByteArraySink(dest); where dest is the pointer to the destination buffer.
options	in	Compression options.

Returns:

Result	Description
Success	Return the number of bytes written.
Failure	Return 0 upon failure or NULL parameters are passed

bool GetUncompressedLength(Source *source, uint32_t *result)#

Find the uncompressed length of the given stream, as given by the header. Note that the true length could deviate from this; the stream could e.g. be truncated.

Parameters	Direction	Description
source	in,out	A Source is an interface that yields a sequence of bytes, you can initialize it by calling snappy::ByteArraySource(inBuf,inBufLen); where inBuf is the pointer to original data and inBufLen is the size of inBuf
result	out	Uncompressed length of the given stream is stored here.

Note

Also note that this leaves “*source” in a state that is unsuitable for further operations, such as RawUncompress(). You will need to rewind or recreate the source yourself before attempting any further calls.

Returns:

Result	Description
Success	If the data inside the source is uncorrupted it will return true.
Failure	It will return false if the data inside the source is corrupted.

bool Uncompress(Source *compressed, Sink *uncompressed)#

Decompresses “compressed” to “*uncompressed”.

Parameters	Direction	Description
compressed	in,out	A Source is an interface that yields a sequence of bytes, you can initialize it by calling snappy::ByteArraySource(inBuf,inBufLen); where inBuf is the pointer to original data and inBufLen is the size of inBuf .
uncompressed	in,out	A Sink is an interface that consumes a sequence of bytes, you can initialize it by calling snappy::UncheckedByteArraySink(dest); where dest is the pointer to the destination buffer.

Returns:

Result	Description
Success	Returns true if successful.
Failure	Returns false if the message is corrupted and could not be decompressed.

size_t UncompressAsMuchAsPossible(Source *compressed, Sink *uncompressed)#

This routine decompresses as much of the “compressed” as possible into sink. It returns the number of valid bytes added to sink (extra invalid bytes may have been added due to errors; the caller should ignore those). The emitted data typically has length GetUncompressedLength(), but may be shorter if an error is encountered.

Parameters	Direction	Description
compressed	in,out	A Source is an interface that yields a sequence of bytes, you can initialize it by calling snappy::ByteArraySource(inBuf,inBufLen); where inBuf is the pointer to original data and inBufLen is the size of inBuf .
uncompressed	in,out	A Sink is an interface that consumes a sequence of bytes, you can initialize it by calling snappy::UncheckedByteArraySink(dest); where dest is the pointer to the destination buffer.

Returns:

Result	Description
Success	It returns the number of valid bytes added to sink (extra invalid bytes may have been added due to errors; the caller should ignore those)
Failure	Returns 0 if the message is corrupted and could not be decompressed or NULL parameters are passed.

Higher-level string based routines.

Higher-level string based routines (should be sufficient for most users)

size_t Compress(const char *input, size_t input_length, std::string *compressed)#

Sets “*compressed” to the compressed version of “input[0,input_length-1]”. Original contents of *compressed are lost.

Parameters	Direction	Description
input	in	This is the buffer where the data we want to compress is accessible.
input_length	in	Length of the input buffer.
compressed	in,out	This is a buffer in which compressed data is stored.

Attention

REQUIRES: “input[]” is not an alias of “*compressed”.

Returns:

Result	Description
Success	Return the number of bytes written.
Failure	Return 0 upon failure or NULL parameters are passed

size_t Compress(const char *input, size_t input_length, std::string *compressed, CompressionOptions options)#

Same as Compress above but takes additional CompressionOptions.

Parameters	Direction	Description
input	in	This is the buffer where the data we want to compress is accessible.
input_length	in	Length of the input buffer.
compressed	in,out	This is a buffer in which compressed data is stored.
options	in	Compression options.

Attention

REQUIRES: “input[]” is not an alias of “*compressed”.

Returns:

Result	Description
Success	Return the number of bytes written.
Failure	Return 0 upon failure or NULL parameters are passed

size_t CompressFromIOVec(const struct iovec *iov, size_t iov_cnt, std::string *compressed)#

Same as Compress above but taking an iovec array as input. Note that this function preprocesses the inputs to compute the sum of iov[0..iov_cnt-1].iov_len before reading. To avoid this, use RawCompressFromIOVec below. First version is to preserve ABI.

Parameters	Direction	Description
iov	in	Input iovec array.
iov_cnt	in	Length of iovec array.
compressed	in,out	This is a buffer in which compressed data is stored.

Returns:

Result	Description
Success	Return the number of bytes written.
Failure	Return 0 upon failure or NULL parameters are passed

size_t CompressFromIOVec(const struct iovec *iov, size_t iov_cnt, std::string *compressed, CompressionOptions options)#

Same as CompressFromIOVec above but takes additional CompressionOptions.

Parameters	Direction	Description
iov	in	Input iovec array.
iov_cnt	in	Length of iovec array.
compressed	in,out	This is a buffer in which compressed data is stored.
options	in	Compression options.

Returns:

Result	Description
Success	Return the number of bytes written.
Failure	Return 0 upon failure or NULL parameters are passed

bool Uncompress(const char *compressed, size_t compressed_length, std::string *uncompressed)#

Decompresses “compressed[0,compressed_length-1]” to “*uncompressed”. Original contents of “*uncompressed” are lost.

Parameters	Direction	Description
compressed	in	This is a buffer which contains compressed data.
compressed_length	in	This is the length of the compressed buffer.
uncompressed	out	Uncompressed data is stored in this buffer.

Attention

REQUIRES: “compressed[]” is not an alias of “*uncompressed”.

Returns:

Result	Description
Success	If the data inside the compressed is successfully decompressed it will return true.
Failure	It will return false if the message is corrupted and could not be decompressed.

Lower-level character array based routines.

These May be useful for efficiency reasons in certain circumstances.

void RawCompress(const char *input, size_t input_length, char *compressed, size_t *compressed_length)#

Takes the data stored in “input[0..input_length]” and stores it in the array pointed to by “compressed”.

“*compressed_length” is set to the length of the compressed output.

Parameters	Direction	Description
input	in	This is the buffer where the data we want to compress is accessible.
input_length	in	Length of the input buffer.
compressed	out	This is a buffer in which compressed data is stored.
compressed_length	out	The length of the data after compression is stored in this.

Attention

REQUIRES: “compressed” must point to an area of memory that is at least “MaxCompressedLength(input_length)” bytes in length.

Note

Example:

char  output = new char[snappy::MaxCompressedLength(input_length)];
size_t output_length;
RawCompress(input, input_length, output, &output_length);
... Process(output, output_length) ...
delete [] output;

Returns:

void

void RawCompress(const char *input, size_t input_length, char *compressed, size_t *compressed_length, CompressionOptions options)#

Same as RawCompress above but takes additional CompressionOptions

Parameters	Direction	Description
input	in	This is the buffer where the data we want to compress is accessible.
input_length	in	Length of the input buffer.
compressed	out	This is a buffer in which compressed data is stored.
compressed_length	out	The length of the data after compression is stored in this.

void RawCompressFromIOVec(const struct iovec *iov, size_t uncompressed_length, char *compressed, size_t *compressed_length)#

Same as RawCompress above but taking an iovec array as input. Note that uncompressed_length is the total number of bytes to be read from the elements of iov (not the number of elements in iov).

Parameters	Direction	Description
iov	in	Input iovec array.
uncompressed_length	in	Total number of bytes to be read from the elements of iov.
compressed	out	This is a buffer in which compressed data is stored.
compressed_length	out	The length of the data after compression is stored in this.

void RawCompressFromIOVec(const struct iovec *iov, size_t uncompressed_length, char *compressed, size_t *compressed_length, CompressionOptions options)#

Same as RawCompressFromIOVec above but takes additional CompressionOptions.

Parameters	Direction	Description
iov	in	Input iovec array.
uncompressed_length	in	Total number of bytes to be read from the elements of iov.
compressed	out	This is a buffer in which compressed data is stored.
compressed_length	out	The length of the data after compression is stored in this.
options	in	Compression options.

bool RawUncompress(const char *compressed, size_t compressed_length, char *uncompressed)#

Given data in “compressed[0..compressed_length-1]” generated by calling the Snappy::Compress routine, this routine stores the uncompressed data to uncompressed[0..GetUncompressedLength(compressed)-1] .

Parameters	Direction	Description
compressed	in	This is a buffer which contains compressed data.
compressed_length	in	This is the length of the compressed buffer.
uncompressed	out	Uncompressed data is stored in this buffer.

Note

When decompressing data that was compressed using multi-threaded APIs, you should call GetUncompressedLengthFromMTCompressedBuffer instead of GetUncompressedLength to correctly allocate memory for the uncompressed buffer.

Returns:

Result	Description
Success	Returns true if successful.
Failure	Returns false if the message is corrupted and could not be decrypted.

bool RawUncompress(Source *compressed, char *uncompressed)#

Given data from the byte source ‘compressed’ generated by calling the Snappy::Compress routine, this routine stores the uncompressed data to uncompressed[0..GetUncompressedLength(compressed,compressed_length)-1] .

Parameters	Direction	Description
compressed	in,out	A Source is an interface that yields a sequence of bytes, you can initialize it by calling snappy::ByteArraySource(inBuf,inBufLen); where inBuf is the pointer to original data and inBufLen is the size of inBuf .
uncompressed	out	Uncompressed data is stored in this buffer.

Returns:

Result	Description
Success	Returns true if successful.
Failure	Returns false if the message is corrupted and could not be decrypted.

bool RawUncompressToIOVec(const char *compressed, size_t compressed_length, const struct iovec *iov, size_t iov_cnt)#

Given data in “compressed[0..compressed_length-1]” generated by calling the Snappy::Compress routine, this routine stores the uncompressed data to the iovec “iov”. The number of physical buffers in “iov” is given by iov_cnt and their cumulative size must be at least GetUncompressedLength(compressed). The individual buffers in “iov” must not overlap with each other.

Parameters	Direction	Description
compressed	in	This is a buffer which contains compressed data.
compressed_length	in	This is the length of the compressed buffer.
iov	in,out	The struct iovec defines one vector element. Normally, this structure is used as an array of multiple elements.
iov_cnt	in,out	This is the number of iovec structures in the array of iov .

Returns:

Result	Description
Success	Returns true if successful.
Failure	Returns false if the message is corrupted and could not be decrypted.

bool RawUncompressToIOVec(Source *compressed, const struct iovec *iov, size_t iov_cnt)#

Given data from the byte source ‘compressed’ generated by calling the Snappy::Compress routine, this routine stores the uncompressed data to the iovec “iov”. The number of physical buffers in “iov” is given by iov_cnt and their cumulative size must be at least GetUncompressedLength(compressed). The individual buffers in “iov” must not overlap with each other.

Parameters	Direction	Description
compressed	in,out	A Source is an interface that yields a sequence of bytes, you can initialize it by calling snappy::ByteArraySource(inBuf,inBufLen); where inBuf is the pointer to original data and inBufLen is the size of inBuf .
iov	in,out	The struct iovec defines one vector element. Normally, this structure is used as an array of multiple elements.
iov_cnt	out	This is the number of iovec structures in the array of iov .

Returns:

Result	Description
Success	Returns true if successful.
Failure	Returns false if the message is corrupted and could not be decrypted.

Helper Functions.

size_t MaxCompressedLength(size_t source_bytes)#

This function determines the maximal size of the compressed representation of input data that is “source_bytes” bytes in length.

Parameters	Direction	Description
source_bytes	in	The size of source in bytes.

Returns:

Result	Description
Success	Returns the maximal size of the compressed representation of input data that is “source_bytes” bytes in length.

bool GetUncompressedLength(const char *compressed, size_t compressed_length, size_t *result)#

Get the Uncompressed Length object.

This operation takes O(1) time.

Parameters	Direction	Description
compressed	in	This is a buffer which contains compressed data.
compressed_length	in	This is the length of the compressed buffer.
result	out	This is the pointer to type size_t where the uncompressed length is stored.

Attention

REQUIRES: “compressed[]” was produced by RawCompress() or Compress().

Returns:

Result	Description
Success	Returns true on successful parsing.
Failure	Returns false on parsing error.

bool GetUncompressedLengthFromMTCompressedBuffer(const char *compressed, size_t compressed_length, size_t *result)#

Get the Uncompressed Length object from the AOCL multithreaded compressor’s compressed buffer.

This operation takes O(1) time.

Parameters	Direction	Description
compressed	in	This is a buffer which contains compressed data. (along with the RAP frame)
compressed_length	in	This is the length of the compressed buffer (including the RAP frame).
result	out	This is the pointer to type size_t where the uncompressed length is stored.

Attention

REQUIRES: “compressed[]” was produced by RawCompress() or Compress() IN AOCL’s MULTITHREADED MODE.

Returns:

Result	Description
Success	Returns true on successful parsing.
Failure	Returns false on parsing error.

bool IsValidCompressedBuffer(const char *compressed, size_t compressed_length)#

Returns true iff the contents of “compressed[]” can be uncompressed successfully. Does not return the uncompressed data. Takes time proportional to compressed_length, but is usually at least a factor of four faster than actual decompression.

Parameters	Direction	Description
compressed	in	This is a buffer which contains compressed data.
compressed_length	in	This is the length of the compressed buffer.

Returns:

Result	Description
Success	Returns true iff the contents of “compressed[]” can be uncompressed successfully.
Failure	Returns false if error.

bool IsValidCompressed(Source *compressed)#

Returns true iff the contents of “compressed” can be uncompressed successfully. Does not return the uncompressed data. Takes time proportional to *compressed length, but is usually at least a factor of four faster than actual decompression. On success, consumes all of *compressed. On failure, consumes an unspecified prefix of *compressed.

Parameters	Direction	Description
compressed	in,out	A Source is an interface that yields a sequence of bytes, you can initialize it by calling snappy::ByteArraySource(inBuf,inBufLen); where inBuf is the pointer to original data and inBufLen is the size of inBuf .

Returns:

Result	Description
Success	Returns true iff the contents of “compressed” can be uncompressed successfully.
Failure	Returns false if error.