uint32_t zofits::compressHUFFMAN16 ( char *  dest, const char *  src, uint32_t  numRows, uint32_t  sizeOfElems, uint32_t  numRowElems  )

inlineprotected

Do huffman encoding

Parameters

dest	the buffer that will receive the compressed data
src	the buffer hosting the transposed data
numRows	number of rows of data in the transposed buffer
sizeOfElems	size in bytes of one data elements
numRowElems	number of elements on each row

Returns

number of bytes written

Definition at line 900 of file zofits.h.

References ___err___, Huffman::Encode(), and gLog.

Referenced by factofits::WriteDrsOffsetsTable().

        {
            std::string huffmanOutput;
            uint32_t previousHuffmanSize = 0;

            //if we have less than 2 elems to compress, Huffman encoder does not work (and has no point). Just return larger size than uncompressed to trigger the raw storage.
            if (numRows < 2)
                return numRows*sizeOfElems*numRowElems + 1000;

            if (sizeOfElems < 2 )
            {
#ifdef __EXCEPTIONS
                throw std::runtime_error("HUFMANN16 can only encode columns with 16-bit or longer types");
#else
                gLog << ___err___ << "ERROR - HUFMANN16 can only encode columns with 16-bit or longer types" << std::endl;
                return 0;
#endif
            }

            uint32_t huffmanOffset = 0;
            for (uint32_t j=0;j<numRowElems;j++)
            {
                Huffman::Encode(huffmanOutput,
                                reinterpret_cast<const uint16_t*>(&src[j*sizeOfElems*numRows]),
                                numRows*(sizeOfElems/2));
                reinterpret_cast<uint32_t*>(&dest[huffmanOffset])[0] = huffmanOutput.size() - previousHuffmanSize;
                huffmanOffset += sizeof(uint32_t);
                previousHuffmanSize = huffmanOutput.size();
            }

            const size_t totalSize = huffmanOutput.size() + huffmanOffset;

            //only copy if not larger than not-compressed size
            if (totalSize < numRows*sizeOfElems*numRowElems)
                memcpy(&dest[huffmanOffset], huffmanOutput.data(), huffmanOutput.size());

            return totalSize;
        }

Here is the call graph for this function:

Here is the caller graph for this function: