New Compression Algorithms and interface A new low-level compression interface has been added to the HDF-netCDF library which allows any data-object to be compressed using a variety of algorithms. This new feature of the library is completely transparent to users once the data has been compressed initially - further data written to an object or read from it are compressed or decompressed internally to the library without user intervention. Currently only three compression algorithms are supported: Run-Length Encoding (RLE), adaptive Huffman, and an LZ-77 dictionary coder (the gzip 'deflation' algorithm). Plans for future algorithms include an Lempel/Ziv-78 dictionary coding, an arithmetic coder and a faster Huffman algorithm. The public interface for this new feature is contained in one new user-level function call, HCcreate. The interface to HCcreate described below: int32 HCcreate(id,tag,ref,model_type,m_info,coder_type,c_info); int32 id; IN: the file id to create the data in (from Hopen) uint16 tag,ref; IN: the tag/ref pair of the data-object which is to be compressed comp_model_t model_type; IN: the type of modeling to use, currently only COMP_MODEL_STDIO is supported, which indicates data is transferred in the same way as C I/O functions operate. model_info *m_info; IN: Information needed for the modeling type chosen Nothing needed for COMP_MODEL_STDIO, so NULL can be used. comp_coder_t coder_type; IN: the type of encoding to use from the following: COMP_CODE_NONE - for no compression COMP_CODE_RLE - for RLE encoding COMP_CODE_SKPHUFF - for adaptive Huffman COMP_CODE_DEFLATE - for gzip 'deflation' coder_info *c_info; IN: Information needed for the encoding type chosen For COMP_CODE_NONE and COMP_CODE_RLE, this is unused and can be set to NULL. For COMP_CODE_SKPHUFF, the structure skphuff in this union needs information about the size of the data elements in bytes (see examples below). For COMP_CODE_DEFLATE, the structure deflate in this union needs information about the "effort" to encode data with. Higher values of 'level' member indicate more compression effort. Values may range from 0 (minimal compression, fastest time) to 9 (maximum compression, slowest time). RETURNS Return an AID to the newly created compressed element, FAIL on error. HCcreate will compress an existing data-object with the specified compression method, or it can create a new data-object which will contain compressed data when it is written to. In either case, Hendaccess must be called to release the AID allocated by HCcreate. In the first two examples below the datasets already exist, in the final example the dataset is created by the HCcreate call. There is currently no FORTRAN equivalent for this function. More details about this function can be found in the HDF reference manual. The following example shows how to compress a scientific dataset data-object (which is composed of multi-dimensional 32-bit integer data) using the adaptive Huffman encoding: { int32 aid; coder_info c_info; c_info.skphuff.skp_size=sizeof(int32); aid=HCcreate(file_id, DFTAG_SD, ref, COMP_MODEL_STDIO, NULL, COMP_CODE_SKPHUFF,&c_info); . . . . Hendaccess(aid); } The following example shows show to compress a raster image data-object using the RLE algorithm: { int32 aid; aid=HCcreate(file_id, DFTAG_RI, ref, COMP_MODEL_STDIO, NULL, COMP_CODE_RLE,NULL); . . . . Hendaccess(aid); } The following example shows how to create a new data-object whose data will compressed as it is written: { int32 aid; aid=HCcreate(file_id, DFTAG_RI, ref, COMP_MODEL_STDIO, NULL, COMP_CODE_RLE,NULL); . . Hwrite(aid,len,data); . . Hendaccess(aid); }