自己写的数据压缩软件wzip

wzip是一款基于BWT变换、波形树和游程编码的块排序压缩工具,压缩比接近bzip2,尤其适用于大多数非音视频文件。相较于bzip2,wzip在单线程或多线程状态下拥有更快的压缩速度。

wzip is a block-sorting compressor,based on BWT tranformation, wavelte tree, run-length code.Its compression ratio is very good and close to bzip2 for most files except for radio and video file. Its compression speed is much faster than bzip2 even in single-thread workstate ,and while in multi-thread workstate it will be more faster.

NAME
    wzip is a block-sorting file compressor based on BWT,wavelet tree,run-legnth code. version 1.0 .

SYNOPSIS
    for compression:
        wzip -c [-b blksize] [-e treetype] [-g node] [-k] [-f] [-p #thread] [-v verbose] filename

    for decompress:
        wzip -d [-k ] [ -f ] [-v verbose ] filename

    for help information:
        wzip -h
    
    for license:
        wzip -L

OPTIONS
    -b blockSiz100k
        while compressing file, we need set the block size of BWT with blockSiz100k. you can set blockSiz100k with 1,2..9,
        and the real BWT block size will be 100000,...900000 bytes. By default, we set the blockSiz100k 9.
    -c
        let wzip works wtih compression state for compressing file

    -d
        let wzip works with decompression state for decompressing file

    -e treeType
        select the kind of wavelet tree , like huffman tree ,balanced tree, hu-tacker tree with treeType 1 ,2,3,separately.
        By default,we set the treeType 3.

    -g nodeCode
To select the code format of the wavelet tree node, like run-length gamma,run-length delta with nodeCode 1,2, separately.
By default, we set the nodeCode 1.

    -f
        force overwrite of output files. Normally, wzip will not overwrite existing output files.

    -k
        do not delete the input file.

    -p #thread
        set the number of threads while compressing file,you can set the value 1,2 4.Noraml the larger #thread is ,the faster wzip will be.
        By default,we set the #thead 2.
    -h
print the use manual of the wzip.

    -v verbose
        verbose mode will show the compression ratio for each file processed and real time.You can set vorvose 0,1,2.
        By default,wo set the verbose 0.

    -L
        Display the software version, license terms and conditions.

DESTRIPTON
    wzip is a block-sorting compressor,based on BWT tranformation, wavelte tree, run-length code.Its compression ratio is very good
    and close to bzip2 for most files except for radio and video file.
    Its compression speed is much faster than bzip2 even in single-thread workstate ,and while in multi-thread workstate it will be more faster.
    
AUTHOR
     goldbeef (goldbeef@163.com) ,xidian university ,CHINA.
      


看了将近30篇论文,作为硕士毕业设计的一部分,开发了wzip这个数据压缩软件,和大家分享下,欢迎大家使用!

github :     https://github.com/goldbeef/wzip/


int tx_format_code_big_pkt(FRAME_INFO *pFrmInfo) { int ret; uint32_t headLen = sizeof(WZ_PROTOCOL_HEAD); uint32_t bodyOffset; uint32_t codesLen; const char *pBarcodeBufStart = gdata_handle->pMyBarRslt;/*V2读码算法结果位置,待移入到FRAME INFO*/ char * pDestTxBuf; //uint32_t *pTxByteLen; workmode_send_data_st wk_data; //SF_LogDebug(">>>>>>finalBarNum = %d, codeNum in buf = %d\n", gdata_handle->finalBarNum, *((int*) gdata_handle->pMyBarRslt)); /*申请缓冲区用于组装待发送的数据*/ pDestTxBuf = (char *)calloc(WORKMODE_BARDATA_LEN, sizeof(char)); if (pDestTxBuf == NULL){ SF_LogError("calloc %dKB buf err!\n", WORKMODE_BARDATA_LEN/1024); return -1; } WZ_PROTOCOL_HEAD *const pHead = (WZ_PROTOCOL_HEAD *)pDestTxBuf; char *const pBody = pDestTxBuf + headLen; char *pCodesValue; /*填充一帧协议头部信息(先填已知部分,数据体长度最后才得知时再填)*/ memcpy(pHead->szMagic, "WZIP", 4); pHead->cmdid = E_WZ_CODE; bodyOffset = 0; codesLen = 0; /*填充数据块【整个读码数据 】, 【数据部分】*/ pCodesValue = pBody + bodyOffset + TYPE_LEN_BYTES; //SF_LogDebug("call fill_codes_data_block_value"); fill_codes_data_block_value(pFrmInfo, pBarcodeBufStart/*V2读码算法结果位置*/, pCodesValue, &codesLen); //【头部部分】 fill_element_type_length(pBody + bodyOffset, L1T_CODES, codesLen); bodyOffset += TYPE_LEN_BYTES + codesLen; /*填充数据块【帧序号 】【头部 + 数据 】*/ fill_element_type_length_value(pBody + bodyOffset, L1T_FRAME_NUM, sizeof(pFrmInfo->txPkgId), (char*)&pFrmInfo->txPkgId); bodyOffset += (TYPE_LEN_BYTES + sizeof(uint32_t)); /*填充数据块【图像宽高】【头部 + 数据 】*/ fill_element_type_length_value(pBody + bodyOffset, L1T_IMG_PIX_W, sizeof(pFrmInfo->w), (char*)&pFrmInfo->w); bodyOffset += (TYPE_LEN_BYTES + sizeof(uint32_t)); fill_element_type_length_value(pBody + bodyOffset, L1T_IMG_PIX_H, sizeof(pFrmInfo->h), (char*)&pFrmInfo->h); bodyOffset += (TYPE_LEN_BYTES + sizeof(uint32_t)); /*填充数据块【自动供包是否剔除、丢弃包裹的判断结果】【头部 + 数据 】*/ SF_LogDebug("rectNum = %d, auto feed discardPkg in frameInfo=%d\n", pFrmInfo->locateRects.rectNums, pFrmInfo->discardPkg); fill_element_type_length_value(pBody + bodyOffset, L1T_AUTO_FEED_DISCARD_PKG/*0xD10B*/, sizeof(pFrmInfo->discardPkg), (char*)&pFrmInfo->discardPkg); bodyOffset += (TYPE_LEN_BYTES + sizeof(uint32_t)); //尾部标识AA 55 pBody[bodyOffset] = 0xAA; pBody[bodyOffset + 1] = 0x55; bodyOffset += 2; /*填入一帧协议的头部信息的body长度*/ memcpy(&pHead->bodyLen, &bodyOffset, sizeof(bodyOffset)); //SF_LogDebug("Code result len = %d\n", bodyOffset); //总发送长度,通过函数参数传递 //*pTxByteLen = headLen + bodyOffset; wk_data.data = pDestTxBuf; wk_data.len = headLen + bodyOffset; if (is_need_jpg()){ #if defined(PHASE_START_END_UIO) //如果是原来的模式逻辑 if ((TRIG_PHASE == gproc_conf->ccd_para.trigMode[0]) && (!cfg_get_is_phase_tx_code_every_frame())) {/*相位每帧都传读码结果的情况例外,要走底下的else分支去生成图片命名*/ if (!pFrmInfo->jpgAlrdySent) { //SF_LogDebug("branch - phase frmae jpg not sent yet\n"); //相位模式为了暂存某种指定的图片(如未识别的第N张,识别的第一张),每帧已经做好jpg文件名字,make_jpg_file_name_for_phase_middle_frame(pFrmInfo->JpgFileName, &(pFrmInfo->nJpgFileNameLen), pFrmInfo->txPkgId, pFrmInfo->phsImgSeq); ret = jpg_send_queue_insert(pFrmInfo->pJpgData, pFrmInfo->nJpgSize, pFrmInfo->JpgFileName, pFrmInfo->nJpgFileNameLen); if (ret < 0){ gdata_handle->stastics.queFullJpgLost++; SF_LogDebug("jpg image insert failed!, %s will be discarded\n", pFrmInfo->JpgFileName); } //pFrmInfo->jpgAlrdySent = 1; } //else { // SF_LogDebug("branch phase frmae jpgAlrdySent\n"); //} } else #endif { SF_LogDebug("jpg name1:%s",pFrmInfo->JpgFileName); tx_make_jpg_name_big_pkt(pFrmInfo); SF_LogDebug("jpg name2:%s",pFrmInfo->JpgFileName); ret = jpg_send_queue_insert(pFrmInfo->pJpgData, pFrmInfo->nJpgSize, pFrmInfo->JpgFileName, pFrmInfo->nJpgFileNameLen); if (ret < 0){ gdata_handle->stastics.queFullJpgLost++; SF_LogDebug("jpg image insert failed!, %s will be discarded\n", pFrmInfo->JpgFileName); } } } if (send_queue_insert(wk_data) < 0){ gdata_handle->stastics.queFullBarLost++; free(wk_data.data); SF_LogDebug("send_queue_insert() err!!!\n"); return -1; } return 0; } //供包台多面扫组网,基于大件组网,另外增加了10位随机数供包拍照Id int tx_format_code_pkg_feeder_cube_shot(FRAME_INFO *pFrmInfo) { int ret; uint32_t headLen = sizeof(WZ_PROTOCOL_HEAD); uint32_t bodyOffset; uint32_t codesLen; const char *pBarcodeBufStart = gdata_handle->pMyBarRslt;/*V2读码算法结果位置,待移入到FRAME INFO*/ char * pDestTxBuf; //uint32_t *pTxByteLen; workmode_send_data_st wk_data; //SF_LogDebug(">>>>>>finalBarNum = %d, codeNum in buf = %d\n", gdata_handle->finalBarNum, *((int*) gdata_handle->pMyBarRslt)); /*申请缓冲区用于组装待发送的数据*/ pDestTxBuf = (char *)calloc(WORKMODE_BARDATA_LEN, sizeof(char)); if (pDestTxBuf == NULL){ SF_LogError("calloc %dKB buf err!\n", WORKMODE_BARDATA_LEN/1024); return -1; } WZ_PROTOCOL_HEAD *const pHead = (WZ_PROTOCOL_HEAD *)pDestTxBuf; char *const pBody = pDestTxBuf + headLen; char *pCodesValue; /*填充一帧协议头部信息(先填已知部分,数据体长度最后才得知时再填)*/ memcpy(pHead->szMagic, "WZIP", 4); pHead->cmdid = E_WZ_CODE; bodyOffset = 0; /*填充数据块【整个读码数据 】, 【数据部分】*/ codesLen = 0; pCodesValue = pBody + bodyOffset + TYPE_LEN_BYTES; //SF_LogDebug("call fill_codes_data_block_value"); fill_codes_data_block_value(pFrmInfo, pBarcodeBufStart/*V2读码算法结果位置*/, pCodesValue, &codesLen); //【头部部分】 fill_element_type_length(pBody + bodyOffset, L1T_CODES, codesLen); bodyOffset += TYPE_LEN_BYTES + codesLen; /*填充数据块【帧序号 】【头部 + 数据 】*/ fill_element_type_length_value(pBody + bodyOffset, L1T_FRAME_NUM, sizeof(pFrmInfo->txPkgId), (char*)&pFrmInfo->txPkgId); bodyOffset += (TYPE_LEN_BYTES + sizeof(uint32_t)); /*填充数据块【10位随机数字符串形式的供包台拍照ID】【头部 + 数据 】, 仅针对供包台多面扫底扫*/ #if defined(PACKAGE_FEEDER_DN) fill_element_type_length_value(pBody + bodyOffset, L1T_FEED_PKG_ID, 10, pFrmInfo->strFeedPkgId); bodyOffset += (TYPE_LEN_BYTES + 10); #endif //尾部标识AA 55 pBody[bodyOffset] = 0xAA; pBody[bodyOffset + 1] = 0x55; bodyOffset += 2; /*填入一帧协议的头部信息的body长度*/ memcpy(&pHead->bodyLen, &bodyOffset, sizeof(bodyOffset)); //SF_LogDebug("Code result len = %d\n", bodyOffset); //总发送长度,通过函数参数传递 //*pTxByteLen = headLen + bodyOffset; wk_data.data = pDestTxBuf; wk_data.len = headLen + bodyOffset; if (is_need_jpg()){ if ((TRIG_PHASE == gproc_conf->ccd_para.trigMode[0]) && (!cfg_get_is_phase_tx_code_every_frame())) {/*相位每帧都传读码结果的情况例外,要走底下的else分支去生成图片命名*/ if (!pFrmInfo->jpgAlrdySent) { //SF_LogDebug("branch - phase frmae jpg not sent yet\n"); //相位模式为了暂存某种指定的图片(如未识别的第N张,识别的第一张),每帧已经做好jpg文件名字,make_jpg_file_name_for_phase_middle_frame(pFrmInfo->JpgFileName, &(pFrmInfo->nJpgFileNameLen), pFrmInfo->txPkgId, pFrmInfo->phsImgSeq); ret = jpg_send_queue_insert(pFrmInfo->pJpgData, pFrmInfo->nJpgSize, pFrmInfo->JpgFileName, pFrmInfo->nJpgFileNameLen); if (ret < 0){ gdata_handle->stastics.queFullJpgLost++; SF_LogDebug("jpg image insert failed!, %s will be discarded\n", pFrmInfo->JpgFileName); } //pFrmInfo->jpgAlrdySent = 1; } //else { // SF_LogDebug("branch phase frmae jpgAlrdySent\n"); //} } else { //供包多面扫使用大件多面扫的图片格式 tx_make_jpg_name_big_pkt(pFrmInfo); ret = jpg_send_queue_insert(pFrmInfo->pJpgData, pFrmInfo->nJpgSize, pFrmInfo->JpgFileName, pFrmInfo->nJpgFileNameLen); if (ret < 0){ gdata_handle->stastics.queFullJpgLost++; SF_LogDebug("jpg image insert failed!, %s will be discarded\n", pFrmInfo->JpgFileName); } } } if (send_queue_insert(wk_data) < 0){ gdata_handle->stastics.queFullBarLost++; free(wk_data.data); SF_LogDebug("send_queue_insert() err!!!\n"); return -1; } return 0; }/////合并这两个函数
最新发布
11-26
评论 2
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值