Bufer compression - decompression

[Iczer]

Basically i need compress Unicode string, transfer it as ASCII string ant then decompress. But for some reason this not working...

 

expandcollapse popup

#include-once
#include <WinAPISys.au3>
#include <WinAPI.au3>
#include <WinAPIDiag.au3>

;==============================================================================================================================================================================================
$sString = 'The _WinAPI_CompressBuffer() function takes as input an uncompressed buffer and produces its compressed equivalent provided that the compressed data fits within the specified destination buffer. 和一些中文詞'

ConsoleWrite("Original: "&$sString & @CRLF & @CRLF)

String_CompressString($sString)
ConsoleWrite("Compressed: "&$sString & @CRLF & @CRLF)


String_ExtractString($sString)
ConsoleWrite("Uncompressed: "&$sString & @CRLF & @CRLF)



;==============================================================================================================================================================================================
Func String_CompressString(ByRef $sString)

    Local $bString = StringToBinary($sString,2)
    ;--------------------------------------------------------------------------------------
    Local $iUncompressedSize = BinaryLen($bString)
    Local $pUncompressedBuffer = _WinAPI_CreateBuffer($iUncompressedSize,0,False)
    ;--------------------------------------------------------------------------------------
    Local $iCompressedSize = $iUncompressedSize
    Local $pCompressedBuffer = _WinAPI_CreateBuffer($iCompressedSize,0,False)
    ;--------------------------------------------------------------------------------------
    DllStructSetData(DllStructCreate('byte[' & $iUncompressedSize & ']', $pUncompressedBuffer), 1, $bString)
    ;--------------------------------------------------------------------------------------
    $iSize = _WinAPI_CompressBuffer($pUncompressedBuffer, $iUncompressedSize, $pCompressedBuffer, $iCompressedSize, BitOR($COMPRESSION_FORMAT_LZNT1,$COMPRESSION_ENGINE_MAXIMUM))
    If Not @error Then
        $sString = BinaryToString(DllStructGetData(DllStructCreate('byte[' & $iSize & ']', $pCompressedBuffer), 1))
    Else
        $iext = @extended
        $sString = "@error = " & @error & "     @extended = " & $iext & "|"& _WinAPI_GetLastErrorMessage() & "|" & _WinAPI_GetErrorMessage(_WinAPI_NtStatusToDosError ($iext)) & "|"
    EndIf
    ;--------------------------------------------------------------------------------------
    _WinAPI_FreeMemory($pCompressedBuffer)
    _WinAPI_FreeMemory($pUncompressedBuffer)
    ;--------------------------------------------------------------------------------------
EndFunc
;==============================================================================================================================================================================================
Func String_ExtractString(ByRef $sString)

    Local $bString = StringToBinary($sString)
    ;--------------------------------------------------------------------------------------
    Local $iCompressedSize = BinaryLen($bString)
    Local $pCompressedBuffer = _WinAPI_CreateBuffer($iCompressedSize,0,False)
    ;--------------------------------------------------------------------------------------
    Local $iUncompressedSize = $iCompressedSize * 8
    Local $pUncompressedBuffer = _WinAPI_CreateBuffer($iUncompressedSize,0,False)
    ;--------------------------------------------------------------------------------------
    DllStructSetData(DllStructCreate('byte[' & $iCompressedSize & ']', $pCompressedBuffer), 1, $bString)
    ;--------------------------------------------------------------------------------------
    $iSize = _WinAPI_DecompressBuffer($pUncompressedBuffer, $iUncompressedSize, $pCompressedBuffer, $iCompressedSize, BitOR($COMPRESSION_FORMAT_LZNT1,$COMPRESSION_ENGINE_MAXIMUM))
    If Not @error Then
        $sString = BinaryToString(DllStructGetData(DllStructCreate('byte[' & $iSize & ']', $pUncompressedBuffer), 1),4)
        $sString = _WinAPI_WideCharToMultiByte($sString,65001,True)
    Else
        $iext = @extended
        $sString = "@error = " & @error & "     @extended = " & $iext & "|"& _WinAPI_GetLastErrorMessage() & "|" & _WinAPI_GetErrorMessage(_WinAPI_NtStatusToDosError ($iext)) & "|"
    EndIf
    ;--------------------------------------------------------------------------------------
    _WinAPI_FreeMemory($pCompressedBuffer)
    _WinAPI_FreeMemory($pUncompressedBuffer)
    ;--------------------------------------------------------------------------------------
EndFunc
;==============================================================================================================================================================================================
Func _ConsoleWrite($sString)
    ConsoleWrite(BinaryToString(StringToBinary($sString,4),1))
EndFunc

 

[j0kky]

Doing a quick check, I saw this:

Local $bString = StringToBinary($sString,2) ;compress function
$sString = BinaryToString(DllStructGetData(DllStructCreate('byte[' & $iSize & ']', $pUncompressedBuffer), 1),4) ;uncompress function

You should use the same UNICODE character set.

Edited December 10, 2016 by j0kky

[Iczer]

thanks! 

but still - compressed string is not ASCII and original string is not equal uncompressed

expandcollapse popup

#include-once
#include <WinAPISys.au3>
#include <WinAPI.au3>
#include <WinAPIDiag.au3>
#include <Array.au3>
;==============================================================================================================================================================================================
$sString = 'The _WinAPI_CompressBuffer() function takes as input an uncompressed buffer and produces its compressed equivalent provided that the compressed data fits within the specified destination buffer. 和一些中文詞'
$sStringOrig = $sString
ConsoleWrite("Original: "&$sString & @CRLF & @CRLF)

String_CompressString($sString)
ConsoleWrite("Compressed: "&$sString & @CRLF & @CRLF)
MsgBox(64,"StringIsASCII = "&StringIsASCII($sString),$sString)

String_ExtractString($sString)
ConsoleWrite("Uncompressed: "&$sString & @CRLF & @CRLF)
MsgBox(64,"Orig string == Uncompressed ? : "&($sStringOrig == $sString),$sStringOrig & @crlf & $sString)


;==============================================================================================================================================================================================
Func String_CompressString(ByRef $sString)

    Local $bString = StringToBinary($sString,2)
    ;--------------------------------------------------------------------------------------
    Local $iUncompressedSize = BinaryLen($bString)
    Local $pUncompressedBuffer = _WinAPI_CreateBuffer($iUncompressedSize,0,False)
    ;--------------------------------------------------------------------------------------
    Local $iCompressedSize = $iUncompressedSize
    Local $pCompressedBuffer = _WinAPI_CreateBuffer($iCompressedSize,0,False)
    ;--------------------------------------------------------------------------------------
    DllStructSetData(DllStructCreate('byte[' & $iUncompressedSize & ']', $pUncompressedBuffer), 1, $bString)
    ;--------------------------------------------------------------------------------------
    $iSize = _WinAPI_CompressBuffer($pUncompressedBuffer, $iUncompressedSize, $pCompressedBuffer, $iCompressedSize, BitOR($COMPRESSION_FORMAT_LZNT1,$COMPRESSION_ENGINE_MAXIMUM))
    If Not @error Then
        $sString = BinaryToString(DllStructGetData(DllStructCreate('byte[' & $iSize & ']', $pCompressedBuffer), 1))
    Else
        $iext = @extended
        $sString = "@error = " & @error & "     @extended = " & $iext & "|"& _WinAPI_GetLastErrorMessage() & "|" & _WinAPI_GetErrorMessage(_WinAPI_NtStatusToDosError ($iext)) & "|"
    EndIf
    ;--------------------------------------------------------------------------------------
    _WinAPI_FreeMemory($pCompressedBuffer)
    _WinAPI_FreeMemory($pUncompressedBuffer)
    ;--------------------------------------------------------------------------------------
EndFunc
;==============================================================================================================================================================================================
Func String_ExtractString(ByRef $sString)

    Local $bString = StringToBinary($sString)
    ;--------------------------------------------------------------------------------------
    Local $iCompressedSize = BinaryLen($bString)
    Local $pCompressedBuffer = _WinAPI_CreateBuffer($iCompressedSize,0,False)
    ;--------------------------------------------------------------------------------------
    Local $iUncompressedSize = $iCompressedSize * 8
    Local $pUncompressedBuffer = _WinAPI_CreateBuffer($iUncompressedSize,0,False)
    ;--------------------------------------------------------------------------------------
    DllStructSetData(DllStructCreate('byte[' & $iCompressedSize & ']', $pCompressedBuffer), 1, $bString)
    ;--------------------------------------------------------------------------------------
    $iSize = _WinAPI_DecompressBuffer($pUncompressedBuffer, $iUncompressedSize, $pCompressedBuffer, $iCompressedSize, BitOR($COMPRESSION_FORMAT_LZNT1,$COMPRESSION_ENGINE_MAXIMUM))
    If Not @error Then
        $sString = BinaryToString(DllStructGetData(DllStructCreate('byte[' & $iSize & ']', $pUncompressedBuffer), 1),2)
        $sString = _WinAPI_WideCharToMultiByte($sString,65001,True)
    Else
        $iext = @extended
        $sString = "@error = " & @error & "     @extended = " & $iext & "|"& _WinAPI_GetLastErrorMessage() & "|" & _WinAPI_GetErrorMessage(_WinAPI_NtStatusToDosError ($iext)) & "|"
    EndIf
    ;--------------------------------------------------------------------------------------
    _WinAPI_FreeMemory($pCompressedBuffer)
    _WinAPI_FreeMemory($pUncompressedBuffer)
    ;--------------------------------------------------------------------------------------
EndFunc
;==============================================================================================================================================================================================
Func _ConsoleWrite($sString)
    ConsoleWrite(BinaryToString(StringToBinary($sString,4),1))
EndFunc

 

[j0kky]

$sString = _WinAPI_WideCharToMultiByte($sString,65001,True)

Why do you add this line? Have you tried to remove it?

Edited December 10, 2016 by j0kky

[Iczer]

it's keep string size in bytes down in comparison to same string in UTF16 (on strings with mixed ascii/non-ascii letters when ascii is most of them)

so now complete version:

expandcollapse popup

#include-once
#include <WinAPISys.au3>
#include <WinAPI.au3>
#include <WinAPIDiag.au3>
#include <Array.au3>
;==============================================================================================================================================================================================
$sStringOrig = 'The _WinAPI_CompressBuffer() function takes as input an uncompressed buffer and produces its compressed equivalent provided that the compressed data fits within the specified destination buffer. 和一些中文詞'
$sStringOrig = $sStringOrig&$sStringOrig&$sStringOrig&$sStringOrig&$sStringOrig&$sStringOrig&$sStringOrig&$sStringOrig&$sStringOrig&$sStringOrig&$sStringOrig&$sStringOrig
$sString = $sStringOrig

ConsoleWrite("Original: "&StringLen($sStringOrig)&"|"&$sString & @CRLF & @CRLF)

String_CompressString($sString)
ConsoleWrite("Compressed: "&StringLen($sString)&"|"&$sString & @CRLF & @CRLF)
MsgBox(64,"StringIsASCII = "&StringIsASCII($sString),$sString)

String_ExtractString($sString)
ConsoleWrite("Uncompressed: "&$sString & @CRLF & @CRLF)
MsgBox(64,"Orig string == Uncompressed ? : "&($sStringOrig == $sString),$sStringOrig & @crlf & $sString)


;==============================================================================================================================================================================================
Func String_CompressString(ByRef $sString)

    $sString = _WinAPI_WideCharToMultiByte($sString,65001,True)
    Local $bString = StringToBinary($sString,4)
    ;--------------------------------------------------------------------------------------
    Local $iUncompressedSize = BinaryLen($bString)
    Local $pUncompressedBuffer = _WinAPI_CreateBuffer($iUncompressedSize,0,False)
    ;--------------------------------------------------------------------------------------
    Local $iCompressedSize = $iUncompressedSize
    Local $pCompressedBuffer = _WinAPI_CreateBuffer($iCompressedSize,0,False)
    ;--------------------------------------------------------------------------------------
    DllStructSetData(DllStructCreate('byte[' & $iUncompressedSize & ']', $pUncompressedBuffer), 1, $bString)
    ;--------------------------------------------------------------------------------------
    $iSize = _WinAPI_CompressBuffer($pUncompressedBuffer, $iUncompressedSize, $pCompressedBuffer, $iCompressedSize, BitOR($COMPRESSION_FORMAT_LZNT1,$COMPRESSION_ENGINE_MAXIMUM))
    If Not @error Then
        $sString = String(DllStructGetData(DllStructCreate('byte[' & $iSize & ']', $pCompressedBuffer), 1))
    Else
        $iext = @extended
        $sString = "@error = " & @error & "     @extended = " & $iext & "|"& _WinAPI_GetLastErrorMessage() & "|" & _WinAPI_GetErrorMessage(_WinAPI_NtStatusToDosError ($iext)) & "|"
    EndIf
    ;--------------------------------------------------------------------------------------
    _WinAPI_FreeMemory($pCompressedBuffer)
    _WinAPI_FreeMemory($pUncompressedBuffer)
    ;--------------------------------------------------------------------------------------
EndFunc
;==============================================================================================================================================================================================
Func String_ExtractString(ByRef $sString)

    Local $bString = Binary($sString)
    ;--------------------------------------------------------------------------------------
    Local $iCompressedSize = BinaryLen($bString)
    Local $pCompressedBuffer = _WinAPI_CreateBuffer($iCompressedSize,0,False)
    ;--------------------------------------------------------------------------------------
    Local $iUncompressedSize = $iCompressedSize * 8
    Local $pUncompressedBuffer = _WinAPI_CreateBuffer($iUncompressedSize,0,False)
    ;--------------------------------------------------------------------------------------
    DllStructSetData(DllStructCreate('byte[' & $iCompressedSize & ']', $pCompressedBuffer), 1, $bString)
    ;--------------------------------------------------------------------------------------
    $iSize = _WinAPI_DecompressBuffer($pUncompressedBuffer, $iUncompressedSize, $pCompressedBuffer, $iCompressedSize, BitOR($COMPRESSION_FORMAT_LZNT1,$COMPRESSION_ENGINE_MAXIMUM))
    If Not @error Then
        $sString = BinaryToString(DllStructGetData(DllStructCreate('byte[' & $iSize & ']', $pUncompressedBuffer), 1),4)
        $sString = _WinAPI_MultiByteToWideChar($sString,65001,0,True)
    Else
        $iext = @extended
        $sString = "@error = " & @error & "     @extended = " & $iext & "|"& _WinAPI_GetLastErrorMessage() & "|" & _WinAPI_GetErrorMessage(_WinAPI_NtStatusToDosError ($iext)) & "|"
    EndIf
    ;--------------------------------------------------------------------------------------
    _WinAPI_FreeMemory($pCompressedBuffer)
    _WinAPI_FreeMemory($pUncompressedBuffer)
    ;--------------------------------------------------------------------------------------
EndFunc
;==============================================================================================================================================================================================
Func _ConsoleWrite($sString)
    ConsoleWrite(BinaryToString(StringToBinary($sString,4),1))
EndFunc

 

[j0kky]

Does it work?

[joiner]

BitOR($COMPRESSION_FORMAT_XPRESS_HUFF, $COMPRESSION_ENGINE_MAXIMUM);@error - 10, @extended - C00000E8

why is it problem?

[Trong]

How to fix if data is ASCII or short string ? Error: The data area passed to a system call is too small.