CRC32 question

[lesmly]

Hi

How to compute CRC32 from data (variable)? I can find CRC32 from file like this:

expandcollapse popup

$bits = "00FF111201A5900000000000FFFFFFFF16952267F1"
$strbits = BinaryToString("0x" & $bits)
$filetemp = "tempcrc32" & ".tst"
If FileExists($filetemp) Then FileDelete($filetemp)
$handlefiletemp = FileOpen($filetemp)
FileWrite($filetemp, $strbits)
FileClose($handlefiletemp)
$CRC32 = _CRC32ForFile($filetemp)
If FileExists($filetemp) Then FileDelete($filetemp)
ConsoleWrite($CRC32 & @CRLF)

; Name...........: _CRC32ForFile
; Description ...: Calculates CRC32 value for the specific file.
; Syntax.........: _CRC32ForFile ($sFile)
; Parameters ....: $sFile - Full path to the file to process.
; Return values .: Success - Returns CRC32 value in form of hex string
;                          - Sets @error to 0
;                  Failure - Returns empty string and sets @error:
;                  |1 - CreateFile function or call to it failed.
;                  |2 - CreateFileMapping function or call to it failed.
;                  |3 - MapViewOfFile function or call to it failed.
;                  |4 - RtlComputeCrc32 function or call to it failed.
; Author ........: trancexx
;
;==========================================================================================

Func _CRC32ForFile($sFile)
    Local $a_hCall = DllCall("kernel32.dll", "hwnd", "CreateFileW", _
            "wstr", $sFile, _
            "dword", 0x80000000, _ ; GENERIC_READ
            "dword", 3, _ ; FILE_SHARE_READ|FILE_SHARE_WRITE
            "ptr", 0, _
            "dword", 3, _ ; OPEN_EXISTING
            "dword", 0, _ ; SECURITY_ANONYMOUS
            "ptr", 0)
    If @error Or $a_hCall[0] = -1 Then
        Return SetError(1, 0, "")
    EndIf
    Local $hFile = $a_hCall[0]
    $a_hCall = DllCall("kernel32.dll", "ptr", "CreateFileMappingW", _
            "hwnd", $hFile, _
            "dword", 0, _ ; default security descriptor
            "dword", 2, _ ; PAGE_READONLY
            "dword", 0, _
            "dword", 0, _
            "ptr", 0)
    If @error Or Not $a_hCall[0] Then
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFile)
        Return SetError(2, 0, "")
    EndIf
    DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFile)
    Local $hFileMappingObject = $a_hCall[0]
    $a_hCall = DllCall("kernel32.dll", "ptr", "MapViewOfFile", _
            "hwnd", $hFileMappingObject, _
            "dword", 4, _ ; FILE_MAP_READ
            "dword", 0, _
            "dword", 0, _
            "dword", 0)
    If @error Or Not $a_hCall[0] Then
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(3, 0, "")
    EndIf
    Local $pFile = $a_hCall[0]
    Local $iBufferSize = FileGetSize($sFile)
    Local $a_iCall = DllCall("ntdll.dll", "dword", "RtlComputeCrc32", _
            "dword", 0, _
            "ptr", $pFile, _
            "int", $iBufferSize)
    If @error Or Not $a_iCall[0] Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(4, 0, "")
    EndIf
    DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
    DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
    Local $iCRC32 = $a_iCall[0]
    Return SetError(0, 0, Hex($iCRC32))
EndFunc   ;==>_CRC32ForFile

but how can I compute CRC32 without creating file? And without machine code like in CRC32 function by Ward...

[rover]

You can remove all of the file handling code and just add a byte buffer.

$bits = "00FF111201A5900000000000FFFFFFFF16952267F1"
$binbits = Binary("0x" & $bits)
$iLen = BinaryLen($binbits)
$tByte = DllStructCreate("byte["&$iLen&"]")
DllStructSetData($tByte, 1, $binbits)
;CRC32, MD4, MD5, SHA1 -for files
;http://www.autoitscript.com/forum/topic/95558-crc32-md4-md5-sha1-for-files/
Local $a_iCall = DllCall("ntdll.dll", "dword", "RtlComputeCrc32", "dword", 0, "struct*", $tByte, "int", $iLen) ;DllStructGetSize($tByte)
If @error Or Not $a_iCall[0] Then Exit ConsoleWrite("! Error: " & @error & @CRLF)
ConsoleWrite(Hex($a_iCall[0], 8) & @CRLF)
MsgBox(262144,'Debug line ~' & @ScriptLineNumber,'Selection:' & @lf & 'Hex($a_iCall[0], 8)' & @lf & @lf & 'Return:' & @lf & Hex($a_iCall[0], 8)) ;### Debug MSGBOX

[lesmly]

I expect that it is just that what I mean, but for now I received an error:

! Error: 1

!>10:16:50 AutoIT3.exe ended.rc:12

[rover]

I expect that it is just that what I mean, but for now I received an error:

! Error: 1

!>10:16:50 AutoIT3.exe ended.rc:12

DllCall error code = 1 unable to use the DLL file.

That means ntdll.dll was not found.

What OS are you trying to run this on?, Vista, WinPE?

If you were able to run the example you posted, you should be able to run what I posted.

RtlComputeCrc32 is undocumented but Geoff Chappell's site says this is available in XP and higher (tried this on XP and Win 7x64)

http://www.geoffchappell.com/studies/windows/win32/ntdll/api/index.htm

This is now in Yashied's WinAPIEx.au3 - _WinAPI_ComputeCrc32()

There is Lazycat's fastcrc32, Ward's crc asm code, Eltorro's crcgen, etc.

If you don't need to get the crc from a large variable, then there are a few non-asm code examples, either using no dll or a non-system dll/console exe.

Searched the forum or use Google site search: crc32 site:http://www.autoitscript.com/forum/

expandcollapse popup

$bits = "00FF111201A5900000000000FFFFFFFF16952267F1"
$binbits = Binary("0x" & $bits)
$iLen = BinaryLen($binbits)
$tByte = DllStructCreate("byte["&$iLen&"]")
DllStructSetData($tByte, 1, $binbits)
$crc2 = Hex(FastCRC32($tByte), 8)
ConsoleWrite("FastCRC32: " & $crc2 & @CRLF)

$binbits = BinaryToString("0x" & $bits)
ConsoleWrite("CRC32: " & CRC32($binbits) & @CRLF)

Func FastCRC32($vBuffer, $nCRC32 = 0xFFFFFFFF)
; Created by Yudin Dmitry (Lazycat)
Local $nLen, $vTemp
If DllStructGetSize($vBuffer) = 0 Then ; String passed
     If IsBinary($vBuffer) Then
         $nLen = BinaryLen($vBuffer)
     Else
         $nLen = StringLen($vBuffer)
     EndIf
     $vTemp = DllStructCreate("byte[" & $nLen & "]")
     DllStructSetData($vTemp, 1, $vBuffer)
     $vBuffer = $vTemp
EndIf

; Machine code hex strings (created by Laszlo)
Local $CRC32Init = "0x33C06A088BC85AF6C101740AD1E981F12083B8EDEB02D1E94A75EC8B542404890C82403D0001000072D8C3"
Local $CRC32Exec = "0x558BEC33C039450C7627568B4D080FB60C08334D108B55108B751481E1FF000000C1EA0833148E403B450C89551072DB5E8B4510F7D05DC3"

; Create machine code stubs
Local $CRC32InitCode = DllStructCreate("byte[" & BinaryLen($CRC32Init) & "]")
DllStructSetData($CRC32InitCode, 1, $CRC32Init)
Local $CRC32ExecCode = DllStructCreate("byte[" & BinaryLen($CRC32Exec) & "]")
DllStructSetData($CRC32ExecCode, 1, $CRC32Exec)

; Structure for CRC32 Lookup table
Local $CRC32LookupTable = DllStructCreate("int["& 256 &"]")

; CallWindowProc under WinXP can have 0 or 4 parameters only, so pad remain params with zeros
; Execute stub for fill lookup table
DllCall("user32.dll", "int", "CallWindowProc", "ptr", DllStructGetPtr($CRC32InitCode), _
                                                 "ptr", DllStructGetPtr($CRC32LookupTable), _
                                                 "int", 0, _
                                                 "int", 0, _
                                                 "int", 0)
; Execute main stub
Local $ret = DllCall("user32.dll", "uint", "CallWindowProc", "ptr", DllStructGetPtr($CRC32ExecCode), _
                                                                 "ptr", DllStructGetPtr($vBuffer), _
                                                                 "uint", DllStructGetSize($vBuffer), _
                                                                 "uint", $nCRC32, _
                                                                 "ptr", DllStructGetPtr($CRC32LookupTable))
Return $ret[0]
EndFunc

Func CRC32($sText)
; Created by Yudin Dmitry (Lazycat)
Local $CRC32Table[256] = [ _
0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419, _
0x706af48f, 0xe963a535, 0x9e6495a3, 0x0edb8832, 0x79dcb8a4, _
0xe0d5e91e, 0x97d2d988, 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, _
0x90bf1d91, 0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de, _
0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7, 0x136c9856, _
0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9, _
0xfa0f3d63, 0x8d080df5, 0x3b6e20c8, 0x4c69105e, 0xd56041e4, _
0xa2677172, 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b, _
0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940, 0x32d86ce3, _
0x45df5c75, 0xdcd60dcf, 0xabd13d59, 0x26d930ac, 0x51de003a, _
0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423, 0xcfba9599, _
0xb8bda50f, 0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924, _
0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d, 0x76dc4190, _
0x01db7106, 0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f, _
0x9fbfe4a5, 0xe8b8d433, 0x7807c9a2, 0x0f00f934, 0x9609a88e, _
0xe10e9818, 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01, _
0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e, 0x6c0695ed, _
0x1b01a57b, 0x8208f4c1, 0xf50fc457, 0x65b0d9c6, 0x12b7e950, _
0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, _
0xfbd44c65, 0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2, _
0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb, 0x4369e96a, _
0x346ed9fc, 0xad678846, 0xda60b8d0, 0x44042d73, 0x33031de5, _
0xaa0a4c5f, 0xdd0d7cc9, 0x5005713c, 0x270241aa, 0xbe0b1010, _
0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f, _
0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17, _
0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad, 0xedb88320, 0x9abfb3b6, _
0x03b6e20c, 0x74b1d29a, 0xead54739, 0x9dd277af, 0x04db2615, _
0x73dc1683, 0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8, _
0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1, 0xf00f9344, _
0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb, _
0x196c3671, 0x6e6b06e7, 0xfed41b76, 0x89d32be0, 0x10da7a5a, _
0x67dd4acc, 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5, _
0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252, 0xd1bb67f1, _
0xa6bc5767, 0x3fb506dd, 0x48b2364b, 0xd80d2bda, 0xaf0a1b4c, _
0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55, 0x316e8eef, _
0x4669be79, 0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236, _
0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f, 0xc5ba3bbe, _
0xb2bd0b28, 0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31, _
0x2cd99e8b, 0x5bdeae1d, 0x9b64c2b0, 0xec63f226, 0x756aa39c, _
0x026d930a, 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713, _
0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38, 0x92d28e9b, _
0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21, 0x86d3d2d4, 0xf1d4e242, _
0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1, _
0x18b74777, 0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c, _
0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45, 0xa00ae278, _
0xd70dd2ee, 0x4e048354, 0x3903b3c2, 0xa7672661, 0xd06016f7, _
0x4969474d, 0x3e6e77db, 0xaed16a4a, 0xd9d65adc, 0x40df0b66, _
0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9, _
0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605, _
0xcdd70693, 0x54de5729, 0x23d967bf, 0xb3667a2e, 0xc4614ab8, _
0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, _
0x2d02ef8d]
$CRCValue = 0xffffffff
For $i = 1 to StringLen($sText)
     $CRCValue = BitXOR($CRC32Table[BitAND(BitXOR($CRCValue, Asc(StringMid($sText, $i, 1))), 0xFF)], BitAND(BitShift($CRCValue, 8), 0x00ffffff))
Next
Return Hex(BitNot($CRCValue), 8)
EndFunc

Edited August 7, 2012 by rover

[lesmly]

My system when I try to write something is WinXp ,but my script has to run on Windows Server 2008, so I don't know whether the method FASTCRC32 will be suitable.

In your code when I change

Local $a_iCall = DllCall("ntdll.dll", "dword", "RtlComputeCrc32", "dword", 0, "struct*", $tByte, "int", $iLen) ;DllStructGetSize($tByte)

to

Local $a_iCall = DllCall("ntdll.dll", "dword", "RtlComputeCrc32", "dword", 0, "ptr*", $tByte, "int", $iLen) ;DllStructGetSize($tByte)

then there is no error, but the CRC32 isn't the same like in my method with creating file. But maybe "ptr*" isn't good choose?

Edited August 7, 2012 by lesmly

[rover]

You should update your AutoIt, struct* is in the current production version 3.3.8.1

Local $a_iCall = DllCall("ntdll.dll", "dword", "RtlComputeCrc32", "dword", 0, "ptr", DllStructGetPtr($tByte), "int", $iLen)

[lesmly]

OK, I see that I should

Now everything works fine.

Anyway THANK YOU !!!

[guinness]

I know rover answered this, but anyone that is using the beta version of AutoIt or WinAPIEx then you can use this >>

#include <WinAPIEx.au3>

ConsoleWrite(_CRC32(@ScriptFullPath) & @CRLF)

Func _CRC32($sFilePath) ; By Yashied, modified by guinness.
    Local $hFileOpen = FileOpen($sFilePath, 0 + 16)
    If $hFileOpen = -1 Then
        Return SetError(1, 0, 0)
    EndIf
    Local $bData = FileRead($hFileOpen)
    FileClose($hFileOpen)
    Local $iLength = BinaryLen($bData), $tData

    If $iLength = 0 Then
        Return SetError(2, 0, 0)
    EndIf
    $tData = DllStructCreate('byte[' & $iLength & ']')
    DllStructSetData($tData, 1, $bData)
    Return Hex(_WinAPI_ComputeCrc32(DllStructGetPtr($tData), $iLength), 8)
EndFunc   ;==>_CRC32