Jump to content

CRC32, MD4, MD5, SHA1 -for files

trancexx

By trancexx,
in AutoIt Example Scripts

 Share

Recommended Posts

CyberMax Seeker

CyberMax

Posted
Posted

DEP issues with Ward's functions are caused by the access protection of the memory he's allocating. Proper way is shown in any recent script written by me on assembly theme. Just read them.

I already explained why CreateFileMapping is used. It has nothing to do with DEP (how could it???).

Hardness here is confusing.

There are only two small things that you need to change to do that with posted functions. For MD5:

Func _MD5ForFirstFileChunk($sFile, $iChunkSize = 0)

    If Not $iChunkSize Then
        $iChunkSize = FileGetSize($sFile)
    EndIf

    Local $a_hCall = DllCall("kernel32.dll", "hwnd", "CreateFileW", _
            "wstr", $sFile, _
            "dword", 0x80000000, _ ; GENERIC_READ
            "dword", 1, _ ; FILE_SHARE_READ
            "ptr", 0, _
            "dword", 3, _ ; OPEN_EXISTING
            "dword", 0, _ ; SECURITY_ANONYMOUS
            "ptr", 0)

    If @error Or $a_hCall[0] = -1 Then
        Return SetError(1, 0, "")
    EndIf

    Local $hFile = $a_hCall[0]

    $a_hCall = DllCall("kernel32.dll", "ptr", "CreateFileMappingW", _
            "hwnd", $hFile, _
            "dword", 0, _ ; default security descriptor
            "dword", 2, _ ; PAGE_READONLY
            "dword", 0, _
            "dword", 0, _
            "ptr", 0)

    If @error Or Not $a_hCall[0] Then
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFile)
        Return SetError(2, 0, "")
    EndIf

    DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFile)

    Local $hFileMappingObject = $a_hCall[0]

    $a_hCall = DllCall("kernel32.dll", "ptr", "MapViewOfFile", _
            "hwnd", $hFileMappingObject, _
            "dword", 4, _ ; FILE_MAP_READ
            "dword", 0, _
            "dword", 0, _
            "dword", $iChunkSize)

    If @error Or Not $a_hCall[0] Then
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(3, 0, "")
    EndIf

    Local $pFile = $a_hCall[0]
    Local $iBufferSize = $iChunkSize

    Local $tMD5_CTX = DllStructCreate("dword i[2];" & _
            "dword buf[4];" & _
            "ubyte in[64];" & _
            "ubyte digest[16]")

    DllCall("advapi32.dll", "none", "MD5Init", "ptr", DllStructGetPtr($tMD5_CTX))

    If @error Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(4, 0, "")
    EndIf

    DllCall("advapi32.dll", "none", "MD5Update", _
            "ptr", DllStructGetPtr($tMD5_CTX), _
            "ptr", $pFile, _
            "dword", $iBufferSize)

    If @error Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(5, 0, "")
    EndIf

    DllCall("advapi32.dll", "none", "MD5Final", "ptr", DllStructGetPtr($tMD5_CTX))

    If @error Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(6, 0, "")
    EndIf

    DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
    DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)

    Local $sMD5 = Hex(DllStructGetData($tMD5_CTX, "digest"))

    Return SetError(0, 0, $sMD5)

EndFunc

MD5 for first 1024 bytes of some file would be:

$sMD5 = _MD5ForFirstFileChunk($sFile, 1024)

Hi trancexx,

Is there a method to read the last 1024 bytes of some file?

Thanks

  • Replies 48
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

trancexx
trancexx

When processing files, especially large ones, you will always lose battle if riding AutoIt. Not to mention RAM suffocation. So, what if I leave the whole process on a level that is few steps below me

DXRW4E
DXRW4E

First thanks for this code i used for a long time this code successfully in my Tool, but unfortunately it has limits, if you put them large files (such as an ISO 5GB) crashes or returns Wrong Hash, in

guinness Universalist

guinness

Posted
Posted

FileRead()?

UDF List:

 
_AdapterConnections() • _AlwaysRun() • _AppMon() • _AppMonEx() • _ArrayFilter/_ArrayReduce • _BinaryBin() • _CheckMsgBox() • _CmdLineRaw() • _ContextMenu() • _ConvertLHWebColor()/_ConvertSHWebColor() • _DesktopDimensions() • _DisplayPassword() • _DotNet_Load()/_DotNet_Unload() • _Fibonacci() • _FileCompare() • _FileCompareContents() • _FileNameByHandle() • _FilePrefix/SRE() • _FindInFile() • _GetBackgroundColor()/_SetBackgroundColor() • _GetConrolID() • _GetCtrlClass() • _GetDirectoryFormat() • _GetDriveMediaType() • _GetFilename()/_GetFilenameExt() • _GetHardwareID() • _GetIP() • _GetIP_Country() • _GetOSLanguage() • _GetSavedSource() • _GetStringSize() • _GetSystemPaths() • _GetURLImage() • _GIFImage() • _GoogleWeather() • _GUICtrlCreateGroup() • _GUICtrlListBox_CreateArray() • _GUICtrlListView_CreateArray() • _GUICtrlListView_SaveCSV() • _GUICtrlListView_SaveHTML() • _GUICtrlListView_SaveTxt() • _GUICtrlListView_SaveXML() • _GUICtrlMenu_Recent() • _GUICtrlMenu_SetItemImage() • _GUICtrlTreeView_CreateArray() • _GUIDisable() • _GUIImageList_SetIconFromHandle() • _GUIRegisterMsg() • _GUISetIcon() • _Icon_Clear()/_Icon_Set() • _IdleTime() • _InetGet() • _InetGetGUI() • _InetGetProgress() • _IPDetails() • _IsFileOlder() • _IsGUID() • _IsHex() • _IsPalindrome() • _IsRegKey() • _IsStringRegExp() • _IsSystemDrive() • _IsUPX() • _IsValidType() • _IsWebColor() • _Language() • _Log() • _MicrosoftInternetConnectivity() • _MSDNDataType() • _PathFull/GetRelative/Split() • _PathSplitEx() • _PrintFromArray() • _ProgressSetMarquee() • _ReDim() • _RockPaperScissors()/_RockPaperScissorsLizardSpock() • _ScrollingCredits • _SelfDelete() • _SelfRename() • _SelfUpdate() • _SendTo() • _ShellAll() • _ShellFile() • _ShellFolder() • _SingletonHWID() • _SingletonPID() • _Startup() • _StringCompact() • _StringIsValid() • _StringRegExpMetaCharacters() • _StringReplaceWholeWord() • _StringStripChars() • _Temperature() • _TrialPeriod() • _UKToUSDate()/_USToUKDate() • _WinAPI_Create_CTL_CODE() • _WinAPI_CreateGUID() • _WMIDateStringToDate()/_DateToWMIDateString() • Au3 script parsing • AutoIt Search • AutoIt3 Portable • AutoIt3WrapperToPragma • AutoItWinGetTitle()/AutoItWinSetTitle() • Coding • DirToHTML5 • FileInstallr • FileReadLastChars() • GeoIP database • GUI - Only Close Button • GUI Examples • GUICtrlDeleteImage() • GUICtrlGetBkColor() • GUICtrlGetStyle() • GUIEvents • GUIGetBkColor() • Int_Parse() & Int_TryParse() • IsISBN() • LockFile() • Mapping CtrlIDs • OOP in AutoIt • ParseHeadersToSciTE() • PasswordValid • PasteBin • Posts Per Day • PreExpand • Protect Globals • Queue() • Resource Update • ResourcesEx • SciTE Jump • Settings INI • SHELLHOOK • Shunting-Yard • Signature Creator • Stack() • Stopwatch() • StringAddLF()/StringStripLF() • StringEOLToCRLF() • VSCROLL • WM_COPYDATA • More Examples...

Updated: 22/04/2018

trancexx Universalist

trancexx

Posted
  • Author
Posted

Yes there is. There are more than one way. You can map only the last 1024 bytes, or map everything and then add offset to reach last 1024 bytes, move file pointer, etc, etc.

♡♡♡

.

eMyvnE

Iczer Universalist

Iczer

Posted
Posted

i'm trying to use trancexx script from first post to get crc32 and md5 for files in my backup project... so 2 (maybe stupid ^_^' ) questions:

- it seems there are some cashing occuring in the first call (64bit):

first run:

! CRC32 1st call took 1673.84318398009 ms
! CRC32 2nd call took 577.403552686165 ms
! MD5 took 587.722741298126 ms

second run:

! CRC32 1st call took 612.193728532537 ms
! CRC32 2nd call took 577.421990783745 ms
! MD5 took 587.62887461954 ms

- is it possible to use that, already cashed file, and write it to the another folder directly (so i save time needed to read file again to be able to copy it to another folder after calculating crc32 and md5 ) ?

- is it possible to make calculations of crc32 and md5 in the same time on different cores ?

- it seems calculations of crc32 and md5 take pretty same time - is it right?

if calculations made in 32bit script? it take:

! CRC32 1st call took 544.092335757757 ms
! CRC32 2nd call took 543.410684877547 ms
! MD5 took 452.442419357768 ms

it somewhat faster... why?

 

up?

  • 4 months later...
Quual Wayfarer

Quual

Posted
Posted

Thanks for the CRC32 Function

I tested it by creating crc32 checksuns for twenty 350MB files (6.8 Gig total data)

Test drive : 7.2 TB Raid 0 volume using 4 HDD in Raid 0 : Max Sustained Xfer rate : ~450 MB/Sec

Here are my results.

Test 01 | Filename [D:TV24Seeason124 1x01 - 1200 A.M.-100 A.M..avi ]  FileSize [345 MB]  Took [0.805 Seconds] at [449217 KB/Sec]
Test 02 | Filename [D:TV24Seeason124 1x02 - 100 A.M.-200 A.M..avi  ]  FileSize [345 MB]  Took [0.814 Seconds] at [444803 KB/Sec]
Test 03 | Filename [D:TV24Seeason124 1x03 - 200 A.M.-300 A.M..avi  ]  FileSize [347 MB]  Took [0.814 Seconds] at [446623 KB/Sec]
Test 04 | Filename [D:TV24Seeason124 1x04 - 300 A.M.-400 A.M..avi  ]  FileSize [345 MB]  Took [0.808 Seconds] at [447888 KB/Sec]
Test 05 | Filename [D:TV24Seeason124 1x05 - 400 A.M.-500 A.M..avi  ]  FileSize [345 MB]  Took [0.788 Seconds] at [459498 KB/Sec]
Test 06 | Filename [D:TV24Seeason124 1x06 - 500 A.M.-600 A.M..avi  ]  FileSize [345 MB]  Took [0.810 Seconds] at [446011 KB/Sec]
Test 07 | Filename [D:TV24Seeason124 1x07 - 600 A.M.-700 A.M..avi  ]  FileSize [346 MB]  Took [0.813 Seconds] at [445751 KB/Sec]
Test 08 | Filename [D:TV24Seeason124 1x08 - 700 A.M.-800 A.M..avi  ]  FileSize [346 MB]  Took [0.839 Seconds] at [431955 KB/Sec]
Test 09 | Filename [D:TV24Seeason124 1x09 - 800 A.M.-900 A.M..avi  ]  FileSize [346 MB]  Took [0.818 Seconds] at [443087 KB/Sec]
Test 10 | Filename [D:TV24Seeason124 1x10 - 900 A.M.-1000 A.M..avi ]  FileSize [346 MB]  Took [0.826 Seconds] at [438751 KB/Sec]
Test 11 | Filename [D:TV24Seeason124 1x11 - 1000 A.M.-1100 A.M..avi]  FileSize [346 MB]  Took [0.797 Seconds] at [454653 KB/Sec]
Test 12 | Filename [D:TV24Seeason124 1x12 - 1100 A.M.-1200 P.M..avi]  FileSize [346 MB]  Took [0.810 Seconds] at [447597 KB/Sec]
Test 12 | Filename [D:TV24Seeason124 1x13 - 1200 P.M.-100 P.M..avi ]  FileSize [347 MB]  Took [0.811 Seconds] at [448204 KB/Sec]
Test 14 | Filename [D:TV24Seeason124 1x14 - 100 P.M.-200 P.M..avi  ]  FileSize [346 MB]  Took [0.842 Seconds] at [430849 KB/Sec]
Test 15 | Filename [D:TV24Seeason124 1x15 - 200 P.M.-300 P.M..avi  ]  FileSize [347 MB]  Took [0.825 Seconds] at [440690 KB/Sec]
Test 16 | Filename [D:TV24Seeason124 1x16 - 300 P.M.-400 P.M..avi  ]  FileSize [348 MB]  Took [0.831 Seconds] at [439273 KB/Sec]
Test 17 | Filename [D:TV24Seeason124 1x17 - 400 P.M.-500 P.M..avi  ]  FileSize [348 MB]  Took [0.843 Seconds] at [432794 KB/Sec]
Test 18 | Filename [D:TV24Seeason124 1x18 - 500 P.M.-600 P.M..avi  ]  FileSize [347 MB]  Took [0.792 Seconds] at [459101 KB/Sec]
Test 19 | Filename [D:TV24Seeason124 1x19 - 600 P.M.-700 P.M..avi  ]  FileSize [348 MB]  Took [0.831 Seconds] at [439418 KB/Sec]
Test 20 | Filename [D:TV24Seeason124 1x20 - 700 P.M.-800 P.M..avi  ]  FileSize [348 MB]  Took [0.818 Seconds] at [446234 KB/Sec]

You function allowed my app to run at ~95% of my raid volume's max speed, I'm impressed.

          Thanks again

  • 7 months later...
DXRW4E Universalist

DXRW4E

Posted
Posted (edited)

First thanks for this code i used for a long time this code successfully in my Tool, but unfortunately it has limits, if you put them large files (such as an ISO 5GB) crashes or returns Wrong Hash, in web there are thousands of examples and threads, when everything very simple ehhhh https://msdn.microsoft.com/en-us/library/windows/desktop/aa382380(v=vs.85).aspx

 

If Not IsDeclared("arDllCall") Then Global Static $arDllCall
Global Const $hKernel32DLL = DllOpen("Kernel32.dll")
Global Const $hAdvapi32Dll = DllOpen("AdvApi32.dll")

Func _MD5ForFile($sFile)
    $arDllCall = DllCall($hKernel32DLL, "hwnd", "CreateFileW", "wstr", $sFile, "dword", 0x80000000, "dword", 3, "ptr", 0, "dword", 3, "dword", 0x80, "ptr", 0)
    If @Error Or $arDllCall[0] = -1 Then
        Return SetError(1, 0, "")
    EndIf
    Local $hFile = $arDllCall[0]
    $arDllCall = DllCall($hAdvapi32Dll, "int", "CryptAcquireContextW", "ULONG_PTR*", 0, "ptr", 0, "ptr", 0, "dword", 1, "dword", 0xF0000000) ; CRYPT_VERIFYCONTEXT
    If @Error Or Not $arDllCall[0] Then
        DllCall($hKernel32DLL, "int", "CloseHandle", "hwnd", $hFile)
        Return SetError(3, 0, "")
    EndIf
    Local $hContext = $arDllCall[1]
    $arDllCall = DllCall($hAdvapi32Dll, "int", "CryptCreateHash", "ULONG_PTR", $hContext, "dword", 0x00008003, "ptr", 0, "dword", 0, "ULONG_PTR*", 0)
    If @Error Or Not $arDllCall[0] Then
        DllCall($hAdvapi32Dll, "int", "CryptReleaseContext", "ULONG_PTR", $hContext, "dword", 0)
        DllCall($hKernel32DLL, "int", "CloseHandle", "hwnd", $hFile)
        Return SetError(3, 0, "")
    EndIf
    Local $hHashMD5 = $arDllCall[5], $bSucceeded = 1, $Buffer = DllStructCreate("byte[65535]")
    While $bSucceeded
        $arDllCall = DllCall($hKernel32DLL, "BOOL", "ReadFile", "HANDLE", $hFile, "ptr", DllStructGetPtr($Buffer), "dword", 65535, "dword*", 0, "ptr", Null)
        If @Error Or $arDllCall[4] = 0 Then ExitLoop
        $bSucceeded = $arDllCall[0]
        $arDllCall = DllCall($hAdvapi32Dll, "int", "CryptHashData", "ULONG_PTR", $hHashMD5, "struct*", $Buffer, "dword", $arDllCall[4], "dword", 0)
        If @Error Or Not $arDllCall[0] Then
            DllCall($hAdvapi32Dll, "int", "CryptDestroyHash", "ULONG_PTR", $hHashMD5)
            DllCall($hAdvapi32Dll, "int", "CryptReleaseContext", "ULONG_PTR", $hContext, "dword", 0)
            DllCall($hKernel32DLL, "int", "CloseHandle", "hwnd", $hFile)
            Return SetError(4, 0, "")
        EndIf
    WEnd
    If Not $bSucceeded Then
        DllCall($hAdvapi32Dll, "int", "CryptDestroyHash", "ULONG_PTR", $hHashMD5)
        DllCall($hAdvapi32Dll, "int", "CryptReleaseContext", "ULONG_PTR", $hContext, "dword", 0)
        DllCall($hKernel32DLL, "int", "CloseHandle", "hwnd", $hFile)
        Return SetError(5, 0, "")
    EndIf
    Local $tOutMD5 = DllStructCreate("byte[16]")
    $arDllCall = DllCall($hAdvapi32Dll, "int", "CryptGetHashParam", "ULONG_PTR", $hHashMD5, "dword", 2, "ptr", DllStructGetPtr($tOutMD5), "dword*", 16, "dword", 0)
    If @Error Or Not $arDllCall[0] Then
        DllCall($hAdvapi32Dll, "int", "CryptDestroyHash", "ULONG_PTR", $hHashMD5)
        DllCall($hAdvapi32Dll, "int", "CryptReleaseContext", "ULONG_PTR", $hContext, "dword", 0)
        DllCall($hKernel32DLL, "int", "CloseHandle", "hwnd", $hFile)
        Return SetError(6, 0, "")
    EndIf
    Local $sSHA1 = Hex(DllStructGetData($tOutMD5, 1))
    DllCall($hAdvapi32Dll, "int", "CryptDestroyHash", "ULONG_PTR", $hHashMD5)
    DllCall($hAdvapi32Dll, "int", "CryptReleaseContext", "ULONG_PTR", $hContext, "dword", 0)
    DllCall($hKernel32DLL, "int", "CloseHandle", "hwnd", $hFile)
    Return SetError(0, 0, $sSHA1)
EndFunc

Func _SHA1ForFile($sFile)
    $arDllCall = DllCall($hKernel32DLL, "hwnd", "CreateFileW", "wstr", $sFile, "dword", 0x80000000, "dword", 3, "ptr", 0, "dword", 3, "dword", 0x80, "ptr", 0)
    If @Error Or $arDllCall[0] = -1 Then
        Return SetError(1, 0, "")
    EndIf
    Local $hFile = $arDllCall[0]
    $arDllCall = DllCall($hAdvapi32Dll, "int", "CryptAcquireContextW", "ULONG_PTR*", 0, "ptr", 0, "ptr", 0, "dword", 1, "dword", 0xF0000000) ; CRYPT_VERIFYCONTEXT
    If @Error Or Not $arDllCall[0] Then
        DllCall($hKernel32DLL, "int", "CloseHandle", "hwnd", $hFile)
        Return SetError(2, 0, "")
    EndIf
    Local $hContext = $arDllCall[1]
    $arDllCall = DllCall($hAdvapi32Dll, "int", "CryptCreateHash", "ULONG_PTR", $hContext, "dword", 0x00008004, "ptr", 0, "dword", 0, "ULONG_PTR*", 0)
    If @Error Or Not $arDllCall[0] Then
        DllCall($hAdvapi32Dll, "int", "CryptReleaseContext", "ULONG_PTR", $hContext, "dword", 0)
        DllCall($hKernel32DLL, "int", "CloseHandle", "hwnd", $hFile)
        Return SetError(3, 0, "")
    EndIf
    Local $hHashSHA1 = $arDllCall[5], $bSucceeded = 1, $Buffer = DllStructCreate("byte[65535]")
    While $bSucceeded
        $arDllCall = DllCall($hKernel32DLL, "BOOL", "ReadFile", "HANDLE", $hFile, "ptr", DllStructGetPtr($Buffer), "dword", 65535, "dword*", 0, "ptr", Null)
        If @Error Or $arDllCall[4] = 0 Then ExitLoop
        $bSucceeded = $arDllCall[0]
        $arDllCall = DllCall($hAdvapi32Dll, "int", "CryptHashData", "ULONG_PTR", $hHashSHA1, "struct*", $Buffer, "dword", $arDllCall[4], "dword", 0)
        If @Error Or Not $arDllCall[0] Then
            DllCall($hAdvapi32Dll, "int", "CryptDestroyHash", "ULONG_PTR", $hHashSHA1)
            DllCall($hAdvapi32Dll, "int", "CryptReleaseContext", "ULONG_PTR", $hContext, "dword", 0)
            DllCall($hKernel32DLL, "int", "CloseHandle", "hwnd", $hFile)
            Return SetError(4, 0, "")
        EndIf
    WEnd
    If Not $bSucceeded Then
        DllCall($hAdvapi32Dll, "int", "CryptDestroyHash", "ULONG_PTR", $hHashSHA1)
        DllCall($hAdvapi32Dll, "int", "CryptReleaseContext", "ULONG_PTR", $hContext, "dword", 0)
        DllCall($hKernel32DLL, "int", "CloseHandle", "hwnd", $hFile)
        Return SetError(5, 0, "")
    EndIf
    Local $tOutSHA1 = DllStructCreate("byte[20]")
    $arDllCall = DllCall($hAdvapi32Dll, "int", "CryptGetHashParam", "ULONG_PTR", $hHashSHA1, "dword", 2, "ptr", DllStructGetPtr($tOutSHA1), "dword*", 20, "dword", 0)
    If @Error Or Not $arDllCall[0] Then
        DllCall($hAdvapi32Dll, "int", "CryptDestroyHash", "ULONG_PTR", $hHashSHA1)
        DllCall($hAdvapi32Dll, "int", "CryptReleaseContext", "ULONG_PTR", $hContext, "dword", 0)
        DllCall($hKernel32DLL, "int", "CloseHandle", "hwnd", $hFile)
        Return SetError(6, 0, "")
    EndIf
    Local $sSHA1 = Hex(DllStructGetData($tOutSHA1, 1))
    DllCall($hAdvapi32Dll, "int", "CryptDestroyHash", "ULONG_PTR", $hHashSHA1)
    DllCall($hAdvapi32Dll, "int", "CryptReleaseContext", "ULONG_PTR", $hContext, "dword", 0)
    DllCall($hKernel32DLL, "int", "CloseHandle", "hwnd", $hFile)
    Return SetError(0, 0, $sSHA1)
EndFunc

2i0ufpz.jpg

Ciao a tutti.

Edited by DXRW4E

apps-odrive.pngdrive_app_badge.png box-logo.png new_logo.png MEGA_Logo.png

  • 8 months later...
ElTiburon Seeker

ElTiburon

Posted
Posted

When processing files, especially large ones, you will always lose battle if riding AutoIt. Not to mention RAM suffocation. So, what if I leave the whole process on a level that is few steps below me and just collect the cream?

 

Script includes functions and small example:

 

 

Opt("MustDeclareVars", 1)

Global $sFile = FileOpenDialog("Choose file", "", "All files (*)")
If @error Then Exit


Global $hTimer, $iTimer, $sData

;------------------------------------------------------------------------
; CRC32:
$hTimer = TimerInit()

$sData = _CRC32ForFile($sFile)
$iTimer = TimerDiff($hTimer)

ConsoleWrite("! CRC32 took " & $iTimer & " ms" & @CRLF)
ConsoleWrite("Result: " & $sData & @CRLF & @CRLF)
;------------------------------------------------------------------------
; MD4:
$hTimer = TimerInit()

$sData = _MD4ForFile($sFile)
$iTimer = TimerDiff($hTimer)

ConsoleWrite("! MD4 took " & $iTimer & " ms" & @CRLF)
ConsoleWrite("Result: " & $sData & @CRLF & @CRLF)
;------------------------------------------------------------------------
; MD5:
$hTimer = TimerInit()

$sData = _MD5ForFile($sFile)
$iTimer = TimerDiff($hTimer)

ConsoleWrite("! MD5 took " & $iTimer & " ms" & @CRLF)
ConsoleWrite("Result: " & $sData & @CRLF & @CRLF)
;------------------------------------------------------------------------
; SHA1:
$hTimer = TimerInit()

$sData = _SHA1ForFile($sFile)
$iTimer = TimerDiff($hTimer)

ConsoleWrite("! SHA1 took " & $iTimer & " ms" & @CRLF)
ConsoleWrite("Result: " & $sData & @CRLF & @CRLF)
;------------------------------------------------------------------------



; Functions...


; #FUNCTION# ;===============================================================================
;
; Name...........: _CRC32ForFile
; Description ...: Calculates CRC32 value for the specific file.
; Syntax.........: _CRC32ForFile ($sFile)
; Parameters ....: $sFile - Full path to the file to process.
; Return values .: Success - Returns CRC32 value in form of hex string
;                          - Sets @error to 0
;                  Failure - Returns empty string and sets @error:
;                  |1 - CreateFile function or call to it failed.
;                  |2 - CreateFileMapping function or call to it failed.
;                  |3 - MapViewOfFile function or call to it failed.
;                  |4 - RtlComputeCrc32 function or call to it failed.
; Author ........: trancexx
;
;==========================================================================================
Func _CRC32ForFile($sFile)

    Local $a_hCall = DllCall("kernel32.dll", "hwnd", "CreateFileW", _
            "wstr", $sFile, _
            "dword", 0x80000000, _ ; GENERIC_READ
            "dword", 3, _ ; FILE_SHARE_READ|FILE_SHARE_WRITE
            "ptr", 0, _
            "dword", 3, _ ; OPEN_EXISTING
            "dword", 0, _ ; SECURITY_ANONYMOUS
            "ptr", 0)

    If @error Or $a_hCall[0] = -1 Then
        Return SetError(1, 0, "")
    EndIf

    Local $hFile = $a_hCall[0]

    $a_hCall = DllCall("kernel32.dll", "ptr", "CreateFileMappingW", _
            "hwnd", $hFile, _
            "dword", 0, _ ; default security descriptor
            "dword", 2, _ ; PAGE_READONLY
            "dword", 0, _
            "dword", 0, _
            "ptr", 0)

    If @error Or Not $a_hCall[0] Then
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFile)
        Return SetError(2, 0, "")
    EndIf

    DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFile)

    Local $hFileMappingObject = $a_hCall[0]

    $a_hCall = DllCall("kernel32.dll", "ptr", "MapViewOfFile", _
            "hwnd", $hFileMappingObject, _
            "dword", 4, _ ; FILE_MAP_READ
            "dword", 0, _
            "dword", 0, _
            "dword", 0)

    If @error Or Not $a_hCall[0] Then
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(3, 0, "")
    EndIf

    Local $pFile = $a_hCall[0]
    Local $iBufferSize = FileGetSize($sFile)

    Local $a_iCall = DllCall("ntdll.dll", "dword", "RtlComputeCrc32", _
            "dword", 0, _
            "ptr", $pFile, _
            "int", $iBufferSize)

    If @error Or Not $a_iCall[0] Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(4, 0, "")
    EndIf

    DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
    DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)

    Local $iCRC32 = $a_iCall[0]

    Return SetError(0, 0, Hex($iCRC32))

EndFunc   ;==>_CRC32ForFile


; #FUNCTION# ;===============================================================================
;
; Name...........: _MD4ForFile
; Description ...: Calculates MD4 value for the specific file.
; Syntax.........: _MD4ForFile ($sFile)
; Parameters ....: $sFile - Full path to the file to process.
; Return values .: Success - Returns MD4 value in form of hex string
;                          - Sets @error to 0
;                  Failure - Returns empty string and sets @error:
;                  |1 - CreateFile function or call to it failed.
;                  |2 - CreateFileMapping function or call to it failed.
;                  |3 - MapViewOfFile function or call to it failed.
;                  |4 - MD4Init function or call to it failed.
;                  |5 - MD4Update function or call to it failed.
;                  |6 - MD4Final function or call to it failed.
; Author ........: trancexx
;
;==========================================================================================
Func _MD4ForFile($sFile)

    Local $a_hCall = DllCall("kernel32.dll", "hwnd", "CreateFileW", _
            "wstr", $sFile, _
            "dword", 0x80000000, _ ; GENERIC_READ
            "dword", 3, _ ; FILE_SHARE_READ|FILE_SHARE_WRITE
            "ptr", 0, _
            "dword", 3, _ ; OPEN_EXISTING
            "dword", 0, _ ; SECURITY_ANONYMOUS
            "ptr", 0)

    If @error Or $a_hCall[0] = -1 Then
        Return SetError(1, 0, "")
    EndIf

    Local $hFile = $a_hCall[0]

    $a_hCall = DllCall("kernel32.dll", "ptr", "CreateFileMappingW", _
            "hwnd", $hFile, _
            "dword", 0, _ ; default security descriptor
            "dword", 2, _ ; PAGE_READONLY
            "dword", 0, _
            "dword", 0, _
            "ptr", 0)

    If @error Or Not $a_hCall[0] Then
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFile)
        Return SetError(2, 0, "")
    EndIf

    DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFile)

    Local $hFileMappingObject = $a_hCall[0]

    $a_hCall = DllCall("kernel32.dll", "ptr", "MapViewOfFile", _
            "hwnd", $hFileMappingObject, _
            "dword", 4, _ ; FILE_MAP_READ
            "dword", 0, _
            "dword", 0, _
            "dword", 0)

    If @error Or Not $a_hCall[0] Then
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(3, 0, "")
    EndIf

    Local $pFile = $a_hCall[0]
    Local $iBufferSize = FileGetSize($sFile)

    Local $tMD4_CTX = DllStructCreate("dword i[2];" & _
            "dword buf[4];" & _
            "ubyte in[64];" & _
            "ubyte digest[16]")

    DllCall("advapi32.dll", "none", "MD4Init", "ptr", DllStructGetPtr($tMD4_CTX))

    If @error Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(4, 0, "")
    EndIf

    DllCall("advapi32.dll", "none", "MD4Update", _
            "ptr", DllStructGetPtr($tMD4_CTX), _
            "ptr", $pFile, _
            "dword", $iBufferSize)

    If @error Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(5, 0, "")
    EndIf

    DllCall("advapi32.dll", "none", "MD4Final", "ptr", DllStructGetPtr($tMD4_CTX))

    If @error Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(6, 0, "")
    EndIf

    DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
    DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)

    Local $sMD4 = Hex(DllStructGetData($tMD4_CTX, "digest"))

    Return SetError(0, 0, $sMD4)

EndFunc   ;==>_MD4ForFile


; #FUNCTION# ;===============================================================================
;
; Name...........: _MD5ForFile
; Description ...: Calculates MD5 value for the specific file.
; Syntax.........: _MD5ForFile ($sFile)
; Parameters ....: $sFile - Full path to the file to process.
; Return values .: Success - Returns MD5 value in form of hex string
;                          - Sets @error to 0
;                  Failure - Returns empty string and sets @error:
;                  |1 - CreateFile function or call to it failed.
;                  |2 - CreateFileMapping function or call to it failed.
;                  |3 - MapViewOfFile function or call to it failed.
;                  |4 - MD5Init function or call to it failed.
;                  |5 - MD5Update function or call to it failed.
;                  |6 - MD5Final function or call to it failed.
; Author ........: trancexx
;
;==========================================================================================
Func _MD5ForFile($sFile)

    Local $a_hCall = DllCall("kernel32.dll", "hwnd", "CreateFileW", _
            "wstr", $sFile, _
            "dword", 0x80000000, _ ; GENERIC_READ
            "dword", 3, _ ; FILE_SHARE_READ|FILE_SHARE_WRITE
            "ptr", 0, _
            "dword", 3, _ ; OPEN_EXISTING
            "dword", 0, _ ; SECURITY_ANONYMOUS
            "ptr", 0)

    If @error Or $a_hCall[0] = -1 Then
        Return SetError(1, 0, "")
    EndIf

    Local $hFile = $a_hCall[0]

    $a_hCall = DllCall("kernel32.dll", "ptr", "CreateFileMappingW", _
            "hwnd", $hFile, _
            "dword", 0, _ ; default security descriptor
            "dword", 2, _ ; PAGE_READONLY
            "dword", 0, _
            "dword", 0, _
            "ptr", 0)

    If @error Or Not $a_hCall[0] Then
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFile)
        Return SetError(2, 0, "")
    EndIf

    DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFile)

    Local $hFileMappingObject = $a_hCall[0]

    $a_hCall = DllCall("kernel32.dll", "ptr", "MapViewOfFile", _
            "hwnd", $hFileMappingObject, _
            "dword", 4, _ ; FILE_MAP_READ
            "dword", 0, _
            "dword", 0, _
            "dword", 0)

    If @error Or Not $a_hCall[0] Then
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(3, 0, "")
    EndIf

    Local $pFile = $a_hCall[0]
    Local $iBufferSize = FileGetSize($sFile)

    Local $tMD5_CTX = DllStructCreate("dword i[2];" & _
            "dword buf[4];" & _
            "ubyte in[64];" & _
            "ubyte digest[16]")

    DllCall("advapi32.dll", "none", "MD5Init", "ptr", DllStructGetPtr($tMD5_CTX))

    If @error Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(4, 0, "")
    EndIf

    DllCall("advapi32.dll", "none", "MD5Update", _
            "ptr", DllStructGetPtr($tMD5_CTX), _
            "ptr", $pFile, _
            "dword", $iBufferSize)

    If @error Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(5, 0, "")
    EndIf

    DllCall("advapi32.dll", "none", "MD5Final", "ptr", DllStructGetPtr($tMD5_CTX))

    If @error Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(6, 0, "")
    EndIf

    DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
    DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)

    Local $sMD5 = Hex(DllStructGetData($tMD5_CTX, "digest"))

    Return SetError(0, 0, $sMD5)

EndFunc   ;==>_MD5ForFile


; #FUNCTION# ;===============================================================================
;
; Name...........: _SHA1ForFile
; Description ...: Calculates SHA1 value for the specific file.
; Syntax.........: _SHA1ForFile ($sFile)
; Parameters ....: $sFile - Full path to the file to process.
; Return values .: Success - Returns SHA1 value in form of hex string
;                          - Sets @error to 0
;                  Failure - Returns empty string and sets @error:
;                  |1 - CreateFile function or call to it failed.
;                  |2 - CreateFileMapping function or call to it failed.
;                  |3 - MapViewOfFile function or call to it failed.
;                  |4 - CryptAcquireContext function or call to it failed.
;                  |5 - CryptCreateHash function or call to it failed.
;                  |6 - CryptHashData function or call to it failed.
;                  |7 - CryptGetHashParam function or call to it failed.
; Author ........: trancexx
;
;==========================================================================================
Func _SHA1ForFile($sFile)

    Local $a_hCall = DllCall("kernel32.dll", "hwnd", "CreateFileW", _
            "wstr", $sFile, _
            "dword", 0x80000000, _ ; GENERIC_READ
            "dword", 3, _ ; FILE_SHARE_READ|FILE_SHARE_WRITE
            "ptr", 0, _
            "dword", 3, _ ; OPEN_EXISTING
            "dword", 0, _ ; SECURITY_ANONYMOUS
            "ptr", 0)

    If @error Or $a_hCall[0] = -1 Then
        Return SetError(1, 0, "")
    EndIf

    Local $hFile = $a_hCall[0]

    $a_hCall = DllCall("kernel32.dll", "ptr", "CreateFileMappingW", _
            "hwnd", $hFile, _
            "dword", 0, _ ; default security descriptor
            "dword", 2, _ ; PAGE_READONLY
            "dword", 0, _
            "dword", 0, _
            "ptr", 0)

    If @error Or Not $a_hCall[0] Then
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFile)
        Return SetError(2, 0, "")
    EndIf

    DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFile)

    Local $hFileMappingObject = $a_hCall[0]

    $a_hCall = DllCall("kernel32.dll", "ptr", "MapViewOfFile", _
            "hwnd", $hFileMappingObject, _
            "dword", 4, _ ; FILE_MAP_READ
            "dword", 0, _
            "dword", 0, _
            "dword", 0)

    If @error Or Not $a_hCall[0] Then
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(3, 0, "")
    EndIf

    Local $pFile = $a_hCall[0]
    Local $iBufferSize = FileGetSize($sFile)

    Local $a_iCall = DllCall("advapi32.dll", "int", "CryptAcquireContext", _
            "ptr*", 0, _
            "ptr", 0, _
            "ptr", 0, _
            "dword", 1, _ ; PROV_RSA_FULL
            "dword", 0xF0000000) ; CRYPT_VERIFYCONTEXT

    If @error Or Not $a_iCall[0] Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(4, 0, "")
    EndIf

    Local $hContext = $a_iCall[1]

    $a_iCall = DllCall("advapi32.dll", "int", "CryptCreateHash", _
            "ptr", $hContext, _
            "dword", 0x00008004, _ ; CALG_SHA1
            "ptr", 0, _ ; nonkeyed
            "dword", 0, _
            "ptr*", 0)

    If @error Or Not $a_iCall[0] Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        DllCall("advapi32.dll", "int", "CryptReleaseContext", "ptr", $hContext, "dword", 0)
        Return SetError(5, 0, "")
    EndIf

    Local $hHashSHA1 = $a_iCall[5]

    $a_iCall = DllCall("advapi32.dll", "int", "CryptHashData", _
            "ptr", $hHashSHA1, _
            "ptr", $pFile, _
            "dword", $iBufferSize, _
            "dword", 0)

    If @error Or Not $a_iCall[0] Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        DllCall("advapi32.dll", "int", "CryptDestroyHash", "ptr", $hHashSHA1)
        DllCall("advapi32.dll", "int", "CryptReleaseContext", "ptr", $hContext, "dword", 0)
        Return SetError(6, 0, "")
    EndIf

    Local $tOutSHA1 = DllStructCreate("byte[20]")

    $a_iCall = DllCall("advapi32.dll", "int", "CryptGetHashParam", _
            "ptr", $hHashSHA1, _
            "dword", 2, _ ; HP_HASHVAL
            "ptr", DllStructGetPtr($tOutSHA1), _
            "dword*", 20, _
            "dword", 0)

    If @error Or Not $a_iCall[0] Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        DllCall("advapi32.dll", "int", "CryptDestroyHash", "ptr", $hHashSHA1)
        DllCall("advapi32.dll", "int", "CryptReleaseContext", "ptr", $hContext, "dword", 0)
        Return SetError(7, 0, "")
    EndIf

    DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
    DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)

    DllCall("advapi32.dll", "int", "CryptDestroyHash", "ptr", $hHashSHA1)

    Local $sSHA1 = Hex(DllStructGetData($tOutSHA1, 1))

    DllCall("advapi32.dll", "int", "CryptReleaseContext", "ptr", $hContext, "dword", 0)

    Return SetError(0, 0, $sSHA1)

EndFunc   ;==>_SHA1ForFile

 

Results are in form of hex strings, but that is easily changed to fit your needs.

 

First time is the hardest - you will see what I mean if you run it (file mapping related).

 

Try it on something big.

Hi, I'm new here. Thanks for your script. I like it. It works for me, but I have an question: How i can make a processbar for creating a checksum?

I mean then the prozess starts e.g. an file with 1GB size it takes a time. What I want is an processbar to see how long it takes.
I think it must be in the callroutine of the DLL process. I'm learning and don't try itself with external DLLs.

I will script 2 processbars. One for the active process to see how long takes the actual file, and one to see how many files to check.
I scripted for me to recursive a directory in an array. From that I take on file and sent it to your routine. After that I save the result in an additional Array.
The progressbar for how many files to complete is easy, but how long it takes to check one file I don't know how I can solve this.
Look at the Programm: "QuickSFV v2.36". I want the same 2 progressbars.

Sorry for my english, I hope you understood me.
Thanks for the time and help for all who can help me to solve this.

With best regards to all
ElTiburon

  • 2 months later...
kaesereibe Wayfarer

kaesereibe

Posted
Posted (edited)

Just for info:

For MD4 and MD5 you use  MDxInit, MDxUpdate & MDxFinal Functions.

For SHA1 you use the CryptX-Functions but you can use here the  A_SHAInit, A_SHAUpdate & A_SHAFinal Functions who are similar to the other two (and should be faster than the CryptX...)

 

Small Hash Example for Strings (need to be rewritten for files like you do for MD4 & MD5): 

MsgBox(0, "SHA", _SHA("test"))

Func _SHA($string)
    Static Local $hDLL = DllOpen("advapi32.dll")
    Static Local $SHA_CTX = DllStructCreate("dword unknown[6];dword state[5];dword count[2];ubyte buffer[64];ubyte digest[20]")
    DllCall($hDLL, "none", "A_SHAInit", "ptr", DllStructGetPtr($SHA_CTX))
    DllCall($hDLL, "none", "A_SHAUpdate", "ptr", DllStructGetPtr($SHA_CTX), "str", $string, "dword", StringLen($string))
    DllCall($hDLL, "none", "A_SHAFinal", "ptr", DllStructGetPtr($SHA_CTX), "ptr", DllStructGetPtr($SHA_CTX) + 0x74)
    return Hex(DllStructGetData($SHA_CTX, "digest"))
EndFunc

 

Ref: https://autoit.de/index.php/Thread/83602-Alternative-Hash-Funktion-f%C3%BCr-MD4-MD5-SHA-1/

Edited by kaesereibe
BIN 2 DEC | ConvertTemp | DEC 2 BIN | GetWeekday | HEX 2 RGB | INT 2 HEX | QueryPerformance

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...