Check MD5 Large Files

[r2du-soft]

Hi, I'm trying to get the md5 value of the files but I'm having trouble with large files.
When I do this, I encounter the Error allocating memory error
I recently found out that it is possible to get the MD5 value via CMD command line.
But I want to use only autoit commands in the program
Is there a way to do this by calling windows functions and dlls to get MD5 files via autoit

[Danyfirex]

Maybe using _Crypt_HashFile.

 

Saludos

[Musashi]

2 hours ago, r2du-soft said:

Is there a way to do this by calling windows functions and dlls to get MD5 files via autoit

Here the Default way and a Speed variant (by @trancexx )

EDIT : Additional info :

I have applied the Default and the Speed variant on a 6 GB file.

The Speed variant throws an error (I assume for files > 2 GB).
The directive #AutoIt3Wrapper_UseX64 = Y ensures that the error message of the Speed variant disappears, but the hash then differs from the Default variant. You should check externally which MD5 hash is the correct one and then add a switch for the respective variant for files > 2 GB.

expandcollapse popup

#include <Crypt.au3>

Opt('MustDeclareVars', 1)

; Filename (full path)
Global $sFilename = @ScriptDir & '\Test.mp4'
Global $dMD5Hash
Global $fTimerStart, $fTimerEnd ; *** just info

; -----------------------------------------
; Default :
; -----------------------------------------
$fTimerStart = TimerInit()  ; *** just info

_Crypt_Startup()
$dMD5Hash =  _Crypt_HashFile($sFilename, $CALG_MD5)
If @error Then
    ConsoleWrite('@@ Error      = <' & @error & '>' & @CRLF)
Else
    ConsoleWrite('@@ MD5 Hash   = <' & Hex($dMD5Hash) & '>' & @CRLF)
EndIf
_Crypt_Shutdown()

$fTimerEnd = TimerDiff($fTimerStart) ; *** just info
ConsoleWrite("Default  -> Time  : " & $fTimerEnd & @CRLF)


; -----------------------------------------
; Speedversion : Uses virtual RAM
; -----------------------------------------
$fTimerStart = TimerInit()  ; *** just info

$dMD5Hash = _HashFile_MD5VMem($sFilename)
If @error Then
    ConsoleWrite('@@ Error V    = <' & @error & '>' & @CRLF)
Else
    ConsoleWrite('@@ MD5 V Hash = <' & Hex($dMD5Hash) & '>' & @CRLF)
EndIf

$fTimerEnd = TimerDiff($fTimerStart) ; *** just info
ConsoleWrite("Virtual -> Time   : " & $fTimerEnd & @CRLF)

; #FUNCTION# ;===============================================================================
;
; Name         : _HashFile_MD5VMem
; Beschreibung : - Calculates MD5 value for the specific file.
;                - Speedversion : Uses virtual RAM
;                - max. Filesize depends on free RAM
; Returns      : Success : @error = 0 , MD5-Hash
;                Error   : @error <> 0 , empty value
;
; Author ........: trancexx
;   see www.autoitscript.com/forum/topic/95558-crc32-md4-md5-sha1-for-files/?page=1
;====================================1======================================================
Func _HashFile_MD5VMem($sFile)
    Local $ahCall , $hFile , $hFileMappingObject, $pFile
    Local $iBufferSize, $tMD5_CTX

    $ahCall = DllCall("kernel32.dll", "hwnd", "CreateFileW", _
            "wstr", $sFile, _
            "dword", 0x80000000, _
            "dword", 3, _
            "ptr", 0, _
            "dword", 3, _
            "dword", 0, _
            "ptr", 0)

    If @error Or $ahCall[0] = -1 Then
        Return SetError(1, 0, "")
    EndIf

    $hFile = $ahCall[0]
    $ahCall = DllCall("kernel32.dll", "ptr", "CreateFileMappingW", _
            "hwnd", $hFile, _
            "dword", 0, _
            "dword", 2, _
            "dword", 0, _
            "dword", 0, _
            "ptr", 0)

    If @error Or Not $ahCall[0] Then
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFile)
        Return SetError(2, 0, "")
    EndIf

    DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFile)

    $hFileMappingObject = $ahCall[0]
    $ahCall = DllCall("kernel32.dll", "ptr", "MapViewOfFile", _
            "hwnd", $hFileMappingObject, _
            "dword", 4, _
            "dword", 0, _
            "dword", 0, _
            "dword", 0)

    If @error Or Not $ahCall[0] Then
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(3, 0, "")
    EndIf

    $pFile = $ahCall[0]
    $iBufferSize = FileGetSize($sFile)

    $tMD5_CTX = DllStructCreate("dword i[2];" & _
            "dword buf[4];" & _
            "ubyte in[64];" & _
            "ubyte digest[16]")

    DllCall("advapi32.dll", "none", "MD5Init", "ptr", DllStructGetPtr($tMD5_CTX))

    If @error Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(4, 0, "")
    EndIf

    DllCall("advapi32.dll", "none", "MD5Update", _
            "ptr", DllStructGetPtr($tMD5_CTX), _
            "ptr", $pFile, _
            "dword", $iBufferSize)

    If @error Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(5, 0, "")
    EndIf

    DllCall("advapi32.dll", "none", "MD5Final", "ptr", DllStructGetPtr($tMD5_CTX))

    If @error Then
        DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
        DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
        Return SetError(6, 0, "")
    EndIf

    DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
    DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)

    Return SetError(0, 0, DllStructGetData($tMD5_CTX, "digest"))
EndFunc   ;==>_HashFile_MD5VMem

 

Edited September 7, 2022 by Musashi
additional info