hash file with progressbar

[Kyan]

Is possible to monitoring a hash process?

I'm using this function to calculate md5 hash of a file:

expandcollapse popup

Func _MD5ForFile($sFile)

Local $a_hCall = DllCall("kernel32.dll", "hwnd", "CreateFileW", _
"wstr", $sFile, _
"dword", 0x80000000, _ ; GENERIC_READ
"dword", 3, _ ; FILE_SHARE_READ|FILE_SHARE_WRITE
"ptr", 0, _
"dword", 3, _ ; OPEN_EXISTING
"dword", 0, _ ; SECURITY_ANONYMOUS
"ptr", 0)

If @error Or $a_hCall[0] = -1 Then
Return SetError(1, 0, "")
EndIf

Local $hFile = $a_hCall[0]

$a_hCall = DllCall("kernel32.dll", "ptr", "CreateFileMappingW", _
"hwnd", $hFile, _
"dword", 0, _ ; default security descriptor
"dword", 2, _ ; PAGE_READONLY
"dword", 0, _
"dword", 0, _
"ptr", 0)

If @error Or Not $a_hCall[0] Then
DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFile)
Return SetError(2, 0, "")
EndIf

DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFile)

Local $hFileMappingObject = $a_hCall[0]

$a_hCall = DllCall("kernel32.dll", "ptr", "MapViewOfFile", _
"hwnd", $hFileMappingObject, _
"dword", 4, _ ; FILE_MAP_READ
"dword", 0, _
"dword", 0, _
"dword", 0)

If @error Or Not $a_hCall[0] Then
DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
Return SetError(3, 0, "")
EndIf

Local $pFile = $a_hCall[0]
Local $iBufferSize = FileGetSize($sFile)

Local $tMD5_CTX = DllStructCreate("dword i[2];" & _
"dword buf[4];" & _
"ubyte in[64];" & _
"ubyte digest[16]")

DllCall("advapi32.dll", "none", "MD5Init", "ptr", DllStructGetPtr($tMD5_CTX))

If @error Then
DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
Return SetError(4, 0, "")
EndIf

DllCall("advapi32.dll", "none", "MD5Update", _
"ptr", DllStructGetPtr($tMD5_CTX), _
"ptr", $pFile, _
"dword", $iBufferSize)

If @error Then
DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
Return SetError(5, 0, "")
EndIf

DllCall("advapi32.dll", "none", "MD5Final", "ptr", DllStructGetPtr($tMD5_CTX))

If @error Then
DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)
Return SetError(6, 0, "")
EndIf

DllCall("kernel32.dll", "int", "UnmapViewOfFile", "ptr", $pFile)
DllCall("kernel32.dll", "int", "CloseHandle", "hwnd", $hFileMappingObject)

Local $sMD5 = Hex(DllStructGetData($tMD5_CTX, "digest"))

Return SetError(0, 0, $sMD5)

EndFunc ;==>_MD5ForFile

there's a better way to calculate a md5 hash and with progress bar?

[BrewManNH]

_Crypt_HashFile or_HashData, _Crypt_EncryptData or _EncryptFile

[Kyan]

_Crypt_HashFile or_HashData, _Crypt_EncryptData or _EncryptFile

_HashData function is not in autoit libraries, right?

for the crypt function, 512 characters equals to 512bytes?

[BrewManNH]

I meant _Crypt_HashData, I was using a shorthand method when I posted same thing for _Crypt_EncryptFile, thought it was more obvious than it was.

What do you mean by your second sentence? What has 512 characters or bytes to do with this?

[Kyan]

I meant _Crypt_HashData, I was using a shorthand method when I posted same thing for _Crypt_EncryptFile, thought it was more obvious than it was.

What do you mean by your second sentence? What has 512 characters or bytes to do with this?

lol, didn't notice, sorry

well, _crypt_HashFile reads the inputed file like this: FileRead($hFile, 512 * 1024), 512*1024 should be the number of characters read each time, i found out that if it is a ansii char. it occupies 1byte, if is unicode char. it occupies 2bytes. Now i looked to fileopen parameter and it is reading in "binary mode", so how much has every chunk size?

EDIT: I need to now how much from the file was read, to now the hash progress

EDIT2: 512/2 = 256 bytes (since every two chars. in binary represents 1byte?

Edited January 27, 2013 by DiOgO

[jchd]

File* operations are independant of encoding, at least regarding sizes. So 512 means bytes. Also remember that a Unicode codepoint is a virtual entity: the actual size of a "Unicode codepoint" representation may range from 1 to 4 bytes in UTF-8, 1 to 2 16-bit words in UTF-16 or 1 32-bit word in UTF-32.

[Kyan]

File* operations are independant of encoding, at least regarding sizes. So 512 means bytes. Also remember that a Unicode codepoint is a virtual entity: the actual size of a "Unicode codepoint" representation may range from 1 to 4 bytes in UTF-8, 1 to 2 16-bit words in UTF-16 or 1 32-bit word in UTF-32.

thank you, so now is easy to do a progress bar

[Kyan]

it takes forever, for 320mb, it takes ~4min

with hashtab for the same 320mb it only takes less than a minute, something is wrong .(

EDIT: it reachs 240% (and keeps goin, I give up to see where it stops), here's the _crypt_hashfile mod:

expandcollapse popup

Func _Crypt_HashFileMOD($sFile, $iALG_ID)
Local $last = 0
$hashread = 0
Local $hFile
Local $iError, $vReturn
Local $hHashObject = 0
Local $bTempData
_Crypt_Startup()

Do
$hFile = FileOpen($sFile, 16)
If $hFile = -1 Then
$iError = 1
$vReturn = -1
ExitLoop
EndIf
$st = TimerInit()
Do
$bTempData = FileRead($hFile, 512 * 1024)
$hashread +=1
If TimerDiff($st)-$last > 3000 Then
$last = TimerDiff($st)
$prog = ($hashread*512*100)/$sFileSize
;If $prog > 100 Then $prog = 100
_GUICtrlListView_SetItem(GUICtrlGetHandle($List1),Round($prog,2)&"%",$k,1)
EndIf
If @error Then
$vReturn = _Crypt_HashData($bTempData, $iALG_ID, True, $hHashObject)
If @error Then
$vReturn = -1
$iError = 2
ExitLoop 2
EndIf
ExitLoop 2
Else
$hHashObject = _Crypt_HashData($bTempData, $iALG_ID, False, $hHashObject)
If @error Then
$vReturn = -1
$iError = 3
ExitLoop 2
EndIf
EndIf
Until False
Until True

_Crypt_Shutdown()
If $hFile <> -1 Then FileClose($hFile)
Return SetError($iError, 0, $vReturn)
EndFunc ;==>_Crypt_HashFile

Edited January 27, 2013 by DiOgO

[BrewManNH]

What is the @error supposed to be responding to in your Do loop?

[AZJIO]

DiOgO

expandcollapse popup

#include <GUIConstantsEx.au3>
#include <Crypt.au3>

; $sPath = @ScriptDir & '\file.avi'

$sPath = FileOpenDialog('Открыть', @DesktopDir, 'All (*.*)', 1)
If @error Then Exit

$hGui = GUICreate('My Program', 350, 110)
$iProgressBar = GUICtrlCreateProgress(10, 10, 330, 20)
$iButton = GUICtrlCreateButton('Start', 250, 50, 90, 28)
$iStatusBar = GUICtrlCreateLabel('StatusBar', 5, 110 - 20, 340, 17)
GUISetState()
While 1
    Switch GUIGetMsg()
        Case $iButton
            $bHash = _HashFile_PrgBar($sPath, $CALG_MD5, $iProgressBar)
            If @error Then
                GUICtrlSetData($iStatusBar, '@error = ' & @error)
            Else
                GUICtrlSetData($iStatusBar, $bHash)
            EndIf
        Case $GUI_EVENT_CLOSE
            Exit
    EndSwitch
WEnd

Func _HashFile_PrgBar($sPath, $iALG_ID, $iProgressBar)
    Local $hFile, $hCryptHash = 0, $bTempData
    _Crypt_Startup()

    $hFile = FileOpen($sPath, 16)
    If $hFile = -1 Then Return SetError(1, 0, _Error())
    $sFileSize = FileGetSize($sPath)
    If $sFileSize > 1048576 Then
        $SizeRead = Ceiling($sFileSize / 100)
        For $i = 1 To 100
            $bTempData = FileRead($hFile, $SizeRead) ; Считываем сотую часть
            If @error = -1 Then ExitLoop ; если достигнут конец файла
            If $i = 100 Then
                $fFinal = True
            Else
                $fFinal = False
            EndIf
            $hCryptHash = _Crypt_HashData($bTempData, $iALG_ID, $fFinal, $hCryptHash)
            If @error Then Return SetError(1, 0, _Error())
            GUICtrlSetData($iProgressBar, $i)
        Next
    Else
        $hCryptHash = _Crypt_HashData($bTempData, $iALG_ID, True)
        If @error Then Return SetError(1, 0, _Error())
        GUICtrlSetData($iProgressBar, 100)
    EndIf
    FileClose($hFile)

    _Crypt_Shutdown()
    Return SetError(0, 0, $hCryptHash)
EndFunc   ;==>_HashFile_PrgBar

Func _Error()
    _Crypt_Shutdown()
    Return 0
EndFunc   ;==>_Error

Edited January 28, 2013 by AZJIO

[guinness]

I thought it was more obvious than it was.

I understood.

Nice concept AZJIO, though the CPU usage was quite high when using it.

[Kyan]

DiOgO

expandcollapse popup

#include <GUIConstantsEx.au3>
#include <Crypt.au3>

; $sPath = @ScriptDir & '\file.avi'

$sPath = FileOpenDialog('Открыть', @DesktopDir, 'All (*.*)', 1)
If @error Then Exit

$hGui = GUICreate('My Program', 350, 110)
$iProgressBar = GUICtrlCreateProgress(10, 10, 330, 20)
$iButton = GUICtrlCreateButton('Start', 250, 50, 90, 28)
$iStatusBar = GUICtrlCreateLabel('StatusBar', 5, 110 - 20, 340, 17)
GUISetState()
While 1
    Switch GUIGetMsg()
        Case $iButton
            $bHash = _HashFile_PrgBar($sPath, $CALG_MD5, $iProgressBar)
            If @error Then
                GUICtrlSetData($iStatusBar, '@error = ' & @error)
            Else
                GUICtrlSetData($iStatusBar, $bHash)
            EndIf
        Case $GUI_EVENT_CLOSE
            Exit
    EndSwitch
WEnd

Func _HashFile_PrgBar($sPath, $iALG_ID, $iProgressBar)
    Local $hFile, $hCryptHash = 0, $bTempData
    _Crypt_Startup()

    $hFile = FileOpen($sPath, 16)
    If $hFile = -1 Then Return SetError(1, 0, _Error())
    $sFileSize = FileGetSize($sPath)
    If $sFileSize > 1048576 Then
        $SizeRead = Ceiling($sFileSize / 100)
        For $i = 1 To 100
            $bTempData = FileRead($hFile, $SizeRead) ; Считываем сотую часть
            If @error = -1 Then ExitLoop ; если достигнут конец файла
            If $i = 100 Then
                $fFinal = True
            Else
                $fFinal = False
            EndIf
            $hCryptHash = _Crypt_HashData($bTempData, $iALG_ID, $fFinal, $hCryptHash)
            If @error Then Return SetError(1, 0, _Error())
            GUICtrlSetData($iProgressBar, $i)
        Next
    Else
        $hCryptHash = _Crypt_HashData($bTempData, $iALG_ID, True)
        If @error Then Return SetError(1, 0, _Error())
        GUICtrlSetData($iProgressBar, 100)
    EndIf
    FileClose($hFile)

    _Crypt_Shutdown()
    Return SetError(0, 0, $hCryptHash)
EndFunc ;==>_HashFile_PrgBar

Func _Error()
    _Crypt_Shutdown()
    Return 0
EndFunc ;==>_Error

nice, works fine, thanks

i modded _crypt udf but occurs infine loops don't know why

[BrewManNH]

How did you modify the UDF, and why?

[guinness]

nice, works fine, thanks

i modded _crypt udf but occurs infine loops don't know why

If you know what you're doing, then great. I also hope you copied the UDF and created an entirely separate version.

Also in the future remember if you post that "the _Crypt UDF isn't working", it could be down to your modification(s).

[Kyan]

How did you modify the UDF, and why?

at %programfiles%AutoIt3Includecrypt.au3, just copied the hashfile function to my script, called with other name and add a var, to indicate how many bytes are read so far...then build up a progress bar

If you know what you're doing, then great. I also hope you copied the UDF and created an entirely separate version.

Also in the future remember if you post that "the _Crypt UDF isn't working", it could be down to your modification(s).

+-. I not understand why the function has Until False and Until True to stop the loop :s

regarding UDF errors, I avoid that by coping the hash function to my script and then modded it (as I said above)

---------------------

just to not create a topic with a simple question, can you tell me what I'm doing wrong here?

StringRegExp('["PP,12","PP,14","PP,15"]','[(?:.*?)(\d{2})(?:.*?)]',3)

in stead of get decimal numbers (12,14,15) I only get a number by each array element, whats happening?

EDIT: I forget to do braket escape, but I could have 1, 2 or 3 numbers and cannot determine a group as optional, for example for 2nd digit, I used a group like: (d{2})? but it does not is recognized

$test = StringRegExp('["PP,12","PP,14","PP,15"]','\[(?:.*?)(\d{2})(?:.*?)(\d{2})(?:.*?)(\d{2})(?:.*?)\]',3)

EDIT2: Well, it with or parameter "|" works, but if I put a two number case, it beggins filling the first 4 array elements with emptiness (LOL, probably it doesn't says like that)

I tried also groupception, but does not work also :s

$test = StringRegExp('["PP,12","PP,14","PP,15"]','\[(?:.*?)(\d{2})(?:.*?)(\d{2})(?:.*?)(\d{2})(?:.*?)|(?:.*?)(\d{2})(?:.*?)(\d{2})(?:.*?)|(?:.*?)(\d{2})(?:.*?)\]',3)

EDIT3: what I'm trying to do is impossible? do I need to check first if exist 3 sets of 2 digits, and if it fails, check again but for 2 sets...?

EDIT4: Here I'm trying...

$test = StringRegExp('["PP,14","PP,15"]','.*?(?:\[)(\d{2}){1,3}(?:\])',3)

Edited January 29, 2013 by DiOgO