RAMzor Posted November 10, 2023 Share Posted November 10, 2023 Hi to all! What I have: I have two binary files with 8bit binary data. Actually there are two 8bit 640x512 grayscale patterns stored as not standard raw files. What is required: Detect tolerance between two files and do this many time in loop. (Actually compare golden pattern file to 300 other in a folder) The question: what is the fastest way to read binary file data to array and compare two large arrays? after many attempt to minimize comparation time, I could only do this (it takes 1300ms on my pc): expandcollapse popup#AutoIt3Wrapper_UseX64=y #include <FileConstants.au3> #include <Array.au3> $sPath1 = "TestPtrn_1.raw" $sPath2 = "TestPtrn_2.raw" $hFileOpen = FileOpen($sPath1, BitOR($FO_READ, $FO_BINARY)) ;~ Local $aBinary1 = FileReadToArray($hFileOpen) ; <-- Not work as expected ? Local $bBinary1 = FileRead($hFileOpen) FileClose($hFileOpen) $hFileOpen = FileOpen($sPath2, BitOR($FO_READ, $FO_BINARY)) ;~ $hFileOpen = FileOpen($sPath2, $FO_READ) Local $bBinary2 = FileRead($hFileOpen) FileClose($hFileOpen) $t = TimerInit() $iMaxDiff = _Compare1($bBinary1, $bBinary2) ConsoleWrite("MAX DIFF: " & $iMaxDiff & " digital level" & @CRLF) ConsoleWrite("Time: " & Round(TimerDiff($t)) & " ms" & @CRLF) ; Return diff in digital level Func _Compare1($bBinary1, $bBinary2) Local $aArr1 = StringSplit($bBinary1, "") Local $aArr2 = StringSplit($bBinary2, "") Local $iDiff, $iMaxDiff = 0 For $i = 1 To StringLen($bBinary2)/2 step 2 $iDiff = Abs(Dec($aArr1[$i] & $aArr1[$i+1]) - Dec($aArr2[$i] & $aArr2[$i+1])) If $iDiff > $iMaxDiff Then $iMaxDiff = $iDiff Next Return $iMaxDiff EndFunc Thanks in advance! TestPtrn_Files_For_Test.7z Link to comment Share on other sites More sharing options...
Nine Posted November 11, 2023 Share Posted November 11, 2023 Got about 20% faster this way, not a big deal but cleaner for sure : #AutoIt3Wrapper_UseX64=y #include <FileConstants.au3> $sPath1 = "TestPtrn_1.raw" $sPath2 = "TestPtrn_2.raw" $iSize = FileGetSize($sPath1) Local $tByte1 = DllStructCreate("byte array[" & $iSize & "]") Local $hFileOpen = FileOpen($sPath1, BitOR($FO_READ, $FO_BINARY)) $tByte1.array = FileRead($hFileOpen) FileClose($hFileOpen) Local $tByte2 = DllStructCreate("byte array[" & $iSize & "]") $hFileOpen = FileOpen($sPath2, BitOR($FO_READ, $FO_BINARY)) $tByte2.array = FileRead($hFileOpen) FileClose($hFileOpen) Local $t = TimerInit(), $iDiff, $iMaxDiff = 0 For $i = 1 To $iSize $iDiff = Abs(DllStructGetData($tByte1, 1, $i) - DllStructGetData($tByte2, 1, $i)) If $iDiff > $iMaxDiff Then $iMaxDiff = $iDiff Next ConsoleWrite("MAX DIFF: " & $iMaxDiff & " digital level" & @CRLF) ConsoleWrite("Time: " & Round(TimerDiff($t)) & " ms" & @CRLF) argumentum 1 “They did not know it was impossible, so they did it” ― Mark Twain Spoiler Block all input without UAC Save/Retrieve Images to/from Text Monitor Management (VCP commands) Tool to search in text (au3) files Date Range Picker Virtual Desktop Manager Sudoku Game 2020 Overlapped Named Pipe IPC HotString 2.0 - Hot keys with string x64 Bitwise Operations Multi-keyboards HotKeySet Recursive Array Display Fast and simple WCD IPC Multiple Folders Selector Printer Manager GIF Animation (cached) Screen Scraping Multi-Threading Made Easy Link to comment Share on other sites More sharing options...
RAMzor Posted November 11, 2023 Author Share Posted November 11, 2023 (edited) Thanx, Nine In the strange way I still have lower runtime in my code 🤷♂️, 1378 ms vs 1272 ms in my case. But that doesn't matter at all, your code much cleaner! I will be happy to use it in my projects where execution time is not an issue In this project, it is necessary to check around 300 files and in this case it takes more than 5-6 minutes every time. This is not acceptable and I must find a faster solution. Maybe there is some external dll or another way to implement this? Edited November 11, 2023 by RAMzor Link to comment Share on other sites More sharing options...
Nine Posted November 11, 2023 Share Posted November 11, 2023 Yes, I have created a DLL for one of my UDF (Screen Scraping) where looping in pure AutoIt was too slow...Ask other forum like FreeBasic or C#. By using DllStruct, it is quite easy to pass those to DLL in any other language. RAMzor 1 “They did not know it was impossible, so they did it” ― Mark Twain Spoiler Block all input without UAC Save/Retrieve Images to/from Text Monitor Management (VCP commands) Tool to search in text (au3) files Date Range Picker Virtual Desktop Manager Sudoku Game 2020 Overlapped Named Pipe IPC HotString 2.0 - Hot keys with string x64 Bitwise Operations Multi-keyboards HotKeySet Recursive Array Display Fast and simple WCD IPC Multiple Folders Selector Printer Manager GIF Animation (cached) Screen Scraping Multi-Threading Made Easy Link to comment Share on other sites More sharing options...
Solution RTFC Posted November 11, 2023 Solution Share Posted November 11, 2023 (edited) @RAMzor I get 15 ms on my crappy old laptop, with a little matrix magic: expandcollapse popup#include "Eigen4AutoIt.au3" $rows = 640 $cols = 512 $colsInt = 512/4 ; has to be divisible by four $elements = $rows * $cols _Eigen_StartUp("int") _Eigen_ResetLogicalBit0only() ; this means use all 32 bits when masking (do not reduce to true/false) ; create work buffers (do this once, prior to any number of file comparisons thereafter) $matA = _Eigen_CreateMatrix($rows, $colsInt) $refA = DllStructCreate("byte[" & $elements & "]", _Eigen_GetPtr($matA, "A", False, False)) $matAx4 = _Eigen_CreateMatrix($rows, $cols) $matB = _Eigen_CreateMatrix($rows, $colsInt) $refB = DllStructCreate("byte[" & $elements & "]", _Eigen_GetPtr($matB, "B", False, False)) $matBx4 = _Eigen_CreateMatrix($rows, $cols) Global $matMask = _Eigen_CreateMatrix_Constant($rows, $cols, 255) _Eigen_CwiseScalarOp_Block_InPlace ($matMask,0,$colsInt,$rows,$colsInt,"shl",8) _Eigen_CwiseScalarOp_Block_InPlace ($matMask,0,$colsInt*2,$rows,$colsInt,"shl",16) _Eigen_CwiseScalarOp_Block_InPlace ($matMask,0,$colsInt*3,$rows,$colsInt,"shl",24) ; file comparison starts here $sPath1 = "TestPtrn_1.raw" $sPath2 = "TestPtrn_2.raw" $hFileOpen = FileOpen($sPath1, BitOR($FO_READ, $FO_BINARY)) DllStructSetData($refA, 1, FileRead($hFileOpen)) FileClose($hFileOpen) $hFileOpen = FileOpen($sPath2, BitOR($FO_READ, $FO_BINARY)) DllStructSetData($refB, 1, FileRead($hFileOpen)) FileClose($hFileOpen) $t1 = TimerInit() ; with expansion _ExpandBytesToInt32($matA, $matAx4) _ExpandBytesToInt32($matB, $matBx4) $t2 = TimerInit() ; without expansion _Eigen_CwiseBinaryOp_InPlace($matAx4, "-", $matBx4) _Eigen_CwiseUnaryOp_InPlace($matAx4, "abs") $iMaxDiff = _Eigen_GetMaxVal($matAx4) ConsoleWrite("MAX DIFF: " & $iMaxDiff & " digital level" & @CRLF) ConsoleWrite("Time with byteToIntExpansion: " & Round(TimerDiff($t1)) & " ms" & @CRLF) ConsoleWrite("Time without byteToIntExpansion: " & Round(TimerDiff($t2)) & " ms" & @CRLF) _Eigen_CloseDown() Func _ExpandBytesToInt32($mat1, $mat4) _Eigen_CreateMatrix_FromA_Tiled ($mat1, 1, 4, $mat4) ; duplicate 4x horizontally _Eigen_CwiseLogicalOp_InPlace($mat4,"and",$matMask) ; remove unwanted bits ; SHR to LSByte _Eigen_CwiseScalarOp_Block_InPlace ($mat4,0,$colsInt,$rows,$colsInt,"shr",8) _Eigen_CwiseScalarOp_Block_InPlace ($mat4,0,$colsInt*2,$rows,$colsInt,"shr",16) _Eigen_CwiseScalarOp_Block_InPlace ($mat4,0,$colsInt*3,$rows,$colsInt,"shr",24) EndFunc Of course, if you can store the binary files as proper int32 in the first place (i.e., 4 times more file space needed) you wouldn't have to do all the masking and shifting here, reducing the processing (at my end) to 3 ms per file comparison. Even faster would be if you could store all binary files in a single dataset which you then block-divide and process. Edited November 11, 2023 by RTFC minor correction, and typos TheXman 1 My Contributions and Wrappers Spoiler BitMaskSudokuSolver BuildPartitionTable CodeCrypter CodeScanner DigitalDisplay Eigen4AutoIt FAT Suite HighMem MetaCodeFileLibrary OSgrid Pool RdRand SecondDesktop SimulatedAnnealing Xbase I/O Link to comment Share on other sites More sharing options...
argumentum Posted November 11, 2023 Share Posted November 11, 2023 (edited) 4 hours ago, RAMzor said: In this project, it is necessary to check around 300 files and in this case it takes more than 5-6 minutes every time if you have 8 cores, it'd take an 8th of the time by forking into 8 running scripts, if you can split the workload and at the end compare the resultant from each the 8 processes. Edit: "Even faster would be if you could store all binary files in a single dataset which you then block-divide and process.". That wasn't there when I posted but yes, it'd be good to delegate the run ( to Eigen4AutoIt ) that way too. If using Eigen4AutoIt is much faster, then maybe splitting/forking the workload is not worth it. Edited November 11, 2023 by argumentum addendum Follow the link to my code contribution ( and other things too ). FAQ - Please Read Before Posting. Link to comment Share on other sites More sharing options...
RAMzor Posted November 11, 2023 Author Share Posted November 11, 2023 1 hour ago, RTFC said: @RAMzor I get 15 ms on my crappy old laptop, with a little matrix magic: expandcollapse popup#include "..\Eigen4AutoIt.au3" $bytemask = 255 $rows = 640 $cols = 512 $colsInt = 512/4 ; has to be divisible by four $elements = $rows * $cols _Eigen_StartUp("int") ; create work buffers $matA = _Eigen_CreateMatrix($rows, $colsInt) $refA = DllStructCreate("byte[" & $elements & "]", _Eigen_GetPtr($matA, "A", False, False)) $matAx4 = _Eigen_CreateMatrix($rows, $cols) $matB = _Eigen_CreateMatrix($rows, $colsInt) $refB = DllStructCreate("byte[" & $elements & "]", _Eigen_GetPtr($matB, "B", False, False)) $matBx4 = _Eigen_CreateMatrix($rows, $cols) Global $matMask = _Eigen_CreateMatrix_Constant($rows, $cols, 255) _Eigen_CwiseScalarOp_Block_InPlace ($matMask,0,$colsInt,$rows,$colsInt,"shl",8) _Eigen_CwiseScalarOp_Block_InPlace ($matMask,0,$colsInt*2,$rows,$colsInt,"shl",16) _Eigen_CwiseScalarOp_Block_InPlace ($matMask,0,$colsInt*3,$rows,$colsInt,"shl",24) ; file comparison starts here $sPath1 = "TestPtrn_1.raw" $sPath2 = "TestPtrn_2.raw" $hFileOpen = FileOpen($sPath1, BitOR($FO_READ, $FO_BINARY)) DllStructSetData($refA, 1, FileRead($hFileOpen)) FileClose($hFileOpen) $hFileOpen = FileOpen($sPath2, BitOR($FO_READ, $FO_BINARY)) DllStructSetData($refB, 1, FileRead($hFileOpen)) FileClose($hFileOpen) $t1 = TimerInit() ; with expansion _ExpandBytesToInt32($matA, $matAx4) _ExpandBytesToInt32($matB, $matAx4) $t2 = TimerInit() ; without expansion _Eigen_CwiseBinaryOp_InPlace($matAx4, "-", $matBx4) _Eigen_CwiseUnaryOp_InPlace($matAx4, "abs") $iMaxDiff = _Eigen_GetMaxVal($matAx4) ConsoleWrite("MAX DIFF: " & $iMaxDiff & " digital level" & @CRLF) ConsoleWrite("Time with byteToIntExpansion: " & Round(TimerDiff($t1)) & " ms" & @CRLF) ConsoleWrite("Time without byteToIntExpansion: " & Round(TimerDiff($t2)) & " ms" & @CRLF) _Eigen_CloseDown() Func _ExpandBytesToInt32($mat1, $mat4) _Eigen_CreateMatrix_FromA_Tiled ($mat1, 1, 4, $mat4) ; duplicate 4x horizontally _Eigen_CwiseLogicalOp_InPlace($mat4,"and",$matMask) ; remove unwanted bits ; SHR to LSByte _Eigen_CwiseScalarOp_Block_InPlace ($mat4,0,$colsInt,$rows,$colsInt,"shr",8) _Eigen_CwiseScalarOp_Block_InPlace ($mat4,0,$colsInt*2,$rows,$colsInt,"shr",16) _Eigen_CwiseScalarOp_Block_InPlace ($mat4,0,$colsInt*3,$rows,$colsInt,"shr",24) EndFunc Of course, if you can store the binary files as proper int32 in the first place (i.e., 4 times more file space needed) you wouldn't have to do all the masking and shifting here, reducing the processing (at my end) to 3 ms per file comparison. Even faster would be if you could store all binary files in a single dataset which you then block-divide and process. OMG! 🤦♂️ So fast! This is not real! Thanks a lot! One question: How can I know which files are required to deploy with a compiled project? Thanks to all for spending your time to help me! Link to comment Share on other sites More sharing options...
RTFC Posted November 11, 2023 Share Posted November 11, 2023 (edited) We aim to please. You'd need to copy the Eigen4AutoIt.ini file (auto-generated when you first run E4A; edit it as you see fit) and the appropriate dll (EigenDense.dll for x86 or EigenDense_x64.dll for x64). Since your data files are tiny (no offense), you probably don't need the extra power that the x64 E4A environment provides, so just compile for x86 and copy EigenDense.dll locally. Note that the dll-path variable in the .ini file needs to point to where the dll resides. If you provide an installer for your package like InnoSetup, you can make it overwrite this path in the .ini file to wherever your installer places the other files during any user-defined local installation. Edited November 11, 2023 by RTFC typos RAMzor 1 My Contributions and Wrappers Spoiler BitMaskSudokuSolver BuildPartitionTable CodeCrypter CodeScanner DigitalDisplay Eigen4AutoIt FAT Suite HighMem MetaCodeFileLibrary OSgrid Pool RdRand SecondDesktop SimulatedAnnealing Xbase I/O Link to comment Share on other sites More sharing options...
RAMzor Posted November 11, 2023 Author Share Posted November 11, 2023 Thanks a lot! Link to comment Share on other sites More sharing options...
RAMzor Posted November 11, 2023 Author Share Posted November 11, 2023 (edited) RTFC, Unfortunately there is an error in your example. The difference between matrix is always 1 even when compared file to itself. Edited November 11, 2023 by RAMzor Link to comment Share on other sites More sharing options...
Andreik Posted November 11, 2023 Share Posted November 11, 2023 If you are running x86 version of AutoIt this function will run in less than 1ms: $tFile1 = ReadBinaryFile('TestPtrn_1.raw') $tFile2 = ReadBinaryFile('TestPtrn_2.raw') $Init = TimerInit() $iMaxDiff = CompareData($tFile1, $tFile2) ConsoleWrite("MAX DIFF: " & $iMaxDiff & " digital level" & @CRLF) ConsoleWrite("Time: " & TimerDiff($Init) & " ms" & @CRLF) Func CompareData(Const ByRef $tFile1, Const ByRef $tFile2) Local $sCode = '0x8B7424048B7C24088B4C240C31D28A068A1F28D8780E38D07F064647E2F0EB0A88C2EBF62A1E88D8EBEC89D0C20C00' Local $dCode = Binary($sCode) Local $iCodeSize = BinaryLen($dCode) Local $tBuffer = DllStructCreate('byte Code[' & $iCodeSize & ']') DllStructSetData($tBuffer, 'Code', $dCode) Local $aCall = DllCallAddress('int', DllStructGetPtr($tBuffer), 'ptr', DllStructGetPtr($tFile1), 'ptr', DllStructGetPtr($tFile2), 'int', DllStructGetSize($tFile1)) Return $aCall[0] EndFunc Func ReadBinaryFile($sPath) Local $iSize = FileGetSize($sPath) Local $tFile = DllStructCreate("byte Data[" & $iSize & "]") Local $hFile = FileOpen($sPath, 16) ; FO_READ + FO_BINARY $tFile.Data = FileRead($hFile) FileClose($hFile) Return $tFile EndFunc RAMzor 1 When the words fail... music speaks. Link to comment Share on other sites More sharing options...
RTFC Posted November 11, 2023 Share Posted November 11, 2023 Thanks for letting me know; I'll have a look at it tomorrow. My Contributions and Wrappers Spoiler BitMaskSudokuSolver BuildPartitionTable CodeCrypter CodeScanner DigitalDisplay Eigen4AutoIt FAT Suite HighMem MetaCodeFileLibrary OSgrid Pool RdRand SecondDesktop SimulatedAnnealing Xbase I/O Link to comment Share on other sites More sharing options...
RAMzor Posted November 11, 2023 Author Share Posted November 11, 2023 55 minutes ago, Andreik said: If you are running x86 version of AutoIt this function will run in less than 1ms: $tFile1 = ReadBinaryFile('TestPtrn_1.raw') $tFile2 = ReadBinaryFile('TestPtrn_2.raw') $Init = TimerInit() $iMaxDiff = CompareData($tFile1, $tFile2) ConsoleWrite("MAX DIFF: " & $iMaxDiff & " digital level" & @CRLF) ConsoleWrite("Time: " & TimerDiff($Init) & " ms" & @CRLF) Func CompareData(Const ByRef $tFile1, Const ByRef $tFile2) Local $sCode = '0x8B7424048B7C24088B4C240C31D28A068A1F28D8780E38D07F064647E2F0EB0A88C2EBF62A1E88D8EBEC89D0C20C00' Local $dCode = Binary($sCode) Local $iCodeSize = BinaryLen($dCode) Local $tBuffer = DllStructCreate('byte Code[' & $iCodeSize & ']') DllStructSetData($tBuffer, 'Code', $dCode) Local $aCall = DllCallAddress('int', DllStructGetPtr($tBuffer), 'ptr', DllStructGetPtr($tFile1), 'ptr', DllStructGetPtr($tFile2), 'int', DllStructGetSize($tFile1)) Return $aCall[0] EndFunc Func ReadBinaryFile($sPath) Local $iSize = FileGetSize($sPath) Local $tFile = DllStructCreate("byte Data[" & $iSize & "]") Local $hFile = FileOpen($sPath, 16) ; FO_READ + FO_BINARY $tFile.Data = FileRead($hFile) FileClose($hFile) Return $tFile EndFunc Work really fast! Thanks a lot! Link to comment Share on other sites More sharing options...
Andreik Posted November 11, 2023 Share Posted November 11, 2023 Yes, it does but it doesn't work with #AutoIt3Wrapper_UseX64=y When the words fail... music speaks. Link to comment Share on other sites More sharing options...
RAMzor Posted November 11, 2023 Author Share Posted November 11, 2023 (edited) 2 hours ago, RTFC said: Thanks for letting me know; I'll have a look at it tomorrow. In addition, I changed the 1st and 2nd byte to 0x05 and 0x06 (vs 0x00 0x00 in above attached files) and the difference is still not changed. updated file "TestPtrn_2_56.raw" for test is attached. Compare it to "TestPtrn_1.raw" - max diff must be 6 TestPtrn_2_56.raw TestPtrn_1.raw Edited November 11, 2023 by RAMzor Link to comment Share on other sites More sharing options...
RAMzor Posted November 11, 2023 Author Share Posted November 11, 2023 (edited) 2 hours ago, Andreik said: If you are running x86 version of AutoIt this function will run in less than 1ms: $tFile1 = ReadBinaryFile('TestPtrn_1.raw') $tFile2 = ReadBinaryFile('TestPtrn_2.raw') $Init = TimerInit() $iMaxDiff = CompareData($tFile1, $tFile2) ConsoleWrite("MAX DIFF: " & $iMaxDiff & " digital level" & @CRLF) ConsoleWrite("Time: " & TimerDiff($Init) & " ms" & @CRLF) Func CompareData(Const ByRef $tFile1, Const ByRef $tFile2) Local $sCode = '0x8B7424048B7C24088B4C240C31D28A068A1F28D8780E38D07F064647E2F0EB0A88C2EBF62A1E88D8EBEC89D0C20C00' Local $dCode = Binary($sCode) Local $iCodeSize = BinaryLen($dCode) Local $tBuffer = DllStructCreate('byte Code[' & $iCodeSize & ']') DllStructSetData($tBuffer, 'Code', $dCode) Local $aCall = DllCallAddress('int', DllStructGetPtr($tBuffer), 'ptr', DllStructGetPtr($tFile1), 'ptr', DllStructGetPtr($tFile2), 'int', DllStructGetSize($tFile1)) Return $aCall[0] EndFunc Func ReadBinaryFile($sPath) Local $iSize = FileGetSize($sPath) Local $tFile = DllStructCreate("byte Data[" & $iSize & "]") Local $hFile = FileOpen($sPath, 16) ; FO_READ + FO_BINARY $tFile.Data = FileRead($hFile) FileClose($hFile) Return $tFile EndFunc Your code accept up to 7 BIT (0x7F). Over this value the Diff is wrong (Diff from 0x00 in one file to 0x80 to another one is 1) Compare attached files TestPtrn_2 vs TestPtrn_2_DEh the diff must be 0xDE (222d). The max diff must be 0xDE but currently the result is 0x22 (34d) TestPtrn_2_DEh.raw TestPtrn_1.raw Edited November 11, 2023 by RAMzor Link to comment Share on other sites More sharing options...
Andreik Posted November 11, 2023 Share Posted November 11, 2023 (edited) Try this: #include <Memory.au3> $tFile1 = ReadBinaryFile('TestPtrn_2_DEh.raw') $tFile2 = ReadBinaryFile('TestPtrn_1.raw') $Init = TimerInit() $iMaxDiff = CompareData($tFile1, $tFile2) ConsoleWrite("MAX DIFF: " & $iMaxDiff & " digital level" & @CRLF) ConsoleWrite("Time: " & TimerDiff($Init) & " ms" & @CRLF) Func CompareData(Const ByRef $tFile1, Const ByRef $tFile2) Local $sCode = '0x8B7424048B7C24088B4C240C31D28A068A1F38D8770688C488D888E328D838D0720288C24647E2E689D0C20C00' Local $dCode = Binary($sCode) Local $iCodeSize = BinaryLen($dCode) Local $tBuffer = DllStructCreate('byte Code[' & $iCodeSize & ']') DllStructSetData($tBuffer, 'Code', $dCode) Local $aCall = DllCallAddress('int', DllStructGetPtr($tBuffer), 'ptr', DllStructGetPtr($tFile1), 'ptr', DllStructGetPtr($tFile2), 'int', DllStructGetSize($tFile1)) Return $aCall[0] EndFunc Func ReadBinaryFile($sPath) Local $iSize = FileGetSize($sPath) Local $tFile = DllStructCreate("byte Data[" & $iSize & "]") Local $hFile = FileOpen($sPath, 16) ; FO_READ + FO_BINARY $tFile.Data = FileRead($hFile) FileClose($hFile) Return $tFile EndFunc Edited November 11, 2023 by Andreik UEZ 1 When the words fail... music speaks. Link to comment Share on other sites More sharing options...
UEZ Posted November 11, 2023 Share Posted November 11, 2023 (edited) If you only want to check if two files are different then you can use the memcmp WinAPI function. ;Coded by UEZ build 2023-11-12 #include <WinAPIFiles.au3> #include <WinAPIHObj.au3> Global $fTimer = TimerInit() Global $bIsDifferent = FastBinCompare("TestPtrn_2_DEh.raw", "TestPtrn_1.raw") ConsoleWrite("Time: " & TimerDiff($fTimer) & " ms" & @CRLF) ConsoleWrite("Is different: " & $bIsDifferent & @CRLF) Func FastBinCompare($sFile1, $sFile2) Local $iFileSize = FileGetSize($sFile1) If $iFileSize <> FileGetSize($sFile2) Or Not $iFileSize Then Return SetError(1, -1, -1) Local $tBuffer1 = DllStructCreate("byte[" & $iFileSize & "]"), $tBuffer2 = DllStructCreate("byte[" & $iFileSize & "]") Local $hFile = _WinAPI_CreateFile($sFile1, 2, 2), $nBytes _WinAPI_ReadFile($hFile, $tBuffer1, $iFileSize, $nBytes) _WinAPI_CloseHandle($hFile) $hFile = _WinAPI_CreateFile($sFile2, 2, 2) _WinAPI_ReadFile($hFile, $tBuffer2, $iFileSize, $nBytes) _WinAPI_CloseHandle($hFile) Local $aResult = DllCall("msvcrt.dll", "int:cdecl", "memcmp", "struct*", $tBuffer1, "struct*", $tBuffer2, "uint", DllStructGetSize($tBuffer1)) If @error Or Not IsArray($aResult) Then Return SetError(2, -2, -2) Return $aResult[0] EndFunc Or here regarding image compare: Edited November 11, 2023 by UEZ mLipok and RAMzor 1 1 Please don't send me any personal message and ask for support! I will not reply! Selection of finest graphical examples at Codepen.io The own fart smells best! ✌Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!¯\_(ツ)_/¯ ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ Link to comment Share on other sites More sharing options...
RTFC Posted November 11, 2023 Share Posted November 11, 2023 (edited) Of course it all worked perfectly at my end (yes, I do get 6 with your second test file), but somehow not on yours... Took a while to realise that I tend to work in full binary mode, thus one of the default switches in my test environment is full 32 bitmode always enabled for bitmasks, whereas for any user installing the package it would initially be evaluating bit0 only, so the difference after masking could never be more than unity in that mode for any comparison. To fix this, you'll need to add one line to the script (now amended in my first post here), just below _Eigen_StartUp(): _Eigen_ResetLogicalBit0only() Edited November 11, 2023 by RTFC My Contributions and Wrappers Spoiler BitMaskSudokuSolver BuildPartitionTable CodeCrypter CodeScanner DigitalDisplay Eigen4AutoIt FAT Suite HighMem MetaCodeFileLibrary OSgrid Pool RdRand SecondDesktop SimulatedAnnealing Xbase I/O Link to comment Share on other sites More sharing options...
Andreik Posted November 12, 2023 Share Posted November 12, 2023 (edited) This version works with both 32/64 bit versions: expandcollapse popup#AutoIt3Wrapper_UseX64=y #include <Memory.au3> $tFile1 = ReadBinaryFile('TestPtrn_1.raw') $tFile2 = ReadBinaryFile('TestPtrn_2_DEh.raw') $Init = TimerInit() $iMaxDiff = CompareData($tFile1, $tFile2) ConsoleWrite("MAX DIFF: " & $iMaxDiff & " digital level" & @CRLF) ConsoleWrite("Time: " & TimerDiff($Init) & " ms" & @CRLF) Func CompareData(Const ByRef $tFile1, Const ByRef $tFile2) Local $sCode, $pMemory, $tBuffer If @AutoItX64 Then $sCode = '0x31C08A440AFF412A4408FF7702F6D838E0720288C4E2EB0FB6C4C3' Else $sCode = '0x8B4C24048B7424088B7C240C31C08A440EFF2A440FFF7702F6D838E0720288C4E2EC0FB6C4C20C00' EndIf Local $dCode = Binary($sCode) Local $iCodeSize = BinaryLen($dCode) $pMemory = _MemVirtualAlloc(0, $iCodeSize, $MEM_COMMIT, $PAGE_EXECUTE_READWRITE) $tBuffer = DllStructCreate('byte Code[' & $iCodeSize & ']', $pMemory) $tBuffer.Code = $dCode Local $aCall = DllCallAddress('int', DllStructGetPtr($tBuffer), _ 'int', DllStructGetSize($tFile1), _ ; Number of bytes 'ptr', DllStructGetPtr($tFile1), _ ; Pointer to first structure 'ptr', DllStructGetPtr($tFile2) _ ; Pointer to second structure ) _MemVirtualFree($pMemory, $iCodeSize, $MEM_DECOMMIT) Return $aCall[0] EndFunc Func ReadBinaryFile($sPath) Local $iSize = FileGetSize($sPath) Local $tFile = DllStructCreate("byte Data[" & $iSize & "]") Local $hFile = FileOpen($sPath, 16) ; FO_READ + FO_BINARY $tFile.Data = FileRead($hFile) FileClose($hFile) Return $tFile EndFunc The results are similar, around 1ms for the files that you provided as samples. Edited November 12, 2023 by Andreik UEZ 1 When the words fail... music speaks. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now