wraithdu Posted November 13, 2011 Author Share Posted November 13, 2011 Hmm. Try to create a small reproducer script with all code req'd and I'll take a look. Maybe you just need a second pair of eyes on it. I've looked through all the path manipulation code in my UDF and tested the regular expressions, and nothing is mangling UNC style paths. Link to comment Share on other sites More sharing options...
llewxam Posted November 14, 2011 Share Posted November 14, 2011 OK, good idea. I'll work on that tomorrow night, and who knows, maybe that will show me the way to correcting it.... Thanks Ian My projects: IP Scanner - Multi-threaded ping tool to scan your available networks for used and available IP addresses, shows ping times, resolves IPs in to host names, and allows individual IPs to be pinged. INFSniff - Great technicians tool - a tool which scans DriverPacks archives for INF files and parses out the HWIDs to a database file, and rapidly scans the local machine's HWIDs, searches the database for matches, and installs them. PPK3 (Persistent Process Killer V3) - Another for the techs - suppress running processes that you need to keep away, helpful when fighting spyware/viruses. Sync Tool - Folder sync tool with lots of real time information and several checking methods. USMT Front End - Front End for Microsoft's User State Migration Tool, including all files needed for USMT 3.01 and 4.01, 32 bit and 64 bit versions. Audit Tool - Computer audit tool to gather vital hardware, Windows, and Office information for IT managers and field techs. Capabilities include creating a customized site agent. CSV Viewer - Displays CSV files with automatic column sizing and font selection. Lines can also be copied to the clipboard for data extraction. MyDirStat - Lists number and size of files on a drive or specified path, allows for deletion within the app. 2048 Game - My version of 2048, fun tile game. Juice Lab - Ecigarette liquid making calculator. Data Protector - Secure notes to save sensitive information. VHD Footer - Add a footer to a forensic hard drive image to allow it to be mounted or used as a virtual machine hard drive. Find in File - Searches files containing a specified phrase. Link to comment Share on other sites More sharing options...
llewxam Posted November 19, 2011 Share Posted November 19, 2011 Thought I should report back. I did start small and had good results, then today I folded all of the tweaks in to my Sync tool and everything was working great - until I tried to exceed the normal 260 character limit using UNC paths - those transfers failed. Here is a stripped-down tester for you: expandcollapse popup#region ;**** Directives created by AutoIt3Wrapper_GUI **** #AutoIt3Wrapper_UseUpx=n #AutoIt3Wrapper_Run_Tidy=y #endregion ;**** Directives created by AutoIt3Wrapper_GUI **** #include <WinAPI.au3> #include <Array.au3> $iToRead = 8 * 1024 * 1024 $Source = "c:testunc" $Target = "c:testunc2" If StringRight($Source, 1) <> "" Then $Source &= "" If StringRight($Target, 1) <> "" Then $Target &= "" $SourceLength = StringLen($Source) If $SourceLength == 3 Then $SourceLength -= 1; gotta have the array elements start with a backslash... $FileList = _FileListToArrayRec($Source, "", "*", "", 1, 2, True) If StringLeft($Source, 2) == "" Then $SourceUNCPrefix = "?UNC" $Source = StringTrimLeft($Source, 2) Else $SourceUNCPrefix = "?" EndIf If StringLeft($Target, 2) == "" Then $TargetUNCPrefix = "?UNC" $Target = StringTrimLeft($Target, 2) Else $TargetUNCPrefix = "?" EndIf _ArrayTrim($FileList, $SourceLength, 0, 1) For $a = 1 To $FileList[0] $CopySource = $SourceUNCPrefix & $Source & $FileList[$a] $CopyTarget = $TargetUNCPrefix & $Target & $FileList[$a] $Copy = _LargeFileCopy($CopySource, $CopyTarget, BitOR(1, 2)) $Error = @error If $Copy == 1 Then $Message = "SUCCESS" Else $Message = $Error EndIf ConsoleWrite($CopySource & " -> " & $CopyTarget & " -> " & $Message & @CR) Next ; #FUNCTION# ==================================================================================================== ; Name...........: _LargeFileCopy ; Description....: Copy large files in such a way as to keep AutoIt GUIs responsive ; Syntax.........: _LargeFileCopy($sSrc, $sDest, $iFlags) ; Parameters.....: $sSrc - Source file name ; $sDest - Destination: may be a file name or directory ; $iFlags - [Optional] Combine flags with BitOR ; | 1 - Overwrite existing file ; | 2 - Create destination directory structure ; | 4 - Flush the destination file buffer before returning ; | 8 - Verify source and destination are identical via bit by bit comparison ; |16 - Verify source and destination are identical via MD5 hash ; |32 - Verify source and destination file size only ; + If more than one verify flag is set, the smallest flag will take precedence ; ; Return values..: Success - 1 ; Failure - 0 and sets @error ; | 1 - Failed to open source file, or source was a directory ; | 2 - Destination file exists and overwrite flag not set ; | 3 - Failed to create destination file ; | 4 - Read error during copy ; | 5 - Write error during copy ; | 6 - Verify failed ; Author.........: Erik Pilsits ; Modified.......: ; Remarks........: MODIFIED BY IAN MAXWELL FOR THIS SPECIFIC SCRIPT, DO NOT COPY THIS FOR RE-USE!!!! ; Related........: ; Link...........: ; Example........: ; =============================================================================================================== Func _LargeFileCopy($sSrc, $sDest, $iFlags = 0) ; check / fix source and dest path syntax _LFC_FixPaths($sSrc, $sDest) ; open file for reading, fail if it doesn't exist or directory Local $hSrc = _LFC_CreateFile($sSrc, $GENERIC_READ, $FILE_SHARE_READ, $OPEN_EXISTING, 0) If Not $hSrc Then Return SetError(1, 0, 0) ; set option flags Local $fOverwrite = (BitAND($iFlags, 1) = 1) Local $fCreate = (BitAND($iFlags, 2) = 2) ; check destination _LFC_CheckDestination($sSrc, $sDest, $fOverwrite, $fCreate) If @error Then _WinAPI_CloseHandle($hSrc) Return SetError(2, 0, 0) EndIf ; create new file for writing, overwrite Local $hDest = _LFC_CreateFile($sDest, BitOR($GENERIC_READ, $GENERIC_WRITE), 0, $CREATE_ALWAYS, 0) If Not $hDest Then _WinAPI_CloseHandle($hSrc) Return SetError(3, 0, 0) EndIf ; check for 0 byte source file Local $iSize = _WinAPI_GetFileSizeEx($hSrc) If $iSize = 0 Then ; done, close handles and return success _WinAPI_CloseHandle($hDest) _WinAPI_CloseHandle($hSrc) Return 1 EndIf ; perform copy Local $iRead, $iWritten, $iTotal = 0, $mSrc = 0, $iReadError = 0, $iWriteError = 0, $iVerifyError = 0 ; allocate buffers Local $hBuffer = DllStructCreate("byte[" & $iToRead & "]") Local $pBuffer = DllStructGetPtr($hBuffer) Do If Not _WinAPI_ReadFile($hSrc, $pBuffer, $iToRead, $iRead) Then $iReadError = 1 ExitLoop EndIf If $iRead = 0 Then ExitLoop ; end of file, edge case if file is an exact multiple of the buffer size If Not _WinAPI_WriteFile($hDest, $pBuffer, $iRead, $iWritten) Or ($iRead <> $iWritten) Then $iWriteError = 1 ExitLoop EndIf Until $iRead < $iToRead _WinAPI_CloseHandle($hDest) _WinAPI_CloseHandle($hSrc) If $iReadError Then Return SetError(4, 0, 0) ElseIf $iWriteError Then Return SetError(5, 0, 0) Else Return 1 EndIf EndFunc ;==>_LargeFileCopy #region INTERNAL FUNCTIONS Func _LFC_CheckDestination($sSrc, ByRef $sDest, $fOverwrite, $fCreate) If (StringRight($sDest, 1) = "") Or StringInStr(FileGetAttrib($sDest), "D") Then ; assume directory If $sSrc = "" Then ; raw copy, must provide a file name Return SetError(2) Else ; use source file name If StringRight($sDest, 1) <> "" Then $sDest &= "" ; add trailing $sDest &= StringRegExpReplace($sSrc, ".*", "") EndIf EndIf ; check overwrite If (Not $fOverwrite) And FileExists($sDest) Then Return SetError(1) ; create destination parent directory ; if create flag not set and destination doesn't exist, CreateFile will fail If $fCreate Then DirCreate(StringRegExpReplace($sDest, "^(.*).*?$", "${1}")) ;~ DirCreate(StringRegExpReplace($sDest, "^(.*).*?$", "${1}")) EndFunc ;==>_LFC_CheckDestination Func _LFC_CreateFile($sPath, $iAccess, $iShareMode, $iCreation, $iFlags) ; open the file with existing HIDDEN or SYSTEM attributes to avoid failure when using CREATE_ALWAYS If $iCreation = $CREATE_ALWAYS Then Local $sAttrib = FileGetAttrib($sPath) If StringInStr($sAttrib, "H") Then $iFlags = BitOR($iFlags, $FILE_ATTRIBUTE_HIDDEN) If StringInStr($sAttrib, "S") Then $iFlags = BitOR($iFlags, $FILE_ATTRIBUTE_SYSTEM) EndIf Local $hFile = DllCall("kernel32.dll", "handle", "CreateFileW", "wstr", $sPath, "dword", $iAccess, "dword", $iShareMode, "ptr", 0, _ "dword", $iCreation, "dword", $iFlags, "ptr", 0) If @error Or ($hFile[0] = Ptr(-1)) Then Return SetError(1, 0, 0) Return $hFile[0] EndFunc ;==>_LFC_CreateFile Func _LFC_FixPath(ByRef $sPath) ; fix possible forward / $sPath = StringRegExpReplace($sPath, "/", "") If Not StringInStr($sPath, ":") Then ; not a full path ; check for bare, relative, or UNC paths If StringLeft($sPath, 1) = "" Then ; check for UNC path If StringLeft($sPath, 2) <> "" Then ; not a UNC path $sPath = "." & $sPath ; else, probably a UNC path, leave it alone EndIf Else ; check for bare or relative path If StringLeft($sPath, 1) <> "." Then ; bare path $sPath = "." & $sPath ; else, probably relative path, leave it alone EndIf EndIf EndIf EndFunc ;==>_LFC_FixPath Func _LFC_FixPaths(ByRef $sSrc, ByRef $sDest) ; remove trailing 's from source only $sSrc = StringRegExpReplace($sSrc, "+$", "") ; check for relative paths in source and dest _LFC_FixPath($sSrc) _LFC_FixPath($sDest) EndFunc ;==>_LFC_FixPaths #endregion INTERNAL FUNCTIONS Func _FileListToArrayRec($sPath, $sExcludeFolderList = "", $sIncludeList = "*", $sExcludeList = "", $iReturnType = 0, $iReturnFormat = 0, $bRecursive = False) Local $sRet = "", $sReturnFormat = "" $sPath = StringRegExpReplace($sPath, "[/]+z", "") & "" If Not FileExists($sPath) Then Return SetError(1, 1, "") If $sIncludeList = "*" Then $sIncludeList = "" Else If StringRegExp($sIncludeList, "[/ :> <|]|(?s)As*z") Then Return SetError(2, 2, "") $sIncludeList = StringRegExpReplace(StringRegExpReplace($sIncludeList, "(s*;s*)+", ";"), "A;|;z", "") $sIncludeList = StringRegExpReplace($sIncludeList, '[.$]', '[0]') $sIncludeList = StringReplace(StringReplace($sIncludeList, "?", "."), "*", ".*?") $sIncludeList = "(?i)A(" & StringReplace($sIncludeList, ";", "$|") & "$)" EndIf If Not ($iReturnType = 0 Or $iReturnType = 1 Or $iReturnType = 2) Then Return SetError(3, 3, "") Local $sOrigPathLen = StringLen($sPath), $aQueue[64] = [1, $sPath], $iQMax = 63 While $aQueue[0] $WorkFolder = $aQueue[$aQueue[0]] $aQueue[0] -= 1 $search = FileFindFirstFile($WorkFolder & "*") ; get full list of current folder If @error Then ContinueLoop Switch $iReturnFormat Case 1 ; relative path $sReturnFormat = StringTrimLeft($WorkFolder, $sOrigPathLen) Case 2 ; full path $sReturnFormat = $WorkFolder EndSwitch While 1 ; process current folder $file = FileFindNextFile($search) If @error Then ExitLoop If @extended Then ; Folder If $bRecursive Then ; push folder onto queue If $aQueue[0] = $iQMax Then $iQMax += 128 ReDim $aQueue[$iQMax + 1] EndIf $aQueue[0] += 1 $aQueue[$aQueue[0]] = $WorkFolder & $file & "" EndIf If $iReturnType = 1 Then ContinueLoop Else ; File If $iReturnType = 2 Then ContinueLoop EndIf If $sIncludeList And Not StringRegExp($file, $sIncludeList) Then ContinueLoop $sRet &= $sReturnFormat & $file & "|" ; append to output WEnd FileClose($search) WEnd If Not $sRet Then Return SetError(4, 4, "") Return StringSplit(StringTrimRight($sRet, 1), "|") EndFunc ;==>_FileListToArrayRec To test, create the $Source folder and put some stuff in there, couple files and small folders, and let it run, and everything will be fine. Then start to add some subfolders to the $Target and it will fail when it gets too long with @error 3. Any further thoughts on how this can be overcome? Thanks! Ian My projects: IP Scanner - Multi-threaded ping tool to scan your available networks for used and available IP addresses, shows ping times, resolves IPs in to host names, and allows individual IPs to be pinged. INFSniff - Great technicians tool - a tool which scans DriverPacks archives for INF files and parses out the HWIDs to a database file, and rapidly scans the local machine's HWIDs, searches the database for matches, and installs them. PPK3 (Persistent Process Killer V3) - Another for the techs - suppress running processes that you need to keep away, helpful when fighting spyware/viruses. Sync Tool - Folder sync tool with lots of real time information and several checking methods. USMT Front End - Front End for Microsoft's User State Migration Tool, including all files needed for USMT 3.01 and 4.01, 32 bit and 64 bit versions. Audit Tool - Computer audit tool to gather vital hardware, Windows, and Office information for IT managers and field techs. Capabilities include creating a customized site agent. CSV Viewer - Displays CSV files with automatic column sizing and font selection. Lines can also be copied to the clipboard for data extraction. MyDirStat - Lists number and size of files on a drive or specified path, allows for deletion within the app. 2048 Game - My version of 2048, fun tile game. Juice Lab - Ecigarette liquid making calculator. Data Protector - Secure notes to save sensitive information. VHD Footer - Add a footer to a forensic hard drive image to allow it to be mounted or used as a virtual machine hard drive. Find in File - Searches files containing a specified phrase. Link to comment Share on other sites More sharing options...
wraithdu Posted November 20, 2011 Author Share Posted November 20, 2011 I'll probably get around to testing this Monday. Link to comment Share on other sites More sharing options...
llewxam Posted November 20, 2011 Share Posted November 20, 2011 That would be great! I have been poking at this for the last 24 hours and haven't got a clue. All I have found is that newer versions of AutoIt are Unicode compliant, and the CreateFileW you call is the Unicode version, sooooooo......why won't a Unicode path work?????? gggrrrrrrrrrr!!!!! Maybe something unrelated? A SciTE wrapper or compiler option to force Unicode that I haven't been finding? I even tried using #RequireAdmin out of exacerbation..... Thanks a million Ian My projects: IP Scanner - Multi-threaded ping tool to scan your available networks for used and available IP addresses, shows ping times, resolves IPs in to host names, and allows individual IPs to be pinged. INFSniff - Great technicians tool - a tool which scans DriverPacks archives for INF files and parses out the HWIDs to a database file, and rapidly scans the local machine's HWIDs, searches the database for matches, and installs them. PPK3 (Persistent Process Killer V3) - Another for the techs - suppress running processes that you need to keep away, helpful when fighting spyware/viruses. Sync Tool - Folder sync tool with lots of real time information and several checking methods. USMT Front End - Front End for Microsoft's User State Migration Tool, including all files needed for USMT 3.01 and 4.01, 32 bit and 64 bit versions. Audit Tool - Computer audit tool to gather vital hardware, Windows, and Office information for IT managers and field techs. Capabilities include creating a customized site agent. CSV Viewer - Displays CSV files with automatic column sizing and font selection. Lines can also be copied to the clipboard for data extraction. MyDirStat - Lists number and size of files on a drive or specified path, allows for deletion within the app. 2048 Game - My version of 2048, fun tile game. Juice Lab - Ecigarette liquid making calculator. Data Protector - Secure notes to save sensitive information. VHD Footer - Add a footer to a forensic hard drive image to allow it to be mounted or used as a virtual machine hard drive. Find in File - Searches files containing a specified phrase. Link to comment Share on other sites More sharing options...
wraithdu Posted November 20, 2011 Author Share Posted November 20, 2011 Well that didn't take long to figure out. I should probably open a ticket for this too. AutoIt's internal DirCreate is reporting success but is truncating the new directory to 255 characters. CreateFile obviously fails if the parent directory does not exist, so there ya go. I'll need to replace the call(s) to DirCreate with the CreateDirectory API which works with UNC style extra long paths. Link to comment Share on other sites More sharing options...
wraithdu Posted November 20, 2011 Author Share Posted November 20, 2011 Hmm, gonna take a little while longer. CreateDirectory does not work recursively, so I have to design that, including a function to crack all the possible path types. Link to comment Share on other sites More sharing options...
llewxam Posted November 20, 2011 Share Posted November 20, 2011 Sounds like a PITA, but yours will be the only routine out there that can do it, I tested Yashied's UDF and it bonked as well. Thought - maybe your revised UDF should internally convert to UNC and just get it out of the way, Seems like you going through all the effort of making it CAPABLE of handling a Long UNC path doesn't make a lot of sense if it doesn't actually do it.... If you do or don't add that feature, it's still awesome that you are making progress. Ian My projects: IP Scanner - Multi-threaded ping tool to scan your available networks for used and available IP addresses, shows ping times, resolves IPs in to host names, and allows individual IPs to be pinged. INFSniff - Great technicians tool - a tool which scans DriverPacks archives for INF files and parses out the HWIDs to a database file, and rapidly scans the local machine's HWIDs, searches the database for matches, and installs them. PPK3 (Persistent Process Killer V3) - Another for the techs - suppress running processes that you need to keep away, helpful when fighting spyware/viruses. Sync Tool - Folder sync tool with lots of real time information and several checking methods. USMT Front End - Front End for Microsoft's User State Migration Tool, including all files needed for USMT 3.01 and 4.01, 32 bit and 64 bit versions. Audit Tool - Computer audit tool to gather vital hardware, Windows, and Office information for IT managers and field techs. Capabilities include creating a customized site agent. CSV Viewer - Displays CSV files with automatic column sizing and font selection. Lines can also be copied to the clipboard for data extraction. MyDirStat - Lists number and size of files on a drive or specified path, allows for deletion within the app. 2048 Game - My version of 2048, fun tile game. Juice Lab - Ecigarette liquid making calculator. Data Protector - Secure notes to save sensitive information. VHD Footer - Add a footer to a forensic hard drive image to allow it to be mounted or used as a virtual machine hard drive. Find in File - Searches files containing a specified phrase. Link to comment Share on other sites More sharing options...
wraithdu Posted November 21, 2011 Author Share Posted November 21, 2011 (edited) Thought - maybe your revised UDF should internally convert to UNC and just get it out of the wayI'm deciding if I want to do that or not. My current thought is that Windows doesn't do that internally, so I shouldn't either. It should be the scripter's responsibility to check path length. Also, if I do it for directory creation, then I should do it for CreateFile as well, and that seems out of scope. Edited November 21, 2011 by wraithdu Link to comment Share on other sites More sharing options...
wraithdu Posted November 21, 2011 Author Share Posted November 21, 2011 (edited) Ok, try this updated version of your example. Please review the new functions to see if anything jumps out at you as being incorrect. expandcollapse popup#include <WinAPI.au3> #include <Array.au3> $iToRead = 8 * 1024 * 1024 $Source = @ScriptDir & "testdir" $Target = @ScriptDir & "asiodfoiwoiawoiefioawejfaawoeifoafjjfejiwaoaiwefjoawiefaoiawoefjofijijewoaoawejfjaw" DirRemove($Target, 1) If StringRight($Source, 1) <> "" Then $Source &= "" If StringRight($Target, 1) <> "" Then $Target &= "" $SourceLength = StringLen($Source) If $SourceLength == 3 Then $SourceLength -= 1; gotta have the array elements start with a backslash... $FileList = _FileListToArrayRec($Source, "", "*", "", 1, 2, True) If StringLeft($Source, 2) == "" Then $SourceUNCPrefix = "?UNC" $Source = StringTrimLeft($Source, 2) Else $SourceUNCPrefix = "?" EndIf If StringLeft($Target, 2) == "" Then $TargetUNCPrefix = "?UNC" $Target = StringTrimLeft($Target, 2) Else $TargetUNCPrefix = "?" EndIf _ArrayTrim($FileList, $SourceLength, 0, 1) For $a = 1 To $FileList[0] $CopySource = $SourceUNCPrefix & $Source & $FileList[$a] $CopyTarget = $TargetUNCPrefix & $Target & $FileList[$a] ConsoleWrite(StringLen($CopySource) & " : " & StringLen($CopyTarget) & @CRLF) $Copy = _LargeFileCopy($CopySource, $CopyTarget, BitOR(1, 2)) $Error = @error If $Copy == 1 Then $Message = "SUCCESS" Else $Message = $Error EndIf ConsoleWrite($CopySource & " -> " & $CopyTarget & " -> " & $Message & @CR) Next ; #FUNCTION# ==================================================================================================== ; Name...........: _LargeFileCopy ; Description....: Copy large files in such a way as to keep AutoIt GUIs responsive ; Syntax.........: _LargeFileCopy($sSrc, $sDest, $iFlags) ; Parameters.....: $sSrc - Source file name ; $sDest - Destination: may be a file name or directory ; $iFlags - [Optional] Combine flags with BitOR ; | 1 - Overwrite existing file ; | 2 - Create destination directory structure ; | 4 - Flush the destination file buffer before returning ; | 8 - Verify source and destination are identical via bit by bit comparison ; |16 - Verify source and destination are identical via MD5 hash ; |32 - Verify source and destination file size only ; + If more than one verify flag is set, the smallest flag will take precedence ; ; Return values..: Success - 1 ; Failure - 0 and sets @error ; | 1 - Failed to open source file, or source was a directory ; | 2 - Destination file exists and overwrite flag not set ; | 3 - Failed to create destination file ; | 4 - Read error during copy ; | 5 - Write error during copy ; | 6 - Verify failed ; Author.........: Erik Pilsits ; Modified.......: ; Remarks........: MODIFIED BY IAN MAXWELL FOR THIS SPECIFIC SCRIPT, DO NOT COPY THIS FOR RE-USE!!!! ; Related........: ; Link...........: ; Example........: ; =============================================================================================================== Func _LargeFileCopy($sSrc, $sDest, $iFlags = 0) ; check / fix source and dest path syntax _LFC_FixPaths($sSrc, $sDest) ; open file for reading, fail if it doesn't exist or directory Local $hSrc = _LFC_CreateFile($sSrc, $GENERIC_READ, $FILE_SHARE_READ, $OPEN_EXISTING, 0) If Not $hSrc Then Return SetError(1, 0, 0) ; set option flags Local $fOverwrite = (BitAND($iFlags, 1) = 1) Local $fCreate = (BitAND($iFlags, 2) = 2) ; check destination _LFC_CheckDestination($sSrc, $sDest, $fOverwrite, $fCreate) If @error Then _WinAPI_CloseHandle($hSrc) Return SetError(2, 0, 0) EndIf ; create new file for writing, overwrite Local $hDest = _LFC_CreateFile($sDest, BitOR($GENERIC_READ, $GENERIC_WRITE), 0, $CREATE_ALWAYS, 0) If Not $hDest Then _WinAPI_CloseHandle($hSrc) Return SetError(3, 0, 0) EndIf ; check for 0 byte source file Local $iSize = _WinAPI_GetFileSizeEx($hSrc) If $iSize = 0 Then ; done, close handles and return success _WinAPI_CloseHandle($hDest) _WinAPI_CloseHandle($hSrc) Return 1 EndIf ; perform copy Local $iRead, $iWritten, $iTotal = 0, $mSrc = 0, $iReadError = 0, $iWriteError = 0, $iVerifyError = 0 ; allocate buffers Local $hBuffer = DllStructCreate("byte[" & $iToRead & "]") Local $pBuffer = DllStructGetPtr($hBuffer) Do If Not _WinAPI_ReadFile($hSrc, $pBuffer, $iToRead, $iRead) Then $iReadError = 1 ExitLoop EndIf If $iRead = 0 Then ExitLoop ; end of file, edge case if file is an exact multiple of the buffer size If Not _WinAPI_WriteFile($hDest, $pBuffer, $iRead, $iWritten) Or ($iRead <> $iWritten) Then $iWriteError = 1 ExitLoop EndIf Until $iRead < $iToRead _WinAPI_CloseHandle($hDest) _WinAPI_CloseHandle($hSrc) If $iReadError Then Return SetError(4, 0, 0) ElseIf $iWriteError Then Return SetError(5, 0, 0) Else Return 1 EndIf EndFunc ;==>_LargeFileCopy #region INTERNAL FUNCTIONS Func _LFC_CheckDestination($sSrc, ByRef $sDest, $fOverwrite, $fCreate) If (StringRight($sDest, 1) = "") Or StringInStr(FileGetAttrib($sDest), "D") Then ; assume directory If $sSrc = "" Then ; raw copy, must provide a file name Return SetError(2) Else ; use source file name If StringRight($sDest, 1) <> "" Then $sDest &= "" ; add trailing $sDest &= StringRegExpReplace($sSrc, ".*", "") EndIf EndIf ; check overwrite If (Not $fOverwrite) And FileExists($sDest) Then Return SetError(1) ; create destination parent directory ; if create flag not set and destination doesn't exist, CreateFile will fail If $fCreate Then _LFC_CreateDirectory(StringRegExpReplace($sDest, "^(.*).*?$", "${1}")) EndFunc ;==>_LFC_CheckDestination Func _LFC_CrackPath($sPath) ; make sure path is terminated If StringRight($sPath, 1) <> "" Then $sPath &= "" ; get prefix Local $aRet = StringRegExp($sPath, "(?i)^?unc|?|", 1) If Not IsArray($aRet) Then $aRet = "" Else $aRet = $aRet[0] EndIf ; capture and remove the root If ($aRet = "") Or ($aRet = "?UNC") Then ; UNC network path $aRet = StringRegExp($sPath, "(?i)^(" & StringReplace(StringReplace($aRet, "", ""), "?", "?") & ".*?.*?)", 1) Else ; $aRet = "" or ? => local path Local $iTrim = StringLen($aRet) + 3 Local $aRet[1] = [StringLeft($sPath, $iTrim)] EndIf Local $aPath = StringTrimLeft($sPath, StringLen($aRet[0])) ; check if path given was just a root If $aPath <> "" Then ; crack path, prepend to get first segment $aPath = StringRegExp("" & $aPath, "(.*?)(?=)", 3) ReDim $aRet[UBound($aPath) + 1] For $i = 0 To UBound($aPath) - 1 $aRet[$i + 1] = $aPath[$i] Next EndIf ; Return $aRet EndFunc Func _LFC_CreateDirectory($sPath) ; check path already exists If FileExists($sPath) Then ; check if it is a direcotry If StringInStr(FileGetAttrib($sPath), "D") Then Return 1 Else Return SetError(1, 0, 0) EndIf EndIf ; Local $aPath = _LFC_CrackPath($sPath) ; make sure root exists Local $sSegment = $aPath[0] If Not FileExists($sSegment) Then Return SetError(2, 0, 0) Local $ret For $i = 1 To UBound($aPath) - 1 $sSegment &= $aPath[$i] & "" If Not FileExists($sSegment) Then ; create sub-path $ret = DllCall("kernel32.dll", "bool", "CreateDirectoryW", "wstr", $sSegment, "ptr", 0) If @error Or Not $ret[0] Then Return SetError(3, 0, 0) EndIf Next Return 1 EndFunc Func _LFC_CreateFile($sPath, $iAccess, $iShareMode, $iCreation, $iFlags) ; open the file with existing HIDDEN or SYSTEM attributes to avoid failure when using CREATE_ALWAYS If $iCreation = $CREATE_ALWAYS Then Local $sAttrib = FileGetAttrib($sPath) If StringInStr($sAttrib, "H") Then $iFlags = BitOR($iFlags, $FILE_ATTRIBUTE_HIDDEN) If StringInStr($sAttrib, "S") Then $iFlags = BitOR($iFlags, $FILE_ATTRIBUTE_SYSTEM) EndIf Local $hFile = DllCall("kernel32.dll", "handle", "CreateFileW", "wstr", $sPath, "dword", $iAccess, "dword", $iShareMode, "ptr", 0, _ "dword", $iCreation, "dword", $iFlags, "ptr", 0) If @error Or ($hFile[0] = Ptr(-1)) Then ConsoleWrite("ERR: " & $hFile[0] & " : " & _WinAPI_GetLastErrorMessage()) ConsoleWrite($sPath & @CRLF) Return SetError(1, 0, 0) EndIf Return $hFile[0] EndFunc ;==>_LFC_CreateFile Func _LFC_FixPath(ByRef $sPath) ; fix possible forward / $sPath = StringRegExpReplace($sPath, "/", "") If Not StringInStr($sPath, ":") Then ; not a full path ; check for bare, relative, or UNC paths If StringLeft($sPath, 1) = "" Then ; check for UNC path If StringLeft($sPath, 2) <> "" Then ; not a UNC path $sPath = "." & $sPath ; else, probably a UNC path, leave it alone EndIf Else ; check for bare or relative path If StringLeft($sPath, 1) <> "." Then ; bare path $sPath = "." & $sPath ; else, probably relative path, leave it alone EndIf EndIf EndIf EndFunc ;==>_LFC_FixPath Func _LFC_FixPaths(ByRef $sSrc, ByRef $sDest) ; remove trailing 's from source only $sSrc = StringRegExpReplace($sSrc, "+$", "") ; check for relative paths in source and dest _LFC_FixPath($sSrc) _LFC_FixPath($sDest) EndFunc ;==>_LFC_FixPaths #endregion INTERNAL FUNCTIONS Func _FileListToArrayRec($sPath, $sExcludeFolderList = "", $sIncludeList = "*", $sExcludeList = "", $iReturnType = 0, $iReturnFormat = 0, $bRecursive = False) Local $sRet = "", $sReturnFormat = "" $sPath = StringRegExpReplace($sPath, "[/]+z", "") & "" If Not FileExists($sPath) Then Return SetError(1, 1, "") If $sIncludeList = "*" Then $sIncludeList = "" Else If StringRegExp($sIncludeList, "[/ :> <|]|(?s)As*z") Then Return SetError(2, 2, "") $sIncludeList = StringRegExpReplace(StringRegExpReplace($sIncludeList, "(s*;s*)+", ";"), "A;|;z", "") $sIncludeList = StringRegExpReplace($sIncludeList, '[.$]', '[0]') $sIncludeList = StringReplace(StringReplace($sIncludeList, "?", "."), "*", ".*?") $sIncludeList = "(?i)A(" & StringReplace($sIncludeList, ";", "$|") & "$)" EndIf If Not ($iReturnType = 0 Or $iReturnType = 1 Or $iReturnType = 2) Then Return SetError(3, 3, "") Local $sOrigPathLen = StringLen($sPath), $aQueue[64] = [1, $sPath], $iQMax = 63 While $aQueue[0] $WorkFolder = $aQueue[$aQueue[0]] $aQueue[0] -= 1 $search = FileFindFirstFile($WorkFolder & "*") ; get full list of current folder If @error Then ContinueLoop Switch $iReturnFormat Case 1 ; relative path $sReturnFormat = StringTrimLeft($WorkFolder, $sOrigPathLen) Case 2 ; full path $sReturnFormat = $WorkFolder EndSwitch While 1 ; process current folder $file = FileFindNextFile($search) If @error Then ExitLoop If @extended Then ; Folder If $bRecursive Then ; push folder onto queue If $aQueue[0] = $iQMax Then $iQMax += 128 ReDim $aQueue[$iQMax + 1] EndIf $aQueue[0] += 1 $aQueue[$aQueue[0]] = $WorkFolder & $file & "" EndIf If $iReturnType = 1 Then ContinueLoop Else ; File If $iReturnType = 2 Then ContinueLoop EndIf If $sIncludeList And Not StringRegExp($file, $sIncludeList) Then ContinueLoop $sRet &= $sReturnFormat & $file & "|" ; append to output WEnd FileClose($search) WEnd If Not $sRet Then Return SetError(4, 4, "") Return StringSplit(StringTrimRight($sRet, 1), "|") EndFunc ;==>_FileListToArrayRec Edit: I've created ticket #2054 for this issue. Edited November 21, 2011 by wraithdu Link to comment Share on other sites More sharing options...
llewxam Posted November 21, 2011 Share Posted November 21, 2011 NICE MAN!! I just ran a test and got a path length of 452 and it was all good! haha, right on! And the new FileListToArrayRec that I have been messing around with did NOT find any files in that huge path, but adding the UNC prefix made it able to find the files, so no horrible mess appears to be needed for that to be happy! GOOD JOB! Ian My projects: IP Scanner - Multi-threaded ping tool to scan your available networks for used and available IP addresses, shows ping times, resolves IPs in to host names, and allows individual IPs to be pinged. INFSniff - Great technicians tool - a tool which scans DriverPacks archives for INF files and parses out the HWIDs to a database file, and rapidly scans the local machine's HWIDs, searches the database for matches, and installs them. PPK3 (Persistent Process Killer V3) - Another for the techs - suppress running processes that you need to keep away, helpful when fighting spyware/viruses. Sync Tool - Folder sync tool with lots of real time information and several checking methods. USMT Front End - Front End for Microsoft's User State Migration Tool, including all files needed for USMT 3.01 and 4.01, 32 bit and 64 bit versions. Audit Tool - Computer audit tool to gather vital hardware, Windows, and Office information for IT managers and field techs. Capabilities include creating a customized site agent. CSV Viewer - Displays CSV files with automatic column sizing and font selection. Lines can also be copied to the clipboard for data extraction. MyDirStat - Lists number and size of files on a drive or specified path, allows for deletion within the app. 2048 Game - My version of 2048, fun tile game. Juice Lab - Ecigarette liquid making calculator. Data Protector - Secure notes to save sensitive information. VHD Footer - Add a footer to a forensic hard drive image to allow it to be mounted or used as a virtual machine hard drive. Find in File - Searches files containing a specified phrase. Link to comment Share on other sites More sharing options...
wraithdu Posted November 21, 2011 Author Share Posted November 21, 2011 Cool Please let me know if you find any problems or edge cases. I'll update the main UDF soon if you do not. Link to comment Share on other sites More sharing options...
islandspapand Posted May 24, 2012 Share Posted May 24, 2012 thanks for a great UDF. a quick noob question how do i make a progressbar-copy.. i have tried a few thing but i do not totaly under stand the function paramter calls that i have tried eksample: #include <Process.au3> #include <_Largefilecopy.au3> #include <Array.au3> Local $Test, $FileCopy, $1, $2,$iWritten, $iTotal,$vVar ; $iTotal=FileGetSize("E:ZS7-168ASM4-KP0_wo3.1.0.0.WIM") ;$iWritten=FileGetSize("E:MSIZS7-168ASM4-KP0_wo3.1.0.0.WIM") $FileCopy=_LargeFileCopy("E:ZS7-168ASM4-KP0_wo3.1.0.0.WIM", "E:MSIZS7-168ASM4-KP0_wo3.1.0.0.WIM", 1,60000,"",test($iWritten, $iTotal)) ConsoleWrite("1 : "&$iWritten&@CR) ConsoleWrite("2 : "&$iTotal&@CR) If $FileCopy=0 Then MsgBox(0,"Copy","Fail to Copy") If $FileCopy=1 Then MsgBox(0,"Copy","Copy'ed Your file") Func test($iWritten, $iTotal) $iTotal=FileGetSize("E:ZS7-168ASM4-KP0_wo3.1.0.0.WIM") $iWritten=FileGetSize("E:MSIZS7-168ASM4-KP0_wo3.1.0.0.WIM") ConsoleWrite("1 : "&$iWritten&@CR) ConsoleWrite("2 : "&$iTotal&@CR) EndFunc hope somebody can help. Thanks in advance Link to comment Share on other sites More sharing options...
wraithdu Posted May 24, 2012 Author Share Posted May 24, 2012 The progress function is automatically called with the 3 parameters described in the function header. You need to change your test function to take all three parameters, even if you don't intend to use them. Then in your call to _LargeFileCopy pass only the name of your progress function as a string, ie "test". Link to comment Share on other sites More sharing options...
Topher Posted May 24, 2012 Share Posted May 24, 2012 Just what I needed. Thank you. [left][hr] $mood = "whimsy" $mode = "confused" $randomChaos = True Do Something() Until $Tired[/left][left]Reflex (Arcade game)[/left][left]IX (Board game)[/left][left]The Word Game (Word game)[/left][left]Plastic Sliding Squares Puzzle (Puzzle)[/left] Link to comment Share on other sites More sharing options...
Syed23 Posted June 10, 2012 Share Posted June 10, 2012 Thanks for this Extradinary UDF... i have a quick question. is there a way to copy the entire folder by using this function? i am following the below task where it fails everytime.. any reason? #include<Array.au3> #include <_LargeFileCopy.au3> Global $coppst[2] $coppst[0] = "c:exchange" $coppst[1] = "C:UsersQ03200AppDataLocalMicrosoftOutlook" For $s = 0 To UBound($coppst)-1 Step +1 ;$m = DirCopy($coppst[$s], "C:backup"&StringTrimLeft($coppst[$s],2),1) ; MsgBox(0, "", $coppst[$s]&"*.* "& "C:backup" & StringTrimLeft($coppst[$s], 2)&"*.psst") $m = _LargeFileCopy($coppst[$s]&"*.*", "C:backup" & StringTrimLeft($coppst[$s], 2)&"*.*", BitOR(2,8)) MsgBox(0,"",$m) Next Thank you,Regards,[font="Garamond"][size="4"]K.Syed Ibrahim.[/size][/font] Link to comment Share on other sites More sharing options...
wraithdu Posted June 25, 2012 Author Share Posted June 25, 2012 (edited) Quick answer... no, this UDF does not natively support recursive copying, and it's not going to. It also does not support use of wildcards. You'll have to implement your own search/loop/copy algorithms. Edited June 25, 2012 by wraithdu Link to comment Share on other sites More sharing options...
Syed23 Posted August 3, 2012 Share Posted August 3, 2012 Quick answer... no, this UDF does not natively support recursive copying, and it's not going to.It also does not support use of wildcards. You'll have to implement your own search/loop/copy algorithms.Thanks Wraithdu! may i know what is the methodology we are using for the copy process? because what i could feel is when i use this function to copy file this is bit faster than normal process. can you give me an overview of the technique you have used in this function? how it is copying in this speed? Thank you,Regards,[font="Garamond"][size="4"]K.Syed Ibrahim.[/size][/font] Link to comment Share on other sites More sharing options...
llewxam Posted August 4, 2012 Share Posted August 4, 2012 Syed23 If you look over the code in this UDF you will see that wraithdu is using WinAPI calls. There are more lines of code in using this method since wraithdu's UDF has to specify the position in the file to open, as well as position to write in the destination, additional error checking, handling UNC paths, etc. but his code is proof that well-written and optimized code can be far faster than fewer lines of poor code. hmm, maybe it is just easier to say that using Windows internal API calls will result in faster code Ian My projects: IP Scanner - Multi-threaded ping tool to scan your available networks for used and available IP addresses, shows ping times, resolves IPs in to host names, and allows individual IPs to be pinged. INFSniff - Great technicians tool - a tool which scans DriverPacks archives for INF files and parses out the HWIDs to a database file, and rapidly scans the local machine's HWIDs, searches the database for matches, and installs them. PPK3 (Persistent Process Killer V3) - Another for the techs - suppress running processes that you need to keep away, helpful when fighting spyware/viruses. Sync Tool - Folder sync tool with lots of real time information and several checking methods. USMT Front End - Front End for Microsoft's User State Migration Tool, including all files needed for USMT 3.01 and 4.01, 32 bit and 64 bit versions. Audit Tool - Computer audit tool to gather vital hardware, Windows, and Office information for IT managers and field techs. Capabilities include creating a customized site agent. CSV Viewer - Displays CSV files with automatic column sizing and font selection. Lines can also be copied to the clipboard for data extraction. MyDirStat - Lists number and size of files on a drive or specified path, allows for deletion within the app. 2048 Game - My version of 2048, fun tile game. Juice Lab - Ecigarette liquid making calculator. Data Protector - Secure notes to save sensitive information. VHD Footer - Add a footer to a forensic hard drive image to allow it to be mounted or used as a virtual machine hard drive. Find in File - Searches files containing a specified phrase. Link to comment Share on other sites More sharing options...
Syed23 Posted September 24, 2012 Share Posted September 24, 2012 Just curious, I have used this function in my GUI and it works perfect, but in some cases when copy bit huge size of file the GUI shows not responding and the marquee progress bar freezes.. could you please help me to over come this? if you have any suggestion how to trouble shoot could you please tell me that so that i can give you the exact scenario where i am facing that problem. I am using this function in Windows 7 64 bit machines. Thanks in advance! Thank you,Regards,[font="Garamond"][size="4"]K.Syed Ibrahim.[/size][/font] Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now