tip Posted February 7, 2012 Share Posted February 7, 2012 (edited) Hi everyone; I don't mean to create a new thread about a subject that has been already widely(wildly) discussed but after reading many many posts I still couldn't figure out which method(function) to prefer... I gathered all the recursive FileListToArray functions I could find in the forums and made a small test package. If you could run the test script and post your results, I think we can create a modest comparison database on how these funcitons work on different Windows versions with different languages. I'm not an experienced scripter and I don't entirely understand the methods/routines used in these function. But I also know that the fastest function is not always the best solution. So I would also like to hear what you have to say about these functions too... Reasons to avoid them or why you would prefer one over the other... For example, I've read that the dos dir command is better not to be used because of its non-english character shotcommings etc... I think it would be really cool if we could find a definitive answer on this subject. My results: OS:WIN_XP Service Pack 3, OS Arch:X86, Language:0409, AutoIt: 3.3.8.1 Dos dir command returned 24296 items in 1.16 seconds. FileListToArrayRecursive returned 24304 items in 0.9 seconds. FileListToArray3 returned 24304 items in 1.52 seconds. RecFileListToArray returned 24304 items in 1.32 seconds. FileListToArrayEx returned 24304 items in 0.8 seconds. FileListToArrayPlus returned 24304 items in 0.71 seconds. FileListTreeToArray returned 24304 items in 2.83 seconds. Sincerely, TipFileListToArray Speed Test.zip Edited February 7, 2012 by tip [center]MsgBox_Tipped: Eye candy msgboxes/inputboxes/loginboxes. | CreateBlankBox: Semi-transparent layers with borders and rounded corners.[/center] Link to comment Share on other sites More sharing options...
BrewManNH Posted February 7, 2012 Share Posted February 7, 2012 Your _RecFileListToArray test is skewed in that you're using the sorting feature fo the function which will slow it down considerably, but you're not sorting with the other tests. Without the sorting these are the results that I get.+>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-> OS:WIN_7 Service Pack 1, OS Arch:X86, Language:0409-> Dos command returned 27303 items in 1 seconds.-> FileListToArrayRecursive returned 27322 items in 2.93 seconds.-> FileListToArray3 returned 27322 items in 5.81 seconds.-> RecFileListToArray returned 27322 items in 1 seconds.-> FileListTreeToArray returned 27322 items in 2.36 seconds.+>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag GudeHow to ask questions the smart way! I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from. Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays. - ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script. - Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label. - _FileGetProperty - Retrieve the properties of a file - SciTE Toolbar - A toolbar demo for use with the SciTE editor - GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI. - Latin Square password generator Link to comment Share on other sites More sharing options...
guinness Posted February 7, 2012 Share Posted February 7, 2012 Without a doubt RecFileListToArray is a very powerful UDF, compared to the rest I've seen. UDF List: _AdapterConnections() • _AlwaysRun() • _AppMon() • _AppMonEx() • _ArrayFilter/_ArrayReduce • _BinaryBin() • _CheckMsgBox() • _CmdLineRaw() • _ContextMenu() • _ConvertLHWebColor()/_ConvertSHWebColor() • _DesktopDimensions() • _DisplayPassword() • _DotNet_Load()/_DotNet_Unload() • _Fibonacci() • _FileCompare() • _FileCompareContents() • _FileNameByHandle() • _FilePrefix/SRE() • _FindInFile() • _GetBackgroundColor()/_SetBackgroundColor() • _GetConrolID() • _GetCtrlClass() • _GetDirectoryFormat() • _GetDriveMediaType() • _GetFilename()/_GetFilenameExt() • _GetHardwareID() • _GetIP() • _GetIP_Country() • _GetOSLanguage() • _GetSavedSource() • _GetStringSize() • _GetSystemPaths() • _GetURLImage() • _GIFImage() • _GoogleWeather() • _GUICtrlCreateGroup() • _GUICtrlListBox_CreateArray() • _GUICtrlListView_CreateArray() • _GUICtrlListView_SaveCSV() • _GUICtrlListView_SaveHTML() • _GUICtrlListView_SaveTxt() • _GUICtrlListView_SaveXML() • _GUICtrlMenu_Recent() • _GUICtrlMenu_SetItemImage() • _GUICtrlTreeView_CreateArray() • _GUIDisable() • _GUIImageList_SetIconFromHandle() • _GUIRegisterMsg() • _GUISetIcon() • _Icon_Clear()/_Icon_Set() • _IdleTime() • _InetGet() • _InetGetGUI() • _InetGetProgress() • _IPDetails() • _IsFileOlder() • _IsGUID() • _IsHex() • _IsPalindrome() • _IsRegKey() • _IsStringRegExp() • _IsSystemDrive() • _IsUPX() • _IsValidType() • _IsWebColor() • _Language() • _Log() • _MicrosoftInternetConnectivity() • _MSDNDataType() • _PathFull/GetRelative/Split() • _PathSplitEx() • _PrintFromArray() • _ProgressSetMarquee() • _ReDim() • _RockPaperScissors()/_RockPaperScissorsLizardSpock() • _ScrollingCredits • _SelfDelete() • _SelfRename() • _SelfUpdate() • _SendTo() • _ShellAll() • _ShellFile() • _ShellFolder() • _SingletonHWID() • _SingletonPID() • _Startup() • _StringCompact() • _StringIsValid() • _StringRegExpMetaCharacters() • _StringReplaceWholeWord() • _StringStripChars() • _Temperature() • _TrialPeriod() • _UKToUSDate()/_USToUKDate() • _WinAPI_Create_CTL_CODE() • _WinAPI_CreateGUID() • _WMIDateStringToDate()/_DateToWMIDateString() • Au3 script parsing • AutoIt Search • AutoIt3 Portable • AutoIt3WrapperToPragma • AutoItWinGetTitle()/AutoItWinSetTitle() • Coding • DirToHTML5 • FileInstallr • FileReadLastChars() • GeoIP database • GUI - Only Close Button • GUI Examples • GUICtrlDeleteImage() • GUICtrlGetBkColor() • GUICtrlGetStyle() • GUIEvents • GUIGetBkColor() • Int_Parse() & Int_TryParse() • IsISBN() • LockFile() • Mapping CtrlIDs • OOP in AutoIt • ParseHeadersToSciTE() • PasswordValid • PasteBin • Posts Per Day • PreExpand • Protect Globals • Queue() • Resource Update • ResourcesEx • SciTE Jump • Settings INI • SHELLHOOK • Shunting-Yard • Signature Creator • Stack() • Stopwatch() • StringAddLF()/StringStripLF() • StringEOLToCRLF() • VSCROLL • WM_COPYDATA • More Examples... Updated: 22/04/2018 Link to comment Share on other sites More sharing options...
BrewManNH Posted February 7, 2012 Share Posted February 7, 2012 Forgot to mention, as guiness stated, RecFileListToArray is also hands down more versatile because of it's search features and filters. If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag GudeHow to ask questions the smart way! I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from. Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays. - ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script. - Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label. - _FileGetProperty - Retrieve the properties of a file - SciTE Toolbar - A toolbar demo for use with the SciTE editor - GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI. - Latin Square password generator Link to comment Share on other sites More sharing options...
guinness Posted February 7, 2012 Share Posted February 7, 2012 Surely this concludes the discussion then? UDF List: _AdapterConnections() • _AlwaysRun() • _AppMon() • _AppMonEx() • _ArrayFilter/_ArrayReduce • _BinaryBin() • _CheckMsgBox() • _CmdLineRaw() • _ContextMenu() • _ConvertLHWebColor()/_ConvertSHWebColor() • _DesktopDimensions() • _DisplayPassword() • _DotNet_Load()/_DotNet_Unload() • _Fibonacci() • _FileCompare() • _FileCompareContents() • _FileNameByHandle() • _FilePrefix/SRE() • _FindInFile() • _GetBackgroundColor()/_SetBackgroundColor() • _GetConrolID() • _GetCtrlClass() • _GetDirectoryFormat() • _GetDriveMediaType() • _GetFilename()/_GetFilenameExt() • _GetHardwareID() • _GetIP() • _GetIP_Country() • _GetOSLanguage() • _GetSavedSource() • _GetStringSize() • _GetSystemPaths() • _GetURLImage() • _GIFImage() • _GoogleWeather() • _GUICtrlCreateGroup() • _GUICtrlListBox_CreateArray() • _GUICtrlListView_CreateArray() • _GUICtrlListView_SaveCSV() • _GUICtrlListView_SaveHTML() • _GUICtrlListView_SaveTxt() • _GUICtrlListView_SaveXML() • _GUICtrlMenu_Recent() • _GUICtrlMenu_SetItemImage() • _GUICtrlTreeView_CreateArray() • _GUIDisable() • _GUIImageList_SetIconFromHandle() • _GUIRegisterMsg() • _GUISetIcon() • _Icon_Clear()/_Icon_Set() • _IdleTime() • _InetGet() • _InetGetGUI() • _InetGetProgress() • _IPDetails() • _IsFileOlder() • _IsGUID() • _IsHex() • _IsPalindrome() • _IsRegKey() • _IsStringRegExp() • _IsSystemDrive() • _IsUPX() • _IsValidType() • _IsWebColor() • _Language() • _Log() • _MicrosoftInternetConnectivity() • _MSDNDataType() • _PathFull/GetRelative/Split() • _PathSplitEx() • _PrintFromArray() • _ProgressSetMarquee() • _ReDim() • _RockPaperScissors()/_RockPaperScissorsLizardSpock() • _ScrollingCredits • _SelfDelete() • _SelfRename() • _SelfUpdate() • _SendTo() • _ShellAll() • _ShellFile() • _ShellFolder() • _SingletonHWID() • _SingletonPID() • _Startup() • _StringCompact() • _StringIsValid() • _StringRegExpMetaCharacters() • _StringReplaceWholeWord() • _StringStripChars() • _Temperature() • _TrialPeriod() • _UKToUSDate()/_USToUKDate() • _WinAPI_Create_CTL_CODE() • _WinAPI_CreateGUID() • _WMIDateStringToDate()/_DateToWMIDateString() • Au3 script parsing • AutoIt Search • AutoIt3 Portable • AutoIt3WrapperToPragma • AutoItWinGetTitle()/AutoItWinSetTitle() • Coding • DirToHTML5 • FileInstallr • FileReadLastChars() • GeoIP database • GUI - Only Close Button • GUI Examples • GUICtrlDeleteImage() • GUICtrlGetBkColor() • GUICtrlGetStyle() • GUIEvents • GUIGetBkColor() • Int_Parse() & Int_TryParse() • IsISBN() • LockFile() • Mapping CtrlIDs • OOP in AutoIt • ParseHeadersToSciTE() • PasswordValid • PasteBin • Posts Per Day • PreExpand • Protect Globals • Queue() • Resource Update • ResourcesEx • SciTE Jump • Settings INI • SHELLHOOK • Shunting-Yard • Signature Creator • Stack() • Stopwatch() • StringAddLF()/StringStripLF() • StringEOLToCRLF() • VSCROLL • WM_COPYDATA • More Examples... Updated: 22/04/2018 Link to comment Share on other sites More sharing options...
tip Posted February 7, 2012 Author Share Posted February 7, 2012 Your _RecFileListToArray test is skewed in that you're using the sorting feature fo the function which will slow it down considerably, but you're not sorting with the other tests.I didn't notice that. Thank you for pointing it out. I've fixed the test file and updated my results...Thank you both for quick replies. Is it normal that I have FileListToArrayRecursive function as the fastest with FileListToArray3 coming second and BrewManNH has those functions as the two slowest of all? [center]MsgBox_Tipped: Eye candy msgboxes/inputboxes/loginboxes. | CreateBlankBox: Semi-transparent layers with borders and rounded corners.[/center] Link to comment Share on other sites More sharing options...
DXRW4E Posted February 7, 2012 Share Posted February 7, 2012 Without a doubt RecFileListToArray is a very powerful UDF, compared to the rest I've seen.+>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-> OS:WIN_8 , OS Arch:X64, Language:0409-> Dos command returned 9416 items in 0.73 seconds.-> FileListToArrayRecursive returned 9512 items in 0.44 seconds.-> FileListToArray3 returned 9512 items in 1.46 seconds.-> RecFileListToArray returned 9512 items in 1.97 seconds.-> FileListTreeToArray returned 9512 items in 2.64 seconds.-> _FileListToArrayEx returned 9512 items in 0.41 seconds.+>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Link to comment Share on other sites More sharing options...
guinness Posted February 7, 2012 Share Posted February 7, 2012 I'm not familiar with the others but looking at the code the only one which has the option to choose the recursion depth is (you've guessed it) RecFileListToArray.I wouldn't really pay much attention to the results because it's going to vary between systems.-> Dos command returned 4475 items in 1.67 seconds.-> FileListToArrayRecursive returned 4487 items in 0.21 seconds.-> FileListToArray3 returned 4487 items in 1.88 seconds.-> RecFileListToArray returned 4487 items in 0.63 seconds.-> FileListTreeToArray returned 4487 items in 0.46 seconds. UDF List: _AdapterConnections() • _AlwaysRun() • _AppMon() • _AppMonEx() • _ArrayFilter/_ArrayReduce • _BinaryBin() • _CheckMsgBox() • _CmdLineRaw() • _ContextMenu() • _ConvertLHWebColor()/_ConvertSHWebColor() • _DesktopDimensions() • _DisplayPassword() • _DotNet_Load()/_DotNet_Unload() • _Fibonacci() • _FileCompare() • _FileCompareContents() • _FileNameByHandle() • _FilePrefix/SRE() • _FindInFile() • _GetBackgroundColor()/_SetBackgroundColor() • _GetConrolID() • _GetCtrlClass() • _GetDirectoryFormat() • _GetDriveMediaType() • _GetFilename()/_GetFilenameExt() • _GetHardwareID() • _GetIP() • _GetIP_Country() • _GetOSLanguage() • _GetSavedSource() • _GetStringSize() • _GetSystemPaths() • _GetURLImage() • _GIFImage() • _GoogleWeather() • _GUICtrlCreateGroup() • _GUICtrlListBox_CreateArray() • _GUICtrlListView_CreateArray() • _GUICtrlListView_SaveCSV() • _GUICtrlListView_SaveHTML() • _GUICtrlListView_SaveTxt() • _GUICtrlListView_SaveXML() • _GUICtrlMenu_Recent() • _GUICtrlMenu_SetItemImage() • _GUICtrlTreeView_CreateArray() • _GUIDisable() • _GUIImageList_SetIconFromHandle() • _GUIRegisterMsg() • _GUISetIcon() • _Icon_Clear()/_Icon_Set() • _IdleTime() • _InetGet() • _InetGetGUI() • _InetGetProgress() • _IPDetails() • _IsFileOlder() • _IsGUID() • _IsHex() • _IsPalindrome() • _IsRegKey() • _IsStringRegExp() • _IsSystemDrive() • _IsUPX() • _IsValidType() • _IsWebColor() • _Language() • _Log() • _MicrosoftInternetConnectivity() • _MSDNDataType() • _PathFull/GetRelative/Split() • _PathSplitEx() • _PrintFromArray() • _ProgressSetMarquee() • _ReDim() • _RockPaperScissors()/_RockPaperScissorsLizardSpock() • _ScrollingCredits • _SelfDelete() • _SelfRename() • _SelfUpdate() • _SendTo() • _ShellAll() • _ShellFile() • _ShellFolder() • _SingletonHWID() • _SingletonPID() • _Startup() • _StringCompact() • _StringIsValid() • _StringRegExpMetaCharacters() • _StringReplaceWholeWord() • _StringStripChars() • _Temperature() • _TrialPeriod() • _UKToUSDate()/_USToUKDate() • _WinAPI_Create_CTL_CODE() • _WinAPI_CreateGUID() • _WMIDateStringToDate()/_DateToWMIDateString() • Au3 script parsing • AutoIt Search • AutoIt3 Portable • AutoIt3WrapperToPragma • AutoItWinGetTitle()/AutoItWinSetTitle() • Coding • DirToHTML5 • FileInstallr • FileReadLastChars() • GeoIP database • GUI - Only Close Button • GUI Examples • GUICtrlDeleteImage() • GUICtrlGetBkColor() • GUICtrlGetStyle() • GUIEvents • GUIGetBkColor() • Int_Parse() & Int_TryParse() • IsISBN() • LockFile() • Mapping CtrlIDs • OOP in AutoIt • ParseHeadersToSciTE() • PasswordValid • PasteBin • Posts Per Day • PreExpand • Protect Globals • Queue() • Resource Update • ResourcesEx • SciTE Jump • Settings INI • SHELLHOOK • Shunting-Yard • Signature Creator • Stack() • Stopwatch() • StringAddLF()/StringStripLF() • StringEOLToCRLF() • VSCROLL • WM_COPYDATA • More Examples... Updated: 22/04/2018 Link to comment Share on other sites More sharing options...
BrewManNH Posted February 7, 2012 Share Posted February 7, 2012 (edited) Here's an another run of the test (without sorting) on a Win7 x64 laptop: +>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -> OS:WIN_7 Service Pack 1, OS Arch:X64, Language:0409 ->-> Dos command returned 13583 items in 1.75 seconds. -> FileListToArrayRecursive returned 13590 items in 0.62 seconds. -> FileListToArray3 returned 13590 items in 3.84 seconds. -> RecFileListToArray returned 13590 items in 0.95 seconds. -> FileListTreeToArray returned 13590 items in 3.46 seconds. +>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Running on a 64 bit Win7, using 32bit AutoIt3 executable +>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -> OS:WIN_7 Service Pack 1, OS Arch:X64, Language:0409 ->-> Dos command returned 13583 items in 1.42 seconds. -> FileListToArrayRecursive returned 13590 items in 0.73 seconds. -> FileListToArray3 returned 13590 items in 3.89 seconds. -> RecFileListToArray returned 13590 items in 1.06 seconds. -> FileListTreeToArray returned 13590 items in 3.72 seconds. +>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ I only use _RecFileListToArray just because I have so much control over what it does and what it can give me. EDIT: Formatting issues...again Edited February 7, 2012 by BrewManNH If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag GudeHow to ask questions the smart way! I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from. Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays. - ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script. - Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label. - _FileGetProperty - Retrieve the properties of a file - SciTE Toolbar - A toolbar demo for use with the SciTE editor - GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI. - Latin Square password generator Link to comment Share on other sites More sharing options...
DXRW4E Posted February 7, 2012 Share Posted February 7, 2012 (edited) Hi guinness, First Sorry for my English I answer you, as you know well autoit, and you understand argument, because I did not want to answer to some other because I will not go down to certain levels I'm not familiar with the others is not another, and that by default in autoit, just been updated but looking at the code and too simple to understand _FileListToArrayEx (uses only 39 line of code) use only 1 While, and so more than logical and more faster he only one which has the option to choose the recursion depth is (you've guessed it) RecFileListToArray. there is not anything that does not support _FileListToArrayEx? I wouldn't really pay much attention to the results because it's going to vary between systems. http://youtu.be/DvJZsZKaUWY However, I want to clarify, this is a discourse without any importance, are not talking about who is better, God forbid (I with Melba23 in autoit us 1 with 10000 ehhhhhh, these my post, I'm not in any way disrespect about Melba23, also looking at its contribution to this community), I like me too much of the work Melba23 (and I was personally useful more than once), and also as a person, always present in the forum and always tries to help everyone without exception at Melba23 as an example but the same goes for all other users sorry again for my English Ciao a tutti. Edited February 7, 2012 by DXRW4E Link to comment Share on other sites More sharing options...
Spiff59 Posted February 7, 2012 Share Posted February 7, 2012 (edited) Not so fast! Add another to your list!I still consider the functions near the end of the June 2009 thread: probably the best examples around. Being buried in a 280-post thread they never seem to get much attention. There were 100 different function names used in the thread, such as "_FileListToArraySuperTurbo5000ExtraPlusProWithMaximumUltraPowerboost ", so I've never really known what best to call this, today it shall be _FileListToArrayPlus(). It has powerful options, the best speed I've seen, and comes in as a single backwards-compatible function of 64 lines:expandcollapse popup; #FUNCTION# ===================================================================================================================== ; _FileListToArrayPlus($sPath, $sInclude = "*", $iFlag = 0, $sExcludeFolder = "", $sExclude = "", $iPathType = 0, $bRecursive = False) ; Name...........: _FileListToArray_Rec ; Parameters ....: $sPath: Folder to search ; $sInclude: String to match on (wildcards allowed, multiples delimited by ;) ; $iFlag: Returned data type. 0 = Files and folders (default), 1 = Files only, 2 = Folders only ; $sExcludeFolder: List of folders to exclude from search (wildcards allowed, multiples delimited by ;) ; $sExclude: List of filenames to exclude from search (wildcards allowed, multiples delimited by ;) ; $iPathType: Returned data format. 0 = Filename only (default), 1 = Path relative to $sPath, 2 = Full path/filename ; $bRecursive: False = Search $sPath folder only (default), True = Search $sPath and all subfolders ; Author ........: Half the Autoit Community (Forum thread #96952) ;=================================================================================================================================== Func _FileListToArrayPlus($sPath, $sInclude = "", $iFlag = 0, $sExcludeFolder = "", $sExclude = "", $iPathType = 0, $bRecursive = False) Local $sRet = "", $sReturnFormat = "" $sPath = StringRegExpReplace($sPath, "[/]+z", "") & "" ; ensure single trailing slash If Not FileExists($sPath) Then Return SetError(1, 1, "") ; Edit include files list If $sInclude = "*" Then $sInclude = "" If $sInclude Then If StringRegExp($sInclude, "[/ :><|]|(?s)As*z") Then Return SetError(2, 2, "") ; invalid characters test $sInclude = StringRegExpReplace(StringRegExpReplace($sInclude, "(s*;s*)+", ";"), "A;|;z", "") ; Strip unwanted whitespace $sInclude = StringRegExpReplace($sInclude, "[][$.+^{}()]", "$0"); Ignore special characters $sInclude = StringReplace(StringReplace(StringReplace($sInclude, "?", "."), "*", ".*?"), ";", "$|") ; Convert ? to ., * to .*?, and ; to | $sInclude = "(?i)A(" & $sInclude & "$)"; case-insensitive, match from first char, terminate strings EndIf ; Edit exclude folders list If $sExcludeFolder Then $sExcludeFolder = StringRegExpReplace(StringRegExpReplace($sExcludeFolder, "(s*;s*)+", ";"), "A;|;z", "") ; Strip unwanted whitespace $sExcludeFolder = StringRegExpReplace($sExcludeFolder, "[][$.+^{}()]", "$0"); Ignore special characters $sExcludeFolder = StringReplace(StringReplace(StringReplace($sExcludeFolder, "?", "."), "*", ".*?"), ";", "$|") ; Convert ?=. *=.*? ;=| $sExcludeFolder = "(?i)A(?!" & $sExcludeFolder & "$)"; case-insensitive, match from first char, terminate strings EndIf ; Edit exclude files list If $sExclude Then $sExclude = StringRegExpReplace(StringRegExpReplace($sExclude, "(s*;s*)+", ";"), "A;|;z", "") ; Strip unwanted whitespace $sExclude = StringRegExpReplace($sExclude, "[][$.+^{}()]", "$0"); Ignore special characters $sExclude = StringReplace(StringReplace(StringReplace($sExclude, "?", "."), "*", ".*?"), ";", "$|") ; Convert ?=. *=.*? ;=| $sExclude = "(?i)A(?!" & $sExclude & "$)"; case-insensitive, match from first char, terminate strings EndIf ; MsgBox(1,"Masks","File include: " & $sInclude & @CRLF & "File exclude: " & $sExclude & @CRLF & "Dir exclude : " & $sExcludeFolder) If Not ($iFlag = 0 Or $iFlag = 1 Or $iFlag = 2) Then Return SetError(3, 3, "") Local $sOrigPathLen = StringLen($sPath), $aQueue[64] = [1,$sPath], $iQMax = 63 While $aQueue[0] $WorkFolder = $aQueue[$aQueue[0]] $aQueue[0] -= 1 $search = FileFindFirstFile($WorkFolder & "*") If @error Then ContinueLoop Switch $iPathType Case 1 ; relative path $sReturnFormat = StringTrimLeft($WorkFolder, $sOrigPathLen) Case 2 ; full path $sReturnFormat = $WorkFolder EndSwitch While 1 $file = FileFindNextFile($search) If @error Then ExitLoop If @extended Then ; Folder If $sExcludeFolder And Not StringRegExp($file, $sExcludeFolder) Then ContinueLoop If $bRecursive Then If $aQueue[0] = $iQMax Then $iQMax += 128 ReDim $aQueue[$iQMax + 1] EndIf $aQueue[0] += 1 $aQueue[$aQueue[0]] = $WorkFolder & $file & "" EndIf If $iFlag = 1 Then ContinueLoop $sRet &= $sReturnFormat & $file & "|" Else ; File If $iFlag = 2 Then ContinueLoop If $sInclude And Not StringRegExp($file, $sInclude) Then ContinueLoop If $sExclude And Not StringRegExp($file, $sExclude) Then ContinueLoop $sRet &= $sReturnFormat & $file & "|" EndIf WEnd FileClose($search) WEnd If Not $sRet Then Return SetError(4, 4, "") Return StringSplit(StringTrimRight($sRet, 1), "|") EndFunc-> Dos command returned 89370 items in 4.26 seconds.-> FileListToArrayRecursive returned 89438 items in 2.89 seconds.-> FileListToArray3 returned 89438 items in 5.06 seconds.-> RecFileListToArray returned 89438 items in 5.15 seconds. (No sort)-> FileListTreeToArray returned 89438 items in 11.1 seconds.-> FileListToArrayPlus returned 89438 items in 2.16 seconds.Sometimes one doesn't want to add 400+ lines and 16 helper functions for just for a simple recursive file search.It's seems to me you can sort an array after it's built with one line of code?edit: fix code tags, changed August to June Edited February 7, 2012 by Spiff59 Link to comment Share on other sites More sharing options...
JohnOne Posted February 7, 2012 Share Posted February 7, 2012 -> Dos command returned 69405 items in 3.51 seconds. -> FileListToArrayRecursive returned 69424 items in 2.27 seconds. -> FileListToArray3 returned 69424 items in 9.69 seconds. -> RecFileListToArray returned 69424 items in 3.08 seconds. -> FileListTreeToArray returned 69424 items in 7.26 seconds. AutoIt Absolute Beginners Require a serial Pause Script Video Tutorials by Morthawt ipify Monkey's are, like, natures humans. Link to comment Share on other sites More sharing options...
Tlem Posted February 7, 2012 Share Posted February 7, 2012 This multitudes of tests is very attractive, but you would not should lose sight of the feature of each of these functions. ^^ 1st : Certain functions have advanced options which can be useful only for one certain number of user. 2nd: Certain functions do not return basic search. 3rd: The gap of the run time of every function is not so important as that (It depends of the search you made). I think that in this precise example, it is not reasonable to choose with regard to a duration of execution. Just to give more explanation about that, if I compare _FileListToArrayEx, _RecFileListToArray, _FileListToArrayPlus (the best challengers in my point of view) : If I use _RecFileListToArray and _FileListToArrayPlus in a recursive search for files and directory, I'v got the list of all directory !!! I know that it is a request, but in real Windows search life, it doesn't exist anywhere. ^^ So if I want to search for files and directory that match a pattern, only _FileListToArrayEx can do the work correctly without additional treatment. So finally, all these functions are great, but they do not still correspond to the need which we have of it. Finaly, for me, I think that _FileListToArrayEx corresponds most to the Windows and AutoIt philosophy, the other functions being specialized in the other processings that those asked originally. Best Regards.Thierry Link to comment Share on other sites More sharing options...
BrewManNH Posted February 7, 2012 Share Posted February 7, 2012 So if I want to search for files and directory that match a pattern, only _FileListToArrayEx can do the work correctly without additional treatment.Not to come off as sounding like RFLTA needs defending (especially by me), but what exactly does this line mean? RFLTA can search for and/or exclude files and folders by filters so that what is returned can match anything you're looking for. What "additional treatement" do these other functions, other than the FLTAEx function, need that isn't needed in FLTAEx? If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag GudeHow to ask questions the smart way! I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from. Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays. - ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script. - Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label. - _FileGetProperty - Retrieve the properties of a file - SciTE Toolbar - A toolbar demo for use with the SciTE editor - GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI. - Latin Square password generator Link to comment Share on other sites More sharing options...
Bowmore Posted February 7, 2012 Share Posted February 7, 2012 I have to agree with the points made by Tiem above. I personally use 3 different file listing functions that I wrote a few years ago. The first is is a slightly modified version of AutoIts built in _FileListToArray which I use when I only need to search a single folder. The second is a recursive version that only uses a simple match pattern e.g. *.txt The third is similar to RecFileListToArray in that it allows very complex searches based on a number of parameters using positive patterns as the first match and then doing a second test against another set of parameters to check if it should be excluded. The point is that there is not one solution and speed is not always the primary reason for choosing other things such as convenience are more important. Another area where prior knowledge of your data can be useful is in choosing which sorting algorithm to use. Most people use the Quick sort algorithm all the time and where you have no prior knowled it's probaly the best choice. But for example if you are adding a few values to an already sorted list then an insertion sort is much quicker algorithm to resort the list. Several posters have commented that some functions user fewer lines of code than another with the implication that fewer lines of code is better. This is a fallacy, it's using the correct logic that is important. Having a longer function with serperate sections of code to handle different situation rather than trying to cram all the logic into only a few lines is usually quicker. tip 1 "Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to build bigger and better idiots. So far, the universe is winning."- Rick Cook Link to comment Share on other sites More sharing options...
tip Posted February 7, 2012 Author Share Posted February 7, 2012 Not so fast! Add another to your list!I still consider the functions near the end of the June 2009 thread: probably the best examples around. Being buried in a 280-post thread they never seem to get much attention. There were 100 different function names used in the thread, such as "_FileListToArraySuperTurbo5000ExtraPlusProWithMaximumUltraPowerboost ", so I've never really known what best to call this, today it shall be _FileListToArrayPlus(). It has powerful options, the best speed I've seen, and comes in as a single backwards-compatible function of 64 lines:Added to the test pack.This multitudes of tests is very attractive, but you would not should lose sight of the feature of each of these functions....Finaly, for me, I think that _FileListToArrayEx corresponds most to the Windows and AutoIt philosophy, the other functions being specialized in the other processings that those asked originally._FileListToArrayExCiao a tutti.Added to the test pack.Thanks everyone for your input. I've just added the two mentioned UDFs and updated my results.I personally think _RecFileListToArray is a very solid and sophisticated UDF and I must add that I don't think 27kb size of the UDF is a disadvantage since after upx compression it doesn't really matter much. But in my tests _FileListToArrayPlus seems to be the fastest. I know this not a real life test and applying include/exclude filters will change results but I believe, it will be very complicated to test for and track outcome of every scenerio. So I tried to keep it simple.Sincerely,Tip [center]MsgBox_Tipped: Eye candy msgboxes/inputboxes/loginboxes. | CreateBlankBox: Semi-transparent layers with borders and rounded corners.[/center] Link to comment Share on other sites More sharing options...
Tlem Posted February 7, 2012 Share Posted February 7, 2012 (edited) @BrewManNH This is an example script test :If Not FileExists("C:MyTestMyTest.txt") Then DirCreate("C:MyTest") FileWrite("C:MyTestMyTest.txt", "My Test File") EndIf $Path = "C:" $Filter = "MyTest*" $Flag = 0 $Recursive = 1 $Begin = TimerInit() $aArray1 = _FileListToArrayEx($Path, $Filter, $Flag, $Recursive) ConsoleWrite("_FileListToArrayEx returned " & $aArray1[0] & " items in " & TimerDiff($begin)/1000 & " seconds " & @CRLF) $Begin = TimerInit() $aArray2 = _RecFileListToArray($Path, $Filter, $Flag, $Recursive) ConsoleWrite("_RecFileListToArray returned " & $aArray2[0] & " items in " & TimerDiff($begin)/1000 & " seconds " & @CRLF) DirRemove("C:MyTest", 1) This is the result on my system :>Running:(3.3.8.0):C:Program FilesAutoIt3autoit3.exe "D:FileListToArrayTestFileListToArrayTest.au3" _FileListToArrayEx returned 2 items in 4.18766156033798 seconds _RecFileListToArray returned 13663 items in 5.11055455372121 seconds +>23:57:10 AutoIT3.exe ended.rc:0 >Exit code: 0 Time: 11.557 My goal is to know witch file(s) and directory(s) have the searched pattern. In a standard syntax, on my system RFLTA return me 13663 items instead of 2 and for me, this is not a request but an inconvenience. Another example :If Not FileExists("C:MyTestMyTestFile.txt") Then DirCreate("C:MyTestMyTestDir") DirCreate("C:MyTestAnotherDir") FileWrite("C:MyTestMyTest.txt", "My Test File") EndIf $Path = "C:MyTest" $Filter = "MyTest*" $Flag = 0 $Recursive = 1 $Begin = TimerInit() $aArray1 = _FileListToArrayEx($Path, $Filter, $Flag, $Recursive) ConsoleWrite("_FileListToArrayEx returned " & $aArray1[0] & " items in " & TimerDiff($begin)/1000 & " seconds " & @CRLF) $Begin = TimerInit() $aArray2 = _RecFileListToArray($Path, $Filter, $Flag, $Recursive) ConsoleWrite("_RecFileListToArray returned " & $aArray2[0] & " items in " & TimerDiff($begin)/1000 & " seconds " & @CRLF) DirRemove("C:MyTest", 1) The result :>Running:(3.3.8.0):C:Program FilesAutoIt3autoit3.exe "D:FileListToArrayTestFileListToArrayTest.au3" _FileListToArrayEx returned 2 items in 0.000978615997284571 seconds _RecFileListToArray returned 3 items in 0.00127194936786659 seconds +>00:34:16 AutoIT3.exe ended.rc:0 >Exit code: 0 Time: 2.002 Notice that RFLTA return 3 items instead of 2. Now, it is good to know that some functions can make more work and give more results with more code in almost the same time. Edited February 7, 2012 by Tlem Best Regards.Thierry Link to comment Share on other sites More sharing options...
BrewManNH Posted February 8, 2012 Share Posted February 8, 2012 Different functions for different usages. If you're going to do an iterative search in subfolders, and you know that what you're looking for is somewhere below "C:Program Files" but you aren't sure which folder it's in, in that folder, but you know it's going to be within the first to 3rd folder deep within that first folder, there's no point to search ALL of the folders under "C:Program Files". _FileListToArrayEX can't limit its search to just 3 folders deep, while _RecFileListToArray can. So, in this not so unusual case, i'm sure it will be a lot faster to return what I'm looking for because it's only looking in the folders I'm expecting it to be in, and not the 300 folders it's not going to be in. Not to mention, if I am looking for something and I know the things I DON'T want, RFLTA has a parameter to filter those things you don't want included, which FLTAEx also doesn't have. If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag GudeHow to ask questions the smart way! I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from. Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays. - ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script. - Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label. - _FileGetProperty - Retrieve the properties of a file - SciTE Toolbar - A toolbar demo for use with the SciTE editor - GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI. - Latin Square password generator Link to comment Share on other sites More sharing options...
Spiff59 Posted February 8, 2012 Share Posted February 8, 2012 (edited) Several posters have commented that some functions user fewer lines of code than another with the implication that fewer lines of code is better. ... Having a longer function with serperate sections of code to handle different situation rather than trying to cram all the logic into only a few lines is usually quicker.I'd have to disagree with that general premise. I think the main goal is to produce well-structured code that is both easily understandable and easily maintained. Poor structure can result from a bloated single function trying to accomplish too much, one that needs to be broken up logically to make it more understandable. I don't see anything meeting that criteria here. Poor structure can also occur from over-fragmenting what would, if left intact, be much more palatable code. In more powerful languages corporate environments I think it's common to print out a source listing, and with marker in hand, study it, edit it, and return to the tube to make your changes. I find Autoit, being intended for less ambitious tasks, tends to be more conducive to (the generally bad practice of) "composing at the tube", where I deal with a monitor worth of code at a time and try to keep the entire program and logic flow in my head as I scroll from screen to screen. For me, that places additional emphasis on striving for a minimum number of simple, concise functions Edited February 8, 2012 by Spiff59 Link to comment Share on other sites More sharing options...
Tlem Posted February 8, 2012 Share Posted February 8, 2012 Different functions for different usages.Yeah, that's what I have said on m'y first message. ^^ Best Regards.Thierry Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now