RecFileListToArray - Deprecated

Melba23 · October 15, 2012

New Version - 15 Oct 2012

Changed: The algorithm to omit hidden and system files/folders has been changed making the UDF much faster when asked to do so. Thanks to guinness for the suggestion. :thumbsup:

Added: Added the ability to omit link/junction folders as well - see function header.

New UDF and zip in first post.

stamandster · October 16, 2012

When I scan C:WindowsINF for INF files (using Windows 7 64bit) it seems to drop two files everytime, inconsistently which ones.

Here's the file search I'm using normally which finds 644, your function (unfortunately because I do like it) only does 642 (and there's nothing weird with the files I've checked).

Func RecursiveFileSearch($RFSstartDir, $RFSFilepattern = ".", $RFSFolderpattern = ".", $RFSFlag = 0, $RFSrecurse = True, $RFSdepth = 0)
;Ensure starting folder has a trailing slash
If StringRight($RFSstartDir, 1) <> "" Then $RFSstartDir &= ""
If $RFSdepth = 0 Then
;Get count of all files in subfolders for initial array definition
$RFSfilecount = DirGetSize($RFSstartDir, 1)
;File count + folder count (will be resized when the function returns)
Global $RFSarray[$RFSfilecount[1] + $RFSfilecount[2] + 1]
EndIf
$RFSsearch = FileFindFirstFile($RFSstartDir & "*.*")
If @error Then Return
;Search through all files and folders in directory
While 1
$RFSnext = FileFindNextFile($RFSsearch)
If @error Then ExitLoop
;If folder and recurse flag is set and regex matches
If StringInStr(FileGetAttrib($RFSstartDir & $RFSnext), "D") Then
If $RFSrecurse And StringRegExp($RFSnext, $RFSFolderpattern, 0) Then
RecursiveFileSearch($RFSstartDir & $RFSnext, $RFSFilepattern, $RFSFolderpattern, $RFSFlag, $RFSrecurse, $RFSdepth + 1)
If $RFSFlag <> 1 Then
     ;Append folder name to array
     $RFSarray[$RFSarray[0] + 1] = $RFSstartDir & $RFSnext
     $RFSarray[0] += 1
EndIf
EndIf
ElseIf StringRegExp($RFSnext, $RFSFilepattern, 0) And $RFSFlag <> 2 Then
;Append file name to array
$RFSarray[$RFSarray[0] + 1] = $RFSstartDir & $RFSnext
$RFSarray[0] += 1
EndIf
WEnd
FileClose($RFSsearch)
If $RFSdepth = 0 Then
ReDim $RFSarray[$RFSarray[0] + 1]
Return $RFSarray
EndIf
EndFunc ;==>RecursiveFileSearchwithoutFileNames

Edited October 16, 2012 by kickarse

Melba23 · October 16, 2012

kickarse,

When I run your function with these parameters:

RecursiveFileSearch("C:WindowsInf", "inf", ".", 1, True)

I too find more files in the C:WindowsInf folder than when I run my UDF.

But the additional files are these:

C:WindowsInfen-USnetsstpa.inf_loc
C:WindowsInfen-USnetsstpt.inf_loc
C:WindowsInfinfpub.dat
C:WindowsInfinfstor.dat
C:WindowsInfinfstrng.dat
C:WindowsInfmdminfot.PNF

Each of those files holds the string "inf", but I would not expect them to be returned from a search for "*.inf" - they should not match in my opinion. So I think you need to take a look at your function to make sure you have the RegEx returning the correct matches.

M23

Edit: If I use "*.inf" in your function I get nothing returned at all - that is why I am looking for only "int". If you can let me have the parameters you used, I can try again.

Edit 2: I tried again with ".inf" - that returned these additional files:

C:WindowsInfen-USnetsstpa.inf_loc
C:WindowsInfen-USnetsstpt.inf_loc
C:WindowsInfmdminfot.PNF

I can see why it might return the first 2, but the third seems completely out-of-place. :wacko:

Edited October 16, 2012 by Melba23

stamandster · October 16, 2012

Wow that was a quick response!

For my current function

RecursiveFileSearch($vDriverLocation, "(?i).(inf|INF|iNF|InF|INf)", ".", 1)

For your function

_RecFileListToArray($vDriverLocation,"*.inf")

I think you're right about the .inf_loc though!

C:WindowsINFen-USnetavpna.inf_loc

C:WindowsINFen-USnetavpnt.inf_loc

Now the question remains is if it's faster

Edited October 16, 2012 by kickarse

Melba23 · October 16, 2012

kickarse,

If you change your RegEx to read like this you do not get those 2 files listed:

ElseIf StringRegExp($RFSnext, $RFSFilepattern & "z", 0) And $RFSFlag <> 2 Then

You are then looking for the end of the string immediately following the "inf".

And you can also reduce your parameter string as well - (?i) makes the match case-insensitive so you can just use:

"(?i).inf"

All clear?

M23

stamandster · October 16, 2012

10-4 good buddy!

Melba23 · October 16, 2012

kickarse,

One final thing if you are after speed - running this line on each return takes quite a while (adds about 150% to the overall from my tests):

If StringInStr(FileGetAttrib($RFSstartDir & $RFSnext), "D") Then

For sometime now FileFindNextFile has returned 1 in @extended if the return is a folder, so looking at that value will save you the FileGteAttrib overhead. Take a look inside my UDF to see how I do it.

M23

stamandster · October 16, 2012

I'll use yours from now on M23. I think I need the "support" too

Melba23 · October 16, 2012

kickarse,

Always very happy to support my UDFs - after all I use them too!

M23

Melba23 · December 2, 2012

New Version - 2 Dec 2012

Changed: The Include_List/Exclude_List/Exclude_Folder_List parameters are now incorporated into a single parameter - the different elements being separated by " | ". So the syntax becomes simpler as you no longer need to add the intervening parameters if they are set to the default values:

; Old syntax:
$aList = _RecFileListToArray($sPath, "*.*", 0, 1, 0, 1, "*.bat;*.ini", "temp*")
; New syntax:
$aList = _RecFileListToArray($sPath, "*.*|*.bat;*.ini|temp*", 0, 1)

Both will search recursively for all files in the tree, ignoring any folders beginning with "temp" and excluding all bat and ini files, and returning a non-sorted list with relative paths.

Note: This syntax change is not script-breaking. The original syntax with separate parameters will still be honoured, but any exclude information added within the initial parameter will override the originals:

; This will honour the existing exclude parameters
$aList = _RecFileListToArray($sPath, "*.*", 0, 1, 0, 1, "*.bat;*.ini", "temp*")
; This will not as blank sections have been added to the first parameter
$aList = _RecFileListToArray($sPath, "*.*||", 0, 1, 0, 1, "*.bat;*.ini", "temp*")

So there is no need to amend your current scripts - but altering them is not too difficult if you decide to do so.

New UDF and zip in the first post.

M23

abberration · December 31, 2012

Melba23, I have been using this script for a while now. I tried the new Dec 2 version today and ran into some troubles. Check out my screenshot:

Posted Image

The new version seems to be having trouble with backslashes and is not looking into folders recursively (perhaps being stopped by error?). Also, I notice that it's listing the folder names as doubles (BackupBackup and SoftwareSoftware). I ran the same script with the two versions of RecFileListToArray. Here is the script I ran:

$i = _RecFileListToArray("C:\Users\Josh\Desktop\raidcalc\", "*", 0, 1)
_ArrayDisplay($i)

I am using Win7 64 bit and running the new script as 64/32 bit does not make a difference. I just thought you might want to know about this problem. For now, I'll keep using the old version.

Thanks!

Melba23 · December 31, 2012

abberration,

This &#!$@#%4 forum editor has eaten all the "" in the posted version - I will try and restore it ASAP. The version in the zip file is not corrupted - and I have confirmed it does work as expected.

M23

abberration · December 31, 2012

abberration,
This &#!$@#%4 forum editor has eaten all the "" in the posted version - I will try and restore it ASAP. The version in the zip file is not corrupted - and I have confirmed it does work as expected.
M23

That would make sense. I didn't download the zip file. But I will now. Thank you!

Edit: the zipped UDF is perfect! Thanks Melba23!

Edited December 31, 2012 by abberration

Mun · February 7, 2013

Thanks Melba23,

I just start using this to parse files and I love it.

I have a question. Is it better to search for both folder and files and separate arrays by * or 2 different call one for files and one for folders only?

Melba23 · February 7, 2013

Mun,

Glad you find it useful.

The answer to your question depends on what you are using as filters - take a look at the "include/exclude options" table on the first page (inside the first spoiler) to see what get returned in the various cases. Remember also that if you add a trailing "" to the path you pass to the UDF, any folders returned also have a trailing "" - that makes it pretty easy to determine whether a returned elements is a file or a folder if you do go for the "file + folder" option.

If you can explain what exactly you are trying to list with the UDF then I can perhaps offer some more focused advice.

M23

johnmcloud · February 7, 2013

Hi Melba,

Why you don't add on the first page ( and in the .zip ) the GUI for _RecFileListToArray?

The GUI need a little update for your last "|" change, and the hidden file/folder...nothing to difficult. I don't remember if was your script or make by another user, but i don't know if make a difference

Edited February 7, 2013 by johnmcloud

Melba23 · February 7, 2013

johnmcloud,

What GUI? I certainly have never written any GUI for this UDF. :huh:

M23

guinness · February 7, 2013

I believe some else did, if I remember I think it was johnmcloud.

Melba23 · February 7, 2013

johnmcloud,

I presume you mean this script. The author has not been around since early last year and I am not interested in maintaining it - do you want to have a go? :huh:

If you do successfully modify it I will add a link to the first post.

M23

johnmcloud · February 7, 2013

#include "RecFileListToArray.au3" ; External UDF by Melba
#include <WindowsConstants.au3>
#include <GUIConstantsEx.au3>
#include <ComboConstants.au3>
#include <EditConstants.au3>
#include <Array.au3>

$hGUI_Parent = GUICreate("RecFileListToArray_GUI", 285, 508, -1, -1)

GUICtrlCreateLabel("Initial path used to generate filelist", 8, 8, 163, 17)
GUICtrlCreateLabel("Files/Folders to include (default ' * ' = [all])", 8, 56, 212, 17)
GUICtrlCreateLabel("Files/Folders to exclude (default ' ' = [none])", 8, 104, 212, 17)
GUICtrlCreateLabel("Optional: specifies whether to return files, folders or both", 8, 201, 267, 17)
GUICtrlCreateLabel("Optional: search recursively in subfolders", 8, 330, 208, 17)
GUICtrlCreateLabel("Optional: sort ordered in alphabetical and depth order", 8, 377, 254, 17)
GUICtrlCreateLabel("Optional: specifies displayed path of results", 8, 426, 206, 17)
GUICtrlCreateLabel("Path exclude folder (default = ' ' [none])", 8, 153, 212, 17)
GUICtrlCreateLabel("Depth", 228, 330, 33, 17)
GUICtrlCreateLabel("Hidden files and folders", 32, 261, 114, 17)
GUICtrlCreateLabel("Link/junction folders", 32, 282, 100, 17)
GUICtrlCreateLabel("System files and folders", 32, 304, 114, 17)

$InputPath = GUICtrlCreateInput("", 8, 24, 193, 21)
GUICtrlSetTip(-1, "Select the path used to generate filelist")
$InputTypeInclude = GUICtrlCreateInput("*", 8, 73, 265, 21)
GUICtrlSetTip(-1, "Separate entries by semicolon. Example: *.ini;*.txt")
$InputTypeExclude = GUICtrlCreateInput("", 8, 120, 265, 21)
GUICtrlSetTip(-1, "Separate entries by semicolon. Example: *.ini;*.txt")
$InputPathExclude = GUICtrlCreateInput("", 8, 168, 193, 21)
GUICtrlSetTip(-1, "Exclude folders matching the filter")
$InputDepth = GUICtrlCreateInput("", 216, 346, 57, 21, BitOR($GUI_SS_DEFAULT_INPUT, $ES_CENTER))
GUICtrlSetTip(-1, "Specific depth that you want to search")

$ComboFileFolder = GUICtrlCreateCombo("Return both files and folders (Default)", 8, 218, 265, 25, BitOR($CBS_DROPDOWN, $CBS_AUTOHSCROLL))
GUICtrlSetData(-1, "Return files only|Return folder only")
$ComboSearchRec = GUICtrlCreateCombo("Do not search in subfolders (Default)", 8, 346, 200, 25, BitOR($CBS_DROPDOWN, $CBS_AUTOHSCROLL))
GUICtrlSetData(-1, "Search in all subfolders (unlimited)|Search subfolders to specified depth")
$ComboSorted = GUICtrlCreateCombo("Not sorted (Default)", 8, 392, 265, 25, BitOR($CBS_DROPDOWN, $CBS_AUTOHSCROLL))
GUICtrlSetData(-1, "Sorted|Sorted with faster algorithm ( requires NTFS drive)")
$ComboPathLen = GUICtrlCreateCombo("Relative to initial path (Default)", 8, 441, 265, 25, BitOR($CBS_DROPDOWN, $CBS_AUTOHSCROLL))
GUICtrlSetData(-1, "File/folder name only|Full path included")

$CheckboxHiddenFiles_Folder = GUICtrlCreateCheckbox("", 16, 259, 17, 17)
$CheckboxSystemFiles_Folder = GUICtrlCreateCheckbox("", 16, 302, 17, 17)
$CheckboxLinkJuction_Folder = GUICtrlCreateCheckbox("", 16, 280, 17, 17)
GUICtrlCreateGroup("Omit special items", 8, 243, 265, 81)
GUICtrlCreateGroup("", -99, -99, 1, 1)

$Button_InputPath = GUICtrlCreateButton("Browse", 208, 22, 65, 25)
$Button_InputPathExclude = GUICtrlCreateButton("Browse", 208, 166, 65, 25)
$Button_Generate = GUICtrlCreateButton("Generate", 8, 472, 265, 25)
GUISetState(@SW_SHOW)

While 1
    $nMsg = GUIGetMsg()
    Switch $nMsg
        Case $GUI_EVENT_CLOSE
            Exit
        Case $Button_InputPath
            _Select_Folder($InputPath)
        Case $Button_InputPathExclude
            _Select_Folder($InputPathExclude, GUICtrlRead($InputPath), 1)
        Case $Button_Generate
            _Generate_RecFileListToArray()
    EndSwitch
    UDF_Syntax()
WEnd

Func UDF_Syntax()
    If GUICtrlRead($ComboSearchRec) = "Search subfolders to specified depth" Then
        If BitAND(GUICtrlGetState($InputDepth), $GUI_DISABLE) = $GUI_DISABLE Then
            GUICtrlSetState($InputDepth, $GUI_ENABLE)
        EndIf
    Else
        If BitAND(GUICtrlGetState($InputDepth), $GUI_ENABLE) = $GUI_ENABLE Then
            GUICtrlSetData($InputDepth, "")
            GUICtrlSetState($InputDepth, $GUI_DISABLE)
        EndIf
    EndIf
    If GUICtrlRead($ComboFileFolder) = "Return both files and folders (Default)" Then
        If GUICtrlRead($ComboSearchRec) <> "Do not search in subfolders (Default)" Then
            If BitAND(GUICtrlGetState($InputPathExclude), $GUI_DISABLE) = $GUI_DISABLE Then
                GUICtrlSetState($InputPathExclude, $GUI_ENABLE)
            EndIf
        Else
            If BitAND(GUICtrlGetState($InputPathExclude), $GUI_ENABLE) = $GUI_ENABLE Then
                GUICtrlSetData($InputDepth, "")
                GUICtrlSetState($InputPathExclude, $GUI_DISABLE)
            EndIf
        EndIf
    EndIf
EndFunc   ;==>UDF_Syntax

Func _Select_Folder($FolderOutput, $StartPath = "", $FolderName = 0)
    $Dir = FileSelectFolder("", $StartPath, 2, @ScriptDir)
    If @error Then
        GUICtrlSetData($FolderOutput, "")
    Else
        GUICtrlSetData($FolderOutput, $Dir)
    EndIf
EndFunc   ;==>_Select_Folder

Func _Generate_RecFileListToArray()
    If GUICtrlRead($InputPath) = "" Then
        MsgBox(16, "Error", "Input file is empty")
        _Select_Folder($InputPath)
        Return
    Else
        If StringLen(GUICtrlRead($InputPath)) >= 260 Then
            $Path = "\\?\" & GUICtrlRead($InputPath)
        Else
            $Path = GUICtrlRead($InputPath)
        EndIf
    EndIf
    If GUICtrlRead($InputPathExclude) = "" Then
        $ExcludeFolder = ""
    Else
        If GUICtrlRead($InputTypeExclude) = "" Then
            $ExcludeFolder = '||' & GUICtrlRead($InputPathExclude)
        Else
            $ExcludeFolder = '|' & GUICtrlRead($InputPathExclude)
        EndIf
    EndIf
    If GUICtrlRead($InputTypeInclude) = "" Then
        MsgBox(16, "Error", "Input type is empty")
        GUICtrlSetData($InputTypeInclude, "*")
        Return
    Else
        $TypeInclude = GUICtrlRead($InputTypeInclude)
    EndIf
    If GUICtrlRead($InputTypeExclude) = "" Then
        $TypeExclude = ""
    Else
        $TypeExclude = '|' & GUICtrlRead($InputTypeExclude)
    EndIf
    If GUICtrlRead($ComboFileFolder) = "Return both files and folders (Default)" Then
        $FileFolder = "0"
    ElseIf GUICtrlRead($ComboFileFolder) = "Return files only" Then
        $FileFolder = "1"
    Else
        $FileFolder = "2"
    EndIf
    Local $HiddenFiles_Folder, $SystemFiles_Folder, $LinkJuction_Folder
    If GUICtrlRead($CheckboxHiddenFiles_Folder) <> $GUI_UNCHECKED Then
        $HiddenFiles_Folder = " + 4"
    EndIf
    If GUICtrlRead($CheckboxSystemFiles_Folder) <> $GUI_UNCHECKED Then
        $SystemFiles_Folder = " + 8"
    EndIf
    If GUICtrlRead($CheckboxLinkJuction_Folder) <> $GUI_UNCHECKED Then
        $LinkJuction_Folder = " + 16"
    EndIf
    $FileFolder &= $HiddenFiles_Folder & $SystemFiles_Folder & $LinkJuction_Folder
    If GUICtrlRead($ComboSearchRec) = "Do not search in subfolders (Default)" Then
        $SearchSubfolder = "0"
    ElseIf GUICtrlRead($ComboSearchRec) = "Search in all subfolders (unlimited)" Then
        $SearchSubfolder = "1"
    Else
        If GUICtrlRead($InputDepth) <> "" Then
            $SearchSubfolder = "-" & GUICtrlRead($InputDepth)
        Else
            MsgBox(16, "Error", "Depth value is empty")
            Return
        EndIf
    EndIf
    If GUICtrlRead($ComboSorted) = "Not sorted (Default)" Then
        $Sorted = "0"
    ElseIf GUICtrlRead($ComboSorted) = "Sorted" Then
        $Sorted = "1"
    Else
        $Sorted = "2"
    EndIf
    If GUICtrlRead($ComboPathLen) = "Relative to initial path (Default)" Then
        $PathLen = "1"
    ElseIf GUICtrlRead($ComboPathLen) = "File/folder name only" Then
        $PathLen = "0"
    Else
        $PathLen = "2"
    EndIf

    $Temp_Output = '_RecFileListToArray("' & $Path & '", "' & $TypeInclude & $TypeExclude & $ExcludeFolder & '", ' & $FileFolder & ', ' & $SearchSubfolder & ', ' & $Sorted & ', ' & $PathLen
    ConsoleWrite("DEBUG: " & $Temp_Output & @CRLF)
    If $PathLen = "1" And $Sorted <> "0" Then
        $FinalOutput = StringTrimRight($Temp_Output, 3) & ")"
    ElseIf $Sorted = "0" And $PathLen = "1" And $SearchSubfolder <> "0" Then
        $FinalOutput = StringTrimRight($Temp_Output, 6) & ")"
    ElseIf $SearchSubfolder = "0" And $Sorted = "0" And $PathLen = "1" And $FileFolder <> "0" Then
        $FinalOutput = StringTrimRight($Temp_Output, 9) & ")"
    ElseIf $FileFolder = "0" And $SearchSubfolder = "0" And $Sorted = "0" And $PathLen = "1" Then
        $FinalOutput = StringTrimRight($Temp_Output, 12) & ")"
    Else
        $FinalOutput = $Temp_Output & ")"
    EndIf

    $hGUI_Child = GUICreate("Output", 410, 144, -1, -1)
    GUICtrlCreateLabel("Display speed result", 24, 81, 123, 17)
    $Edit = GUICtrlCreateEdit("", 8, 16, 393, 57, BitOR($ES_READONLY, $WS_HSCROLL))
    GUICtrlSetData(-1, "")
    GUICtrlSetBkColor(-1, 0xFFFFFF)
    $Button_TestCode = GUICtrlCreateButton("Test _RecFileListToArray", 8, 104, 145, 33)
    $Button_Clipboard = GUICtrlCreateButton("Save to the Clipboard", 249, 104, 153, 33)
    $Checkbox_Timer = GUICtrlCreateCheckbox("", 8, 80, 16, 16)
    GUISetState(@SW_SHOW)
    GUISetState(@SW_DISABLE, $hGUI_Parent)

    GUICtrlSetData($Edit, $FinalOutput)
    ConsoleWrite($FinalOutput & @CRLF)

    While 1
        $nMsg = GUIGetMsg()
        Switch $nMsg
            Case $GUI_EVENT_CLOSE
                ClipPut($FinalOutput)
                GUISetState(@SW_ENABLE, $hGUI_Parent)
                GUIDelete($hGUI_Child)
                ExitLoop
            Case $Button_Clipboard
                ClipPut($FinalOutput)
                GUISetState(@SW_ENABLE, $hGUI_Parent)
                GUIDelete($hGUI_Child)
                ExitLoop
            Case $Button_TestCode
                If GUICtrlRead($Checkbox_Timer) = $GUI_CHECKED Then
                    Local $begin = TimerInit()
                EndIf
                $aArray = _RecFileListToArray($Path, $TypeInclude & $TypeExclude & $ExcludeFolder, Number($FileFolder), Number($SearchSubfolder), Number($Sorted), Number($PathLen))
                If @error Then
                    MsgBox(16, "Error", "Failure - Extended: " & @extended & @CRLF & _
                            "1 = Path not found or invalid" & @CRLF & _
                            "2 = Invalid $sInclude_List" & @CRLF & _
                            "3 = Invalid $iReturn" & @CRLF & _
                            "4 = Invalid $iRecur" & @CRLF & _
                            "5 = Invalid $iSort" & @CRLF & _
                            "6 = Invalid $iReturnPath" & @CRLF & _
                            "7 = Invalid $sExclude_List" & @CRLF & _
                            "8 = Invalid $sExclude_List_Folder" & @CRLF & _
                            "9 = No files/folders found")
                Else
                    If GUICtrlRead($Checkbox_Timer) = $GUI_CHECKED Then
                        Local $dif = TimerDiff($begin)
                        MsgBox(64, "Result", "Speed result is:" & @CRLF & $dif & @CRLF & "Informations are copied to the clipboard")
                        ClipPut($FinalOutput & "Speed result: " & $dif)
                    EndIf
                    _ArrayDisplay($aArray, "")
                EndIf
        EndSwitch
    WEnd
EndFunc   ;==>_Generate_RecFileListToArray

Edited June 12, 2013 by johnmcloud

Sign In

RecFileListToArray - Deprecated

Recommended Posts

Melba23

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Melba23

Melba23

Melba23

Posted Images

stamandster

Melba23

stamandster

Melba23

stamandster

Melba23

stamandster

Melba23

Melba23

abberration

Melba23

abberration

Mun

Melba23

johnmcloud

Melba23

guinness

Melba23

johnmcloud

Recently Browsing 0 members

Browse

AutoIt Resources

Release

Beta