Improvement of included _FileListToArray function.

Spiff59 · June 29, 2009

How about this for the last paragraph?

The $sWorkPath test makes sure the error test is only executed once at the end, after all the recursion loops are complete.

It seems to work fine regardless of the value of $iRetFormat, and would make this function behave identical to the production version of FLTA (and the proposed beta version of FLTA).

;Set according return value
  If (Not $sWorkPath) And (Not $sFileList) Then Return SetError(4, 4, ""); the parens arent necessary...
  Switch $iRetFormat
    Case 0
      Return $sFileList
    Case 1, 2
        Return StringSplit(StringTrimLeft($sFileList, 1), "|", $iRetFormat)
  EndSwitch

Edit: It would also simplify the "Return value(s)" section of the function description, and that area of the helpfile would remain unchanged.

Edit2: I agree with you, @error =1 and @error = 2 are not pertinent with this function. I kinda like getting rid of @error = 3 also, by handling values of 1 and 2 for $iSrchType and defaulting everything else via a "case else" to a value of 0 (files and folders). That way, the only @error returned would be when no files/folders are found and it could be tested with a simple "If @error", rather than specifying "If @error = 4". Testing for the other existing errors doesn't make much sense to me. I can't see anyone writing code into their script to test if they've sent a function a valid flag.

Edited June 29, 2009 by Spiff59

BaKaMu · June 29, 2009

How about this for the last paragraph?
The $sWorkPath test makes sure the error test is only executed once at the end, after all the recursion loops are complete.
It seems to work fine regardless of the value of $iRetFormat, and would make this function behave identical to the production version of FLTA (and the proposed beta version of FLTA).
;Set according return value
  If (Not $sWorkPath) And (Not $sFileList) Then Return SetError(4, 4, ""); the parens arent necessary...
  Switch $iRetFormat
    Case 0
      Return $sFileList
    Case 1, 2
        Return StringSplit(StringTrimLeft($sFileList, 1), "|", $iRetFormat)
  EndSwitch
Edit: It would also simplify the "Return value(s)" section of the function description, and that area of the helpfile would remain unchanged.

Hm, interesting but needs some discussion.

How do you handle the situation where more than one path is present?

If on does not exist and the next exist?

Where should SetError(4, 4, "") returned? After the first path or after the second path?

And if i can not decide witch path don't exists, so the error information is normally useless.

This are only some thoughts, i don't know a generality answer.

For me the best compromise is just returning no results for a non existing path (and no error setting)

And in your code don't forget a

Case Else

Return SetError(4, 4, "")

at the end for situations where $iRetFormat is not 0 or 1 or 2

The

Case 1, 2

Return StringSplit(StringTrimLeft($sFileList, 1), "|", $iRetFormat)

looks good (more compatibility to _FileListToArray)

And the community has to decide witch version should developed further on.

_FileListToArrayEx3 or _FileListToArrayEx4

(it makes no sense to work on both)

I like both approaches (both have advantage and disadvantage) and therefore i have no preference.

I hope that other forum members make this decision.

Edited June 29, 2009 by BaKaMu

BaKaMu · June 29, 2009

Edit2: That way, the only @error returned would be when no files/folders are found and it could be tested with a simple "If @error", rather than specifying "If @error = 4".

I have a little problem with setting error for no files/folders are found.

For me if no files/folders are found it is just no result (but not an error)

Example: if i am searching for a file "abcdefg.hij" and this file does not exist, i have nothing made wrong, there is simply no result.

So the result should be an empty string or an empty array.

But my opinion maybe wrong.

Edited June 29, 2009 by BaKaMu

jpm · June 29, 2009

To my opinion, FileListToArrayEx() is a new function. The original function must has to remain such which.

But it can be improved by few modification like the use of @Extended and my first suggestion to not use the array in the function but strings, and the return of the full path.

The line for Windows 98 is not any more right to be because AutoIt does not support any more Win98

So I suggest the optimized version by Spiff59. This version is really more faster than the original (2X), but it's the same.

Func _FileListToArray($sPath, $sFilter = "*", $iFlag = 0)
    Local $hSearch, $sFile, $sFileList, $sDelim = "|"

    If Not FileExists($sPath) Then Return SetError(1, 1, "")
    If StringRegexp($sFilter, "[\\/:<>|]") Or (Not StringStripWS($sFilter, 8)) Then Return SetError(2, 2, "")
    If StringRight($sPath, 1) <> "\" Then $sPath &= "\" 

    $hSearch = FileFindFirstFile($sPath & $sFilter)
    If $hSearch = -1 Then Return SetError(4, 4, "")

    If $iFlag > 3 Then
        $sDelim &= $sPath & "\"
        $iflag -= 4
    EndIf

    Switch $iFlag
        Case 0; Files and Folders
            While 1
                $sFile = FileFindNextFile($hSearch)
                If @error Then ExitLoop
                $sFileList &= $sDelim & $sFile
            WEnd
        Case 1; Files Only
            While 1
                $sFile = FileFindNextFile($hSearch)
                If @error Then ExitLoop
                If @extended Then ContinueLoop; bypass folder
                $sFileList &= $sDelim & $sFile
            WEnd
            If Not $sFileList Then Return SetError(4, 4, "")
        Case 2; Folders Only
            While 1
                $sFile = FileFindNextFile($hSearch)
                If @error Then ExitLoop
                If @extended = 0 Then ContinueLoop; bypass file
                $sFileList &= $sDelim & $sFile
            WEnd
            If Not $sFileList Then Return SetError(4, 4, "")
        Case Else
            Return SetError(3, 3, "")
    EndSwitch

    FileClose($hSearch)
    Return StringSplit(StringTrimLeft($sFileList, 1), "|")
EndFunc;==>_FileListToArray

What do you think about that jpm ?

I think is the parameter are compatible with optional ones doing extra work we can stay with only one

The naming of the parameter is not important just the relative position

Cheers

jpm · June 29, 2009

I have a little problem with setting error for no files/folders are found.
For me if no files/folders are found it is just no result (but no error)
So the result should be an empty string or an empty array.
But my opinion maybe wrong.

I am ok with this error handling

GEOSoft · June 29, 2009

I have a little problem with setting error for no files/folders are found.
For me if no files/folders are found it is just no result (but not an error)
Example: if i am searching for a file "abcdefg.hij" and this file does not exist, i have nothing made wrong, there is simply no result.
So the result should be an empty string or an empty array.
But my opinion maybe wrong.

I think it's better to return the error.

$aFiles = _FileListToArrayEx(whatever params)
If NOT @Error Then
    For $i = 1 To Ubound($aFiles)-1
       ;; Do something here.
    Next
EndIf

jpm · June 29, 2009

I think it's better to return the error.

$aFiles = _FileListToArrayEx(whatever params)
If NOT @Error Then
    For $i = 1 To Ubound($aFiles)-1
      ;; Do something here.
    Next
EndIf

in Fact

$aFiles = _FileListToArrayEx(whatever params)
If NOT ($aFiles = "") Then
    For $i = 1 To Ubound($aFiles)-1
      ;; Do something here.
    Next
EndIf

work too but it look nicer with @error

Spiff59 · June 29, 2009

Hm, interesting but needs some discussion.
I hope that other forum members make this decision.

I wish more people would get their two-cents in also!

How do you handle the situation where more than one path is present?
If on does not exist and the next exist?
Where should SetError(4, 4, "") returned? After the first path or after the second path?

In my opinion, if you're allowing multiple paths, then the results of an individual path should be inconsequential. Only if the total of all specified paths results in no files/folders would the "not find" error condition be set. You either get "something" returned, or you get "nothing" returned (and an @error condition). That's how the current FLTA behaves.

And in your code don't forget a
Case Else
Return SetError(4, 4, "")
at the end for situations where $iRetFormat is not 0 or 1 or 2
And the community has to decide witch version should developed further on.
_FileListToArrayEx3 or _FileListToArrayEx4
(it makes no sense to work on both)
I like both approaches (both have advantage and disadvantage) and therefore i have no preference.

I can think of instances where the relative or partial path option would be of use, accessing files from the @scriptdir or @workingdir would only require a relative path, or simply listing file hierarchy from a certain point onward.

If I had to elect an option that I can't find a use for, it's the 0 version of $iRetFormat. I'm not sure what anyone can do with a huge |-delimited list of items other than StringSplit it into an array? In the freakish case they wanted a string, one statement, _ArrayToString, would achieve that. I think $iRetFormat would be better as "0 = Return a 0-based array" and "1 = Return a 1-based array".

I don't see the harm with, instead of validating every parameter and returning a number of different error codes, simply treating an out-of-bounds parameter as the default. Many Autoit functions do not test and error every parameter. If someone can't figure out how to specify "1" for "Files only" or "2" for "Folders only", then give them the default of "0 = Files and Folders". Or if $iRetFormat is not "0", then return the default of "1", a 1-based array. I don't think error handling need cater to hand-holding of the complete novice. If someone can't specify a proper function call, they wouldn't know where to look for an @error code either. Understanding a list of valid parameter values and what they do from the helpfile takes no more effort than understanding a list of error messages and how to retrieve and interpret them. As long as the function never terminates abnormally, and has predictable results, I don't see much value in processing error messages that will almost never occur.

I'm going to go hide until Wednesday and let others (hopefully) chime in!

Edited June 29, 2009 by Spiff59

Danny35d · June 29, 2009

I'm not sure if I'm doing something wrong, but when I compare the script from post #84 with the original _FileListToArray() function I get different results. Below is the script I use to test both functions. I rename the script from Post #84 to _FileListToArrayEx.

#include <File.au3>
#include <Array.au3>

For $x = 0 To 2
    $aFiles = _FileListToArray('C:', '*', $x)
    $error = @error
    If IsArray($aFiles) Then
        _ArrayDisplay($aFiles, 'Original Flag = ' & $x)
    Else
        ConsoleWrite('Original Error: ' & $error & ' Flag = ' & $x & @CRLF)
    EndIf

    $aFiles = _FileListToArrayEx('C:', '*', $x)
    $error = @error
    If IsArray($aFiles) Then
        _ArrayDisplay($aFiles, 'New Flag = ' & $x)
    Else
        ConsoleWrite('New Error: ' & $error & ' Flag = ' & $x & @CRLF)
    EndIf
NextoÝ÷ ØYZ@h¶îËb¢{!£azÆ¦z·¬º[E¨5)^.+-N+¬ ÷¦yø¥zÊ'!bââ²ÔèºÚÈL`÷¦zØ^±©jÇåj
W¬jw_¢W^®Áej
WËS 
ëk("½é~]z»(WËS 
ëk!1+Þéè¶§©Ýën®w«®øN¬r¸©µ·¥£¬¶¨âyéð¢¹"
'~íéZ²Ûaç¦¢{a~,b¥Ø^nè,!Æ§~)Ý"Ø¬}«-Êjwhç.®È¯y©Ý"ØbB¥Ø^nè,jëh×6; #FUNCTION# ====================================================================================================================
; Name...........: _FileListToArrayEx
; Description ...: Lists files and\or folders in a specified path (Similar to using Dir with the /B Switch)
; Syntax.........: _FileListToArrayEx($sPath[, $sFilter = "*"[, $iFlag = 0, $iRecursive = 0]])
; Parameters ....: $sPath   - Path to generate filelist for.
;                  $sFilter - Optional the filter to use, default is *. Search the Autoit3 helpfile for the word "WildCards" For details.
;                  $iRecursive - Search files in specified directory and all subdirectories.
;                  $iRecursive = 0 (Default) Search in specified directory.
;                  $iRecursive = 1 Search in specified and all subdirectories.
;                  $iFlag   - Optional: specifies whether to return files folders or both
;                  |$iFlag = 0(Default) Return both files and folders
;                  |$iFlag = 1 Return files only
;                  |$iFlag = 2 Return Folders only
; Return values .: @Error - 1 = Path not found or invalid
;                  |2 = Invalid $sFilter
;                  |3 = Invalid $iFlag
;                  |4 = No File(s) Found
; Author ........: SolidSnake <MetalGX91 at GMail dot com>
; Modified.......: Danny35d <Danny35d at GMail dot com>
; Remarks .......: The array returned is one-dimensional and is made up as follows:
;                                $array[0] = Number of Files\Folders returned
;                                $array[1] = 1st File\Folder
;                                $array[2] = 2nd File\Folder
;                                $array[3] = 3rd File\Folder
;                                $array[n] = nth File\Folder
; Related .......:
; Link ..........;
; Example .......; Yes
; ===============================================================================================================================
Func _FileListToArrayEx($sPath, $sFilter = '*.*', $iFlag = 0, $iRecursive = 0, $iRunFirstTime = 1)
    Local $aFileList = '', $aFolderList = '', $Tmp = '', $sMask

    $sPath = StringRegExpReplace($sPath, '\\+$', '') ; Removed trailing backslash
    If Not FileExists($sPath) Then Return SetError(1, 1, '')
    If StringRegExp($sFilter, '[\\/ :> <\|]') Then Return SetError(2, 2, '')
    If Not StringRegExp($iFlag, '[012]') Then Return SetError(3, 3, '')
    $sMask = '(?i)^' & StringReplace(StringReplace(StringReplace($sFilter, '.', '\.'), '*', '.*'), '?', '.')
    $iFirstFile = FileFindFirstFile($sPath & '\*')
    If @error Then Return

    While True
        $iNextFile = FileFindNextFile($iFirstFile)
        If @error Then ExitLoop
        $sFullPath = $sPath & '\' & $iNextFile

        If StringInStr(FileGetAttrib($sFullPath), 'D') Then
            If $iFlag <> 1 And ($sFilter = '*' Or $sFilter = '*.*') Then $aFileList &= $sFullPath & @CRLF
            If $iRecursive Then
                $Tmp = _FileListToArrayEx($sFullPath, $sFilter, $iFlag, $iRecursive, 0)
                If $Tmp <> Chr(38) And $Tmp <> ChrW(38) Then $aFileList &= $Tmp
            EndIf
        Else
            If Not StringRegExp($iNextFile, $sMask) Then ContinueLoop
            If Not StringRegExp(StringRegExpReplace($sFilter, '^.*\.', ''), '[?*]') Then
                If StringRegExpReplace($sFilter, '^.*\.', '') <> StringRegExpReplace($iNextFile, '^.*\.', '') Then ContinueLoop
            EndIf
            If $iFlag <> 2 Then $aFileList &= $sFullPath & @CRLF
        EndIf
    WEnd
    FileClose($iFirstFile)

    If $iRunFirstTime Then
        $aFileList = StringTrimRight($aFileList, 2)
        If StringLen($aFileList) = 0 Then Return SetError(4, 4, '') ; Not Files/Folder found.
        $aFileList = StringSplit($aFileList, @CRLF, 1)
    EndIf
    Return ($aFileList)
EndFunc   ;==>_FileListToArrayEx

BaKaMu · June 29, 2009

In my opinion, if you're allowing multiple paths, then the results of an individual path should be inconsequential. Only if the total of all specified paths results in no files/folders would the "not find" error condition be set. You either get "something" returned, or you get "nothing" returned (and an @error condition). That's how the current FLTA behaves.

OK, let's show an @error condition. In my opinion the compatibility to FLTA is important.

If I had to elect an option that I can't find a use for, it's the 0 version of $iRetFormat. I'm not sure what anyone can do with a huge |-delimited list of items other than StringSplit it into an array? In the freakish case they wanted a string, one statement, _ArrayToString, would achieve that. I think $iRetFormat would be better as "0 = Return a 0-based array" and "1 = Return a 1-based array".

Oh no, the option $iRetFormat = 0 (returning a string) is essentially for correct working of recursion (please don't touch this value)

As long as the function never terminates abnormally, and has predictable results, I don't see much value in processing error messages that will almost never occur.

This are really true words, i agree complete with this statement.

And this

... As long as the function never terminates abnormally...

needs now testing, testing, testing and again .... testing ;-)

BaKaMu · June 29, 2009

OK, let's show an @error condition. In my opinion the compatibility to FLTA is important.

The return part of _FileListToArrayEx3/4 (with error handling in case of no files/folders were found) looks now like this:

...
  ;Set according return value
  Switch $iRetFormat
    Case 0  ;return a delimited string
      If $sFileList Then
        Return $sFileList
      Else
        Return SetError(4, 4, "")
      EndIf
    Case 1  ;return a 1-based array
      If $sFileList Then
        Return StringSplit(StringTrimLeft($sFileList, 1), "|", $iRetFormat)
      Else
        Local $aRet[1] = [0]
        Return SetError(4, 4, $aRet)
      EndIf
    Case 2  ;return a 0-based array
      If $sFileList Then
        Return StringSplit(StringTrimLeft($sFileList, 1), "|", $iRetFormat)
      Else
        Local $aRet[0]
        Return SetError(4, 4, $aRet)
      EndIf
    Case Else  ;return if $iRetFormat is wrong
      Return SetError(4, 4, "")
  EndSwitch
...

Does everyone agree with this ?

Otherwise please bring forward new proposals.

Tlem · June 29, 2009

@Danny35d

Did you test it with the beta version ?

The last dev function can work only with the new version of AutoIt (>3.3.1.0).

PS: I have edited my post #77 to correct a forgetting think.

Danny35d · June 29, 2009

@Danny35d
Did you test it with the beta version ?
The last dev function can work only with the new version of AutoIt (>3.3.1.0).
PS: I have edited my post #77 to correct a forgetting think.

No I didn't. Spiff59 send me a PM and let me know I need the latest AutoIT Beta. After installing AutoIT Beta and running a test your script was faster that mine not for much, but I have the choice of recursive using my script.

Spiff59 · June 29, 2009

Argh! I can't help myself. I don't know how I get anything done at work...

...
      Case Else  ;return if $iRetFormat is wrong
      Return SetError(4, 4, "")
...

Technically, the invalid $iRetFormat value is unrelated to a "not found" condition, so I don't think it should be considered the same error?

Here's a revised FileListToArrayEx4 for your perusal. Main Changes:

1. I thought duplicating almost the entire function for each conditon of the $iExclude flag makes function maintenance difficult, so one "If $iExclude" test is added per file/folder in the FileReadNextFile loop. I think the very minor degradation to performance may be worth the smaller, simpler function?

2. Error handling. I've set it so that invalid flag values will use the default rather than error out. If nothing is found, nothing is returned, and @error is set (to 4), which conforms to the current function in production.

3. I built a complete $sTExclude in the function initialization, prior to any loops, which allowed for the simplification of the SRE/SRER statements for excludes (should be useful in FileListToArrayEx3 also).

4. I made the $iRetFormat values to be: 0 = 0-based array, and 1 = 1-based array, which seems more logical/elegant (but the necessary functionality to support recursion is retained)

Now I too need to do some testing, testing, and more testing

; #FUNCTION# ===========================================================================================
; Name:          _FileListToArrayEx4
; Description:    full compatible _FileListToArray replacement  (with greater performance and additional features)
;                  additional: multi-path, multi-filter, multi-exclude-filter, path format options, recursive search
; Syntax:          _FileListToArrayEx4([$sPath = @ScriptDir, [$sFilter = "*", [$iSearchType, [$bRecursiv = False, [$sExclude = "", [$iRetFormat = 1]]]]]])
; Parameter(s):  $sPath = optional: Search path(s), semicolon delimited (default: @ScriptDir)
;                            (Example: "C:\Tmp;D:\Temp")
;                  $sFilter = optional: Search filter(s), semicolon delimited . Wildcards allowed. (default: "*")
;                             (Example: "*.exe;*.txt")
;                  $iSearchType = Include in search: 0 = Files and Folder, 1 = Files Only, 2 = Folders Only
;                  $iPathType = Returned element format: 0 = file/folder name only, 1 = relative path, 2 = full path
;                  $bRecursiv = optional: True: recursive search including all subdirectories
;                                         False (default): search only in specified folder
;                  $sExclude = optional: Exclude filter(s), semicolon delimited. Wildcards allowed.
;                               (Example: "Unins*" will remove all files/folders that begin with "Unins")
;                  $iRetFormat =  optional: return format
;                                 0 = one-dimensional array, 0-based
;                                 1 = one-dimensional array, 1-based (default)
;                  $sWorkPath = *** internal use only (do not use) ***
; Requirement(s):   none
; Return Value(s):  on success: 1-based or 0-based array (dependent on $iRetFormat)
;                   If no data is found, @error is set (to 4, for backwards compatibility)
; Author(s):        Half the AutoIt Community
; ====================================================================================================




Func _FileListToArrayEx4($sPath = @ScriptDir, $sFilter = "*", $iSearchType = 0, $iPathType = 0, $bRecursiv = False, $sExclude = "", $iRetFormat = 1, $sWorkPath = "")
  Local $hSearch, $iPCount, $iFCount, $sFile, $sFileList = "", $sTExclude = "", $sTWorkPath

  If $sPath = -1 Or $sPath = Default Then $sPath = @ScriptDir
  If $sFilter = -1 Or $sFilter = Default Then $sFilter = "*"
  If $iSearchType = -1 Or $iSearchType = Default Then $iSearchType = 0
  If $iPathType = -1 Or $iPathType = Default Then $iPathType = 0
  If $bRecursiv = Default Then $bRecursiv = False
  If $sExclude = -1 Or $sExclude = Default Then $sExclude = ""
  If $iRetFormat = -1 Or $iRetFormat = Default Then $iRetFormat = 1

  Local $aPath = StringSplit($sPath, ';');paths array
  Local $aFilter = StringSplit($sFilter, ';');filters array

  If $sExclude Then
  ;Strip leading and trailing spaces and spaces between semi colon delimiter
    $sTExclude = StringStripWS(StringRegExpReplace($sExclude, "\s*;\s*", ";"), 3)
  ;convert $sExclude to fit StringRegExp (not perfect but useable)
    $sTExclude = StringRegExpReplace($sTExclude, '([\Q\.+[^]$(){}=!\E])', '\\$1');thanks KaFu and Ascend4nt
    $sTExclude = StringReplace($sTExclude, "?", ".")
    $sTExclude = StringReplace($sTExclude, "*", ".*?")
    $sTExclude = "(?i)\A" & StringReplace($sTExclude, ";", "|") & "\z"

    For $iPCount = 1 To $aPath[0];Path loop
      Local $sPathItem = StringStripWS($aPath[$iPCount], 3)
      Local $sDelim = "|"
      If StringRight($sPathItem, 1) <> "\" Then $sPathItem = $sPathItem & "\";ensure trailing slash
      If $iPathType = 2 Then $sDelim &= $sPathItem;return full-path
    
      For $iFCount = 1 To $aFilter[0];Filter loop
        Local $FilterItem = StringStripWS($aFilter[$iFCount], 3)
        If StringRegExp($FilterItem, "[\\/:<>|]") Then ContinueLoop;bypass filters with invalid chars
        $hSearch = FileFindFirstFile($sPathItem & $FilterItem)
        If @error Then ContinueLoop
        Switch $iSearchType
          Case 1;Files Only
            While True
              $sFile = FileFindNextFile($hSearch)
              If @error Then ExitLoop
              If @extended Then ContinueLoop;bypass folder
              If StringRegExp(StringRegExpReplace($sFile, "(.*?[\\/]+)*(.*?\z)", "\2"), $sTExclude) Then ContinueLoop
              $sFileList &= $sDelim & $sWorkPath & $sFile
            WEnd
          Case 2;Folders Only
            While True
              $sFile = FileFindNextFile($hSearch)
              If @error Then ExitLoop
              If @extended = 0 Then ContinueLoop;bypass file
              If StringRegExp(StringRegExpReplace($sFile, "(.*?[\\/]+)*(.*?\z)", "\2"), $sTExclude) Then ContinueLoop
              $sFileList &= $sDelim & $sWorkPath & $sFile
            WEnd
          Case Else;Files and Folders
            While True
              $sFile = FileFindNextFile($hSearch)
              If @error Then ExitLoop
              If StringRegExp(StringRegExpReplace($sFile, "(.*?[\\/]+)*(.*?\z)", "\2"), $sTExclude) Then ContinueLoop
              $sFileList &= $sDelim & $sWorkPath & $sFile
            WEnd
        EndSwitch
        FileClose($hSearch)
      Next;$iFCount - next filter
    ;optional recursive search
      If $bRecursiv Then
        $hSearch = FileFindFirstFile($sPathItem & "*.*")
        If Not @error Then
          While True
            $sFile = FileFindNextFile($hSearch)
            If @error Then ExitLoop
            If @extended = 0 Then ContinueLoop;bypass file
            If StringRegExp(StringRegExpReplace($sFile, "(.*?[\\/]+)*(.*?\z)", "\2"), $sTExclude) Then ContinueLoop
          ;call recursive search
            If $iPathType = 1 Then $sTWorkPath = $sWorkPath & $sFile & "\"
            $sFileList &= _FileListToArrayEx4($sPathItem & $sFile, $sFilter, $iSearchType, $iPathType, $bRecursiv, $sExclude, 2, $sTWorkPath)
          WEnd
          FileClose($hSearch)
        EndIf
      EndIf
    Next;$iPCount - next path
  
  Else; not exclude
  
    For $iPCount = 1 To $aPath[0];Path loop
      Local $sPathItem = StringStripWS($aPath[$iPCount], 3)
      Local $sDelim = "|"
      If StringRight($sPathItem, 1) <> "\" Then $sPathItem = $sPathItem & "\";ensure trailing slash
      If $iPathType = 2 Then $sDelim &= $sPathItem;return full-path
    
      For $iFCount = 1 To $aFilter[0];Filter loop
        Local $FilterItem = StringStripWS($aFilter[$iFCount], 3)
        If StringRegExp($FilterItem, "[\\/:<>|]") Then ContinueLoop;bypass filters with invalid chars
        $hSearch = FileFindFirstFile($sPathItem & $FilterItem)
        If @error Then ContinueLoop
        Switch $iSearchType
          Case 1;Files Only
            While True
              $sFile = FileFindNextFile($hSearch)
              If @error Then ExitLoop
              If @extended Then ContinueLoop;bypass folder
              $sFileList &= $sDelim & $sWorkPath & $sFile
            WEnd
          Case 2;Folders Only
            While True
              $sFile = FileFindNextFile($hSearch)
              If @error Then ExitLoop
              If @extended = 0 Then ContinueLoop;bypass file
              $sFileList &= $sDelim & $sWorkPath & $sFile
            WEnd
          Case Else;Files and Folders
            While True
              $sFile = FileFindNextFile($hSearch)
              If @error Then ExitLoop
              $sFileList &= $sDelim & $sWorkPath & $sFile
            WEnd
        EndSwitch
        FileClose($hSearch)
      Next;$iFCount - next filter

    ;optional recursive search
      If $bRecursiv Then
        $hSearch = FileFindFirstFile($sPathItem & "*.*")
        If Not @error Then
          While True
            $sFile = FileFindNextFile($hSearch)
            If @error Then ExitLoop
            If @extended = 0 Then ContinueLoop;bypass file
          ;call recursive search
            If $iPathType = 1 Then $sTWorkPath = $sWorkPath & $sFile & "\"
            $sFileList &= _FileListToArrayEx4($sPathItem & $sFile, $sFilter, $iSearchType, $iPathType, $bRecursiv, $sExclude, 2, $sTWorkPath)
          WEnd
          FileClose($hSearch)
        EndIf
      EndIf
    Next;$iPCount - next path
  Endif
  
  If $iRetFormat = 2 Then Return $sFileList;If recursion loop, return as string, not array  
  If Not $sFileList Then Return SetError(4, 4, "");Last loop. No data.
  If $iRetFormat Then Return StringSplit(StringTrimLeft($sFileList, 1), "|", 0);Last loop. Data found, return 1-based array
  Return StringSplit(StringTrimLeft($sFileList, 1), "|", 2);Last loop. Data found, return 0-based array
EndFunc ;==>_FileListToArrayEx4

Edited June 30, 2009 by Spiff59

BaKaMu · June 29, 2009

Technically, the invalid $iRetFormat value is unrelated to a "not found" condition, so I don't think it should be considered the same error?

OK, should be error #3 (wrong parameter)

Here's a revised FileListToArrayEx4 for your perusal. Main Changes:
1. I thought duplicating almost the entire function for each conditon of the $iExclude flag makes function maintenance difficult, so one "If $iExclude" test is added per file/folder in the FileReadNextFile loop. I think the very minor degradation to performance may be worth the smaller, simpler function?

Don't agree, if the function is stable then there is no need for maintenance (if we have done a good work), so my preference is speed.

(and to maintain 2 branches is not so difficult)

For myself i have made a function with same functionality that have a speed advantage of over 10% (reduced error checking)

2. Error handling. I've set it so that invalid flag values will use the default rather than error out. If nothing is found, nothing is returned, and @error is set (to 4), which conforms to the current function in production.

OK

3. I built a complete $sTExclude in the function initialization, prior to any loops, which allowed for the simplification of the SRE/SRER statements for excludes (should be useful in FileListToArrayEx3 also).

Ah yes, good idea!

4. I made the $iRetFormat values to be: 0 = 0-based array, and 1 = 1-based array, which seems more logical/elegant (but the necessary functionality to support recursion is retained)

maybe possible, but you loss the compatibility to StringSplit (flag = 2 returns 0-based array)

(ok i know, this is not a very strong argument)

--------------------

And last not least, i am sorry, your function does not work for me (i don't know why)

The function returns only 24 items where it should return 12144.

(filter="*", all files/folders, full path, recursive, no exclude)

It may be my mistake, but i have checked it more than once.

So i stop now and take a new look tomorrow,

Thanks for your improvement suggestions :-)

Spiff59 · June 29, 2009

The function returns only 24 items where it should return 12144.

I hadn't tried all the flag combinations after making some big changes. I'd introduced a recursion bug, it's corrected in the above post now.

Well, somewhere there's a balance between size/complexity and speed. I'm usually all for speed too. I'll have to run a comparison and see if the extra "If $iExclude" makes more than a negligible change to performance when $iExclude is not specified (which will be most of the time).

KaFu · June 29, 2009

Some thoughts on this version _FileListToArrayEx4(). Is this really necessary

If $sPath = -1 Or $sPath = Default Then $sPath = @ScriptDir

or wouldnt something like this

If $sPath = -1 Then $sPath = @ScriptDir

just do? I think the assignment of the Default value is already done in the parameter definition of the function.

Well, somewhere there's a balance between size/complexity and speed...

Who says that? Hmmm, didn't read anywhere that UDF's need to be simple and small in size... as long as they are thoroughly tested it should be fine. How about introducing a switch, letting the user decided to use a fast+simple search or a slow+extensive one?

Spiff59 · June 30, 2009

Who says that? Hmmm, didn't read anywhere that UDF's need to be simple and small in size...

I was just floating the idea, but the vote is 0-2, so I'll revert and add the 60 lines back in and regain the prior performance.

Maintainability is huge in the corporate world, and I did have a bugtrac, on this same function, shot down a couple months ago for that exact reason. My natural tendency is to go for the speed. I've amended post #94.

Edit: Regarding the "Or $variable = Default" tests. Those tests were introduced a while back and are inherited to this version I'm promoting (it's main difference being expanding the PathType option to include an option to return a relative path as well as just file/folder name, and full path). I hadn't really noticed, but as written now, this routine could be called without any parms, since they are all defaulted in the Func command. Reading the helpfile, it looks like all the "$variable = Default" tests could be canned?

Edited June 30, 2009 by Spiff59

Tlem · June 30, 2009

I made some tests between the optimized version of _FileListToArray (#77) and _FileListToArrayEx4.

This is the results (on my laptop) :

For 50000 files with $iFlag = 0 :

_FileListToArray = 646

_FileListToArrayEx4 = 683

For 50000 files with $iFlag = 1 :

_FileListToArray = 1333

_FileListToArrayEx4 = 1408

For 100000 files with $iFlag = 0 :

_FileListToArray = 1350

_FileListToArrayEx4 = 1426

For 100000 files with $iFlag = 1 :

_FileListToArray = 1339

_FileListToArrayEx4 = 1554

Like we can see, for 100000 files with $iFlag = 1, we lose about 200ms ... I think that I will survive on this lost.

If we compare with the original version of _FileListToArray, this new function remains almost twice as fast in spite of all its options.

Take you very much mister Spiff59 and BaKaMu. Great work.

jpm · June 30, 2009

I made some tests between the optimized version of _FileListToArray (#77) and _FileListToArrayEx4.
This is the results (on my laptop) :
For 50000 files with $iFlag = 0 :
_FileListToArray = 646
_FileListToArrayEx4 = 683
For 50000 files with $iFlag = 1 :
_FileListToArray = 1333
_FileListToArrayEx4 = 1408
For 100000 files with $iFlag = 0 :
_FileListToArray = 1350
_FileListToArrayEx4 = 1426
For 100000 files with $iFlag = 1 :
_FileListToArray = 1339
_FileListToArrayEx4 = 1554
Like we can see, for 100000 files with $iFlag = 1, we lose about 200ms ... I think that I will survive on this lost.
If we compare with the original version of _FileListToArray, this new function remains almost twice as fast in spite of all its options.
Take you very much mister Spiff59 and BaKaMu. Great work.

are we at the stage _FileListToArrayEx4 will replace _FileListToArray with extra functionality?

Awsome Works

Improvement of included _FileListToArray function.

Recommended Posts

Link to comment

Share on other sites

Top Posters In This Topic

Top Posters In This Topic

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members