_FileListToArrayRec and regular expresions

[wljdflgio]

According to the manual, _FileListToArrayRec uses regular expressions to check the mask values. Nevertheless a correct mask according to https://www.autoitscript.com/autoit3/docs/tutorials/regexp/regexp.htm

does not work.

If I use mask "*.001;*.002;" then files a.001 and a.002 are matched. If I use ".[0-9]{3} They are not. Am I doing something wrong or is _FileListToArrayRec just not fully supporting Regex?

Thanks

Edited August 30, 2020 by wljdflgio

[Nine]

I believe there is a documentation error in _FileListToArray help file.  After looking at _FileListToArrayRec code, there is no RegEx involved in the function.  

[pseakins]

I think the help for _FileListToArray may be a misleading. It says that _FileListToArrayRec "compares all subfolders to the specified filter using a RegEx" but this does not necessarily mean that you can use a regular expression. I think it means that the function internally converts the arguments into some form of regular expression which it then applies.

[Melba23]

Hi, 

pseakins has it correctly. The filter you enter is internally converted into a RegEx to speed up the comparison code when searching - but the filter must be of the form indicated in the Help file.

Of interest I did produce a version of the function which would accept a RegEx as filter but it was never adopted - do people think this might be worth looking into again?

M23 

[Musashi]

16 minutes ago, Melba23 said:

Of interest I did produce a version of the function which would accept a RegEx as filter but it was never adopted - do people think this might be worth looking into again?

A crystal clear YES

[AspirinJunkie]

I am pretty sure that a wide range of user-defined functions already exists for exactly this use case.
Like many others, i have written a function that lists files/folders recursively with the possibility to filter by RegEx.
But it has a few extras: On the one hand you can directly pass multiple search locations. On the other hand you can do much more complex filtering, because you can pass your own matching function:

expandcollapse popup

; #FUNCTION# ======================================================================================
; Name ..........: _listFilesFlexible()
; Description ...: Finds files and/or folders in a directory tree with multiple filtering options
; Syntax ........: _listFilesFlexible($sSD, [Const $sPat = '', [Const $iF = 3, [Const $1BASED = 0]]])
; Parameters ....: $sSD           - search location[s] (you can separate multiple with "|")
;                  Const $sPat    - [optional] regular expression for filename matching or a function pointer for more complex matching
;                                   remark: RegEx is normally case-sensitive - add (?i) at pattern start to switch to case-insensitive
;                                   "" (Default): No filename matching - all elements will be retrieved
;                  Const $iF      - [optional] 1=only files, 2=only folders, 3=files and folders (default:3)
;                  Const $1BASED  - [optional] 0= zero-based - no count in $Array[0] (Default), 1= 1-based - count in $Array[0]
; Return values .: Success - Return Array with Matches
;                  Failure - Return "" and set @error
; Author ........: AspirinJunkie
; =================================================================================================
Func _listFilesFlexible($sSD, Const $sPat = '', Const $iF = 3, Const $1BASED = 0)
    Local $cEl = $1BASED, $cAS = 1 ; counter for number of elements vs. current array size
    Local $aRet[1] = [0]

    Local $aSD = StringSplit($sSD, '|', 1) ; stack for search folders
    Local $FFFF, $FFNF, $sDir, $sP

    Local $hDLL = DllOpen('kernel32.dll')
    Local $p_F = DllCall($hDLL, "ptr", "GetProcAddress", "handle", DllCall($hDLL, "handle", "GetModuleHandleW", "wstr", "kernel32.dll")[0], "str", "GetFileAttributesW")[0] ; get the function address of GetModuleHandleW
    If Not ($iF = 3 Or $iF = 1 Or $iF = 2) Then Return SetError(3, 0, "")

    If $sPat = "" Then
        Do
            ; pull next dir from stack
            $sDir = $aSD[$aSD[0]] & "\"
            $aSD[0] -= 1

            $FFFF = FileFindFirstFile($sDir & '*')
            If $FFFF <> -1 Then
                Do
                    $FFNF = FileFindNextFile($FFFF)
                    If @error Then ExitLoop
                    $sP = $sDir & $FFNF
                    If @extended Then ; folder

                        If BitAND($iF, 2) Then ; matched dir
                            If $cEl >= $cAS Then
                                $cAS += $cAS
                                ReDim $aRet[$cAS]
                            EndIf
                            $aRet[$cEl] = $sP
                            $cEl += 1
                        EndIf

                        ; ignore reparse points
                        If BitAND(DllCallAddress('dword', $p_F, 'wstr', $sP)[0], 0x400) Then ContinueLoop

                        ; push subfolder to folder stack
                        $aSD[0] += 1
                        If $aSD[0] = UBound($aSD) Then ReDim $aSD[UBound($aSD) * 2]
                        $aSD[$aSD[0]] = $sP

                    ElseIf BitAND($iF, 1) Then ; matched file
                        If $cEl >= $cAS Then
                            $cAS += $cAS
                            ReDim $aRet[$cAS]
                        EndIf
                        $aRet[$cEl] = $sP
                        $cEl += 1
                    EndIf
                Until 0
                FileClose($FFFF)
            EndIf
        Until $aSD[0] = 0

    ElseIf IsFunc($sPat) Then ; $sPat = function
        Do
            ; pull next dir from stack
            $sDir = $aSD[$aSD[0]] & "\"
            $aSD[0] -= 1

            $FFFF = FileFindFirstFile($sDir & '*')
            If $FFFF <> -1 Then
                Do
                    $FFNF = FileFindNextFile($FFFF)
                    If @error Then ExitLoop
                    $sP = $sDir & $FFNF
                    If @extended Then ; folder
                        If BitAND($iF, 2) And $sPat($sP) Then ; match by function
                            If $cEl >= $cAS Then
                                $cAS += $cAS
                                ReDim $aRet[$cAS]
                            EndIf
                            $aRet[$cEl] = $sP
                            $cEl += 1
                        EndIf

                        ; ignore reparse points
                        If BitAND(DllCallAddress('dword', $p_F, 'wstr', $sP)[0], 0x400) Then ContinueLoop

                        ; push subfolder to folder stack
                        $aSD[0] += 1
                        If $aSD[0] = UBound($aSD) Then ReDim $aSD[UBound($aSD) * 2]
                        $aSD[$aSD[0]] = $sP

                        ; file
                    ElseIf BitAND($iF, 1) And $sPat($sP) Then ; match by function
                        If $cEl >= $cAS Then
                            $cAS += $cAS
                            ReDim $aRet[$cAS]
                        EndIf
                        $aRet[$cEl] = $sP
                        $cEl += 1
                    EndIf
                Until 0
                FileClose($FFFF)
            EndIf
        Until $aSD[0] = 0

    Else ; $sPat = regex
        Do
            ; pull next dir from stack
            $sDir = $aSD[$aSD[0]] & "\"
            $aSD[0] -= 1

            $FFFF = FileFindFirstFile($sDir & '*')
            If $FFFF <> -1 Then
                Do
                    $FFNF = FileFindNextFile($FFFF)
                    If @error Then ExitLoop
                    $sP = $sDir & $FFNF
                    If @extended Then ; folder
                        If BitAND($iF, 2) And StringRegExp($FFNF, $sPat) Then ; match by regex
                            If $cEl >= $cAS Then
                                $cAS += $cAS
                                ReDim $aRet[$cAS]
                            EndIf
                            $aRet[$cEl] = $sP
                            $cEl += 1
                        EndIf

                        ; ignore reparse points
                        If BitAND(DllCallAddress('dword', $p_F, 'wstr', $sP)[0], 0x400) Then ContinueLoop

                        ; push subfolder to folder stack
                        $aSD[0] += 1
                        If $aSD[0] = UBound($aSD) Then ReDim $aSD[UBound($aSD) * 2]
                        $aSD[$aSD[0]] = $sP

                    ElseIf BitAND($iF, 1) And StringRegExp($FFNF, $sPat) Then ; match by regex
                        If $cEl >= $cAS Then
                            $cAS += $cAS
                            ReDim $aRet[$cAS]
                        EndIf
                        $aRet[$cEl] = $sP
                        $cEl += 1
                    EndIf

                Until 0
                FileClose($FFFF)
            EndIf
        Until $aSD[0] = 0
    EndIf

    DllClose($hDLL)
    If $1BASED = 1 Then $aRet[0] = $cEl - 1
    ReDim $aRet[$cEl]
    Return $aRet
EndFunc   ;==>_listFilesFlexible

 

Example for simple RegEx usage:

#include <Array.au3>

; all dll-files in system dir
$aFiles = _listFilesFlexible(@SystemDir, "(?i)\.dll$", 1)
_ArrayDisplay($aFiles)

More complex example (multiple search folders with search by file size):

#include <Array.au3>

; all dll files in system dir AND user profile dir which are greater than 5 mb
$aFiles = _listFilesFlexible(@SystemDir & "|" & @UserProfileDir, _myCheck, 1)
_ArrayDisplay($aFiles)


Func _myCheck(Const $sPath)
    If StringRight($sPath, 4) <> ".dll" Then Return 0
    If FileGetSize($sPath) > 5242880 Then Return 1
    Return 0
EndFunc

 

Edited September 24, 2020 by AspirinJunkie
performance improved of function

[wljdflgio]

Interesting, because wildcards * and ? did work for me, so some sub-set of RegEx seems to work.

[AspirinJunkie]

2 hours ago, wljdflgio said:

Interesting, because wildcards * and ? did work for me, so some sub-set of RegEx seems to work.

These are not complete regular expressions but only two placeholders provided by the underlying Windows API function >>FindFirstFileA<<. There is no more than these two.