True Recursive _FileListToArray

[mLipok]

Error 1 extended 9

[Deye]

it's worth to be noted that in the period of time which I have used autoit
occasionally the commands where acting like broken, especially commands that deal with paths
I never got behind the root cause for this, even if I have deleted the whole dir structure, when rebuilt in same structure under the same folder name !, things simply wouldn't work ..
Now for instance if you changed to do the same but from a different root folder name .. Things would be back to normal  ..go figure

Maybe try doing the same from a different root folder name ..

[KaFu]

17 hours ago, Earthshine said:

I just edited the above c# program and ran it to find all dlls on c:\ and tested it against a Command Prompt with a stopwatch

CMD = 58 sec

my app = 32 seconds

fast.

Could you download SMF and stopwatch it too? I'm curious on a benchmark.

https://funk.eu/wp-content/plugins/download-monitor/download.php?id=232

Enter "dll" into the "for extensions" filter in the main GUI to speed up the search.

Edited November 11, 2017 by KaFu

[ptrex]

Hi mLipok 

Since PS and C# is mentioned you have the tools available to make it work one way or the or, correct ?

 

https://www.autoitscript.com/forum/topic/188158-net-common-language-runtime-clr-framework/

rgds

ptrex

 

[Melba23]

mLipok,

That error certainly means that no files were found. Sorry to sound flippant, but have you checked to make sure that the files are actually there - that the entire file/folder structure is present? What does FileExists say?

M23

[mLipok]

3 hours ago, Deye said:

Maybe try doing the same from a different root folder name ..

I was trying many things.

2 hours ago, ptrex said:

Hi mLipok 

Since PS and C# is mentioned you have the tools available to make it work one way or the or, correct ?

 

https://www.autoitscript.com/forum/topic/188158-net-common-language-runtime-clr-framework/

rgds

ptrex

 

Thanks, but I would to like solve the problem and stick with native AutoIt.

 

[mLipok]

On ‎11‎/‎11‎/‎2017 at 11:31 AM, Melba23 said:

mLipok,

That error certainly means that no files were found. Sorry to sound flippant, but have you checked to make sure that the files are actually there - that the entire file/folder structure is present? What does FileExists say?

M23

New testing procedure/example:

expandcollapse popup

#include <File.au3>
#include <Array.au3>

Global $g_sTestingDir = @ScriptDir & '\TEST_DIR_FOR_PDF_LIST ' & @YEAR & '-' & @MON & '-' & @MDAY

_Example_CreateTestingEnvironment()
_Example_TryingToFindFiles()
_Example_ListingFilesManually()

Func _Example_CreateTestingEnvironment()
    ConsoleWrite(@CRLF & '_Example_CreateTestingEnvironment()' & @CRLF)
    ConsoleWrite("! FileExists($g_sTestingDir)=" & FileExists($g_sTestingDir) & @CRLF)
    If FileExists($g_sTestingDir) Then Return

    Local $aSubdirs[4] = ['Katowice', 'Warszawa', 'Kraków', 'Ostrowiec Świętokrzyski']
    For $iDir_idx = 0 To UBound($aSubdirs) - 1
        DirCreate($g_sTestingDir & '\' & $aSubdirs[$iDir_idx])
        For $iFile_idx = 1 To 9
            FileWrite( _
                    $g_sTestingDir & '\' & $aSubdirs[$iDir_idx] & '\' & _
                    $iFile_idx & '__' & @HOUR & @MIN & @SEC & '.' & @MSEC & '.pdf' _
                    , '')
        Next
    Next

EndFunc   ;==>_Example_CreateTestingEnvironment

Func _Example_TryingToFindFiles()
    ConsoleWrite(@CRLF & '_Example_TryingToFindFiles()' & @CRLF)
    Local $aPDFFileList = _FileListToArrayRec($g_sTestingDir, '*.pdf', $FLTAR_FILES, $FLTAR_RECUR, $FLTAR_SORT, $FLTAR_FULLPATH)
    If @error Then ConsoleWrite('! ---> @error=' & @error & '  @extended=' & @extended & ' : _FileListToArrayRec()' & @CRLF)



    _ArrayDisplay($aPDFFileList, '$aPDFFileList')
EndFunc   ;==>_Example_TryingToFindFiles

Func _Example_ListingFilesManually()
    ConsoleWrite(@CRLF & '_Example_ListingFilesManually()' & @CRLF)
    Local $aSubdirs = _FileListToArrayRec($g_sTestingDir, '*', $FLTAR_FOLDERS, $FLTAR_RECUR, $FLTAR_SORT, $FLTAR_FULLPATH)
    If @error Then
        ConsoleWrite('! External1 ---> @error=' & @error & '  @extended=' & @extended & ' : ' & @CRLF)
    EndIf

    $aSubdirs = _FileListToArray($g_sTestingDir, '*', $FLTA_FOLDERS, True)
    If @error Then
        ConsoleWrite('! External2 ---> @error=' & @error & '  @extended=' & @extended & ' : ' & @CRLF)
        Return
    EndIf
    ConsoleWrite("> $aSubdirs[0]=" & $aSubdirs[0] & @CRLF)

    Local $aPDFFileList
    For $iDir_idx = 1 To $aSubdirs[0]
        ConsoleWrite("+ $aSubdirs[$iDir_idx]=" & $aSubdirs[$iDir_idx] & @CRLF)
        $aPDFFileList = _FileListToArrayRec($aSubdirs[$iDir_idx], '*.pdf', $FLTAR_FILES, $FLTAR_RECUR, $FLTAR_SORT, $FLTAR_NOPATH)
        If @error Then
            ConsoleWrite('! Internal2 ---> @error=' & @error & '  @extended=' & @extended & ' : ' & @CRLF)
            Return
        EndIf

        For $iFile_idx = 1 To $aPDFFileList[0]
            ConsoleWrite("-    " & $aPDFFileList[$iFile_idx] & @CRLF)
        Next


    Next

EndFunc   ;==>_Example_ListingFilesManually

I delete testing directory and run this example twice.

And the results for second run is:

expandcollapse popup

>"C:\Program Files (x86)\AutoIt3\SciTE\..\AutoIt3.exe" "C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /run /prod /ErrorStdOut /in "Z:\TOOLs\Macro\PROGRAMY\__TEST\_FileListToArrayRec.au3" /UserParams    
+>14:10:24 Starting AutoIt3Wrapper v.17.224.935.0 SciTE v.3.7.3.0   Keyboard:00000415  OS:WIN_10/  CPU:X64 OS:X64  Environment(Language:0415)  CodePage:0  utf8.auto.check:4
+>         SciTEDir => C:\Program Files (x86)\AutoIt3\SciTE   UserDir => C:\Users\MichaÅ‚\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper   SCITE_USERHOME => C:\Users\MichaÅ‚\AppData\Local\AutoIt v3\SciTE 
>Running AU3Check (3.3.15.2)  from:C:\Program Files (x86)\AutoIt3  input:Z:\TOOLs\Macro\PROGRAMY\__TEST\_FileListToArrayRec.au3
+>14:10:24 AU3Check ended.rc:0
>Running:(3.3.14.2):C:\Program Files (x86)\AutoIt3\autoit3.exe "Z:\TOOLs\Macro\PROGRAMY\__TEST\_FileListToArrayRec.au3"    
--> Press Ctrl+F11 to Restart or Ctrl+Break -or- F11 to Stop


_Example_CreateTestingEnvironment()
! FileExists($g_sTestingDir)=1

_Example_TryingToFindFiles()
! ---> @error=1  @extended=9 : _FileListToArrayRec()

_Example_ListingFilesManually()
! External1 ---> @error=1  @extended=9 : 
> $aSubdirs[0]=4
+ $aSubdirs[$iDir_idx]=Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Katowice
-    1__140210.799.pdf
-    2__140210.799.pdf
-    3__140210.799.pdf
-    4__140210.803.pdf
-    5__140210.803.pdf
-    6__140210.803.pdf
-    7__140210.803.pdf
-    8__140210.807.pdf
-    9__140210.807.pdf
+ $aSubdirs[$iDir_idx]=Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Kraków
-    1__140210.811.pdf
-    2__140210.811.pdf
-    3__140210.811.pdf
-    4__140210.815.pdf
-    5__140210.815.pdf
-    6__140210.815.pdf
-    7__140210.815.pdf
-    8__140210.815.pdf
-    9__140210.819.pdf
+ $aSubdirs[$iDir_idx]=Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Ostrowiec Œwiêtokrzyski
-    1__140210.819.pdf
-    2__140210.819.pdf
-    3__140210.823.pdf
-    4__140210.823.pdf
-    5__140210.823.pdf
-    6__140210.823.pdf
-    7__140210.827.pdf
-    8__140210.827.pdf
-    9__140210.827.pdf
+ $aSubdirs[$iDir_idx]=Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Warszawa
-    1__140210.807.pdf
-    2__140210.807.pdf
-    3__140210.807.pdf
-    4__140210.811.pdf
-    5__140210.811.pdf
-    6__140210.811.pdf
-    7__140210.811.pdf
-    8__140210.811.pdf
-    9__140210.811.pdf
+>14:10:24 AutoIt3.exe ended.rc:0
+>14:10:24 AutoIt3Wrapper Finished.
>Exit code: 0    Time: 0.8553

 

Edited November 12, 2017 by Melba23
Altered quote to code tags

[Earthshine]

ptrex converted me to c#, I blame you guys... lol i like to mix it up now.

[Earthshine]

ptrex converted me to c#, I blame you guys... lol i like to mix it up now.

i have about 30 pdfs in folders on my d: drive, the program.cs version i found runs it at under 2s, with paths included. you could make that an assembly to load and run, get rid of the main of course. , rename it and pass it two strings, drive and filespec (any windows filespec with wildcards works same as CMD)

then your app would rule at search speed. easier solution would be to just compile that console app and call it on the command line and have it pipe to a file, then read in your file to an array

 

example

Search.exe c:\ *.pdf > files.txt

 

Edited November 11, 2017 by Earthshine

[Melba23]

mLipok,

What a strange set of results! I still get the _ArrayDisplay of all 36 files on the initial search as well as the individual listings later on.

I can only think that the search is not recurring when you run it in the _Example_TryingToFindFiles function - but I have no idea why this might be the case. Thinking cap firmly in place.....

M23

Edit: A quick question: does the function find the files if you search for "*.*" rather then "*.pdf"?

Edit 2: Can you please run this script which has a modified function to tell us whether it is indeed searching in the subfolders:

expandcollapse popup

#include <File.au3>
#include <Array.au3>

Global $g_sTestingDir = @ScriptDir & '\TEST_DIR_FOR_PDF_LIST ' & @YEAR & '-' & @MON & '-' & @MDAY

_Example_CreateTestingEnvironment()
_Example_TryingToFindFiles()
_Example_ListingFilesManually()

Func _Example_CreateTestingEnvironment()
    ConsoleWrite(@CRLF & '_Example_CreateTestingEnvironment()' & @CRLF)
    ConsoleWrite("! FileExists($g_sTestingDir)=" & FileExists($g_sTestingDir) & @CRLF)
    If FileExists($g_sTestingDir) Then Return

    Local $aSubdirs[4] = ['Katowice', 'Warszawa', 'Kraków', 'Ostrowiec Swietokrzyski']
    For $iDir_idx = 0 To UBound($aSubdirs) - 1
        DirCreate($g_sTestingDir & '\' & $aSubdirs[$iDir_idx])
        For $iFile_idx = 1 To 9
            FileWrite( _
                    $g_sTestingDir & '\' & $aSubdirs[$iDir_idx] & '\' & _
                    $iFile_idx & '__' & @HOUR & @MIN & @SEC & '.' & @MSEC & '.pdf' _
                    , '')
        Next
    Next

EndFunc   ;==>_Example_CreateTestingEnvironment

Func _Example_TryingToFindFiles()
    ConsoleWrite(@CRLF & '_Example_TryingToFindFiles()' & @CRLF)
    Local $aPDFFileList = _FileListToArrayRec_Test($g_sTestingDir, '*.pdf', $FLTAR_FILES, $FLTAR_RECUR, $FLTAR_SORT, $FLTAR_FULLPATH)
    If @error Then ConsoleWrite('! ---> @error=' & @error & '  @extended=' & @extended & ' : _FileListToArrayRec_Test()' & @CRLF)



    _ArrayDisplay($aPDFFileList, '$aPDFFileList')
EndFunc   ;==>_Example_TryingToFindFiles

Func _Example_ListingFilesManually()
    ConsoleWrite(@CRLF & '_Example_ListingFilesManually()' & @CRLF)
    Local $aSubdirs = _FileListToArrayRec_Test($g_sTestingDir, '*', $FLTAR_FOLDERS, $FLTAR_RECUR, $FLTAR_SORT, $FLTAR_FULLPATH)
    If @error Then
        ConsoleWrite('! External1 ---> @error=' & @error & '  @extended=' & @extended & ' : ' & @CRLF)
    EndIf

    $aSubdirs = _FileListToArray($g_sTestingDir, '*', $FLTA_FOLDERS, True)
    If @error Then
        ConsoleWrite('! External2 ---> @error=' & @error & '  @extended=' & @extended & ' : ' & @CRLF)
        Return
    EndIf
    ConsoleWrite("> $aSubdirs[0]=" & $aSubdirs[0] & @CRLF)

    Local $aPDFFileList
    For $iDir_idx = 1 To $aSubdirs[0]
        ConsoleWrite("+ $aSubdirs[$iDir_idx]=" & $aSubdirs[$iDir_idx] & @CRLF)
        $aPDFFileList = _FileListToArrayRec_Test($aSubdirs[$iDir_idx], '*.pdf', $FLTAR_FILES, $FLTAR_RECUR, $FLTAR_SORT, $FLTAR_NOPATH)
        If @error Then
            ConsoleWrite('! Internal2 ---> @error=' & @error & '  @extended=' & @extended & ' : ' & @CRLF)
            Return
        EndIf

        For $iFile_idx = 1 To $aPDFFileList[0]
            ConsoleWrite("-    " & $aPDFFileList[$iFile_idx] & @CRLF)
        Next


    Next

EndFunc   ;==>_Example_ListingFilesManually

Func _FileListToArrayRec_Test($sFilePath, $sMask = "*", $iReturn = $FLTAR_FILESFOLDERS, $iRecur = $FLTAR_NORECUR, $iSort = $FLTAR_NOSORT, $iReturnPath = $FLTAR_RELPATH)
    If Not FileExists($sFilePath) Then Return SetError(1, 1, "")

    ; Check for Default keyword
    If $sMask = Default Then $sMask = "*"
    If $iReturn = Default Then $iReturn = $FLTAR_FILESFOLDERS
    If $iRecur = Default Then $iRecur = $FLTAR_NORECUR
    If $iSort = Default Then $iSort = $FLTAR_NOSORT
    If $iReturnPath = Default Then $iReturnPath = $FLTAR_RELPATH

    ; Check for valid recur value
    If $iRecur > 1 Or Not IsInt($iRecur) Then Return SetError(1, 6, "")

    Local $bLongPath = False
    ; Check for valid path
    If StringLeft($sFilePath, 4) == "\\?\" Then
        $bLongPath = True
    EndIf

    Local $sFolderSlash = ""
    ; Check if folders should have trailing \ and ensure that initial path does have one
    If StringRight($sFilePath, 1) = "\" Then
        $sFolderSlash = "\"
    Else
        $sFilePath = $sFilePath & "\"
    EndIf

    Local $asFolderSearchList[100] = [1]
    ; Add path to folder search list
    $asFolderSearchList[1] = $sFilePath

    Local $iHide_HS = 0, _
            $sHide_HS = ""
    ; Check for H or S omitted
    If BitAND($iReturn, 4) Then
        $iHide_HS += 2
        $sHide_HS &= "H"
        $iReturn -= 4
    EndIf
    If BitAND($iReturn, 8) Then
        $iHide_HS += 4
        $sHide_HS &= "S"
        $iReturn -= 8
    EndIf

    Local $iHide_Link = 0
    ; Check for link/junction omitted
    If BitAND($iReturn, 16) Then
        $iHide_Link = 0x400
        $iReturn -= 16
    EndIf

    Local $iMaxLevel = 0
    ; If required, determine \ count for max recursive level setting
    If $iRecur < 0 Then
        StringReplace($sFilePath, "\", "", 0, $STR_NOCASESENSEBASIC)
        $iMaxLevel = @extended - $iRecur
    EndIf

    Local $sExclude_List = "", $sExclude_List_Folder = "", $sInclude_List = "*"
    ; Check mask parameter
    Local $aMaskSplit = StringSplit($sMask, "|")
    ; Check for multiple sections and set values
    Switch $aMaskSplit[0]
        Case 3
            $sExclude_List_Folder = $aMaskSplit[3]
            ContinueCase
        Case 2
            $sExclude_List = $aMaskSplit[2]
            ContinueCase
        Case 1
            $sInclude_List = $aMaskSplit[1]
    EndSwitch

    Local $sInclude_File_Mask = ".+"
    ; Create Include mask for files
    If $sInclude_List <> "*" Then
        If Not __FLTAR_ListToMask($sInclude_File_Mask, $sInclude_List) Then Return SetError(1, 2, "")
    EndIf

    Local $sInclude_Folder_Mask = ".+"
    ; Set Include mask for folders
    Switch $iReturn
        Case 0
            ; Folders affected by mask if not recursive
            Switch $iRecur
                Case 0
                    ; Folders match mask for compatibility
                    $sInclude_Folder_Mask = $sInclude_File_Mask
            EndSwitch
        Case 2
            ; Folders affected by mask
            $sInclude_Folder_Mask = $sInclude_File_Mask
    EndSwitch

    Local $sExclude_File_Mask = ":"
    ; Create Exclude List mask for files
    If $sExclude_List <> "" Then
        If Not __FLTAR_ListToMask($sExclude_File_Mask, $sExclude_List) Then Return SetError(1, 3, "")
    EndIf

    Local $sExclude_Folder_Mask = ":"
    ; Create Exclude mask for folders
    If $iRecur Then
        If $sExclude_List_Folder Then
            If Not __FLTAR_ListToMask($sExclude_Folder_Mask, $sExclude_List_Folder) Then Return SetError(1, 4, "")
        EndIf
        ; If folders only
        If $iReturn = 2 Then
            ; Folders affected by normal mask
            $sExclude_Folder_Mask = $sExclude_File_Mask
        EndIf
    Else
        ; Folders affected by normal mask
        $sExclude_Folder_Mask = $sExclude_File_Mask
    EndIf

    ; Verify other parameters
    If Not ($iReturn = 0 Or $iReturn = 1 Or $iReturn = 2) Then Return SetError(1, 5, "")
    If Not ($iSort = 0 Or $iSort = 1 Or $iSort = 2) Then Return SetError(1, 7, "")
    If Not ($iReturnPath = 0 Or $iReturnPath = 1 Or $iReturnPath = 2) Then Return SetError(1, 8, "")

    ; Prepare for DllCall if required
    If $iHide_Link Then
        Local $tFile_Data = DllStructCreate("struct;align 4;dword FileAttributes;uint64 CreationTime;uint64 LastAccessTime;uint64 LastWriteTime;" & _
                "dword FileSizeHigh;dword FileSizeLow;dword Reserved0;dword Reserved1;wchar FileName[260];wchar AlternateFileName[14];endstruct")
        Local $hDLL = DllOpen('kernel32.dll'), $aDLL_Ret
    EndIf

    Local $asReturnList[100] = [0]
    Local $asFileMatchList = $asReturnList, $asRootFileMatchList = $asReturnList, $asFolderMatchList = $asReturnList
    Local $bFolder = False, _
            $hSearch = 0, _
            $sCurrentPath = "", $sName = "", $sRetPath = ""
    Local $iAttribs = 0, _
            $sAttribs = ''
    Local $asFolderFileSectionList[100][2] = [[0, 0]]
    ; Search within listed folders
    While $asFolderSearchList[0] > 0

        ; Set path to search
        $sCurrentPath = $asFolderSearchList[$asFolderSearchList[0]]

        ConsoleWrite("Searching: " & $sCurrentPath & @CRLF)

        ; Reduce folder search list count
        $asFolderSearchList[0] -= 1
        ; Determine return path to add to file/folder name
        Switch $iReturnPath
            ; Case 0 ; Name only
            ; Leave as ""
            Case 1 ;Relative to initial path
                $sRetPath = StringReplace($sCurrentPath, $sFilePath, "")
            Case 2 ; Full path
                If $bLongPath Then
                    $sRetPath = StringTrimLeft($sCurrentPath, 4)
                Else
                    $sRetPath = $sCurrentPath
                EndIf
        EndSwitch

        ; Get search handle - use code matched to required listing
        If $iHide_Link Then
            ; Use DLL code
            $aDLL_Ret = DllCall($hDLL, 'handle', 'FindFirstFileW', 'wstr', $sCurrentPath & "*", 'struct*', $tFile_Data)
            If @error Or Not $aDLL_Ret[0] Then
                ContinueLoop
            EndIf
            $hSearch = $aDLL_Ret[0]
        Else
            ; Use native code
            $hSearch = FileFindFirstFile($sCurrentPath & "*")
            ; If folder empty move to next in list
            If $hSearch = -1 Then
                ContinueLoop
            EndIf
        EndIf

        ; If sorting files and folders with paths then store folder name and position of associated files in list
        If $iReturn = 0 And $iSort And $iReturnPath Then
            __FLTAR_AddToList($asFolderFileSectionList, $sRetPath, $asFileMatchList[0] + 1)
        EndIf
        $sAttribs = ''

        ; Search folder - use code matched to required listing
        While 1
            ; Use DLL code
            If $iHide_Link Then
                ; Use DLL code
                $aDLL_Ret = DllCall($hDLL, 'int', 'FindNextFileW', 'handle', $hSearch, 'struct*', $tFile_Data)
                ; Check for end of folder
                If @error Or Not $aDLL_Ret[0] Then
                    ExitLoop
                EndIf
                ; Extract data
                $sName = DllStructGetData($tFile_Data, "FileName")
                ; Check for .. return - only returned by the DllCall
                If $sName = ".." Then
                    ContinueLoop
                EndIf
                $iAttribs = DllStructGetData($tFile_Data, "FileAttributes")
                ; Check for hidden/system attributes and skip if found
                If $iHide_HS And BitAND($iAttribs, $iHide_HS) Then
                    ContinueLoop
                EndIf
                ; Check for link attribute and skip if found
                If BitAND($iAttribs, $iHide_Link) Then
                    ContinueLoop
                EndIf
                ; Set subfolder flag
                $bFolder = False
                If BitAND($iAttribs, 16) Then
                    $bFolder = True
                EndIf
            Else
                ; Reset folder flag
                $bFolder = False
                ; Use native code
                $sName = FileFindNextFile($hSearch, 1)
                ; Check for end of folder
                If @error Then
                    ExitLoop
                EndIf
                $sAttribs = @extended
                ; Check for folder
                If StringInStr($sAttribs, "D") Then
                    $bFolder = True
                EndIf
                ; Check for Hidden/System
                If StringRegExp($sAttribs, "[" & $sHide_HS & "]") Then
                    ContinueLoop
                EndIf
            EndIf

            ; If folder then check whether to add to search list
            If $bFolder Then
                Select
                    Case $iRecur < 0 ; Check recur depth
                        StringReplace($sCurrentPath, "\", "", 0, $STR_NOCASESENSEBASIC)
                        If @extended < $iMaxLevel Then
                            ContinueCase ; Check if matched to masks
                        EndIf
                    Case $iRecur = 1 ; Full recur
                        If Not StringRegExp($sName, $sExclude_Folder_Mask) Then ; Add folder unless excluded
                            __FLTAR_AddToList($asFolderSearchList, $sCurrentPath & $sName & "\")
                        EndIf
                        ; Case $iRecur = 0 ; Never add
                        ; Do nothing
                EndSelect
            EndIf

            If $iSort Then ; Save in relevant folders for later sorting
                If $bFolder Then
                    If StringRegExp($sName, $sInclude_Folder_Mask) And Not StringRegExp($sName, $sExclude_Folder_Mask) Then
                        __FLTAR_AddToList($asFolderMatchList, $sRetPath & $sName & $sFolderSlash)
                    EndIf
                Else
                    If StringRegExp($sName, $sInclude_File_Mask) And Not StringRegExp($sName, $sExclude_File_Mask) Then
                        ; Select required list for files
                        If $sCurrentPath = $sFilePath Then
                            __FLTAR_AddToList($asRootFileMatchList, $sRetPath & $sName)
                        Else
                            __FLTAR_AddToList($asFileMatchList, $sRetPath & $sName)
                        EndIf
                    EndIf
                EndIf
            Else ; Save directly in return list
                If $bFolder Then
                    If $iReturn <> 1 And StringRegExp($sName, $sInclude_Folder_Mask) And Not StringRegExp($sName, $sExclude_Folder_Mask) Then
                        __FLTAR_AddToList($asReturnList, $sRetPath & $sName & $sFolderSlash)
                    EndIf
                Else
                    If $iReturn <> 2 And StringRegExp($sName, $sInclude_File_Mask) And Not StringRegExp($sName, $sExclude_File_Mask) Then
                        __FLTAR_AddToList($asReturnList, $sRetPath & $sName)
                    EndIf
                EndIf
            EndIf

        WEnd

        ; Close current search
        If $iHide_Link Then
            DllCall($hDLL, 'int', 'FindClose', 'ptr', $hSearch)
        Else
            FileClose($hSearch)
        EndIf

    WEnd

    ; Close the DLL if needed
    If $iHide_Link Then
        DllClose($hDLL)
    EndIf

    ; Sort results if required
    If $iSort Then
        Switch $iReturn
            Case 2 ; Folders only
                ; Check if any folders found
                If $asFolderMatchList[0] = 0 Then Return SetError(1, 9, "")
                ; Correctly size folder match list
                ReDim $asFolderMatchList[$asFolderMatchList[0] + 1]
                ; Copy size folder match array
                $asReturnList = $asFolderMatchList
                ; Simple sort list
                __ArrayDualPivotSort($asReturnList, 1, $asReturnList[0])
            Case 1 ; Files only
                ; Check if any files found
                If $asRootFileMatchList[0] = 0 And $asFileMatchList[0] = 0 Then Return SetError(1, 9, "")
                If $iReturnPath = 0 Then ; names only so simple sort suffices
                    ; Combine file match lists
                    __FLTAR_AddFileLists($asReturnList, $asRootFileMatchList, $asFileMatchList)
                    ; Simple sort combined file list
                    __ArrayDualPivotSort($asReturnList, 1, $asReturnList[0])
                Else
                    ; Combine sorted file match lists
                    __FLTAR_AddFileLists($asReturnList, $asRootFileMatchList, $asFileMatchList, 1)
                EndIf
            Case 0 ; Both files and folders
                ; Check if any root files or folders found
                If $asRootFileMatchList[0] = 0 And $asFolderMatchList[0] = 0 Then Return SetError(1, 9, "")
                If $iReturnPath = 0 Then ; names only so simple sort suffices
                    ; Combine file match lists
                    __FLTAR_AddFileLists($asReturnList, $asRootFileMatchList, $asFileMatchList)
                    ; Set correct count for folder add
                    $asReturnList[0] += $asFolderMatchList[0]
                    ; Resize and add file match array
                    ReDim $asFolderMatchList[$asFolderMatchList[0] + 1]
                    _ArrayConcatenate($asReturnList, $asFolderMatchList, 1)
                    ; Simple sort final list
                    __ArrayDualPivotSort($asReturnList, 1, $asReturnList[0])
                Else
                    ; Size return list
                    Local $asReturnList[$asFileMatchList[0] + $asRootFileMatchList[0] + $asFolderMatchList[0] + 1]
                    $asReturnList[0] = $asFileMatchList[0] + $asRootFileMatchList[0] + $asFolderMatchList[0]
                    ; Sort root file list
                    __ArrayDualPivotSort($asRootFileMatchList, 1, $asRootFileMatchList[0])
                    ; Add the sorted root files at the top
                    For $i = 1 To $asRootFileMatchList[0]
                        $asReturnList[$i] = $asRootFileMatchList[$i]
                    Next
                    ; Set next insertion index
                    Local $iNextInsertionIndex = $asRootFileMatchList[0] + 1
                    ; Sort folder list
                    __ArrayDualPivotSort($asFolderMatchList, 1, $asFolderMatchList[0])
                    Local $sFolderToFind = ""
                    ; Work through folder list
                    For $i = 1 To $asFolderMatchList[0]
                        ; Add folder to return list
                        $asReturnList[$iNextInsertionIndex] = $asFolderMatchList[$i]
                        $iNextInsertionIndex += 1
                        ; Format folder name for search
                        If $sFolderSlash Then
                            $sFolderToFind = $asFolderMatchList[$i]
                        Else
                            $sFolderToFind = $asFolderMatchList[$i] & "\"
                        EndIf
                        Local $iFileSectionEndIndex = 0, $iFileSectionStartIndex = 0
                        ; Find folder in FolderFileSectionList
                        For $j = 1 To $asFolderFileSectionList[0][0]
                            ; If found then deal with files
                            If $sFolderToFind = $asFolderFileSectionList[$j][0] Then
                                ; Set file list indexes
                                $iFileSectionStartIndex = $asFolderFileSectionList[$j][1]
                                If $j = $asFolderFileSectionList[0][0] Then
                                    $iFileSectionEndIndex = $asFileMatchList[0]
                                Else
                                    $iFileSectionEndIndex = $asFolderFileSectionList[$j + 1][1] - 1
                                EndIf
                                ; Sort files if required
                                If $iSort = 1 Then
                                    __ArrayDualPivotSort($asFileMatchList, $iFileSectionStartIndex, $iFileSectionEndIndex)
                                EndIf
                                ; Add files to return list
                                For $k = $iFileSectionStartIndex To $iFileSectionEndIndex
                                    $asReturnList[$iNextInsertionIndex] = $asFileMatchList[$k]
                                    $iNextInsertionIndex += 1
                                Next
                                ExitLoop
                            EndIf
                        Next
                    Next
                EndIf
        EndSwitch
    Else ; No sort
        ; Check if any file/folders have been added
        If $asReturnList[0] = 0 Then Return SetError(1, 9, "")
        ; Remove any unused return list elements from last ReDim
        ReDim $asReturnList[$asReturnList[0] + 1]

    EndIf

    Return $asReturnList
EndFunc   ;==>_FileListToArrayRec_Test

_Example_TryingToFindFiles()
Searching: M:\Program\Au3 Scripts\TEST_DIR_FOR_PDF_LIST 2017-11-11\
Searching: M:\Program\Au3 Scripts\TEST_DIR_FOR_PDF_LIST 2017-11-11\Warszawa\
Searching: M:\Program\Au3 Scripts\TEST_DIR_FOR_PDF_LIST 2017-11-11\Ostrowiec Swietokrzyski\
Searching: M:\Program\Au3 Scripts\TEST_DIR_FOR_PDF_LIST 2017-11-11\Kraków\
Searching: M:\Program\Au3 Scripts\TEST_DIR_FOR_PDF_LIST 2017-11-11\Katowice\

 

Edited November 11, 2017 by Melba23

[mLipok]

Bingo:

expandcollapse popup

>"Z:\AutoItPortable\App\SciTE\..\AutoIt3.exe" "Z:\AutoItPortable\App\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /run /prod /ErrorStdOut /in "Z:\TOOLs\Macro\PROGRAMY\__TEST\_FileListToArrayRec_Melba23.au3" /UserParams    
+>19:20:11 Starting AutoIt3Wrapper v.17.224.935.4 SciTE v.4.0.2.0   Keyboard:00000415  OS:WIN_10/  CPU:X64 OS:X64  Environment(Language:0415)  CodePage:0  utf8.auto.check:4
+>         SciTEDir => Z:\AutoItPortable\App\SciTE   UserDir => C:\Users\MichaÅ‚\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper   SCITE_USERHOME => C:\Users\MichaÅ‚\AppData\Local\AutoIt v3\SciTE 
>Running AU3Check (3.3.15.4)  from:Z:\AutoItPortable\App  input:Z:\TOOLs\Macro\PROGRAMY\__TEST\_FileListToArrayRec_Melba23.au3
+>19:20:11 AU3Check ended.rc:0
>Running:(3.3.14.2):Z:\AutoItPortable\App\autoit3.exe "Z:\TOOLs\Macro\PROGRAMY\__TEST\_FileListToArrayRec_Melba23.au3"    
--> Press Ctrl+F11 to Restart or Ctrl+Break -or- F11 to Stop


_Example_CreateTestingEnvironment()
! FileExists($g_sTestingDir)=1

_Example_TryingToFindFiles()
Searching: Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\
Searching: Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Warszawa\
Searching: Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Ostrowiec Œwiêtokrzyski\
Searching: Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Kraków\
Searching: Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Katowice\

_Example_ListingFilesManually()
Searching: Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\
Searching: Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Warszawa\
Searching: Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Ostrowiec Œwiêtokrzyski\
Searching: Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Kraków\
Searching: Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Katowice\
> $aSubdirs[0]=4
+ $aSubdirs[$iDir_idx]=Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Katowice
Searching: Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Katowice\
-    1__140210.799.pdf
-    2__140210.799.pdf
-    3__140210.799.pdf
-    4__140210.803.pdf
-    5__140210.803.pdf
-    6__140210.803.pdf
-    7__140210.803.pdf
-    8__140210.807.pdf
-    9__140210.807.pdf
+ $aSubdirs[$iDir_idx]=Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Kraków
Searching: Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Kraków\
-    1__140210.811.pdf
-    2__140210.811.pdf
-    3__140210.811.pdf
-    4__140210.815.pdf
-    5__140210.815.pdf
-    6__140210.815.pdf
-    7__140210.815.pdf
-    8__140210.815.pdf
-    9__140210.819.pdf
+ $aSubdirs[$iDir_idx]=Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Ostrowiec Œwiêtokrzyski
Searching: Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Ostrowiec Œwiêtokrzyski\
-    1__140210.819.pdf
-    2__140210.819.pdf
-    3__140210.823.pdf
-    4__140210.823.pdf
-    5__140210.823.pdf
-    6__140210.823.pdf
-    7__140210.827.pdf
-    8__140210.827.pdf
-    9__140210.827.pdf
+ $aSubdirs[$iDir_idx]=Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Warszawa
Searching: Z:\TOOLs\Macro\PROGRAMY\__TEST\TEST_DIR_FOR_PDF_LIST 2017-11-11\Warszawa\
-    1__140210.807.pdf
-    2__140210.807.pdf
-    3__140210.807.pdf
-    4__140210.811.pdf
-    5__140210.811.pdf
-    6__140210.811.pdf
-    7__140210.811.pdf
-    8__140210.811.pdf
-    9__140210.811.pdf
+>19:20:17 AutoIt3.exe ended.rc:0
+>19:20:17 AutoIt3Wrapper Finished.
>Exit code: 0    Time: 6.745

Currently I do not have time to go into the changes, so I'll ask straight:
Where the magic is ?

.... I know, I know in your mind

 

Edited November 12, 2017 by Melba23
Altered quote to code tags

[Melba23]

mLipok,

All I did was add a ConsoleWrite to show which folder was being searched. Adding a short delay like that makes me wonder if you are working on a drive which does not update quickly enough to allow _Example_TryingToFindFiles to find the files, whereas by the time _Example_ListingFilesManually runs, they are visible.

Can you please retry your original script using the standard _FLTAR function, but run the following 2 tests:

1. Simply reverse the order of the 2 function calls, so the UDF runs second
2. Keep the current order, but add a Sleep(100) before calling _Example_TryingToFindFiles

and see what happens.

M23

Edited November 11, 2017 by Melba23
Fixed formatting

[argumentum]

try this:

expandcollapse popup

#include <File.au3>
#include <Array.au3>

Global $g_sTestingDir = @ScriptDir & '\TEST_DIR_FOR_PDF_LIST ' & @YEAR & '-' & @MON & '-' & @MDAY

;~ _Example_CreateTestingEnvironment()
_Example_TryingToFindFiles()
;~ _Example_ListingFilesManually()
Local $t = TimerInit()
Local $a = ExampleRecursive($g_sTestingDir, ".pdf")
$t = TimerDiff($t)
_ArrayDisplay($a, "ExampleRecursive() [" & Round($t, 5) & " ms.]")

Func _Example_CreateTestingEnvironment()
    ConsoleWrite(@CRLF & '_Example_CreateTestingEnvironment()' & @CRLF)
    ConsoleWrite("! FileExists($g_sTestingDir)=" & FileExists($g_sTestingDir) & @CRLF)
    If FileExists($g_sTestingDir) Then Return
    Local $aSubdirs[4] = ['Katowice', 'Warszawa', 'Kraków', 'Ostrowiec Świętokrzyski']
    For $iDir_idx = 0 To UBound($aSubdirs) - 1
        DirCreate($g_sTestingDir & '\' & $aSubdirs[$iDir_idx])
        For $iFile_idx = 1 To 9
            FileWrite( _
                    $g_sTestingDir & '\' & $aSubdirs[$iDir_idx] & '\' & _
                    $iFile_idx & '__' & @HOUR & @MIN & @SEC & '.' & @MSEC & '.pdf' _
                    , '')
        Next
    Next

EndFunc   ;==>_Example_CreateTestingEnvironment

Func _Example_TryingToFindFiles()
    ConsoleWrite(@CRLF & '_Example_TryingToFindFiles()' & @CRLF)
    Local $t = TimerInit()
    Local $aPDFFileList = _FileListToArrayRec($g_sTestingDir, '*.pdf', $FLTAR_FILES, $FLTAR_RECUR, $FLTAR_SORT, $FLTAR_FULLPATH)
    $t = TimerDiff($t)
    If @error Then ConsoleWrite('! ---> @error=' & @error & '  @extended=' & @extended & ' : _FileListToArrayRec()' & @CRLF)
    _ArrayDisplay($aPDFFileList, '$aPDFFileList - ' & Round($t, 5))
EndFunc   ;==>_Example_TryingToFindFiles

Func _Example_ListingFilesManually()
    ConsoleWrite(@CRLF & '_Example_ListingFilesManually()' & @CRLF)
    Local $aSubdirs = _FileListToArrayRec($g_sTestingDir, '*', $FLTAR_FOLDERS, $FLTAR_RECUR, $FLTAR_SORT, $FLTAR_FULLPATH)
    If @error Then
        ConsoleWrite('! External1 ---> @error=' & @error & '  @extended=' & @extended & ' : ' & @CRLF)
    EndIf

    $aSubdirs = _FileListToArray($g_sTestingDir, '*', $FLTA_FOLDERS, True)
    If @error Then
        ConsoleWrite('! External2 ---> @error=' & @error & '  @extended=' & @extended & ' : ' & @CRLF)
        Return
    EndIf
    ConsoleWrite("> $aSubdirs[0]=" & $aSubdirs[0] & @CRLF)

    Local $aPDFFileList
    For $iDir_idx = 1 To $aSubdirs[0]
        ConsoleWrite("+ $aSubdirs[$iDir_idx]=" & $aSubdirs[$iDir_idx] & @CRLF)
        $aPDFFileList = _FileListToArrayRec($aSubdirs[$iDir_idx], '*.pdf', $FLTAR_FILES, $FLTAR_RECUR, $FLTAR_SORT, $FLTAR_NOPATH)
        If @error Then
            ConsoleWrite('! Internal2 ---> @error=' & @error & '  @extended=' & @extended & ' : ' & @CRLF)
            Return
        EndIf

        For $iFile_idx = 1 To $aPDFFileList[0]
            ConsoleWrite("-    " & $aPDFFileList[$iFile_idx] & @CRLF)
        Next


    Next

EndFunc   ;==>_Example_ListingFilesManually

Func ExampleRecursive($sFilePath, $sFilePattern)
;~  ConsoleWrite('Func ExampleRecursive("' & $sFilePath & '", "' & $sFilePattern & '")' & @CRLF)
    Local Static $uReturn = 0, $aReturn[1] = [0], $iReturnAdd = 10 ; <-- this should be greater in production
    Local $iReturnArray = 0
    If $uReturn = 0 Then
        ReDim $aReturn[$iReturnAdd]
        $uReturn = $iReturnAdd - 1
        $iReturnArray = 1
    EndIf

    ; Assign a Local variable the search handle of all files in the current directory.
    Local $hSearch = FileFindFirstFile($sFilePath & "\*")

    ; Check if the search was successful, if not display a message and return False.
    If $hSearch = -1 Then
        MsgBox($MB_SYSTEMMODAL, "", "Error: No files/directories matched the search pattern.")
        Return False
    EndIf

    ; Assign a Local variable the empty string which will contain the files names found.
    Local $sFileName = "", $iResult = 0

    While 1
        $sFileName = FileFindNextFile($hSearch)
        ; If there is no more file matching the search.
        If @error Then ExitLoop

        If $aReturn[0] = $uReturn Then
            $uReturn += $iReturnAdd
            ReDim $aReturn[$uReturn + 1]
        EndIf

;~      ConsoleWrite(">" & $sFilePath & "\" & $sFileName & @TAB & FileGetAttrib($sFilePath & "\" & $sFileName) & @CRLF)

        If StringInStr(FileGetAttrib($sFilePath & "\" & $sFileName), "D") Then
            ExampleRecursive($sFilePath & "\" & $sFileName, $sFilePattern)
        Else
            If StringRight($sFilePath & "\" & $sFileName, StringLen($sFilePattern)) = $sFilePattern Then
                $aReturn[0] += 1
                $aReturn[$aReturn[0]] = $sFilePath & "\" & $sFileName
            EndIf
        EndIf

;~         ; Display the file name.
;~         $iResult = MsgBox(BitOR($MB_SYSTEMMODAL, $MB_OKCANCEL), "", "File: " & $sFileName)
;~         If $iResult <> $IDOK Then ExitLoop ; If the user clicks on the cancel/close button.
    WEnd

    ; Close the search handle.
    FileClose($hSearch)
    If $iReturnArray Then
        ReDim $aReturn[$aReturn[0] + 1]
        Return $aReturn
    Else
        Return SetError(1, 0, "")
    EndIf
EndFunc   ;==>ExampleRecursive

looks faster

[LarsJ]

mLipok, Are you sure there are no issues with your disks? There was an issue here too.

[mLipok]

15 hours ago, Melba23 said:

Simply reverse the order of the 2 function calls, so the UDF runs second

_Example_CreateTestingEnvironment()
;~ Sleep(1000)
_Example_ListingFilesManually()
_Example_TryingToFindFiles()

RESULT: The same problem.

15 hours ago, Melba23 said:

Keep the current order, but add a Sleep(100) before calling _Example_TryingToFindFiles

_Example_CreateTestingEnvironment()
Sleep(1000)
_Example_TryingToFindFiles()
_Example_ListingFilesManually()

RESULT: The same problem.

 

[Melba23]

mLipok,

I am out of ideas I am afraid. I have no idea why adding a simple ConsoleWrite internally made the function work - it seems to work for everyone else without it.

M23

[Factfinder]

_FileListToArrayRec is a great function and I have used it a lot. In rare cases on some systems it didn't properly worked or even hanged.  In those cases omitting symbolic link/junctions from the search helped. You may give this a try too by adding +$FLTAR_NOLINK to the filter:

Local $aPDFFileList = _FileListToArrayRec($g_sTestingDir, '*.pdf', $FLTAR_FILES+$FLTAR_NOLINK, $FLTAR_RECUR, $FLTAR_SORT, $FLTAR_FULLPATH)

 

[Melba23]

Factfinder,

Interesting - thanks for that bit of info. If you use $FLTAR_NOLINK then the UDF uses the FileAttributes element of a struct created by a FindNextFileW call to kernel32 DLL to determine whether the returned name is a link - and also whether it is a folder. If you do not use that parameter, the UDF uses the native FileFindNextFile function which returns the attributes in @extended - which value is used to determine if it is a folder. So I can well understand why there might be a difference between the 2 cases (with/without $FLTAR_NOLINK) and it definitely worth mLipok giving it a try.

M23

 

 

 

[mLipok]

I find out what was the issue, and I know now that this was my fault.
I describe in a moment what happend.

 

[mLipok]

There was a Time when I was back to Au3.3.12.x in some of my projects, and I'm still using this version from about 2 years.

I was trying to work with the same INCLUDE so I found that in my cases, the only one problem was related to _FileListToArrayRec().

In Au3.3.12.x this line:
 

$sName = FileFindNextFile($hSearch, 1)

will fire errors as in Au3.3.12.x there was only one parameter.

So I just now notice that I still have:

$sName = FileFindNextFile($hSearch)

instead:

$sName = FileFindNextFile($hSearch, 1)

And when recently I start writing simple script with Au3.3.14.x this came to me with exactly this problem (as I described in this thread), which in this case I had from about 1 week.

Sorry for confusion, this is really my own issue.

 

Edited November 12, 2017 by mLipok