True Recursive _FileListToArray

[mLipok]

I know there is _FileListToArrayRec()

but in help FIle you can read:
Despite the name, this UDF is iterative, not recursive.

and of course I mean real recursive.

I need to find all PDF files in all subfolders.

Do you know any solution ?

 

[Earthshine]

well, you could blend C# together for better/faster searching results, no?

https://stackoverflow.com/questions/14305581/method-to-get-all-files-within-folder-and-subfolders-that-will-return-a-list

 

 

Edited November 9, 2017 by Earthshine

[TheDcoder]

I doubt it is possible in pure AutoIt, because of the stack overflow error. The interpreter will at some point stop recursive function calls to prevent stack overflow . A reliable way of doing this would be to use Windows API (Win32, WMI etc.)

[Melba23]

mLipok,

There is a working recursive file listing script in the "Recursion" tutorial in the Wiki.

M23

[BrewManNH]

2 hours ago, mLipok said:

I need to find all PDF files in all subfolders.

And _FileListToArrayRec doesn't do this already?

[mLipok]

16 minutes ago, BrewManNH said:

And _FileListToArrayRec doesn't do this already?

No

[mLipok]

17 minutes ago, Melba23 said:

mLipok,

There is a working recursive file listing script in the "Recursion" tutorial in the Wiki.

M23

Thank you. I will look there.

[orbs]

@mLipok, to my experience, the iterative method is by far faster than the recursive one (some 30%-50% faster). and if _FileListToArrayRec does not do what you want, there must be a malfunction somewhere, because it is supposed to. this works for me:

#include <File.au3>
#include <Array.au3>
Global $aResult = _FileListToArrayRec('D:\', '*.pdf', $FLTAR_FILES, $FLTAR_RECUR)
_ArrayDisplay($aResult)

 

Edited November 9, 2017 by orbs

[iamtheky]

Ps?

local $Dir , $File

_PsFileSearch_Rec("C:\Users" , "*.pdf")

Func _PsFileSearch_Rec($Dir , $File)
    run('cmd /k "powershell "dir -Path ' & $Dir & ' -Filter ' & $File & ' -Recurse" | %{$_.FullName}')
EndFunc ;PsFileSearch_Rec

 

[BrewManNH]

27 minutes ago, iamtheky said:

Ps?

Now you're just showing off.

[Earthshine]

you guys rule.

[Melba23]

mLipok,

I did not realise from your initial post you felt that _FileListToArrayRec does not function correctly in your specific case. As the author of the function, could I ask for more details of why you believe this to be true and sight of the code that you used.

M23

[Earthshine]

In case you cared, I found this code on Stack Exchange for searching with C#, check it... fast searching. Uses Threading to parallelise the search! I love that site. You could modify it to return an array or whatever.

I made it a console app. you give it a drive followed by a space then the file type, you can use wildcards.

such as ConsoleApp1 C:\ *.pdf and watch it fly! just need to add a try/catch to catch when the access to a given path is denied by security, you can pipe that to a file once you sort that out and then you could read your file into an array, or just use this from AutoIt, adding in the trap for access denied. also needs to check that you passed parameters before running. lol. samples.

 

 

Program.cs

Edited November 14, 2017 by Earthshine
updated source to check for arguments passed, also, if you target .NET 4.6.2 there is no MaxPath errors

[mLipok]

1 hour ago, Melba23 said:

mLipok,

I did not realise from your initial post you felt that _FileListToArrayRec does not function correctly in your specific case. As the author of the function, could I ask for more details of why you believe this to be true and sight of the code that you used.

M23

Sure give me few hours .

 

[Earthshine]

I just edited the above c# program and ran it to find all dlls on c:\ and tested it against a Command Prompt with a stopwatch

CMD = 58 sec

my app = 32 seconds

fast.

Edited November 10, 2017 by Earthshine
did an average of time tests

[iamtheky]

can we see the code for your test? I am surprised the improvement was only double. 

[Earthshine]

here. i posted it as a download link above too. on my home notebook it runs great, 20 secs to find all dlls vs 60.

my first test was on a build machine with MANY thousands of DLLs

expandcollapse popup

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
    class Program
    {

        public static void Main(string[] args)
        {
            if (args.Length < 2 )
            {
            Console.WriteLine(value: "ERROR: args = null please provide a path and a file specification to search for.");
            Console.WriteLine(value: "Example: list c:\\ *.dll");
        }
        else
            {
               DirectoryInfo testdr = new DirectoryInfo(path: args[0]);
               WalkDirectoryTree(testdr, searchname: args[1]);
            }
        }
    
    public static void WalkDirectoryTree(DirectoryInfo dr, string searchname)
    {
        System.IO.FileInfo[] files = null;
        DirectoryInfo[] subDirs = null;
        try
        {
            files = dr.GetFiles(searchname);
        }
        catch (Exception ex)
        {
            //Console.WriteLine(ex.Message);
        }

        if (files != null)
        {
            foreach (FileInfo fi in files)
            {
                string dir = fi.DirectoryName;
                string file = fi.ToString();
                Console.WriteLine(value: dir + file);
            }
            subDirs = dr.GetDirectories();

            Parallel.ForEach(subDirs, dir => WalkDirectoryTree(dir, searchname));
        }
    }

    public override bool Equals(object obj)
    {
        return base.Equals(obj);
    }

    public override int GetHashCode()
    {
        return base.GetHashCode();
    }

    public override string ToString()
    {
        return base.ToString();
    }
}

 

Edited November 14, 2017 by Earthshine
check for arguments

[Earthshine]

here is the whole project so you don't need to fart around. mind you, this was 100 percent googled code off of StackOverflow... lol i try not to reinvent wheels. You can get a free Visual Studio 2017 to run this, heck I bet you could do it with Visual Code, also free and adorable. love that program.

use the Debug exe, was last one compiled... lol

 

ConsoleApp1.zip

Edited November 14, 2017 by Earthshine
check for arguments, targets .Net 4.6.2 so no Maximum path bullcrap

[mLipok]

On 9.11.2017 at 9:14 PM, mLipok said:

Sure give me few hours .

little late...

#include <File.au3>
#include <Array.au3>

Global $g_sTestingDir = @ScriptDir & '\TEST_DIR_FOR_PDF_LIST ' & @YEAR & '-' & @MON & '-' & @MDAY

_Example_CreateTestingEnvironment()
_Example_TryingToFindFiles()

Func _Example_CreateTestingEnvironment()
    if FileExists($g_sTestingDir) then Return

    Local $aSubdirs[4] = ['Katowice', 'Warszawa', 'Kraków', 'Ostrowiec Świętokrzyski']
    For $iDir_idx = 0 To UBound($aSubdirs) - 1
        DirCreate($g_sTestingDir & '\' & $aSubdirs[$iDir_idx])
        For $iFile_idx = 1 To 9
            FileWrite( _
                    $g_sTestingDir & '\' & $aSubdirs[$iDir_idx] & '\' & _
                    $iFile_idx & '__' & @HOUR & @MIN & @SEC & '.' & @MSEC & '.pdf' _
                    , '')
        Next
    Next

EndFunc   ;==>_Example_CreateTestingEnvironment

Func _Example_TryingToFindFiles()
    Local $aPDFFileList = _FileListToArrayRec($g_sTestingDir, '*.pdf', $FLTAR_FILES, $FLTAR_RECUR, $FLTAR_SORT, $FLTAR_FULLPATH)
    If @error Then MsgBox($MB_ICONERROR, '_FileListToArrayRec()', '@error = ' & @error & @CRLF & '@extended = ' & @extended)

    _ArrayDisplay($aPDFFileList, '$aPDFFileList')
EndFunc   ;==>_Example_TryingToFindFiles

EDIT1: Example changed

Edited November 11, 2017 by mLipok

[Melba23]

mLipok,

That example works fine for me (as I would expect) and I get all 36 pdf files returned. What do you see?

M23