UDF: traversing a directory tree

[thomasl]

I am currently translating some of my local Perl utilities to "pure" AU3: the goal is to learn the language and to see whether AU3 can replace some of the more complex Perl stuff I wrote over the years. (I especially would like to integrate my utilities with the AU3 GUI..() functions: Perl can be rather dry in that respect -- think command line.)

Anyway, among the first things I moved to AutoIt3 was a smallish UDF that helps with finding files and directories. These functions work w/o calling any external programs, they only use AU3 APIs.

They either do a single directory or will traverse a whole tree; they support both conventional wildcards (i.e. "w*.do?") and regular expressions ("^(?i)w.+\.(exe|dll|bat)$" will find w*.exe, w*.dll and w*.bat [1]).

There is not much in the way of finding filenames these functions can't do (if you do find something, tell me ). The price for that flexibility is that the advanced mode can be a little tricky to master.

The simple mode is easy: call the main function with a a path, pattern, the right flags... and it will return an array with the matching file and directory names:

$files=_DirWalk("c:\","*",1) ; reads all of c:\

The advanced mode is where the real fun starts: for that you have to define one or two callback functions that will be called for every file and directory that is found. The callbacks can do whatever you want, including terminating the search. (In fact, the easy mode is implemented in terms of these two callbacks, so there is a working example of this in the library itself.)

One significant advantage of the callback approach is that this allows complete integration into the GUI message loop. See the included GUI example if that's all Greek to you. But even for-non GUI uses the callback approach has advantages as it gives full control over the whole process.

Well, the proof is in the eating, as they say, so here's the GUI example code:

expandcollapse popup

#include <GUIConstants.au3>
#include <DirWalk.au3>

AutoItSetOption("MustDeclareVars",1)

Global $ed_Path,$ed_Pattern,$chk_Recurse,$chk_RE,$btn_Go,$btn_Stop,$btn_Copy,$ed_Out,$msg,$flags

GUICreate("DirWalkGUI",1024,768,197,115,$WS_SIZEBOX+$WS_SYSMENU)
GUICtrlCreateLabel("Path:",8,8,34,20)
$ed_Path=GUICtrlCreateInput("c:\",64,8,593,24)
GUICtrlCreateLabel("Pattern:",8,46,49,20)
$ed_Pattern=GUICtrlCreateInput("*",64,46,593,24)
$chk_Recurse=GUICtrlCreateCheckbox("Recurse",672,8,81,25)
$chk_RE=GUICtrlCreateCheckbox("Regular expression",672,48,153,25)
$btn_Go=GUICtrlCreateButton("Search!",872,8,81,24,0)
$btn_Stop=GUICtrlCreateButton("Stop!",872,46,81,24,0)
$btn_Copy=GUICtrlCreateButton("Copy to clipboard",8,96,153,25,0)
$ed_Out=GUICtrlCreateEdit("",8,128,1001,609)
GUICtrlSetState($btn_Stop,$GUI_DISABLE)
GUICtrlSetState($ed_Path,$GUI_FOCUS)
GUICtrlSetLimit($ed_Out,1024*1024)
GUISetState(@SW_SHOW)

Func FileCB($name,$isdir,$level)
    Local $msg
    $msg=GUIGetMsg()
    If $msg=$GUI_EVENT_CLOSE Then Exit
    If $msg=$btn_Stop Then Return $DW_BREAK_ALL
    If $isdir Then
        ; do nothing
    Else
        GUICtrlSetData($ed_Out,$name&@CRLF,1)
    Endif
    Return $DW_CONTINUE
EndFunc

While 1
    $msg=GUIGetMsg()
    Select
        Case $msg=$GUI_EVENT_CLOSE
            Exit
        Case $msg=$btn_Go
            GUICtrlSetData($ed_Out,"")
            GUICtrlSetState($btn_Go,$GUI_DISABLE)
            GUICtrlSetState($btn_Copy,$GUI_DISABLE)
            GUICtrlSetState($btn_Stop,$GUI_ENABLE)
            $flags=0
            If GUICtrlRead($chk_Recurse)=$GUI_CHECKED Then $flags=1
            If GUICtrlRead($chk_RE)=$GUI_CHECKED Then $flags+=2
            _DirWalk(GUICtrlRead($ed_Path),GUICtrlRead($ed_Pattern),$flags,"FileCB")
            GUICtrlSetState($btn_Stop,$GUI_DISABLE)
            GUICtrlSetState($btn_Copy,$GUI_ENABLE)
            GUICtrlSetState($btn_Go,$GUI_ENABLE)
        Case $msg=$btn_Copy
            ClipPut(GUICtrlRead($ed_Out))
    EndSelect
WEnd
oÝ÷ Ø(ì!jÛ"^­Ê'²^½êìÚºÚ"µÍÚ[ÛYH   Ð^K]LÉÝÂÚ[ÛYH Ñ[K]LÉÝÂÚ[ÛYH ÑØ[Ë]LÉÝÂ]]Ò]Ù]Ü[Û    ][ÝÓ]ÝXÛUÉ][ÝËJB]]Ò]Ù]Ü[Û    ][ÝÑ^[Ý[ÜÉ][ÝËJBÛØ[    ÌÍÜK ÌÍÜ  ÌÍÜÂY   ÌÍÐÛY[VÌOLÈ[RY    ÌÍÐÛY[VÌWOI][ÝË]É][ÝÈ[BIÌÍÜOIÌÍÐÛY[VÌBBWÑÜ]
    ÌÍÜK ÌÍÜBQ[ÙBBIÌÍÜOIÌÍÐÛY[VÌWBBIÌÍÜIÌÍÐÛY[VÌBQ[YIÌÍÜÏIÌÍÐÛY[VÌ×B[ÙBIÌÍÜOI][ÝØÎÌLÉ][ÝÈÈÛÚÈ[ÎÌLÂIÌÍÜI][ÝÝÚJ][ÝÈÈÛÚÈÜÚJIÌÍÜÏLHÈXÝÙHÝÛHÚÛHYB[YÛÛÛÛUÜ]J ][ÝÐÛÛXÝ[È[Ë][ÝÊBÛØ[ ÌÍÙWÑØ[Ê  ÌÍÜK ÌÍÜ  ÌÍÜÊBY  ÌÍÙÌH[PÛÛÛÛUÜ]J    ][ÝÐÔ^H[ÙNÔ    ][ÝÊBQÜ  ÌÍÚOLHÈ ÌÍÙÌBBPÛÛÛÛUÜ]J    ][ÝÈ  ][ÝÉ[ÉÌÍÙÉÌÍÚWI[ÐÔBS^[ÙBPÛÛÛÛUÜ]J ][ÝÐÔÝ[ÈÝ[Ô  ][ÝÊB[YÛÛÛÛUÜ]JÔBÈÈÙÈHØ[XÚÂ[È[PÐ   ÌÍÛ  ÌÍÙ  ÌÍÛ
BRY ÌÍÙ[BPÛÛÛÛUÜ]J  ][ÝÈ  ][ÝÊBQ[ÙBBPÛÛÛÛUÜ]J ][ÝÈ[N    ][ÝÊBQ[YPÛÛÛÛUÜ]J    ][ÝÉÌÍÛÌÍÈÉÌÍÛ  ÌÍ×PÔ   ][ÝÊBT]   ÌÍÑ×ÐÓÓSQB[[ÂÛÛÛÛUÜ]J  ][ÝÐØ[XÚÈ[ÙNÔ    ][ÝÊBÑØ[Ê  ÌÍÜK ÌÍÜ  ÌÍÜË    ][ÝÑ[PÐ][ÝÊBoÝ÷ ض¬µ»­ey«-*ÖjY»{ay@ÅÛö®¶­sb6æ6ÇVFRÖöæ6P¢6æ6ÇVFRfÇC´fÆRæS2fwC°£²6æ6ÇVFRfÇC´FV'Vuw&FRæS2fwC° ¤6öç7Bb33c´Euô4ôåDåTSÓ¤6öç7Bb33c´Euô%$T³Ó¤6öç7Bb33c´Euô%$TµôÄÃÓ  £²ÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒУ²fWrçFW&æÂgVæ7Föç0 ¤6öç7Bb33cµôEuô%$ô$õTäCÓc@¤vÆö&Âb33c¶uöGuGFW&âÂb33c¶uöGtfÆW5³ÒÂb33c¶uöGu&V7W'6RÂb33c¶uöGtçfW'BÂb33c¶uöGtfÆT4"Âb33c¶uöGtF$4"Âb33c¶uöGtÆWfVÀ £²çFW&æÂFVfVÇB6ÆÆ&6²f÷"fÆW0£²b33c¶æÖR2FRgVÆÂfÆVæÖRæRâæ6ÇVFærF£²b33c¶6F"2fÇC²fwC³bb33c¶æÖR2F&V7F÷'£²b33c¶ÆWfVÂ2FRF&V7F÷'ÆWfVÂvööBb6ÆÆVB&V7W'6fVÇ£°£²&WGW&ç2b33c´Euô4ôåDåTRFò6öçFçVRFRvÆ°£²÷"b33c´Euô%$T²Fò7F÷7W'&VçBF&V7F÷'vÆ°£²÷"b33c´Euô%$TµôÄÂFò7F÷vƶær6ö×ÆWFVǤgVæ2ôçFW&æÄfÆT4"b33c¶æÖRÂb33c¶6F"Âb33c¶ÆWfV b33c¶uöGtfÆW5³Ò³Ó bb33c¶uöGtfÆW5³ÓÕT&÷VæBb33c¶uöGtfÆW2FVâ&TFÒb33c¶uöGtfÆW5µT&÷VæBb33c¶uöGtfÆW2²b33cµôEuô%$ô$õTäEÐ b33c¶uöGtfÆW5²b33c¶uöGtfÆW5³ÕÓÒb33c¶æÖP bb33c¶6F"FVà ´FV'Vuw&FRgV÷C´F#¢W2VgV÷C²Âb33c¶æÖRÂb33c¶ÆWfV VÇ6P ´FV'Vuw&FRgV÷C´fÆS¢W2VgV÷C²Âb33c¶æÖRÂb33c¶ÆWfV VæD` &WGW&âb33c´Euô4ôåDåTP¤VæDgVæ0 £²çFW&æÂFVfVÇB6ÆÆ&6²f÷"F&V7F÷&W0£²b33c¶æÖR2FRgVÆÂF&V7F÷'æÖRæRâæ6ÇVFærF£²b33c¶ÆWfVÂ2FRF&V7F÷'ÆWfVÀ£°£²&WGW&âfÇVW26VRôçFW&æÄfÆT4"¤gVæ2ôçFW&æÄF$4"b33c¶æÖRÂb33c¶ÆWfV¢bb33c¶ÆWfVÂfwC³FVà ´FV'Vuw&FRgV÷C´VçG'¢W2VgV÷C²Âb33c¶æÖRÂb33c¶ÆWfV VÇ6P ´FV'Vuw&FRgV÷C´WB¢W2VgV÷C²Âb33c¶æÖRÂb33c¶ÆWfV VæD` &WGW&âb33c´Euô4ôåDåTP¤VæDgVæ0 £²çFW&æÂv÷&¶÷'6RÂ6â6ÆÂG6VÆb&V7W'6fVǤgVæ2ôF%vƵ÷&V7W'6Rb33c·F Æö6Âb33c¶ÓÂb33c¶g2Âb33c¶gVÆÂÂb33c¶fÆW5²b33cµôEuô%$ô$õTäEÒÂb33c¶2Âb33c¶@ b7G&æu&vBb33c·FÃfÇC²fwC²gV÷C²b3#²gV÷C²FVâb33c·Ff׳ÒgV÷C²b3#²gV÷C° b33c¶g3ÔfÆTfæDf'7DfÆRb33c·FfײgV÷C²¢gV÷C² bb33c¶g3ÒÓFVâ&WGW&à vÆR b33c¶³Ó bb33c¶ÕT&÷VæBb33c¶fÆW2FVâ&TFÒb33c¶fÆW5µT&÷VæBb33c¶fÆW2²b33cµôEuô%$ô$õTäEÐ b33c¶fÆW5²b33c¶ÓÔfÆTfæDæWDfÆRb33c¶g2 bW'&÷"FVâWDÆö÷ tVæ@ fÆT6Æ÷6Rb33c¶g2 b33c¶fÆW5³ÓÒb33c¶Ó f÷"b33c¶ÓFòb33c¶fÆW5³Ð b33c¶gVÆÃÒb33c·Ffײb33c¶fÆW5²b33c¶Ð b33c¶CÕô4F&V7F÷'b33c¶gVÆ bb&E÷"b33c¶uöGtçfW'BÅ7G&æu&VtWb33c¶fÆW5²b33c¶ÒÂb33c¶uöGuGFW&âFVà b33c¶3Ô6ÆÂb33c¶uöGtfÆT4"Âb33c¶gVÆÂÂb33c¶BÂb33c¶uöGtÆWfV bb33c¶2fÇC²fwC²b33c´Euô4ôåDåTRFVâWDÆö÷ VæD` bb33c¶BæBb33c¶uöGu&V7W'6RFVà b33c¶uöGtÆWfVÂ³Ó b33c¶3Òb33c´Euô4ôåDåTP b6ÆÂb33c¶uöGtF$4"Âb33c¶gVÆÂÂb33c¶uöGtÆWfVÂÓFVâb33c¶3ÕôF%vƵ÷&V7W'6Rb33c¶gVÆ 6ÆÂb33c¶uöGtF$4"Âb33c¶gVÆÂÂÒb33c¶uöGtÆWfV b33c¶uöGtÆWfVÂÓÓ bb33c¶3Òb33c´Euô%$TµôÄÂFVâWDÆö÷ VæD` æW@ &WGW&âb33c¶0¤VæDgVæ0 £²ÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒУ²WFW&æÂgVæ7Föç0 £²ô4F&V7F÷'b33c·2£²VÇW"gVæ7Föã¢&WGW&ç2fÇVRfwC³bb33c·22F&V7F÷'¤gVæ2ô4F&V7F÷'b33c·2 &WGW&â7G&ætå7G"fÆTvWDGG&"b33c·2ÂgV÷C´BgV÷C²¤VæDgVæ0 £²ôF%vƲb33c·FÒgV÷C²âb3#²gV÷C²Âb33c·GFW&ãÒgV÷C²¢gV÷C²Âb33c¶fÆw3ÓÂb33c¶fÆV6#ÒgV÷CµôçFW&æÄfÆT4"gV÷C²Âb33c¶F&6#ÒgV÷CµôçFW&æÄF$4"gV÷C²£²vÆ·2F&V7F÷'æB6â&WGW&âFRfÆW2öF&V7F÷&W2f÷Væ@£°£²b33c·F2FRFFòÆöö²ã²F26â&R3¢b3#²÷"C¢b3#µvæF÷w0£²b33c·GFW&â2FRGFW&âFòÆöö²f÷#²F26âVFW"&Râõ2×GRvÆF6&@£²æRâ¢æFóò÷"&VwVÆ"W&W76öâ6VRb33c¶fÆw2£°£²FRfÆw2&R&BÖ6öFVBfÇVW2æB6â&Rg&VVÇ6öÖ&æVC £²bb33c¶fÆw3ÓFVâb33c·F2vƶVBF÷vâ&V7W'6fVÇ£²bb33c¶fÆw3Ó"FVâb33c·GFW&â2&VwVÆ"W&W76öâFöâb33·Bf÷&vWBââââb33c²`£²÷RvçBFRGFW&âæ6÷&VB£²bb33c¶fÆw3ÓBFVâFR&W7VÇBöbFRGFW&âÖF62çfW'FVBæRâr¢æWP£²vÆÂ&WGW&âÆÂfÆW2¦W6WB¢r¢æWRfÆW2£°£²bb33c¶fÆV6"2FVfæVBB×W7B&RFRæÖRöbgVæ7FöâFB2W6VB2£²6ÆÆ&6²f÷"ÆÂfÆW2æBF&V7F÷&W2f÷VæC²6VRôçFW&æÄfÆT4"£²bb33c¶F&6"2FVfæVBB×W7B&RFRæÖRöbgVæ7FöâFB2W6VB2£²6ÆÆ&6²&Vf÷&RögFW"&V7W'6ærF÷vâ÷WF&V7F÷'²6VRôçFW&æÄF$4"£²bVFW"6ÆÆ&6²2æ÷BFVfæVBFRçFW&æÂFVfVÇB2W6VC¢ôçFW&æÄfÆT4"£²7F÷&W2ÆÂfÆW2æBF&V7F÷&W2ââ'&v62&WGW&æVB'F2gVæ7FöࣲôçFW&æÄF$4"FöW2æ÷Færà¤gVæ2ôF%vƲb33c·FÒgV÷C²âb3#²gV÷C²Âb33c·GFW&ãÒgV÷C²¢gV÷C²Âb33c¶fÆw3ÓÂb33c¶fÆV6#ÒgV÷CµôçFW&æÄfÆT4"gV÷C²Âb33c¶F&6#ÒgV÷CµôçFW&æÄF$4"gV÷C² b33c¶uöGu&V7W'6SÔ&DæBb33c¶fÆw2à b33c¶uöGtçfW'CÓ b&DæBb33c¶fÆw2ÃBFVâb33c¶uöGtçfW'CÓ²ÖÆöö²6ÆÇ'WBFöâb33·B6ævP b33c¶uöGtfÆW5³ÓÓ b33c¶uöGtfÆT4#Òb33c¶fÆV6  b33c¶uöGtF$4#Òb33c¶F&6  b&DæBb33c¶fÆw2Ã"FVà b33c¶uöGuGFW&ãÒb33c·GFW&à VÇ6P b33c·GFW&ãÕ7G&æu&WÆ6Rb33c·GFW&âÂgV÷C²âgV÷C²ÂgV÷C²b3#²âgV÷C² b33c·GFW&ãÕ7G&æu&WÆ6Rb33c·GFW&âÂgV÷C³ògV÷C²ÂgV÷C²âgV÷C² b33c¶uöGuGFW&ãÕ7G&æu&WÆ6Rb33c·GFW&âÂgV÷C²¢gV÷C²ÂgV÷C²â¢gV÷C² b33c¶uöGuGFW&ãÒgV÷CµâögV÷C²fײb33c¶uöGuGFW&âfײgV÷C²b33c²gV÷C° VæD` b33c¶uöGtÆWfVÃÓ ôF%vƵ÷&V7W'6Rb33c·F &WGW&âb33c¶uöGtfÆW0¤VæDgVæ0 £²ôF%7ÆB'&Vbb33c·GFW&âÄ'&Vbb33c·F£²VÇW"gVæ7Föã¢æFÆ6W2vÆF6&BGFW&â÷F6öÖ&æFöâ7V6F@£²B6âW6VBf÷"ôF%vƲ¢Wöâ6ÆÆærÂb33c·GFW&âöÆG2F&V7F÷'£²æBö÷"GFW&ââF227ÆBÂ6òFRWöâ&WGW&âb33c·GFW&â2§W7BFP£²GFW&â'BæBb33c·FâââvVÆÂÂFRF¢Òââ÷FW"v÷&G2Âb33c·F0£²÷fW'w&GFVâ'F26öFR⣲F2gVæ7FöâFöW2äõB7W÷'B&VwVÆ"W&W76öç2ÂöæÇvÆF6&G2b333°¤gVæ2ôF%7ÆB'&Vbb33c·GFW&âÄ'&Vbb33c·F Æö6Âb33c¶CÂb33c¶C"Âb33c¶C0 bb33c·GFW&âfÇC²fwC²gV÷C²¢gV÷C²FVà bô4F&V7F÷'b33c·GFW&âFVà b33c·FÒb33c·GFW&à b33c·GFW&ãÒgV÷C²¢gV÷C° VÇ6P õF7ÆBb33c·GFW&âÂb33c·FÂb33c¶CÂb33c¶C"Âb33c¶C2 ´FV'Vuw&FRgV÷C´gFW"7ÆC¢b333²W2b333²W2b333²W2b333²W2b333²gV÷C²Âb33c·FÂb33c¶CÂb33c¶C"Âb33c¶C2 b33c·Ff׳Òb33c¶C b33c·GFW&ãÒb33c¶C"fײb33c¶C0 bb33c·GFW&ãÒgV÷C²gV÷C²FVâb33c·GFW&ãÒgV÷C²¢gV÷C° VæD` VÇ6P b33c·FÒgV÷C²âb3#²gV÷C° VæD`¤VæDgVæ0

All three files in one zip: DirWalk.zip

Please give it a whirl and tell me what you think. (AU3-wise I am a noob so any hints how to do things more AU3-like are appreciated.)

---

[1] For those who are not (yet ) into regular expressions, this beast says the following:

^     match from the beginning of the string
(?i)   ignore case
w     match w
.+   match any character, one or more times (that's the * in wildcards)
\.   match a literal dot
(     match either
 exe   exe
|dll   or dll
|bat   or com
)
$     match end of string

That's all there is to it.

EDIT: Changed DirWalk.au3 so that it now includes <File.au3> on its own. Thanks gcriaco.

EDIT 2: Included a new flag for inverting the result of the pattern match and uploaded new version

Edited October 4, 2006 by thomasl

[gcriaco]

Compilation error:

DirWalkGui.au3:

DirWalk.au3(141,41) : ERROR: _PathSplit(): undefined function.

You should rename:

DirWalkCon.auc => DirWalkCon.au3

I dont' know Reg. Expressions: can you help me with an example for listing in a given folder every exe file except those ones that match the pattern unins*.exe?

Thanks

Peppe

[thomasl]

Compilation error:

DirWalkGui.au3:

DirWalk.au3(141,41) : ERROR: _PathSplit(): undefined function.

Oops. Changed DirWalk.au3 so that it now #includes <file.au3>. For the time being just #include this before you #include DirWalk.au3.

I dont' know Reg. Expressions: can you help me with an example for listing in a given folder every exe file except those ones that match the pattern unins*.exe?

Off the top of my head try this one: "^(??!unins.*\.exe).)*$" It works here. The problem is that this expression only works with the new regex engine which is still in beta. So that won't help you for the time being.

But I will prepare another version of DirWalk that'll allow you to invert the result of the pattern matching... this should make this sort of thing easier. Stay tuned.

[thomasl]

With the 3.2.1.7 beta finally out (which has the PCRE regular expression engine included) problems of the kind gcriaco described can be easily solved with an expression like that:

(?i)(?!^unins.*\.exe$)^.*\.exe$

This finds all .exe files but not uninst*.exe

I have also included another flag in the code that inverses the search pattern. This is probably more useful for the simpler wildcard matches:

$files=_DirWalk("c:\","*.exe",5)

returns all files that are not .exes.

[gcriaco]

With the 3.2.1.7 beta finally out (which has the PCRE regular expression engine included) problems of the kind gcriaco described can be easily solved with an expression like that:

(?i)(?!^unins.*\.exe$)^.*\.exe$

This finds all .exe files but not uninst*.exe

I have also included another flag in the code that inverses the search pattern. This is probably more useful for the simpler wildcard matches:

$files=_DirWalk("c:\","*.exe",5)

returns all files that are not .exes.

It works very fine. Please consider to submit your code to devs for inclusion in the AutoIt package as standard UDF (see UDF Standards)

Thanks

Peppe

Edited October 6, 2006 by gcriaco

[Yourcells]

Thomas, thank you very much for your great codes.