FileSystemMonitor UDF

[seangriffin]

This is a UDF that allows users to monitor the file system, using a combination of two internal Windows API Functions:

• ReadDirectoryChangesW; and
• SHChangeNotifyRegister

The purpose of this UDF is to record all file system events occurring within a given path.

REQUIREMENTS:

• AutoIt3 3.2 or higher

LIST OF FUNCTIONS:

_FileSysMonSetup($monitor_type = 3, $dir_monitor_path = "C:\", $shell_monitor_path = "")

_FileSysMonSetDirMonPath($dir_monitor_path = "C:\")

_FileSysMonSetShellMonPath($dir_monitor_path = "")

_FileSysMonDirEventHandler()

_FileSysMonShellEventHandler()

EXAMPLES:

The following example is of a GUI that dynamically updates when files, folders and drives change within the file system (outside of the GUI).

The "Drives" list automatically updates when new drives are connected to the computer, and when existing drives are disconnected. The "Directory Listing" automatically updates whenever an external application (ie. Windows Explorer) adds, renames or deletes files and folders in the directory currently displayed.

When the "Directory Listing" is automatically updated with additions or changes, the list is automatically scrolled to that change.

The user may click on an item in the "Drives" list to change the "Directory Listing" to that drive. The user may also double click on a directory (<DIR>) in the "Directory Listing" to navigate to that directory. Double-clicking on "<DIR> .." will navigate back to the previous directory. The user may also press "F2" to rename an item in the "Directory Listing", and press "Delete" to delete/recycle an item in the "Directory Listing".

FileSystemMonitor Explorer example.au3

The following example monitors file system events. Run this script, and then use various applications (ie. Windows Explorer) whilst the script is running, to manipulate files, folders and drives within "C:\". The changes you make should get recorded in the GUI of this example. The user may also change the Path and click "Use Path" to change monitoring to another path, other than "C:\".

FileSystemMonitor example.au3

DOWNLOAD:

Latest Version - v0.4 (02/05/10)

FileSystemMonitor.au3

BACKGROUND:

The script for ReadDirectoryChangesW comes from the following brilliant topic:

[NOW WORKING] a (broken) monitor file changes script which uses ReadDirectoryChangeW

The script for SHChangeNotifyRegister comes from the equally brilliant topic:

SHChangeNotifyRegister() Receive notifications of shell changes.

The combination of the two functions allows monitoring of most I/O functions in Windows, including:

• folder and file renames
• folder and file creates
• folder and file deletes
• folder and file updates
• drive additions
• drive removals

I began this script by using SHChangeNotifyRegister alone, but then found that it didn't catch file creations from many apps (like Internet Explorer or Opera file downloads). A detailed description of the problem is mentioned in this article:

Shell Notifications

Here's an excerpt:

"...The Origin of Events

So now, you know how to receive any of these shell notifications that are floating around, but who is actually generating them? According to the documentation, 'An application should use this function (SHChangeNotify) if it performs an action that may affect the shell'. But that seems to be a bit of wishful thinking. I can't imagine there are many applica tion developers that really give a damn whether the shell is kept informed of their actions...

...The result is that these notifications tend to be a little bit unreliable. The likelyhood of you getting an event for something, may depend on what explorer windows happen to be open at the time. The shell also only has a 10 item event buffer, and may replace some events with a generic SHCNE_UPDATEDIR in case of an overflow. In short: don't depend on these notifications for any mission-critical applications..."

I then switched to ReadDirectoryChangesW, but found that it lacks in features, such as detecting removable drive additions and removals.

The result I implemented was a combination of the two.

I prefer the Shell Change Notify method much more than the ReadDirectoryChangeW method, though because of the issues with it (as described above) I've had to implement both approaches to catch all events.

I like the fact that the Shell Change Notify method is very asynchronous in nature, running in the background as a registered Windows Message, with a seperate AutoIT function being activated whenever an event occurs, unlike ReadDirectoryChangeW's polling nature. I tried to put the monitoring of ReadDirectoryChangeW inside the MY_SHNOTIFY function above, but it didn't work. I'm still unsure why? If anyone can work it out please let me know. According to other forum topics, each call to ReadDirectoryChangeW should pull the next unprocessed event (that has occurred) out of it's queue (if that's the right word for it). So in theory, it shouldn't matter where it's called, or when. However calling it from within MY_SHNOTIFY doesn't work (even after changing the above code to make all variables global, and therefore accessible to MY_SHNOTIFY). Yet cutting and pasting the ReadDirectoryChangeW script into the Main Loop (While 1) does work. Strange.

Edited May 2, 2010 by seangriffin

[guinness]

This is brilliant! I have read posts that have tried to do this, but problems have occurred with the program hanging or worse crashing.

Thanks.

[KaFu]

Looks really good ...

[Andreik]

This script is very useful. Is really good.

[Shafayat]

Very useful script. Thanks.

[Manko]

This is really great!

Some wonderings:

"Drive free space has changed" - Can you report which drive has changed?

Also, if the folder to be monitored doesn't exist at start, the polling gave nothing...

You never checked if the handle returned by CreateFile was valid...

Upon creation of that folder, it should try to CreateFile again...

...or it could have warned, or app could have created folder... it all depends on what one wants I guess.

Keep up the good work!

/Manko

[WeMartiansAreFriendly]

Really nice. UDF's would be great.

[ripdad]

I get this error when opening some applications

See attached png image

[slayerz]

Nice script seangriffin.

BTW, when I test it on my laptop, it will list the file/folder operation on all drives regardless of path that I set. Does it supposed to be like that?

[seangriffin]

This is really great!

Some wonderings:

"Drive free space has changed" - Can you report which drive has changed?

Also, if the folder to be monitored doesn't exist at start, the polling gave nothing...

You never checked if the handle returned by CreateFile was valid...

Upon creation of that folder, it should try to CreateFile again...

...or it could have warned, or app could have created folder... it all depends on what one wants I guess.

Keep up the good work!

/Manko

Fixed the script to now report on the drive for which free space has changed. See the updated script at the top of this topic.

[SeF]

Thanks sean!

This will be very useful for me!

[seangriffin]

Really nice. UDF's would be great.

The script is now a UDF!

Starting it's life as v0.1. Hope you all like it. Please provide feedback.

It's a very early release. I haven't tested it much. Global variable names are quite ugly, but it seems to work.

[seangriffin]

Nice script seangriffin.

BTW, when I test it on my laptop, it will list the file/folder operation on all drives regardless of path that I set. Does it supposed to be like that?

You are right. I didn't include a specific path for shell monitoring in the script, and therefore all shell events across the file system were being reported.

I think I've addressed the path issues in v0.1 of the UDF now. Give this a try and let me know how it goes.

Cheers,

Sean.

Edited April 29, 2010 by seangriffin

[Tlem]

Thank you very much.

Great stuf, but in UDF you can delete include not needed (all but WinAPI.au3) and put them on the sample code.

It seems that the function _FileSysMonActionEvent is missing.

A deleted file or folder is not indicate. There just a message that the size of folder have been changed ...

[seangriffin]

Thank you very much.

Great stuf, but in UDF you can delete include not needed (all but WinAPI.au3) and put them on the sample code.

Good pickup. Have fixed this now in v0.2 (to be posted soon)

It seems that the function _FileSysMonActionEvent is missing.

This is intentional. It's up to the user to define in their own script, so they can action the events as they like. I guess I didn't make this clear in the UDF. Have updated remarks in v0.2.

A deleted file or folder is not indicate. There just a message that the size of folder have been changed ...

I just ran the example script myself, and deleted a file in "C:\" and it was reported. Screenshot attached below:

Did you perform the delete via Windows Explorer, and use the $monitor_type = 3 param?

[slayerz]

A deleted file or folder is not indicate. There just a message that the size of folder have been changed ...

I got the same problem... I delete the file using Windows Explorer, and use the $monitor_type = 3 but it only report drive free space has changed.

[Tlem]

Did you perform the delete via Windows Explorer, and use the $monitor_type = 3 param?

I use your example on the first post.

So yes ...

Another question about the UDF.

Is it possible to have the name of a modified (content) file?

[seangriffin]

Version 0.2 is now available for download.

Changes include:

• Removed unused external UDF references.
• Updated remarks to indicate that the user must create _FileSysMonActionEvent.
• Removed the use of $old_Filename (a failed attempt to hide duplicate events).
• Fixed a bug where _FileSysMonSetDirMonPath() was still reporting the next
• change in the previous path.

Also added a great Windows Explorer like example in the EXAMPLES section at the top of this topic. I think it is a great demonstration of the UDF's capabilities.

Edited May 1, 2010 by seangriffin

[seangriffin]

Version 0.3 is now available for download.

Changes include:

• Removed the $shell_gui param from functions. Not required, now managed internally.
• Updated "FileSystemMonitor Explorer example.au3" to select drive "C" if the currently selected drive is disconnected.
• Fixed error in "FileSystemMonitor Explorer example.au3" when an inactive drive is selected.
• Updated "FileSystemMonitor Explorer example.au3" to use $scroll_checkbox.
• Updated "FileSystemMonitor Explorer example.au3" to allow file/folder rename (F2) and deletion (Delete).

[seangriffin]

Also, if the folder to be monitored doesn't exist at start, the polling gave nothing...

You never checked if the handle returned by CreateFile was valid...

Upon creation of that folder, it should try to CreateFile again...

...or it could have warned, or app could have created folder... it all depends on what one wants I guess.

Manko,

I've done quite a few changes to the UDF now, and added two good examples. Can you try and reproduce your problem above in v0.3 of the UDF. Hopefully it's fixed.

Cheers,

Sean.