Jump to content

SMF - The fastest duplicate files finder... [Updated 2024-Oct-20]


KaFu
 Share

Recommended Posts

Link to comment
Share on other sites

You know more RegEx than me.

That returns a lot of other things like

just-cause-2-20101001001102509.jpg

and

263535345.jpg

I only want files that are exactly "7 numbers" + .jpg

Link to comment
Share on other sites

This is because the regexp is not anchored to the left, thus it matches if there are at least 7 digits followed by ".jpg" the whole thing being possibly preceeded and/or followed by something else.

Try making the RegExp "(?i)^d{7}.jpg$"

which breaks down into:

(?i) makes the match case-insensitive

^ matches only at start of subject

d{7}.jpg followed by 7 digits and then ".jpg"

$ matches only at end of subject

That should not match abcd1234567.jpg nor 123456789.jpg nor 1234567.jpg.old

I wonder if the capturing parenthesis around the filename are required?!?

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Link to comment
Share on other sites

  • 2 weeks later...

Hi,

I'm using the v1.8 portable on a win7 Pro x64 and I perhaps found a small bug.

I find out that full MD5 hashes aren't right. I tested with big and small files too, to check with short MD5 and it seems to have the same problem.

I used my own program to check out (which uses default AutoIt _Crypt_HashFile() ), the CorZ checksum, and an online md5() too, just to be sure.

The hashes seem to be consistent, but the numbers returned aren't right. By consistent I mean that, in example, when doing a duplicate search the dupe files returned are alright.

I'll follow this topic, but as it doesn't keep this setting for long, don't hesitate to ask me by PM if you need informations, or reprimand me because I made a big wrong discovery :).

Link to comment
Share on other sites

Hi AwA,

you are absolutely right, full MD5 hashes (full CRC32 and full SHA1 hashes too) are wrong in SMF 1.8. Thanks for reporting the issue ;), I didn't saw that!

And as you said they're consistently wrong. I've checked the calculation and quickly found the issue.

The hashes are calculated in 1 Megabyte chunks. I do this to make the calculation interruptable.

Now the issue was that all chunks were exactly 1MB large, even the last chunk which is only partially filled with the rest, e.g. 12.467 Bytes. The rest of the last chunk was filled with zeros.

That's why the hashes were still consistent, but not comparable with the ones created with other calculation methods. I've fixed this in the upcoming version 1.9, which I'll release as soon as AutoIt 3.3.8.0 is released :D...

Btw, I've looked at your page. "UAC Pass" is a really nice piece of software, I've started coding something similar I called "UAC Elevator" (using the "Schedule.Service" object), but development on that one stalled :). Why don't you post it on the forums?

Best Regards

KaFu

Link to comment
Share on other sites

Hi KaFu,

Thank you about explaining what was going wrong, it is always interesting to understand things.

About UAC Pass, thanks for the flowers :).

I guess it is a shy attitude, this is really an insignificant tool compared to SMF.

Also the source are a bit messy... well, they are messy, and I see you, AutoIt MVP, coming with your whips ^^. Well, it makes me realize that I push back this moment since the beginning (cleaning the source code), I'll publish them on my site as they are, and then I'll make a topic here. I'll buy protection against whips, and clean the sources when I will have finished my current project.

I take advantage of this post to propose some ideas about SMF.

I see 2 things which could be nice, the first could help me with my current duplicates project (as I try a lot, without success, to convert the functions to AutoIt), the second could make it useless, but I don't care ;)

- Why not list all hard links to a file in a tooltip when hovering the hard link number of a file (FindFirstFileNameW / FindNextFileNameW),

- Why not propose, after auto-selecting the 1st duplicate of a file, to hard link the other dupes to the 1st one? (either with a DLL call, either with integrated AutoIt FileCreateNTFSLink() )

This second idea includes that dupes must be on the same drive, which must be NTFS formatted.

Cheers,

AvvA

Edited by AvvA
Link to comment
Share on other sites

No worries on UAC Pass. I've seen something similar made in Delphi, but this is to my knowledge the first implementation in AutoIt... and the number of bugs and potential improvements in SMF diffinitly is somewhere in the thousands :) ...

As to your suggestion... I gave them some thoughts and came to the conclusion I won't implement them in SMF, sorry. FindFirstFileName requires Vista+ and thus would be of an overall reduced usefulness. The second suggestions is imho so special (I personally never have worked with NTFS links) that the effort would greatly outrun the benefits (for me personally ;) ). But don't hesitate to PM me if you've got some questions on the functions themselves, I'll do my best to support you.

Thanks for making suggestions at all! In general I'll love to see SMF improved with unique features :D...

Best Regards

Link to comment
Share on other sites

  • 2 weeks later...
  • 1 month later...

Thanks for the feedback. I know of these, these either don't have any effects (1, the function is a string and thus can be obfuscated => "will/could lead to errors" applies to calls using a variable as the function name) or are in functions that are not used in SMF and thus stripped in the obfuscation run itself (all others). Start the compiled exe and you will see that it works without problems (at least non generate by these errors :))...

Edited by KaFu
Link to comment
Share on other sites

  • 8 months later...

The upcoming Version 2.1 will have improved search speed and especially a reworked report functionality. Now the displayed preview icons can be changed on the fly and you can select between detail report, icon and tile view (with many style related customizing options, like adjust spacings, colors and fonts).

I'm currently in the process to wrap up all changes and retest all functionalities to make it fit for an official 2.1 release. However there are some many functions now, with so many interdependencies, that it's truly hard to say when it's really ready :).

So everyone is invited to an open Beta test, here's a link to the current executable (source will be released with official version change again).

SMF_2.1_Beta.html

Here are three sample screenshots of the report, one in detail report view, one in icon view and one in tile view, showing exactly the same result set.

Posted Image

Posted Image

Posted Image

Best Regards

Link to comment
Share on other sites

  • 2 months later...

2012-Dec-27, Changelog 2.0 > 3.0

  • Reworked the overall search loop and underlying SQL statements to improve speed
  • Redesigned SMF_TNP.exe to create thumbnails using AutoIt native functions / does not use AutoItObject dll anymore
  • Added Quicksave and restore functions
  • Reworked the Export functions
  • Report: Added icon and tile view
  • Report: Added several new style and option settings
  • Report: Updated "Secure Delete" function, uses wraithdu's excellent
  • Report: Updated "Copy & Move" functions, uses Yashied's excellent
  • Fixed a lot of bugs and improved overall performance
  • For sure implemented a lot of new bugs :whistle:, please keep reporting
  • And much, much more I can not remember, but a whole version number is justified for this release :)...
  • Updated to SQLite DLL to v3.7.15.1
  • Updated MediaInfo DLL to version 0.7.61
  • Updated TrID Definitions to version 2012 Dec 23
Source and Executable are available at

http://www.funk.eu

Best Regards

Updated first Post... Enjoy :)...

Edited by KaFu
Link to comment
Share on other sites

  • 2 months later...

2013-Mar-20, Changelog 3.0 > 3.1

  • All SMF functionalities should now work with long filenames too (max buffer = 4096)
  • Added optional password to recover from tray
  • Report: Added ToolTip Previewer for large previews of the currently selected file
  • Report: Fixed bug which occasionally froze whole report (WM_COMMAND loopback)
  • Report: Updated "Copy & Move" operations to handle long filenames, removed copy.dll and added own wrapper for wraithdu's
  • Fixed a lot of other bugs and improved overall performance
  • Updated to SQLite DLL to v3.7.16
  • Updated MediaInfo DLL to version 0.7.62
  • Updated TrID Definitions to version 2013 Mar 15

Example of the ToolTip Previewer in action:

Posted Image

Source and Executable are available at

http://www.funk.eu

Best Regards

Updated first Post... Enjoy :)...

Edited by KaFu
Link to comment
Share on other sites

When I attempted to delete individual duplicate files, SMF deleted my entire Pictures folder. Luckily, it was simply moved to the Recycling folder, so I was able to restore it. But needless to say, I'm going to stop using SMF until you figure out what's wrong.

Link to comment
Share on other sites

  • KaFu changed the title to SMF - The fastest duplicate files finder... [Updated 2024-Oct-20]

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...