SMF - The fastest duplicate files finder... [Updated 2024-Oct-20]

[KaFu]

2023-Jun-03, Changelog v14 > v15

• Updated - SQLite Dll to 3.42.0
• Updated - MediaInfo Dll to 23.04
• Updated -   TrID Definitions to version 2023-May-14
• Updated -   Added dHash visual similarity check
• Updated -   Improved thumbnail creation speed
• Fixed       -   Caching of hashes to speed up duplicate search
• Fixed       -   Report crash on stats change
• Fixed       -   CTRL+ALT+F explorer integration
• Updated -   Lots of other small bug fixes and style changes

Source and Executable are available at https://funk.eu

Best Regards
Updated first Post... Enjoy ...

[goodmanjl531]

woudl this program have the ability to search for multple files using a set oif searches?

i.e.

*filename1234.txt

*filename1235.txt

*filename1247.txt

*filename1248.txt

*filename2240.txt

 

find all 5 of theses files then give you an option to copy them to another folder?

[KaFu]

Yes, SMF can do that. Select the folders to search, do a search and open the report.

Go to the yellow filter box above the "FileName Long" column and enter this:

filename1234.txt|filename1235.txt|filename1247.txt|filename1248.txt|filename2240.txt

The "|" sign is a delimiter for an OR condition in the LIKE statement. You can also select "LIKE Expressions" in the drop-down above the yellow field and enter the filenames one per line.

Select all required files in the report and either Ctrl+C copy them or use the right mouse click menu for more complex copy operations (e.g. "Copy to...").

Edited April 28 by KaFu

[goodmanjl531]

@KaFu

 

THANK YOU so much. You just helped me make a 45 minute manual job drop to 5 minutes.  😀

[KaFu]

Glad to hear it works fine for you 🙂👍

[jchd]

Hi @KaFu, I've a real-world use case for detecting and managing duplicates (same content but different filenames) in a large folder tree.

My case is as follows: I gather payware technical documentation for a very large number of appliances from many makers. Each model has it's own set of docs but very often the maker issues the same docs for a number of similar machines in the same series. Many makers use a number of brands, often offering almost identical machine under different names and brand names. I organize the docs based on brand, then type then model number. Hence I store many dups in either the same folder or different folders. I also store datasheet of electronic components , but this is a much smaller volume and large duplicates are rare. All this accumulates over years and I'd like to minimize storage and backups sizes. Curently, 85833 files in 14667 folders, worth 13Gb of disk storage.

The way I think of it is to allow SMF to let me decide, given a number of docs having the same content, which file will remain the source and convert dups to links (hard or soft) while retaining the path and name of the dups.

Of course I could write a dedicated program to do precisely that, but if SMF could do that by itself, I wouldn't object 😁

Do you think it's both possible and a valuable feature to other users?

[KaFu]

Hi @jchd, SMF can do all the initial steps. but it can not replace the duplicates detected with (soft or hard) links. Only workaround would be to do a search and then export the results including the hash values to CSV as an input for a separate script, which replaces all but the first matches per file with links.

[jchd]

Understood, that's what I was heading to.

[KaFu]

2024-Oct-9, Changelog v15 > v16

• Updated   -   SQLite Dll to 3.46.1
• Updated   -   MediaInfo Dll to 24.06
• Updated   -   TrID Definitions to version 2024-Oct-9
• Updated   -   dHash > added support for video files, if ffmpeg.exe exists in directory "SMF_Files\Bins"
• Updated   -   dHash > added support for WebP image files, based on WebP UDF by UEZ
• Updated   -   Lots of other small bug fixes and style changes

Source and Executable are available at https://funk.eu

Best Regards
Updated first Post... Enjoy ...

Edited October 14 by KaFu

[KaFu]

2024-Oct-20, Changelog v16 > v17

• Fixed         -   "Datatype mismatch" crash in "Hash Cache" function of duplicates search

Sorry for any inconvenience caused, typical case of over-optimization.

Source and Executable are available at https://funk.eu

Best Regards
Updated first Post... Enjoy ...