Dup Scout, available as a free limited version and a professional version, is a powerful program to locate duplicate files on the system. The free version is limited to scanning 500,000 files and a maximum storage capacity of 2 Terabytes. This should be sufficient for most uses.

You can start a check for duplicate files with a click on the duplicates button in the main interface. The program can search for duplicates in a selected folder, on all hard drives, on servers and NAS devices, or all servers on a network.

You may want to click on options before starting the search to configure search parameters. Here you can change performance related options or exclude directories from the scan. The free version lists options only available to users who have purchased the pro version of the software. This includes actions that are run automatically on duplicate files (for instance deleting the oldest files, or replacing the newest files with links pointing to the original copy of the file).

The process speeds depends largely on the hard drive the files are stored on and the processor of the system. Processing time is usually very fast though.

All duplicate files are displayed in list form in the interface. The program indicates the original file name, the total number of files (2 or more), the individual file size and the total space wasted by the duplicates.

A click on a row opens the the duplicate file set in a new window. All file paths, creation and last modification dates are displayed here. This is also the place to select an action that you would like to perform. Available for selection are to delete the duplicate files, replace them with links, move them to a directory, compress the duplicates or to compress and move them.

You can alternative highlight some or all rows at once and pick an action from the actions button at the top of the interface.

Dup Scout displays statistics below the duplicate file listing and with a click on charts in the toolbar.

The chart image can be copied to the clipboard or printed. Duplicate files can not only be sorted by file extension, but also by file size, access time or user name.

A click on Preview opens a preview window where all actions are displayed. The program lists the total number of duplicates and the space that you save when all actions are performed. Checkboxes are available to block a duplicate from being processed. Once you have made that final selection you can press the execute button to start the process.

Users can create duplicate file reports which can be saved as HTML reports by free users. The commercial version adds XML, SQL and other exporting options.

Dup Scout is a fast duplicate file finder for all recent versions of the Windows operating system. Free users, at least at the beginning, may be irritated by the presence of pro only features.

Windows users can download the latest version from the developer website.

It is usually not a good idea to search on all hard drives and storage devices for all kinds of duplicate files, considering that you will end up with many duplicate Windows system files in the process.

The Search Type filter helps in this regard. You can switch it from All Files to Music Files or Directories, or click the customise button to add your own filters to the application. You could for instance create a media filter to find duplicates for all your media files, or a document filter for Office documents.

Configuring the right file filters can speed up the duplicate file search tremendously. The customise search menu can be used to add or delete file filters. Each filter consists of a descriptive name and one or multiple file extensions. Only the file extensions listed under the filter will be checked for duplicate files if that filter is selected in the main program window.

NoDupe displays the results in a table. Identical files are shown as groups in the interface, along with their length, filename, directory, extension and data.

You can select individual files for removal and deletion, or use the Auto Mark feature to let the program do that for you. A click on Delete removes the selected files and folders from the system.

Advanced options, linked to from the main interface, can be used to further customize the duplicate file finding process. Here it is possible to select a minimum or maximum file size that you want to be considered, configure file deletion warnings or enable permanent deletion of deleted files.

Windows users looking for a straightforward but nevertheless sophisticated duplicate file finder should take a closer look at NoDupe. The program is compatible with 32-bit and 64-bit editions of the Windows 7 operating system. It can be downloaded from the developer website.

Duplicate Photo Finder’s premise is to be a comfortable and easy to use application to find and manage duplicate images stored on the PC.

The interface of the application can be best described as minimalistic but sufficient for most users. It begins with the selection of a comparison method at the top. Duplicate Photo Finder supports comparison by file sizes or file hashes. The first option is fast, the second more reliable but slower.

A scan can then be started once the original folder and the duplicate folder have been selected from the folder browser. The program will not search through folders recursively which can limit its use if photos are stored in a lot of subfolders. It would be nice if the developer could add the option to include subfolders and their images in the scan.

All duplicate images that are found in the process are displayed in a listing at the bottom of the screen. The interesting option here is that it is possible to select them to display both photos next to each other in the interface which makes it easy to look through the duplicate images before they are deleted.

The program offers controls to delete some or all of the duplicate images at once. Duplicate Photo Finder is handy software program for the Windows operating system. It requires the Microsoft .NET Framework 4.0 and runs fine on 32-bit and 64-bit editions of Windows. Downloads are available at the homepage of the developer.

Duplicate Cleaner is a sophisticate duplicate file finder with lots of options to configure the scanning and duplicate file checking. The main program window is used to configure the search criteria. Duplicate Cleaner can be configured to find files based on the following parameters:

• Same content (checked by comparing MD5 hashes)
• Same file name
• Same size
• Same created date
• Same modified date
• For music files: same artist, same title, same album

Filters can be added to reduce the number of files to be scanned by the program. It is possible to configure a file size filter, which will only scan files of a specific size range, file dates, which will only scan files that have been created or modified between dates specified and file name filters that can be used to include or exclude files based on names and extensions. All file scanning parameters can be combined in the scan.

Folders, partitions and drives can be added easily in the same window. Only the selected folders will be scanned by the duplicate file finder.

The time it takes to scan the folder depends on the speed of the hard drive and computer, as well as the files and folders and how much disk space they occupy on the selected drives. A report is displayed at the end of the scan listing the number of files that have been scanned, the time it took to scan the files and the number and size of the duplicate files that have been found.

A listing of all duplicate files is then displayed to the user. Individual files can be selected manually, and a preview is available for images for a visual aid.

The Selection Assistant can aid the user if many duplicate files have been found on the system.

Here it is possible to select multiple files at once, based on lots of different criteria including:

• Shortest or longest filename
• Oldest or newest file based on creation / modification
• Smallest or largest file
• By audio attributes (keep highest bitrate, highest sample rate, longest length)
• By text patterns in file name, path, sitze and lots of other variables. Supports regular expressions
• By location based on a selected path

Once the duplicate files have been selected for removal it is possible to configure deletion options. Standard options exist like deleting the duplicates to the recycle bin / without moving them to the recycle bin. Interesting options are available to automatically rename the duplicate files, move or copy them instead or create hardlinks (only on NTFS file systems). The option to create hardlinks could be interesting for users who fear that the removal of files could result in dead links or accessibility problems.

It is additionally possible to remove empty folders in the process. Duplicate Cleaner is an impressive program that comes with lots of additional options that have not been mentioned yet. It is for instance possible to change the file comparison algorithm, include network drives in the search and to import and export the results to CSV files.

Here is the feature list of Duplicate Cleaner as listed on the developer website:

• Find Duplicate Files by content, fast!
• Cleaner hard drives means faster performance
• Fast Scanning and accurate duplicate comparison (Uses the MD5 Hash algorithm)
• Deep scans music – locate duplicate music formats (MP3, iTunes, etc)
• Flexible search parameters
• Intelligent Selection Assistant
• Search home or networked drives
• Search multiple locations
• Export / Import results in CSV format
• View Path,Size and Date of files
• Large, resizable list window
• Image preview window
• Delete duplicate files to the Recycle bin
• Move dupe files to a new location
• Turn duplicate file into hard links (NTFS file systems only)
• Protection for Windows and system files
• Supports multiple languages

Duplicate Cleaner Download And Compatibility

It is surprising that this high quality application is available free of charge. It is one of the best duplicate cleaners available for the Windows operating system. The duplicate file finder is compatible with 32-bit and 64-bit editions of Microsoft Windows 2000, Windows XP, Windows Vista and Windows 7. It should also run fine on Windows Server installations. Downloads and additional information are available at the developer website at DigitalVolcano.co.uk.

Duplicate Commander is a tool that can do that. I reviewed it a few days back, and a commenter mentioned the application Duplicate File Hard Linker in the comments.

Duplicate File Hard Linker is a command line tool, that can replace duplicate files with hard links. The program basically runs recursively through folders on one partition to detect duplicate files. Files are compared byte-per-byte to make sure they are identical, and if they are all duplicate files are deleted and replaced with hard links.

If you have read carefully you have noticed that Duplicate File Hard Linker can only perform that operation on a single partition. That’s the main difference to Duplicate Commander, which does not have that restriction.

Then again, Duplicate File Hard Linker is a command line utility which means it can be used in batch files and other scripting languages to automate the duplicate file finding process.

Command Line Parameters

Usage:

dfhl [options] [path] [...]
/? Displays the command line help.
/a File attributes must match for linking. Files with different file attributes will be ignored.
/d Debug Mode, extended output of console.
/h Process hidden files. Hidden files will be skipped per default.
/j Also follow junctions (=reparse points) in file system. Per default, junctions will be skipped and not followed.
/l Process Hard Links for Files. If not specified, tool will just read (test) for duplicates.
/m Also Process small files <1024 bytes, they are skipped by default.
/o List duplicate file result to stdout.
/q Silent Mode.
/r Runs recursive through the given folder list.
/s Process system files. System files will be skipped per default if this option is ommitted
/t Time + Date of files must match before comparing files.
/v Verbose Mode
/w Print further statistics after processing

Limitations

* Hard Links can only be created on same partition. A permission overriding link creation is not possible due to physical limitations.
* Only NTFS is supported as file system for hard links.
* All hard linked files share the same set of file attributes, security descriptors and timestamps. Changes on one file Meta attribute will reflect to the other linked file immediately. This is limited by the underlying file system.
* When removing files which are hard linked, the space will only be freed on hard drive when the last reference of the file is going to be removed.
* Already linked files can not be linked again to gain space – of course…

duplicate files

The last update of the program dates back to 2006. Despite that, it works fine on Microsoft’s latest operating systems Windows 7 and Windows Server 2008 R2. May require an elevated command line prompt. The command line utility can be downloaded from the developer’s site.

Duplicate Commander is a free software for Windows for this exact purpose. The program is compatible with 32-bit and 64-bit editions, and requires the Microsoft .net Framework 4.0 to run.

On first glance, it looks like a standard duplicate file finder. A click on Search opens a folder browser to add folders to the search. Only a handful of options are provided, including the ability to filter by file type or size. A byte by byte comparison can be launched from the listing of results.

finding duplicate files

Duplicate Commander then processes the files in the selected directories based on the configured rules. This can take some time, depending on the speed of the hard drive, the performance of the computer and the amount of files stored in those directories. The status window displays the progress of the operation. It includes a progress bar, and information about the number of duplicates found, and their size on the disk.

duplicate files

Once the scan has finished a list containing all duplicate files found is displayed to the user in the main program interface. Duplicate files are linked, and checkboxes are provided to select some or even all to deal with them. Controls are provided to make the selection easier. It is for instance possible to click on a button to automatically select specific duplicates from the list, for instance the first or last of the group, the oldest or newest, or the duplicate with the smallest sizes.

Once duplicate files have been selected, it is possible to click on the delete button to remove the files from the system, or use the unique hard links option to create hard links to the remaining files.

Creating hard links offers an interesting alternative to deleting the files outright. Creating those links ensures that the system still acts as if the files exist in the location, even though they have been deleted by the duplicate file finder.

This obviously makes a lot of sense if the user is unsure about the consequences of deleting the files outright.

Duplicate Commander is available for download at the developer website. The hard link feature makes it a very interesting option for users who want to delete duplicate files from their system.

A duplicate file scanner like Similarity can aid in the detection of duplicates to free up disk space and make the music collection more accessible.

Similarity uses three different detection types for finding duplicate music files. It looks at the content, music tags and an experimental networks mode which basically utilizes P2P technologies to discover duplicates.

Usage of Similarity is pretty straightforward. You point the program to one or multiple directories and hit the start button afterwards. Supported are local directories, network directories, removable storage devices and Windows 7 libraries.

The application then scans all music files in the selected directories, a process that can take a while depending on the number of music files and the speed of the hard drive, memory and processor.

Similarity scans several music formats including mp3, wma, ogg, flac and wav and supports ID3, WMF and Vorbis music tags.

The results listing displays the potentially duplicate music files. Files are listed with their path and name, the percentages that they are similar in content (using sound analyzing and not byte to byte comparison), tags or precise, duration and size.

The standard parameters for detecting duplicate music files will result in lots of false positives. The program came up with about 7500 duplicate music files after scanning 4800 songs.

It is recommended to change the lower thresholds in the Tools > Options menu to avoid this. Another possibility is to sort the duplicates using the content, tags or precise column so that the files with the greatest similarities are displayed on top.

Similarity not only finds identical music but also different versions of a song. This is the main reason why the initial scanning variables have been set to 75%. This basically means that songs that match a 75% similarity in content or tags will be displayed as duplicates.

You can adjust sensivity of searching criteria to find exact or similar files. In this way it is possible to identify similarly titles with smaller differences. Duplicate files can be deleted or browsed. The list of duplicate files can be sorted, exported to playlist, and acted upon.

Other features of the duplicate music files scanner are a tag editor, tag and file renaming tool and exporting results into playlists.

Similarity is compatible with 32-bit and 64-bit editions of the Windows operating system. The software can be downloaded from the developer’s website.

Experienced users know that this methodology can only uncover identical files but not duplicate files that do not have the same file size. Two photos that show the same scenery would for instance not be detected as duplicates if their file size would not be identical (that is the case if they have different resolutions for instance).

The hash generation on the other hand ensures that only absolute identical files will be detected as duplicates which might be exactly what the user wants to achieve. The program uses Java which is the reason why it is compatible with Windows, Linux and Mac. It does however mean that Java needs to be installed on the computer system.

The user can add one or multiple directories in the program interface. All, including their subdirectories, will be scanned for duplicate files. An option is provided to exclude files from the scan by either using wildcards or regular expressions.

All duplicate files are displayed after the scan in the interface. They are grouped together to make the identification easier. Each result can be checked individually or by using the pulldown menu at the bottom to make the selection (for instance all but the newest, all but the first or all files).

Duper is available from the developer’s website. It is compatible with 32-bit and 64-bit editions of Windows. (via Addictive Tips)

SoftChronizer is a duplicate file finder for Windows that can scan the PC and compare files found to display the duplicates in its interface.

The left sidebar of the program contains links to the program’s options, search parameters and file filters. Each have to be configured and selected before the search.

The duplicate file filter can filter files by type with pdf, mp3 and pictures offered besides a wildcard search that displays and checks all file types in the selected directory.

SoftChronizer supports normal and binary comparisons and can also compare by file size and file name again with the option to do a basic size or name comparison or use Fuzzy Logic to do so.

The scan of the target directory is very fast. It is possible to select a partition of the computer or a folder on the hard drive. The duplicate file finder will automatically scan the subdirectories of the selected root folder as well, a parameter that can be changed in the program’s options.

All duplicate files will then be displayed in the program’s interface after the very fast scan of the selected folder. The application lists the directory, file name, size and other information in its interface.

A pulldown menu is available for each item in the results that can be used to delete or move the selected file. The duplicates are color coded to make the detection easier. It is however not possible to select multiple duplicates at once, use automatic duplicate file deletion, sort the results or to select another file filter.

This makes the program impracticable for large amounts of data that needs to be compared and cleaned of duplicates.

There is also no build in preview feature to take a look at pictures or documents right from the program’s interface. A double-click will launch the selected file in the default viewer which is not very comfortable.

Installation

This is not the usual Giveaway of the Day installation. The zip file contains the setup program to install the duplicate file finder, a readme with instructions on how to install, activate.exe to activate the program and the file gacasreg.bin which needs to be moved into the program directory after installation.

Positive

• Very fast scan
• Fuzzy Logic to find duplicate files even if their names are not identical

Negative

• Cannot scan multiple drives at once
• Results cannot be sorted, changing filters is not possible
• No build in preview of results
• No automatic cleaning
• Multiple selections not possible
• Duplicate file scan parameters limited to file size, name and binary comparison.

Clone Spy is a free alternative duplicate file finder that supports many of the features that Softchronizer does not. It can for instance automatically delete duplicate files, exclude files and folders from the scan and more.

Softchronizer is a fast duplicate file finder with several usability issues that make the program less than ideal for most uses. It is currently offered for free at the Giveaway of the Day website. The developer’s website is accessible here.

The Duplicate File Finder by software developer Auslogic is one of the duplicate file finders available for the Windows operating system. The interface has been designed with ease of use in mind. It begins by selecting folders or hard drives that should be checked for duplicates. It is possible to select multiple folders and drives or just one.

The options of the duplicate file scan are configured in the next screen once the folder and drive selection has been made. The options are divided in the three categories matching criteria, advanced criteria and file types.

Matching criteria defines the aspects of the files that should be matched with each other. These are the name, date and time and size by default with the option to add contents as well (resulting in a slower scan) or remove one of the other criteria.

Advanced criteria can be used to filter out files with a size lower than the size specified in this section of the configuration. File types finally makes it possible to only scan specific file types or all files in the selected folders.

A list of results is then displayed after the program finished the scan. File names, sizes, locations and last modified dates are displayed with the option to select some or all of the files so that they can be deleted. The Duplicate File Finder can be downloaded from the Auslogics website.

Once the folders have been selected they can be immediately checked for duplicates. Once a duplicate has been detected it can be deleted or moved immediately. Other options include the creation of a list of duplicate files and the creation of a batch file. The second option is interesting as well. Files can be compared to checksum files that have been previously generated which results in a faster processing speed. Ideal for instance if you store files that will stay on your system and that you only add new files to.

Clone Spy provides several automatic options on how to handle duplicate files. It can delete older or newer files or files with a longer or shorter file name or path.

The process can be further customized in the program’s options. There the user can include or exclude file types from being included in the scan, if and how file attributes affect the scanning and if specific folders should be ignored.

Certain file types can be excluded from scan, the default configuration exlcudes dll, sys, vxd and inf files from the scan. You could add more files that are important in your case or remove any of the default ones. You could on the other hand limit the scan to files matching a certain filter. It would be easy to setup a scan that would only scan for images for instance using this filter. (good way to check for duplicates in image collections of 100000+ images)

Last but not least you can leave out files that have a certain file attribute such as read only, system or hidden. System files are excluded from scans by default. A last parameter makes it easy to define a rule that files will not be scanned if the file size exceeds or undercuts a give size.

The lenghts of a scan depends on various factors such as hard drive space, filters that have been set and speed of hard drive. Doublekiller is a clever program that makes it easy to scan through a large list of files and free up disk space in the progress.

I also have a rather large amount of images on my pc, could well be over 30000 and I always had the impression that there have to be duplicates all over the place. Never had the time to manually check them and delete the duplicates, would take forever.

Therefor I decided to take a look at tools that compare and delete duplicate files automatically. I found two that suit my needs, one is more of a general duplicate finder and one is specifically for images.

Clonespy is able to locate duplicate files even if they have a different file name or size. Its freeware of course, simply unzip to a location and run the file. The program gives you some options how to handle the duplicate files, move them, delete them and exporting a list of the files without removing any.

The second tool which is unfortunately nagware is called Unique Filer. I have not found a freeware tool that is able to perform the same operations as Unique Filer. It is able to find images dupes even if the images have been resized, rescaled, resampled and color corrected. It also finds normal dupes of course, but that’s not what makes it so special.

If you have many images that you downloaded over the years I recommend Unique Filer to you, you can use all its functions without registering, but see a nagscreen all 10 files that you opt to delete. And those are a lot, from my 30000 files more than 5000 have been dupes..