Find and Remove Duplicate Images - gHacks Tech News

Find and Remove Duplicate Images

If you download many images from the Internet, bulk collections for instance from Usenet or P2P, then you will eventually face the problem that you have duplicate images in your collection. Manually exploring and deleting duplicate images should be out of the question if the collection contains 10K+ images unless you really have lots of time at hand. The best way would be to use a software that scans the entire image collection and either automatically deletes all dupes or displays possible hits so that the user only needs to confirm if it is indeed a dupe.

Deduper can do that. This software is easy to use and scans a folder and all its subfolders for images. It gives the user the choice to automatically run and delete images or display ever duplicate that was found so that the user can finalize the process. The user has the choice to select a folder that is save so to speak which means that the duplicate of an image in that folder will be deleted and not the image in the folder specified.

One weak point of Deduper is the fact that it only recognizes binary identical images which means that it does not recognize a dupe if it finds an image in two different sizes or one with text and one without text but the same size. I guess that's why it's so fast. A test run on a folder with 18000 wallpaper images did not take longer than a few minutes and had the result that almost 50% of the images were dupes resulting in a saving of more than 700 Megabytes on my hard drive.

duplicate images

Deduper is a nice fast program that definitely has its uses if you regularly download images from the Internet or already have many images on your hard drives.

Update: Deduper is no longer available. I suggest you use AntiDupl instead which provides you with similar options to find duplicate images on your system.

We need your help

Advertising revenue is falling fast across the Internet, and independently-run sites like Ghacks are hit hardest by it. The advertising model in its current form is coming to an end, and we have to find other ways to continue operating this site.

We are committed to keeping our content free and independent, which means no paywalls, no sponsored posts, no annoying ad formats or subscription fees.

If you like our content, and would like to help, please consider making a contribution:

Comments

  1. ismaelj said on April 7, 2008 at 9:32 pm
    Reply

    A better program is Similar Images:

    http://celebnamer.celebworld.ws/similarimages/

    Not only checks for file size, but performs a real analysis of images and can detect similar images with different sizes, resolution, etc.

    It hasn’t been updated since 2006 but the las available version still works OK.

    It has worked very well for me!

  2. Rarst said on April 8, 2008 at 5:57 am
    Reply

    Visipics
    http://www.visipics.info/

    is the best (and freeware) program I had seen for such task. It analyzes images, not binary data (analyzing images is kinda key thing for… analyzing images :) ). Very fast and allows to easily control degree of search – it can look for images that are almost exact match or that are loosely alike or anything in between.

    Only minor downside that interface is more suitable for automatic deleting of duplicates (it chooses files to keep based on file type and resolution) and is not very suitable if you want to manually compare found duplicates.

  3. Simon Templar said on April 14, 2008 at 10:14 pm
    Reply

    There is another, Unique Filer, at http://www.uniquefiler.com/ , I don’t think its been updated since 2001 but its the fastest software of its kind I’ve used, Its very fast, I haven’t used it in a long while, since storage is virtually free now, but back in my 40GB HD days, I used it to prune my image collection, it went through around 10,000 images in an hour on my ancient PC and not only did it nail every dupe image ( not binary alone but visual similarity ), it alo displayed the results according to the degree of similarity, it even managed to match thumbnails to full sized images, awesome program, worth a try if even for novelty purpose.

Leave a Reply

Check the box to consent to your data being stored in line with the guidelines set out in our privacy policy

Please note that your comment may not appear immediately after you post it.