A quick way to remove duplicates?

catface
Posts: 35
Joined: 22 Oct 15 19:38

A quick way to remove duplicates?

Post by catface » 27 Oct 15 23:57

Is there a quicker way to remove duplicate files from photo supreme then manually go through all? Some script or present function?

Mike Buckley
Posts: 4561
Joined: 10 Jul 08 14:18

Re: A quick way to remove duplicates?

Post by Mike Buckley » 28 Oct 15 2:52

You'll receive more helpful responses if you define "duplicate file." If the duplicate is the same image but has a different file name, that's one situation. If the duplicate is the same image and file name, that's another situation.

Hert
Posts: 21020
Joined: 13 Sep 03 7:24

Re: A quick way to remove duplicates?

Post by Hert » 28 Oct 15 7:19

In addition to Mike; did you already find CATALOG-By State? There are two states there: duplicates and similar images
This is a User-to-User forum which means that users post questions here for other users.
Feature requests, change suggestions, or bugs can be logged in the ticketing system

catface
Posts: 35
Joined: 22 Oct 15 19:38

Re: A quick way to remove duplicates?

Post by catface » 28 Oct 15 10:53

Mike Buckley wrote:You'll receive more helpful responses if you define "duplicate file." If the duplicate is the same image but has a different file name, that's one situation. If the duplicate is the same image and file name, that's another situation.
It's the same images with same filenames.

catface
Posts: 35
Joined: 22 Oct 15 19:38

Re: A quick way to remove duplicates?

Post by catface » 28 Oct 15 10:53

IDimager wrote:In addition to Mike; did you already find CATALOG-By State? There are two states there: duplicates and similar images
I'm referring to the duplicates, not the similar images.

vlad
Posts: 967
Joined: 01 Sep 08 15:20

Re: A quick way to remove duplicates?

Post by vlad » 28 Oct 15 11:06

You could switch the collection view to Grid (look for the View button towards the top right corner) and then sort your (duplicate) images by FilePath. Based on what criteria would you expect a script to (automatically) remove duplicated files?

catface
Posts: 35
Joined: 22 Oct 15 19:38

Re: A quick way to remove duplicates?

Post by catface » 30 Oct 15 15:48

vlad wrote:You could switch the collection view to Grid (look for the View button towards the top right corner) and then sort your (duplicate) images by FilePath. Based on what criteria would you expect a script to (automatically) remove duplicated files?

I'm just wondering if there is an easier way for me to delete the images that have the exact same filenames as each other, than just going thorugh them manually and delete all the extras?

Couldn't a script be able to read the filenames and delete all of the ones with the same filenames but one?


Because right now I have to scroll through over 7k files and look for the dobbeltgangers and delete them manually.
Image

vlad
Posts: 967
Joined: 01 Sep 08 15:20

Re: A quick way to remove duplicates?

Post by vlad » 30 Oct 15 17:23

Couldn't a script be able to read the filenames and delete all of the ones with the same filenames but one?
I think it could, but I don't understand: don't you care which image should be kept and which images should be deleted (based on their filepath)? How did you get the duplicates in the first place - did you import the corresponding folders on purpose or by accident?

catface
Posts: 35
Joined: 22 Oct 15 19:38

Re: A quick way to remove duplicates?

Post by catface » 30 Oct 15 17:37

All of the images are from a harddrive, I don't know how I got this many duplicates in the first place. I may very well have done something wrong on the import, since this is my first time working with this program. I don't care which image is deleted, as long as it is the exact copy of the other that is kept.

Do you suppose I might have managed to import some of them twice or something?

This is what the file paths look like;

Image

vlad
Posts: 967
Joined: 01 Sep 08 15:20

Re: A quick way to remove duplicates?

Post by vlad » 30 Oct 15 18:29

It seems to me that your image organization on the hard drive is a bit of a mess (because you have duplicates even across folders with different dates - see for example DSC_4292.jpg). Do you have a plan for the folder structure where you would like your (non-duplicated) images to be stored?

It is not advisable to store your images all over the place even if you manage to get rid of duplicates (since you would still have the same mess on your hard disk, so you become entirely dependent on Photo Supreme). Try to decide on a clean folder structure for your pictures and then we could further advise you.

(Well, one tip I have is to use a program like Duplicate Cleaner - it is free and very good. But IMHO you still ought to clean up your folder structure, in addition to duplicates.)

Mike Buckley
Posts: 4561
Joined: 10 Jul 08 14:18

Re: A quick way to remove duplicates?

Post by Mike Buckley » 30 Oct 15 18:36

I would be very tempted in that situation to create a new catalog and to import all of the images into it, being especially careful to devise a system that ensures that no image is imported more than once. Doing so would probably be both simpler, less time-consuming and error-free compared to devising a system that eliminates duplicates.

Relative to the issues mentioned by Vlad pertaining to your file and folder structure, I strongly encourage you to devise a file-naming convention that ensures that no two files in your system have the same name. For best practices about that, review the following website: http://dpbestflow.org/links/38 My hunch is that you could attend to that after creating a new catalog, as Supreme provides a very powerful and flexible file renaming tool.

vlad
Posts: 967
Joined: 01 Sep 08 15:20

Re: A quick way to remove duplicates?

Post by vlad » 30 Oct 15 18:50

I agree with Mike.
Mike Buckley wrote:I would be very tempted in that situation to create a new catalog and to import all of the images into it, being especially careful to devise a system that ensures that no image is imported more than once.
If 'catface' is willing to create a new folder structure upon importing into the new catalog, that should be possible by creating and employing a carefully devised import profile. He could enable Copy images to new location and make sure that the Duplicate Handling option is set to Skip file. (Well, I forgot: does that option simply check for duplicate filenames or does it also involve a binary check?)

catface
Posts: 35
Joined: 22 Oct 15 19:38

Re: A quick way to remove duplicates?

Post by catface » 30 Oct 15 18:52

These aren't actually my files or folder systems, it's a gathering of images all the way from 2004 that are my sisters. I'm doing this as a job for her.

My main task was to just tag the images to make it easier for her to find the ones she's looking for. But it seems this turns into a larger task by the day.

So I should actually start organizing her whole external hard drive before I even import the images?

This will become a huge task since there are over 100k images all over the place that I'm supposed to tag. By importing them all, removing the duplicates and then tagging them. At least she'll be able to find them with photo supreme though, right? Without the whole folder system reorganizing?

fbungarz
Posts: 3163
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: A quick way to remove duplicates?

Post by fbungarz » 30 Oct 15 19:36

Hi,
it may indeed be easier to remove the duplicates before importing. You don't necessarily have to do this manually though.
If the photos are exactly the same, i.e., truly identical duplicates, some duplicate finder like http://www.extrabit.com/spaceman99/ can fairly quickly do that job for you.
I hope that helps,
Frank

Mke
Posts: 433
Joined: 15 Jun 14 15:39

Re: A quick way to remove duplicates?

Post by Mke » 30 Oct 15 19:40

Going through 100k images will certainly be a major task! Wish I had a sibling who fancied doing the same for my back catalog :)
catface wrote:So I should actually start organizing her whole external hard drive before I even import the images?
I'm sure that all of us here would recommend that.
Personally I'd also do any basic file renaming first too (using FreeCommaner 2009), which may help to cut down on false duplicates.
catface wrote:This will become a huge task since there are over 100k images all over the place that I'm supposed to tag. By importing them all, removing the duplicates and then tagging them. At least she'll be able to find them with photo supreme though, right? Without the whole folder system reorganizing?
Yes, you could go ahead as you planned, and yes, she'll be able to find them. And, once you've done that, you could then move images and folders from within PS. However if you (or she) subsequently moves them from outside PS, it will loose track of where they are. Now PS does have tools to enable you to find them again, but if everything is logically labelled and organized first, that problem goes away.
Having everything in organized folders is also likely to make tagging easier, I would imagine.

Post Reply