Duplicates and Similar Images

Post Reply
jtiasgaard
Posts: 5
Joined: 16 Jun 20 4:47

Duplicates and Similar Images

Post by jtiasgaard » 16 Jun 20 4:52

I bought Photo Supreme after reading DAM and being recommended the program as a replacement for Media Pro. I am impressed with its ability to handle duplicates, however, one thing does not make sense to me. I currently have a catalog of over 100,000 images. It shows 0 "duplicates". However, under "similar images" it has 20,000... many of which are EXACT duplicates.

Is there something you can do to re-run and look for duplicates again?

Also, is there a way to see just "duplicates" or "similar images" by year or specific file? with 20,000+ images it is a daunting task to have them all together at once.

Hert
Posts: 6369
Joined: 13 Sep 03 7:24

Re: Duplicates and Similar Images

Post by Hert » 16 Jun 20 10:11

A warm welcome to the forum.

"Duplicates" and "Similar images" are not the same.

Duplicates: These are exact binary duplicate files. For example a 1:1 copy of a file
Similar images: These are images that "look" the same. For instance a b/w version and a full color version, or a large and a downsized version
Apparently your catalog contains 20000 images that look the same, but none of them are exact binary copies.

What I recommend you do is:
1. Optionally empty the Image Basket
2. Open States -> Similar Images
3. Use the filter bar to filter all within one year (see attached screenshot)
4. Select all thumbs (Cmd+A or Ctrl+A)
5. Add them to the Image Basket (Shift+B)

Now work from the image basket. Once you worked through the year, repeat the steps for the next year.

Hert
Filter per year
Filter per year
advanced_filter.png (833.59 KiB) Viewed 1735 times
This is a User-to-User forum which means that users post questions here for other users.
Feature requests, change suggestions, or bugs can be logged in the ticketing system

jtiasgaard
Posts: 5
Joined: 16 Jun 20 4:47

Re: Duplicates and Similar Images

Post by jtiasgaard » 18 Jun 20 6:09

Thank you Hert!

I have attached a screenshot of an example set of images listed in "similar images".
Screen Shot 2020-06-17 at 10.03.53 PM.jpg
Screen Shot 2020-06-17 at 10.03.53 PM.jpg (124.14 KiB) Viewed 1664 times


I cannot see how they differ as files except that they are in different folders? Perhaps there is some reason that they are being seen as similar but are not duplicates as you said...

Another thing has just happened. When I hit "x" to mark files, it does not mark them until I do it 2 or 3 times. There is some strange activity in the bottom left of my program window, but the "x" mark does not get applied the first time. Any suggestions for that?

Thanks!

KeesdH
Posts: 40
Joined: 26 Jun 08 19:53

Re: Duplicates and Similar Images

Post by KeesdH » 18 Jun 20 7:07

Hi, the date of the files is different.

Hope this helps.

Hert
Posts: 6369
Joined: 13 Sep 03 7:24

Re: Duplicates and Similar Images

Post by Hert » 18 Jun 20 8:15

In addition to Kees's reply: a file is considered identical (duplicate) when the binary content of the file is identical and the file size is exactly the same.
If you think these files are exactly the same, then could you send me a copy of both files so I can verify them here? If needed, you can upload them to your cloud drive and send a download link to support@idimager.com
Thank you
There is some strange activity in the bottom left of my program window, but the "x" mark does not get applied the first time. Any suggestions for that?
Are these files fully imported? The files is out-of-sync (see the orange dot on the thumb), could be that you didn't let the import process finish. To manually import the metadata of these files, select the thumbs, and right click -> metadata -> Read metadata from file". The orange dot should disappear, indicating that the catalog info is in sync with the file.
You can also manually import all out-of-sync images by selecting "Tools -> Read metadata to catalog for all out-of-sync images" from the hamburger menu.
BTW; not imported yet could also be a reason for not having duplicate files identified.
This is a User-to-User forum which means that users post questions here for other users.
Feature requests, change suggestions, or bugs can be logged in the ticketing system

jtiasgaard
Posts: 5
Joined: 16 Jun 20 4:47

Re: Duplicates and Similar Images

Post by jtiasgaard » 19 Jun 20 14:32

Thank you Hert as well as KeesdH!

That was the problem. It had not processed the information - so it was seeing identical images as "similar". Now, as the metadata slowly is moving into folder, it is recognizing them as duplicates.

The process is taking a lot of time because it keeps making my mac say "out of memory" so I will work on that next - but your help was incalculable! Thank you so much!

jtiasgaard
Posts: 5
Joined: 16 Jun 20 4:47

Re: Duplicates and Similar Images

Post by jtiasgaard » 04 Sep 20 15:02

I am back because, now I have imported all of my photos and there are tens of thousands of duplicates that are not being seen. The files differ in name. Is there something I need to do to the catalog to get it to find the duplicates?

Example screenshot attached.
Attachments
Screen Shot 2020-09-04 at 6.57.58 AM.jpg
Screen Shot 2020-09-04 at 6.57.58 AM.jpg (99.06 KiB) Viewed 1121 times

Hert
Posts: 6369
Joined: 13 Sep 03 7:24

Re: Duplicates and Similar Images

Post by Hert » 07 Sep 20 8:25

I had to look into this. A files gets into the "Duplicates" section when the file and its sidecar files are identical. For instance for RAW files, the metadata is read from a XMP sidecar file, which should also be identical in order to end up in duplicates. I didn't think of that in my earlier reply.
This is a User-to-User forum which means that users post questions here for other users.
Feature requests, change suggestions, or bugs can be logged in the ticketing system

jtiasgaard
Posts: 5
Joined: 16 Jun 20 4:47

Re: Duplicates and Similar Images

Post by jtiasgaard » 07 Sep 20 22:42

Thank you for your reply...

So, I assume, I should delete all of the sidecar files in order for them to then show up as duplicates? So it is just comparing the RAW file and not differing XMPs?

Hert
Posts: 6369
Joined: 13 Sep 03 7:24

Re: Duplicates and Similar Images

Post by Hert » 08 Sep 20 14:31

I wouldn't do that. Keep in mind that PSU generates the signature on import. PSU uses the signature to detect changes. And that's why the sidecar files are taken into account as well. Because when the XMP changes, PSU also has to consider that file as changed and be able to detect such changes. If you'd delete all the XMP files then you'd also have to reimport the files (manually with Cmd+Opt+S or Ctrl+Alt+S). Only afterwards the signature is updated.
I am currently considering if PSU should perhaps keep two signatures. One for the file only, one for difference detection.

For now, I recommend using "Similar images". Those should include all your duplicates because that looks at the content of the image.
This is a User-to-User forum which means that users post questions here for other users.
Feature requests, change suggestions, or bugs can be logged in the ticketing system

Post Reply