Problem with "build missing thumbnails and previews"

PhilBurton
Posts: 313
Joined: 12 Sep 10 18:47
Location: CA, USA

Re: Problem with "build missing thumbnails and previews"

Post by PhilBurton » 19 Nov 15 5:30

Mike Buckley wrote:
PhilBurton wrote:Is there any way to run an "image validate" function that would detect and optionally remove corrupted images?
Not that I'm aware of.
Is this something that should go into Mantis?

Phil
Photo Supreme user
Home built i7 3930, 32 GB RAM, Win 10 Pro 64, latest version of Photo Supreme 3, Lightroom 6 and Photoshop CS 6 (perpetual licenses)

vlad
Posts: 969
Joined: 01 Sep 08 15:20

Re: Problem with "build missing thumbnails and previews"

Post by vlad » 19 Nov 15 14:29

I don't know to what extent PSU (or any other program) could detect corrupted images and signal them as such. I guess one question is how is "corruption" defined and what would be the scope of corruption detection.

(For example, this discussion concerns thumbnail (re)building, but aren't those images which require "Convert Metadata to XMP" corrupt in some way (i.e. non-compliant with the standard), too? I don't want to hijack this thread, but I have always wondered how could I know to run that operation and on which image selection, unless I find out which images are missing proper XMP data in the first place.)

Mike Buckley
Posts: 4561
Joined: 10 Jul 08 14:18

Re: Problem with "build missing thumbnails and previews"

Post by Mike Buckley » 19 Nov 15 15:29

vlad wrote:how could I know to run that operation [ "Convert Metadata to XMP"] and on which image selection, unless I find out which images are missing proper XMP data in the first place.
I don't think it will hurt anything to run it, so you could run it on all images. As an example, I always have to run it on all vertically oriented raw files that have been edited in Capture NX2. Rather than take the time to select only the vertically oriented images, I select all of them.

PhilBurton
Posts: 313
Joined: 12 Sep 10 18:47
Location: CA, USA

Re: Problem with "build missing thumbnails and previews"

Post by PhilBurton » 19 Nov 15 20:31

vlad wrote:I don't know to what extent PSU (or any other program) could detect corrupted images and signal them as such. I guess one question is how is "corruption" defined and what would be the scope of corruption detection.
Let's define corruption as some issue with the image file or its sidecar XMP file that causes PSu to misfunction or even crash. The misfunction could be an inability to display the thumbnail, inability to import more images, or a general freezeup.

Maybe what's needed is some kind of diagnostic reporting, like the kind you get when Windows bluescreens.

Phil
Photo Supreme user
Home built i7 3930, 32 GB RAM, Win 10 Pro 64, latest version of Photo Supreme 3, Lightroom 6 and Photoshop CS 6 (perpetual licenses)

n8vz
Posts: 65
Joined: 09 Oct 09 18:39

Re: Problem with "build missing thumbnails and previews"

Post by n8vz » 19 Nov 15 20:40

I don't know of any such routine, either. It would be a nice addition in a future update. It seemed to me that there used to be more "fix it" routines in the old IDI than in PSU.

That brings me back to the problem. I tried Mike's latest suggestion, and I still have not resolved the issue. I've also noted that PSU seems to be running slower than it used to.

Hert
Posts: 21548
Joined: 13 Sep 03 7:24

Re: Problem with "build missing thumbnails and previews"

Post by Hert » 19 Nov 15 22:16

Slower? Compact the database regularly, a weekly frequency is a good average maintenance cycle.
This is a User-to-User forum which means that users post questions here for other users.
Feature requests, change suggestions, or bugs can be logged in the ticketing system

Mike Buckley
Posts: 4561
Joined: 10 Jul 08 14:18

Re: Problem with "build missing thumbnails and previews"

Post by Mike Buckley » 20 Nov 15 3:04

IDimager wrote:Slower? Compact the database regularly, a weekly frequency is a good average maintenance cycle.
If it really should be done weekly, that information should be included in the "Maintenance" Quick Manual. It only says that compacting should be done "on a regular basis." A regular basis could be every six months or every week.

Personally, I have gotten to the point that I compact only when I make changes to the system of catalog labels though I would also compact if I used the custom XMP capabilities and the like. I have never experienced any of the problems experienced by people who don't compact.

Hert
Posts: 21548
Joined: 13 Sep 03 7:24

Re: Problem with "build missing thumbnails and previews"

Post by Hert » 20 Nov 15 6:52

There's no reason why you should compact after changing your catalog label system or using custom XMP. Main reason to compact is to clean up fragmentation in the database file(s).
This is a User-to-User forum which means that users post questions here for other users.
Feature requests, change suggestions, or bugs can be logged in the ticketing system

n8vz
Posts: 65
Joined: 09 Oct 09 18:39

Re: Problem with "build missing thumbnails and previews"

Post by n8vz » 20 Nov 15 18:03

IDimager wrote:Slower? Compact the database regularly, a weekly frequency is a good average maintenance cycle.
I do this regularly.

Mke
Posts: 558
Joined: 15 Jun 14 15:39

Re: Problem with "build missing thumbnails and previews"

Post by Mke » 23 Nov 15 0:45

Mike Buckley wrote:
PhilBurton wrote:Is there any way to run an "image validate" function that would detect and optionally remove corrupted images?
Not that I'm aware of.
No - but fixity checking to guard against digital decay is certainly something that I'd like to see in PSU. Without it, it could be years before a corruption problem is discovered, by which time it is likely that any backups would also contain corrupted versions of the file, unless you use abnormally robust archiving. Only a small sample, but doing some quick math on the Getty Research Institute report at http://blogs.getty.edu/iris/preventing-digital-decay/ suggests that there is a likely complete corruption of around 2 images per decade per 1,000 images. (In fact maybe that rate would now be higher, since image files are now much bigger - so more data to corrupt?)

With an in-depth knowledge of the image file structure would I imagine it would be possible to separate the data for the image (which wouldn't be expected to change) with the metadata in the file (which would, for those of use who write metadata for the files) and to create a checksum for the former. I thought I'd put in a Mantis request for it already, but as I can't find it it looks like I didn't.

The only photo software that offers a partial solution (restricted to DNG files, which have inbuilt checksums) is Lightroom (from version 5).

Of course if you never change your files once their created, you can do it with 3rd party solutions - such as ExactFile http://www.exactfile.com/ (donationware).
Or for the ultimate solution... ...archive your images to analog film, perhaps (https://web.archive.org/web/20150330191 ... _sheet.htm)?

Mike Buckley
Posts: 4561
Joined: 10 Jul 08 14:18

Re: Problem with "build missing thumbnails and previews"

Post by Mike Buckley » 23 Nov 15 2:13

Mke wrote:there is a likely complete corruption of around 2 images per decade per 1,000 images.
That's only 0.02% per year! Considering that PhotoSupreme is available via a perpetual license of only $100, if I was the developer that statistic would be the best economic motivation I could imagine for not taking the effort to build the corruption-detection capability into the software. I would leave that up to a third-party provider. That's especially true considering that there are free checksum products.

Mke
Posts: 558
Joined: 15 Jun 14 15:39

Re: Problem with "build missing thumbnails and previews"

Post by Mke » 23 Nov 15 15:09

Mike Buckley wrote:
Mke wrote:there is a likely complete corruption of around 2 images per decade per 1,000 images.
That's only 0.02% per year! Considering that PhotoSupreme is available via a perpetual license of only $100, if I was the developer that statistic would be the best economic motivation I could imagine for not taking the effort to build the corruption-detection capability into the software. I would leave that up to a third-party provider. That's especially true considering that there are free checksum products.
It's also a total loss of 200 images out a collection of 100,000 over 50 years - plus all the partial corruptions - more if file size plays a part, as I suspect; whether that's good or bad is subjective. Compared to the total loss of zero images in my old analog collection (which dates back over 130 years - no I didn't take them all) it doesn't seem particularly good to me.

In marketing terms, adding a feature that makes a real difference in protecting your images would be a good selling point. Adobe seem to think so, even if they've only picked off an easy partial solution.

Mike Buckley
Posts: 4561
Joined: 10 Jul 08 14:18

Re: Problem with "build missing thumbnails and previews"

Post by Mike Buckley » 23 Nov 15 15:52

Mke wrote:In marketing terms, adding a feature that makes a real difference in protecting your images would be a good selling point. Adobe seem to think so, even if they've only picked off an easy partial solution.
Actually, I looked for the selling point at Adobe's website and couldn't find it. Even if it's buried somewhere so deeply that I missed it, it's clear to me that they don't think it's a particularly good selling point. If it was, it would be displayed more prominently or at all.

n8vz
Posts: 65
Joined: 09 Oct 09 18:39

Re: Problem with "build missing thumbnails and previews"

Post by n8vz » 08 Dec 15 17:51

I still have not solved this problem. I can't find the remaining apparently corrupt files that are presumably at fault. This is very frustrating. PSU really needs a utility to identify the files that it is having trouble with and that may be corrupt.

Mike Buckley
Posts: 4561
Joined: 10 Jul 08 14:18

Re: Problem with "build missing thumbnails and previews"

Post by Mike Buckley » 08 Dec 15 19:20

Have you tried the suggestions I made in my 5th and 6th posts in the thread?

Post Reply