anyone having problems with Verify Folders for Files?

fbungarz
Posts: 1628
Joined: 08 Dec 06 5:03
Location: Arizona, USA

anyone having problems with Verify Folders for Files?

Post by fbungarz » 15 Dec 20 17:55

I have tried this two times now:
I imported images into one database on one of my computers, used the importer to change the file names, add metadata, versioning. After compacting the database my previous complaints about this no longer working have gone away - great :D

But then I copied the files over to a different computer and tried to add the files there by using "right-click, verify folder, scan for missing folders and files". The program detects the missing folders and files and I hit "add to catalog", the progress bar opens in the activities window and then the application completely freezes. "Not Responding" according to the task manager. Last night I let it run for two hours. No progress at all.
I finally closed it and re-started, and all the images are there :shock:

This morning another attempt with a different set of images: I get an error that version detection failed ???
And the program freezes again. Upon restart only some folders were imported... not versioned, no thumbnails...
Perhaps I interrupted the process to early. The first batch imported last night were all unversioned files; the second batch were images that have been versioned on the other computer. Perhaps I need to turn off version detection when using verify folders?

Anyone else having similar problems???

Thanks,
Frank

fbungarz
Posts: 1628
Joined: 08 Dec 06 5:03
Location: Arizona, USA

Re: anyone having problems with Verify Folders for Files?

Post by fbungarz » 15 Dec 20 19:27

Hhmm, seems to work now with the version detection switched of. I am using the exact same version placeholder setup in both databases and thus would have expected this not to cause any problems for the files versioned on a different computer in a different database...
And I definitely did not think having the option enabled for files that are obviously not versioned might make any difference.
Quite obviously it does.
With the version detection turned off, the whole procedure now was pretty fast. No hick up at all...

vkfoto
Posts: 199
Joined: 19 Oct 16 3:51

Re: anyone having problems with Verify Folders for Files?

Post by vkfoto » 15 Dec 20 21:09

Don't know if this is all related and also if it's due to the recent update BUT
  • Sometimes I'll start PSU, click on the Preferences option in the hamburger menu and PSU becomes unresponsive. According to Task manager, there is no CPU activity by PSU
  • I'll be going through the database (279k+ images), updating the keywords. With automatic sync turned on, sometimes the changes are made and other times, the activity bar will say that it's updating but nothing happens and the progress bar stays put.
  • Ran into a database corruption and after sending the report, I restarted PSU and attempted the restore from a previous backup. Nothing happened. I manually replaced the files from the backup folder and PSU appeared to start normally albeit at an earlier state.

fbungarz
Posts: 1628
Joined: 08 Dec 06 5:03
Location: Arizona, USA

Re: anyone having problems with Verify Folders for Files?

Post by fbungarz » 15 Dec 20 23:38

I think my database got corrupted just now as well! The FIRST time this ever happened since I used IdImager and then PhotoSupreme !!!
:(

After everything seemed to have imported just well without any glitches when turning off version detection. I ran version detection on those unversioned file. Worked just fine. Then I noticed that a lot of images were now suddenly "out-of-sync", so I selected them all and used CTRL+S. This crashed midway through and threw and "exception error".
The folder with the database now has four files in addition to the thumbs and image databases themselves:
idimager.cat.db-shm = 32 KB
idimager.cat.db-wal = 0 KB
idimager.thumbs.db-shm = 32 KB
idimager.thumbs.db-wal = 1,420 KB

Upon restart the program is unresponsive. Everything just stalled. I tried deleting those files. Same difference.
What can I do? Do I need to go back to the backup???
Also, I am a little scared that some of the files did perhaps not get synced correctly, is there any chance for metadata corruption?

Thanks,
Frank


User avatar
G8DHE
Posts: 297
Joined: 21 Aug 17 13:58

Re: anyone having problems with Verify Folders for Files?

Post by G8DHE » 16 Dec 20 12:19

Not had a problem with Verify and I use that most of the time, however I have had several occurrences of the hanging on startup in the last few days, I can't duplicate it everytime (had this before a while back as well) but it seems if I leave it 10 seconds after all my external link Icons appear I don't get a problem, but if I start clicking for things as soon as the screen appears then its a 50/50 chance it will hang and go non-responsive. Last time this started happening it seemed to sort itself possibly after a further update. See viewtopic.php?f=57&t=28071
Geoff Mather (G8DHE)

fbungarz
Posts: 1628
Joined: 08 Dec 06 5:03
Location: Arizona, USA

Re: anyone having problems with Verify Folders for Files?

Post by fbungarz » 16 Dec 20 16:12

Geoff,
thanks for getting back, but it is not necessarily a startup issue. At least not a simple one. I have let it run for half an hour or more and it still stalls...

I have now made another backup of what I assumed were corrupted database(s). Then I used SQLite to do a vacuum on both databases, which took almost all night to finish, but it did finally complete. That would suggest the databases are structurally intact.

Just now, I try to start it again. Same difference. Program hangs, not just for seconds, but minutes, hours, eternally...
I can only assume that there is something wrong with the startup parameter. Perhaps these parameters were somehow written to the registry? Some corrupted startup value that keeps PSu from starting up correctly ???

Hert - any suggestions. I would MUCH rather not have to go back to a months old backup.

Thanks !!!

Frank

fbungarz
Posts: 1628
Joined: 08 Dec 06 5:03
Location: Arizona, USA

Re: anyone having problems with Verify Folders for Files?

Post by fbungarz » 16 Dec 20 16:16

PS: If it helps troubleshooting - after I terminate the application via Task Manager, the *.db-smh and *.db-wal files remain present in the folder together with the main database files. Normally these temporary files would be deleted upon close.

vkfoto
Posts: 199
Joined: 19 Oct 16 3:51

Re: anyone having problems with Verify Folders for Files?

Post by vkfoto » 16 Dec 20 16:37

I've got a whole pile of keywords to update and with the up coming holidays, I thought I would spend some time doing that since we'll be in lock down for a while. Is there an easy way to roll back the latest PSU update? Otherwise, I have a version 5 installer for version 5.0.0.2423 which I could reinstall while waiting for this unresponsive issue to be resolved.

fbungarz
Posts: 1628
Joined: 08 Dec 06 5:03
Location: Arizona, USA

Re: anyone having problems with Verify Folders for Files?

Post by fbungarz » 16 Dec 20 16:38

I just got it to work. The database is intact and starts up without problems now.
Here is what happened:
For years I have been using a custom registry, which allows me to transfer database settings more easily from one computer to the next.
Following Michael Weidner's tweaks, I have set up my program shortcut to use the following target:
"...\Photo Supreme x64\IDimagerSU.exe" -regdb "...\PSU.idreg" -A "C...\PSU Catalog"
[not adding the complete paths here for security reasons]
The "-regdb" parameter tells the program to create a custom registry file, the "-A" parameter tells it where to look for the catalog database files.
After deleting the "-regdb" parameter the program seems to just now starting up normal again. This suggested the crash might have corrupted my custom registry file.

Some more sleuthing:
Looking at the folder with the custom registry file, I notice there were also two -shm and -wal files there. Deleting those and changing the shortcut back to using the custom registry, the program starts up just fine. So, it seems the custom registry file wasn't actually corrupted, but some temporary settings that weren't properly deleted when the program crashed are to blame.

Now, Hert will likely tell me simply not to use a custom registry any longer, right? But I guess I can call myself lucky that I did!? Else the corrupt registry settings might have been written to the Windows registry, right? Which I guess would imply the only way would have been to do a complete remove and re-install?

Well, all that is just guesswork on my side of course.

I will now need to do some checking to see if everything is there and working as expected ...

fbungarz
Posts: 1628
Joined: 08 Dec 06 5:03
Location: Arizona, USA

Re: anyone having problems with Verify Folders for Files?

Post by fbungarz » 16 Dec 20 16:45

Otherwise, I have a version 5 installer for version 5.0.0.2423 which I could reinstall while waiting for this unresponsive issue to be resolved.
Do you have similar problems? Else, why go back to a previous version?
I am not sure, actually, if going back to a previous version is always possible. When updates are executed PSu occasionally tells you that your database needs to be updated. If you accept that, the database will likely no longer run under the previous version. In that case you would not just have to re-install the older version of the program, but also use a backup of a previous version of your database.

Unless you have experienced similar issues as I have, I would not recommend doing this. In my experience PSU (and previously IDI) has always been extremely resilient against any corruption and my experience now actually confirms again that it is pretty difficult to damage the database itself. The crash obviously did not corrupt the database files themselves, but somehow the startup parameters got messed up.

vkfoto
Posts: 199
Joined: 19 Oct 16 3:51

Re: anyone having problems with Verify Folders for Files?

Post by vkfoto » 16 Dec 20 16:58

When I get back to my photo computer, I can check but I don't think I'm using any custom registry parameters except for pointing the temp folder to another specific location since my C: drive is too small.

fbungarz
Posts: 1628
Joined: 08 Dec 06 5:03
Location: Arizona, USA

Re: anyone having problems with Verify Folders for Files?

Post by fbungarz » 16 Dec 20 21:15

Unfortunately this is not resolved yet.
:(

There seems to be something off, with how the program now syncs versioned files (built 5.6.0.3337).
No idea if that only affects my database or whether this is a general issue.

I typically have three versions as part of a version set for most of my images:
The original JPG+NEF pair from the camera + a DNG. Sometimes there are additional versions, identified by a _pring or _edited suffix

My cellphone photos are a single file only, no versions. The all sync without a problem. The problem are the files imported via verify folder. As outlined before version detection during verify failed. So I imported these files not as versions and then used versioning to create the versions. Afterwards all files added to the database via "verify" were out of sync. Trying to synchronize these files the program crashed and as outlined above I was only able to get it back to starting up by deleting all -smh and -wall files, including those of the custom registry.

But, back up running the same happens again. The sync process stalls, the program hangs, closing the application the temporary -smh and -wall files do not disappear, but need to be manually deleted.
So, I decided to try syncing one file at a time. Non-versioned files sync without a glitch, but the versions don't. The JPG from a JPG+NEF+DNG triplet gets synced, then the program stalls (at 40% of the sync process). I can only close out via task manager, delete the temporary -smh and -wall files manually and then start again...

fbungarz
Posts: 1628
Joined: 08 Dec 06 5:03
Location: Arizona, USA

Re: anyone having problems with Verify Folders for Files?

Post by fbungarz » 16 Dec 20 21:16

CORRECTION:
The program does sync, but each single image takes several minutes to sync.

fbungarz
Posts: 1628
Joined: 08 Dec 06 5:03
Location: Arizona, USA

Re: anyone having problems with Verify Folders for Files?

Post by fbungarz » 16 Dec 20 21:24

It seems that I found the culprit. There was one version set that would not sync. I split the version set and re-versioned. It synced and it seems that now the other files sync as well.

Post Reply