download/import skips images

[Hert]

The last version of the script implements a separate lock and ruled out that this is a threading race condition issue. If these files already exist on the hard drive then that is the only thing that makes sense to me right now.

You say they don't and so other things may be at play here. Could it be that writing to the registry has delays due to a virus scanner that keeps track of registry changes? Afaik writing to the registry should not have delays....and if you are 100% sure that the files do not pre-exist then the registry having delays after a write (meaning a written value is not persisted until after a delay of the write statement) would be the only explanation.

Do I remember correctly that you're using a registry file with a startup command???? With a registry file, the registry file is updated every time that a value is written and I can imagine that during import the file is rewritten a LOT of times meaning you can get delay issues with virus scanners but also with Windows' cache system. If using a registry file then try removing that startup command so that the standard registry is being utilized. Does it then work?

[fbungarz]

Thanks for still helping with the troubleshooting!!!

I believe this indeed MUST have something to do with me using a custom registry.
After running the profile once, which resulted in the download of 24 instead of 38 images, skipping 14, I now tried downloading the files that were skipped immediately again, with leaving the ones downloaded in place. I would expect that the downloader starts renaming these previously skipped files using the last counter number stored by the script in the registry.

This does not happen. The counter starts all over again with number "1". I have been testing this on a completely new download profile, with a new file prefix. This means the first download would of course start with one, but downloading 24 images the next number should then start at 25 and not again at number 1.
So, the second part of the script (or more correctly) the pre-script, not the renaming script, which keeps tracks of the numbers storing the counts in the registry, clearly does not work. It does not store the numbers in my custom registry.

Do I remember correctly that you're using a registry file with a startup command????

Yes. I am using this:
"..\Program Files\IDimager Products\Photo Supreme x64\IDimagerSU.exe" -regfile "..\PSU Catalog\mydata.idreg" -A "..\PSU Catalog"

Could it be that writing to the registry has delays due to a virus scanner that keeps track of registry changes?

I am using ESET NOD32 and the folder, where the database and registry file reside is excluded from scanning. A while back, I believe, someone on the forum recommended excluding the PSU database folder from virus scanning to improve speed. I have not changed that configuration since.

With a registry file, the registry file is updated every time that a value is written and I can imagine that during import the file is rewritten a LOT of times meaning you can get delay issues with virus scanners but also with Windows' cache system.

Both Windows own registry resides on the same SSD as the custom regfile as well. So, why would caching to the custom registry be any different to the main registry?
Has anything with writing to the registry changed in version 4? Why would this be different from version 3?

If using a registry file then try removing that startup command so that the standard registry is being utilized. Does it then work?

The reason why I decided to use a custom registry was to more easily exchange the registry settings of the database between my Laptop and Desktop computers. I am using the same database with identical settings on both, using SyncBack to keep both synchronized. If I disable the regfile startup command, how can I make sure these registry parameters are still synchronized between the computers? And: will I not loose the current regristry settings in the custom registry? How do I avoid that? As you pointed out before PSu's regfile format is not identical with the one in Windows, so importing these values into Windows registry does not work, right?

I am quite happy to switch to using the standard windows registry instead of the custom PSu file. And I can test, if the pre-script then remembers the number count correctly. But before I do, I'd like to know the implications. Will it then be just as easy to switch back and forth between Laptop and Desktop using SyncBack to keep the programs synchronized?

I assume that the registry file tells PSu where to look for download profiles, scripts, renaming rules, filters, etc. etc. I have these all stored in the directory that I keep synchronized back and forth. Wouldn't switching to a standard registry mean that these settings can no longer be exchanged that easily between Laptop and Desktop?

[fbungarz]

BTW - actually running the profile and resetting the counter manually the previously skipped images get downloaded!

So, I believe this has nothing to do with the files already having been downloaded or PSu skipping them because it considers them duplicates. It seems to me that it must be related to not updating the counter values in the registry. So, either this is a problem with storing the values in the custom registry, or it is a problem with the renaming rule not correctly telling where and how to store these values, or a problem with the pre-skript not updating these values...

[Hert]

And: will I not loose the current regristry settings in the custom registry? How do I avoid that? As you pointed out before PSu's regfile format is not identical with the one in Windows, so importing these values into Windows registry does not work, right?

You can't avoid that. But that's worry for later. First check if it works with the Windows registry.

But before I do, I'd like to know the implications. Will it then be just as easy to switch back and forth between Laptop and Desktop using SyncBack to keep the programs synchronized?

First check if switching to Windows registry makes a difference. And if it makes the difference, then worry. Though to me that is a no brainer: use a registry file and not being able to import or not use a registry file and being able to import.

[fbungarz]

Though to me that is a no brainer: use a registry file and not being able to import or not use a registry file and being able to import.

It is more like:
Use a custom registry and to keep PSu Settings synchronized on two different computers, but not being able trust the import (and thus have to do the renaming and sorting into folders manually outside and use Verify Folders to get files into PSu - which is a real pain, but which I have had to do for quite some time now) ...

But OK, I understand that I will have to test this as the next troubleshooting step

[fbungarz]

OK, I just tried it. Getting rid of the custom registry, all files are downloaded, not one is skipped.
Hurray!


Now my question:
What settings are stored in the custom registry? Are there any "essentials", which I need to transfer to another computer when keeping databases on two PCs in-sync?
My biggest worry is this:
All my download profiles, share settings, print templates, scripts, etc. are currently stored in folders that reside in the same folder where the database is located, and NOT in Windows APP data folder. It is relatively simple to keep that folder with the database and the other stuff synchronized across different PCs. Having to share a folder in Windows APP data folder is a real pain, running into issues with file access permissions...

I'd just like to know: do I loose any important settings, if I can no longer share the custom registry across computers?
Thanks for clarifying...

And: thanks for all the troubleshooting!!!

PSU support rules

[Hert]

There's no important information in the registry. It mainly is used to remember last visited folders etc. So you may have to select the file from the folder and onward the folders will be remembered again.

PS. in the next update I'll open up the new registry file type that I've wanting to release for a while. That new type allows you to use a registry file without the disadvantages of using the flat file registry saving/loading under high transaction conditions.

[fbungarz]

PS. in the next update I'll open up the new registry file type that I've wanting to release for a while. That new type allows you to use a registry file without the disadvantages of using the flat file registry saving/loading under high transaction conditions.

Sounds great!
I'll try it out then...