download/import skips images

fbungarz
Posts: 3250
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: download/import skips images

Post by fbungarz » 05 Oct 18 16:14

If you remove the copy script, will it then copy all your images?
It is not a copy script, it only renames the files. But I have not tried this, because I rely on having those files renamed correctly. Else, I could just import the folder - which works. I had to to that with the previous batch of files that I copied from the memory card via FastPictureViewer. They were simply too many to use the downloader. So, I manually renamed them and then simply used VerifyFolder to import them.

Do you suggest I should have tried importing them instead without using the script? That would imply I also change the download profile...

I haven't tried creating a new download profile from scratch. Perhaps I should...
And you've never used the import feature in V4 before this?
Nope!

Hert
Posts: 21197
Joined: 13 Sep 03 7:24

Re: download/import skips images

Post by Hert » 05 Oct 18 17:13

We can’t just guess. If we want to know what could cause this we’ll have to rule out things. So if it works without the script then we know for sure that it’s in the script or the script related features
This is a User-to-User forum which means that users post questions here for other users.
Feature requests, change suggestions, or bugs can be logged in the ticketing system

fbungarz
Posts: 3250
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: download/import skips images

Post by fbungarz » 16 Oct 18 23:32

It's the script. The images renaming gets messed up and as a result some files are not being downloaded. Also, the script counter does not remember the last number assigned. It fails to store it in the registry.
Last edited by fbungarz on 21 Nov 18 0:50, edited 1 time in total.

fbungarz
Posts: 3250
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: download/import skips images

Post by fbungarz » 16 Oct 18 23:38

...and it is always the 5th file that is not being downloaded. A maximum of 4 images = 2 image pairs (NEF+JPG) are being correctly renamed.

fbungarz
Posts: 3250
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: download/import skips images

Post by fbungarz » 20 Nov 18 0:55

OK, I have been away collecting specimens and taking more images, so I could not follow up on this.
I tried it out again today with the newest version 4.3.0.1755.
If I use the script the counter number is not saved to the registry and ever so often some of the files are skipped and the image renaming therefore messed up.
Disabling the script unfortunately does not work either!
If I mark 318 out of a total of 318 images, only 130 images get downloaded. Oddly enough the images still get renamed according to the rename rule, despite having disabled that option and without any box popping up that showed a number with the counter :?
To me it looks like the renaming rule from the script is still being applied even though I disabled the option to run the scrip...
Very strange.

fbungarz
Posts: 3250
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: download/import skips images

Post by fbungarz » 20 Nov 18 0:58

Here the screenshot that shows that not all files that were marked got downloaded and that despite disabling the script the images were nevertheless still renamed...
Download-result.jpg
Download-result.jpg (250 KiB) Viewed 368 times

fbungarz
Posts: 3250
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: download/import skips images

Post by fbungarz » 20 Nov 18 1:06

PS: Just to explain what I did...
I chose an existing import profile which downloads the images into folders by year and then date, e.g.:
2018
|--2018-11-19
That profile also applies my renaming script.
Now, to disable the script I saved removed the checkmark from the box that says "run script" and saved the profile under a new name.
Result: the files are still being renamed and and not all files are downloaded...
When I try downloading using the same new Import Profile again, the program freezes and I get an exception error, being asked if I want to send the error. However, I cannot send the error message or cancel but the need to kill the program via Task Manager, because everything is frozen...

Username
Posts: 166
Joined: 18 Feb 18 22:21

Re: download/import skips images

Post by Username » 20 Nov 18 7:22

I will try this later today on my Mac and see what happens.
I shoot NEF+jpeg as well but into separate memory cards.

What happens if you manually move your images to a folder on disk and import/rename them via script from there.
Still problems?
- I'm the user

fbungarz
Posts: 3250
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: download/import skips images

Post by fbungarz » 21 Nov 18 0:33

I am downloading from a folder, not a memory card. The JPG+NEF pairs are copied from the card using FastPictureViewer.
This worked flawlessly in previous versions of PSu.
For me this means the downloader is now completely broken. The only way I can be sure that all files are imported is by manually creating a folder in Windows File Explorer, copying the files there and manually rename the pairs and then have PSu scan for missing folders.
:cry:

fbungarz
Posts: 3250
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: download/import skips images

Post by fbungarz » 21 Nov 18 0:50

For now my workaround is this:
(1) In Windows File Explorer create the Folders manually.
(2) Use XYplorer rename functionality with rename mask (e.g., HX_<#2707>) to first rename all JPGs, then all NEFs. This means I now have the file renamed in pairs.
(3) Use "verify" in PSu to add have the folders and files added...
Bit of a pain for something that previously just required using the built-in downloader.
I first thought this was caused by using a corrupted rename script, but it is not even possible to disable using that script...

fbungarz
Posts: 3250
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: download/import skips images

Post by fbungarz » 21 Nov 18 0:54

I will probably need to test this with a completely new download profile...
No script, no nothing, just the default.
:?

fbungarz
Posts: 3250
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: download/import skips images

Post by fbungarz » 30 Nov 18 23:16

OK, still troubleshooting, what is going on...

(1)
I now have created a new download profile and used a new batch of images to test. The download profile uses the same settings as the other profiles, but NOT the script.
This works. No file is skipped. :D

(2)
In the next step I am trying to add the script to the profile. No success :(
  • I open the download profile, select "more copy settings"
  • Checking the box "Scripting" opens up the options to run a script "before" or "after" or use a "file copy script".
  • I choose "before" and select the script "downloadbyPrefix_noButton" from the drop down.
  • Then I run the profile.
  • The pop-up box from the script shows up and allows me to enter the prefix.
  • I use a new dummy prefix and the counter shows number "1".
  • I hit OK and the import starts...
  • The images get downloaded, BUT not renamed !!!
  • The next time the counter for the script is still stuck at number "1".
So - when I create a completely new download profile from scratch, enabling the rename script has no effect, images do NOT get renamed...
If I modify an existing profile to NOT use the script, the script does still run, images get renamed, but some of the images are being skipped ...
:(

What do I do now to continue troubleshooting ?

:mrgreen:

fbungarz
Posts: 3250
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: download/import skips images

Post by fbungarz » 01 Dec 18 20:08

OK, Hert suggested I share the import profiles and the script, so that hopefully someone here can try them out and see if they get the same results.

First the script:
The way this script is supposed to work (and always worked for me perfectly before upgrading) is this. You add it to the Download Profile it as pre-script. When you run the profile, a pop-up box asks you for a prefix (e.g., "ASU_", "DX_", whatever...). For each prefix the script then generates a four digit counter number (with leading zeros) and then stores the last number after renaming in the registry. That way, if you run the script again using the same prefix it remembers what the last number was and starts renaming at the next number.
The script also renames pairs of JPG+NEF files with the same counter number, i.e.:

_DSC0789.jpg ---> DX_0001.jpg
_DSC0789.jpg ---> DX_0001.nef
_DSC0790.jpg ---> DX_0002.jpg
_DSC0790.jpg ---> DX_0002.nef
_DSC0791.jpg ---> DX_0003.jpg
_DSC0791.jpg ---> DX_0003.nef
_DSC0792.jpg ---> DX_0004.jpg
_DSC0792.jpg ---> DX_0004.nef
etc.

Here that script:
DownloadByPrefix_noButton.psc
(2.88 KiB) Downloaded 4 times
[Needs to be placed into \Scripts\ImportScripts]

Now the download profile(s):
My profiles are very simple. They download the files into a subfolder with the date, which resides inside a folder named by the year, thus images shot today would end up inside:

G:/Photos/2018/2018-12-01/

Here the download profile that works with the script, but unfortunately is not reliable, because it skips some of the images and thus makes a mess out of the renaming (and does not remember the correct, new, highest counter number):
HX_ImportProfile_2018-12-01.zip
(6.28 KiB) Downloaded 2 times
Then supposedly the one where the script is disabled, but oddly enough files are still renamed (and skipped!):
HX_ImportProfile_noScript_2018-12-01.zip
(6.3 KiB) Not downloaded yet
And, finally a new Import Profile, that downloads the images without skipping any, but does not rename, even though the script has been added:
ASU_Macro_2018-12-01.zip
(1.42 KiB) Downloaded 1 time
Thanks everybody for testing and reporting the results. I really hope to figure out, why this no longer works...

Frank

[FYI - I had to ZIP these import profiles, else the forum would not let me upload them; unzip and place them into \\PSU Catalog\ImportProfiles\]

fbungarz
Posts: 3250
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: download/import skips images

Post by fbungarz » 03 Dec 18 16:41

Hert just pointed out that the renaming in my new profile does not work, because the script needs to interact with the following rename code being entered into the custom define file rename rule:

Code: Select all

//    - This script assumes the list of processed  filenames will be ordered
//      such that like basenames are consecutively processed.
//    - This script is designed to be used with the downloader pre-script
//      "Download by Prefix", as it manages some of the registry keys used
//      by this script.
// JAG, 16-Apr-2007
// HVZ, 23-Jun-2016 (converted to Photo Supreme)

var
  lastNumber: Integer;
  lastFile:   WideString;
  prefix:     WideString;
 
begin
  prefix     := ReadFromRegistry ('dlByPrefixCurrent', 'prefix', '');
  lastNumber := ReadFromRegistry ('dlByPrefixCurrent', 'value', 0);
  lastFile   := ReadFromRegistry ('dlByPrefixCurrent', 'filename', '');
  if (prefix = '') then prefix := 'UNKNOWN_'; 

  // If the current filename doesn't equal the last stored filename
  // increment our number and store both values to the registry
  if (lastFile <> '%FileName') then
    begin
       Inc(lastNumber);
       WriteToRegistry ('dlByPrefixLastVals', prefix, lastNumber);
       WriteToRegistry ('dlByPrefixCurrent', 'filename', '%FileName');       
       WriteToRegistry ('dlByPrefixCurrent', 'value', lastNumber);
  end

  // return our newly constructed filename
  result := prefix + AddLeadingChars (IntToStr(lastNumber), '0', 4, False);
end;
So, I guess, I'll try again and report back...

:wink:

fbungarz
Posts: 3250
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: download/import skips images

Post by fbungarz » 03 Dec 18 17:20

OK, I just added the renaming rule to the new profile.

Now, files are being renamed, but again not all files are downloaded, some are being skipped by the downloader and the re-nameing thus is a complete mess. These are just the same results as using any of my my other profiles.

The screenshot below illustrates this:
The NEF file that should have been downloaded and re-named ASU_0005.nef was skipped. This messed up the naming of the following files son that ASU_0006.jpg got paired with ASU_0007.nef and ASU_0006.nef with ASU_0007.jpg
StillSkipped.jpg
StillSkipped.jpg (254.58 KiB) Viewed 67 times
Not only the re-naming gets messed up, also not all files are being downloaded. In this particular case only 39 files out of a total of 68 files were downloaded, i.e., almost half of all files were skipped...

This means:
With the pre-script and the renaming rule in place, some files are not being downloaded .

Without the pre-script and only the renaming rule in place the results are the same, files are being skipped.

With the pre-script and no renaming rule, all files are downloaded.

Without pre-script and without re-nameing rule, all files are downloaded.

To me this suggests, some part of the renaming rule is no longer working, preventing all files from being downloaded and correctly being renamed.

Post Reply