Custom XMP in Info vs Details Panel?

Post Reply
vlad
Posts: 969
Joined: 01 Sep 08 15:20

Re: Custom XMP in Info vs Details Panel?

Post by vlad » 01 Jun 15 19:52

fbungarz wrote: Turning off versioning (with ICS read enabled) results in images being imported as non-versioned, i.e., not arranged in version sets. It also displays the imported images as "out-of-sync". Apparently the metadata are thus not being read during the import.
Which metadata are you talking about? Just ICS or other metadata too?
This, with auto-sync being enabled, seems odd.

I agree. Perhaps I (still) misunderstand the full story of auto-sync, but I've thought that no image should keep staying as OOS with that setting enabled. (Let's not forget, however, that the setting technically - and nominally - refers only to the write syncing of metadata.)
However, it perhaps makes a certain sense: to avoid overwriting the ICS schema PSU apparently sets the images out-of-sync, thus suggesting you might still decide if you want to read the ICS data into the database...
Hmm, why would that be the case?
Now, subsequently reading the metadata from the JPGs, DNGs and NEFs results in these images now being versioned (the JPG as main, DNG and NEF as subversions).
What do you mean by: "now, subsequently reading the metadata"? How do you subsequently read it? Do you select the images and manually invoke the metadata reading (Ctrl+Alt+S)?
Nevertheless, both DNG and NEF are still not assigned to a placeholder.
This actually makes some sense. IDI5 does not write the ICS placeholder tag to the images. This tag is missing and PSU obviously therefore cannot read it from the ICS schema. IDI writes tags which images is the main version and which other ones are subversions. Therefore, reading the metadata from the file into the database after import results in the images being versioned correctly, yet still not assigned to placeholders.
Yep, that part makes sense to me.
Simple. you destroy the version set and then one by one drag first the DNG onto the main JPG and then the NEF assigning placeholders while you do it.
How do you assign the placeholders during DnD?
The rather random assignment of placeholders despite the custim masks correctly configured is definitely reproducible and in my opinion qualifies as a serious bug (which I have now reported in Mantis). I have no idea of course, if this is related to the way DNGs and NEFs are rather randomly assigned one time to the first, other times to the second version. It is only a hunch.
I think I've already asked you in Mantis, but I'm going to do it again, just to remove any possible confusion: when you're reporting the "random assignment of placeholders", are you talking about version detection as performed *after* the import and on non-versioned images? Could you also confirm that you have all other version placeholders set to No Matching?
in my case versioning is rather straight forward. the DNG custom mask is *.dng, the NEF is *.nef and I always have just one single DNG and just one single NEF in each folder with the same name as the main JPG version, e.g.: DX_0001.jpg + DX_0001.dng + DX_0001.nef, DX_0002.jpg + DX_0002.dng + DX_0002.nef, etc.
Again, make sure that you don't have any placeholder set to "First Version Found" or something like this.

vlad
Posts: 969
Joined: 01 Sep 08 15:20

Re: Custom XMP in Info vs Details Panel?

Post by vlad » 01 Jun 15 20:20

fbungarz wrote:OK, I am becoming convinced that this must be related.

(1) I destroy the version set.
(2) I select all images (CTRL+A) and run version detection (CTRL+SHIFT+V).
(3) Some NEFs are assigned to the first subversion. They are assigned the placeholder "Original". NEFs assigned to the second subversion get no placeholder assigned. The DNGs generally get no placeholder assigned, whether they belong to the first or the second subversion makes no difference.
That seems wrong and I would focus my bug report solely on this sequence. The subsequent steps introduce noise, IMHO (see bellow).
(4) I select one triplet (JPG+DNG+NEF) at a time and run run version detection (CTRL+SHIFT+V) again.
(5) For each triplet the "Original" placeholder is assigned to the NEF, which becomes the first subversion, the DNG becomes the second subversion but remains without placeholder.
I am not sure it's valid to run version detection on an existing version set - or, more generally, on any selection that already contains at least one versioned image. (It's an interesting issue, but let's not add complexity here.) Until Hert confirms that is supported too, I would ignore the result of step (5).
(6) I filter the folder to just show me the DNGs, whcih still lack placeholders and on these files now run run version detection (CTRL+SHIFT+V) again.
(7) Now the DNGs are all being assigned their corresponding placeholder "Digital Negative", but the placeholder "Original" previously assigned to the NEFs is again removed from these files!
Ditto for step 7 too.
As far as I can tell this is reproducible. It seems to suggest that version detection (CTRL+SHIFT+V) does not work correctly on more than two files.
If confirmed, that's an interesting finding. Personally I've never used more than one sub-version (except for script testing purposes), but I remember fellow users who employ multiple versions (...successfully, I presume).

fbungarz
Posts: 3305
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: Custom XMP in Info vs Details Panel?

Post by fbungarz » 01 Jun 15 20:23

vlad wrote:
fbungarz wrote: Turning off versioning (with ICS read enabled) results in images being imported as non-versioned, i.e., not arranged in version sets. It also displays the imported images as "out-of-sync". Apparently the metadata are thus not being read during the import.
Which metadata are you talking about? Just ICS or other metadata too?
ICS
This, with auto-sync being enabled, seems odd.

I agree. Perhaps I (still) misunderstand the full story of auto-sync, but I've thought that no image should keep staying as OOS with that setting enabled. (Let's not forget, however, that the setting technically - and nominally - refers only to the write syncing of metadata.)
In IDI the indicator referred to both read and write "out-of-sync".
However, it perhaps makes a certain sense: to avoid overwriting the ICS schema PSU apparently sets the images out-of-sync, thus suggesting you might still decide if you want to read the ICS data into the database...
Hmm, why would that be the case?
I agree it makes not a lot of sense. If one turns reading of ICS one one would expect it to being read during import, right? But perhaps, because reading the ICS schema can mess up your catalog if permanently enabled (I experienced this before in IDI5) the default is set to not read the data unless specifically invoking it manually.
Now, subsequently reading the metadata from the JPGs, DNGs and NEFs results in these images now being versioned (the JPG as main, DNG and NEF as subversions).
What do you mean by: "now, subsequently reading the metadata"? How do you subsequently read it? Do you select the images and manually invoke the metadata reading (Ctrl+Alt+S)?
Yes, you select the thumbnail, right-click and select Metadata - Read Metadata From File (CTRL+ALT+S).
Nevertheless, both DNG and NEF are still not assigned to a placeholder.
This actually makes some sense. IDI5 does not write the ICS placeholder tag to the images. This tag is missing and PSU obviously therefore cannot read it from the ICS schema. IDI writes tags which images is the main version and which other ones are subversions. Therefore, reading the metadata from the file into the database after import results in the images being versioned correctly, yet still not assigned to placeholders.
Yep, that part makes sense to me.
Simple. you destroy the version set and then one by one drag first the DNG onto the main JPG and then the NEF assigning placeholders while you do it.
How do you assign the placeholders during DnD?
Manually dragging a thumb of an unversioned image onto another unversioned one a dialog pops up asking you to which particular version you want to assign that image.
The rather random assignment of placeholders despite the custim masks correctly configured is definitely reproducible and in my opinion qualifies as a serious bug (which I have now reported in Mantis). I have no idea of course, if this is related to the way DNGs and NEFs are rather randomly assigned one time to the first, other times to the second version. It is only a hunch.
I think I've already asked you in Mantis, but I'm going to do it again, just to remove any possible confusion: when you're reporting the "random assignment of placeholders", are you talking about version detection as performed *after* the import and on non-versioned images? Could you also confirm that you have all other version placeholders set to No Matching?
I am talking about the following things:
(1) With versioning enabled during import, images are correctly versioned, but placeholders are not assigned.
(2) When I select all imported & versioned images and run version detection (CTRL+SHIFT+V) some subversions are assigend to placeholder, others not (and that is apparently "randomly" so = I can not recognize any consistent pattern!)
(3) The same happens if I first destroy the version sets again after import and then run version detection again.
Yes, all other placehoders are set to No matching!
in my case versioning is rather straight forward. the DNG custom mask is *.dng, the NEF is *.nef and I always have just one single DNG and just one single NEF in each folder with the same name as the main JPG version, e.g.: DX_0001.jpg + DX_0001.dng + DX_0001.nef, DX_0002.jpg + DX_0002.dng + DX_0002.nef, etc.
Again, make sure that you don't have any placeholder set to "First Version Found" or something like this.
Yes, all other placehoders are set to No matching!

vlad
Posts: 969
Joined: 01 Sep 08 15:20

Re: Custom XMP in Info vs Details Panel?

Post by vlad » 01 Jun 15 21:05

In IDI the indicator referred to both read and write "out-of-sync".
That's changed in Photo Supreme: the OOS indicator only tells there are metadata changes in the catalog that are not yet saved (synced) to the physical file. You may have to unlearn one thing or two from IDI - and that includes using it as a frame of reference for the Photo Supreme functionality :wink:
If one turns reading of ICS one one would expect it to being read during import, right?
Right.
But perhaps, because reading the ICS schema can mess up your catalog if permanently enabled (I experienced this before in IDI5) the default is set to not read the data unless specifically invoking it manually.
We're getting in speculation territory here.
I am talking about the following things:
(1) With versioning enabled during import, images are correctly versioned, but placeholders are not assigned.
(2) When I select all imported & versioned images and run version detection (CTRL+SHIFT+V) some subversions are assigend to placeholder, others not (and that is apparently "randomly" so = I can not recognize any consistent pattern!)
(3) The same happens if I first destroy the version sets again after import and then run version detection again.
Yes, all other placehoders are set to No matching!
Ok, your bug report seems pretty clear now and I have nothing further to comment on it. Until the flawed behavior is confirmed and eventually fixed by Hert, I would focus my efforts on practical workarounds.

Here's an idea of what I would perhaps try next:
(1) Import images without version detection
(2) Specify only the Original placeholder with the *.nef mask (that is, temorarily delete the Digital Negative placeholder).
(3) Run version detection. Does it correctly detect and assign the NEF files (and only those) to Original?
(4) If yes, then add the Digital Negative placeholder with the *.dng mask.
(5) Run version detection again. Does it correctly detect and assign the DNG files to Digital Negative? If yes, then make sure the NEF files are still assigned to Original.

(Again, I'm not entirely sure if step (5) is supposed to work - but it's worth trying to specify and detect versions incrementally, just in case it achieves what you want. If it does not, then I'm going to suggest another slight variation of the above scenario.)

fbungarz
Posts: 3305
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: Custom XMP in Info vs Details Panel?

Post by fbungarz » 01 Jun 15 22:11

I tried what you suggested:
(1) Done. 129 Images imported. 43 JPG, 43 DNG, 43 NEF
(2) Done.
(3) PSU detects all NEF files and all DNGs as subversion of Main version JPGs. BUT: the placeholder "Digital Negative" is assigned only to 31 out of 43 NEF files.
...

fbungarz
Posts: 3305
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: Custom XMP in Info vs Details Panel?

Post by fbungarz » 01 Jun 15 22:23

OK, I tried it again:
(3) It works, but ONLY if remove the file mask from the placeholder Digital Negative AND disable the placeholder by removing the checkmark in front of the placeholder.
(4) Done.
(5) After adding the mask and enabling the DNG placeholder again, and running version detection:
Of 43 NEFs 10 have their placeholders removed, that leaves 33 NEFs that still have the correct placeholder still assigned. Of 43 DNGs only 13 have the placeholder "Digital Negative" assigned. The others don't.

vlad
Posts: 969
Joined: 01 Sep 08 15:20

Re: Custom XMP in Info vs Details Panel?

Post by vlad » 01 Jun 15 22:28

Frank, did you turn ICS reading off, just to prevent any interference from the (incomplete) ICS versioning info?

vlad
Posts: 969
Joined: 01 Sep 08 15:20

Re: Custom XMP in Info vs Details Panel?

Post by vlad » 01 Jun 15 22:43

fbungarz wrote: (3) It works, but ONLY if remove the file mask from the placeholder Digital Negative AND disable the placeholder by removing the checkmark in front of the placeholder.
Ok, that's good progress.
(4) Done.
(5) After adding the mask and enabling the DNG placeholder again, and running version detection:
Of 43 NEFs 10 have their placeholders removed, that leaves 33 NEFs that still have the correct placeholder still assigned. Of 43 DNGs only 13 have the placeholder "Digital Negative" assigned. The others don't.
Ok, here's one more tweaking suggestion: in step (4), try also unchecking the Original placeholder for NEF files. Then run again step (5).

If this still doesn't work, I have yet one more idea for step (4): try also to restrict the Version Set detection scope to %FileName*.dng instead of %FileName*.*. I am thinking that might prevent PSU to revisit altogether the existing NEF versions, which were successfully assigned in step (3).

fbungarz
Posts: 3305
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: Custom XMP in Info vs Details Panel?

Post by fbungarz » 02 Jun 15 6:16

If this still doesn't work, I have yet one more idea for step (4): try also to restrict the Version Set detection scope to %FileName*.dng instead of %FileName*.*. I am thinking that might prevent PSU to revisit altogether the existing NEF versions, which were successfully assigned in step (3).
No idea what you are taling about. How do I do that?
Last edited by fbungarz on 02 Jun 15 7:21, edited 2 times in total.

vlad
Posts: 969
Joined: 01 Sep 08 15:20

Re: Custom XMP in Info vs Details Panel?

Post by vlad » 02 Jun 15 7:13

fbungarz wrote:
If this still doesn't work, I have yet one more idea for step (4): try also to restrict the Version Set detection scope to %FileName*.dng instead of %FileName*.*. I am thinking that might prevent PSU to revisit altogether the existing NEF versions, which were successfully assigned in step (3).
No idea what you are taling about. How do I do that?
Tools->Preferences->Catalog: Version Set detection scope is the 2nd setting above Color label names. (Minor interface remark: imho, It would be better to group all version options together - that is, move Version place holders right above Color label names; will submit the suggestion to Mantis.)

Do I understand correctly that the previous suggestion (disabling the NEF placeholder before step 5) did not help? Had you started with fresh images, imported with ICS reading disabled?

fbungarz
Posts: 3305
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: Custom XMP in Info vs Details Panel?

Post by fbungarz » 02 Jun 15 7:21

ICS is OFF.
I tried, what you suggested on a new folder.
(1) Images are imported non-versioned.
(2) DNG placeholder and file name mask disabled, running version detection for the NEFs: all NEFs get assigned as subversion, but one NEF does NOT get assigned the placeholder original. I assign the placeholder manually.
(3) NEF placeholder and file name mask disabled, DNG placeholder and file name mask re-enabled, running version detection for the DNGs results in the program freezing up and giving me the message "Access Violation (0)"

I think we probably best stop troubleshooting now. This no longer seems productive. Clearly Versioning in PSU is defective. It essentially does not work consistenly! At this point I do not think either you or I can resolve this issue. It is a bug and only Hert can fix it.
I do much appreciate all your patience trying to help. Thank you!

fbungarz
Posts: 3305
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: Custom XMP in Info vs Details Panel?

Post by fbungarz » 02 Jun 15 7:29

BTW - trying it the other way around (running detection on the DNGs first, then the NEFs) the same happens: the program hangs with "Access Violation (0)" when running version detection the second time.

vlad
Posts: 969
Joined: 01 Sep 08 15:20

Re: Custom XMP in Info vs Details Panel?

Post by vlad » 02 Jun 15 7:35

I think we probably best stop troubleshooting now. This no longer seems productive.

I agree.
Clearly Versioning in PSU is defective. It essentially does not work consistenly!

Refinement: version detection involving custom filemasks is inconsistent, therefore defective.
At this point I do not think either you or I can resolve this issue. It is a bug and only Hert can fix it.
Agreed. What I could probably try is write a custom script which automatically assigns the NEF and DNG version placeholders, as desired. Unfortunately, I'm quite busy with my own work (despite the time I've recently spent on this forum) - plus, the real solution is for the custom filemasks to be properly supported.
I do much appreciate all your patience trying to help. Thank you!
Frank, you are welcome.

fbungarz
Posts: 3305
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: Custom XMP in Info vs Details Panel?

Post by fbungarz » 02 Jun 15 7:48

What I could probably try is write a custom script which automatically assigns the NEF and DNG version placeholders, as desired.
Well, scripting is beyond me. So I would appreciate such a script, but I understand you are busy. Therefore: thanks again...
Unfortunately this means I still have to put moving from IDI5 to PSU3 on hold. I have now spend nearly two weeks trying to do this, also trying to investigate which raw converter I want to use. Using a raw converter from within IDI is a bit of a pain, and PSU runs so much more smoothly. So I had really hoped I could move forward, start using PSU. For now the whole conversion process has been nothing but a headache. I do believe PSU is the better, more mature program and I would love to be able to use it. Unfortunately I need to migrate what I have in IDI into PSU and until that works smoothly I am stuck. Converting the whole database would have been the most straightforward solution. That did not work, so I was happy to try using Hert's script to move the catalog and then run the import on one folder at a time - even that with a catalog of more than 80,000 images will take me days to complete. It is good to know that the labels and custom XMP makes it across without problem, but versioning is broke and it is simply too time consuming having to do this manually. I would still have to do it on hundreds of folders - even if I do it using Mike Buckley's suggestion to select several thumbs of the same file types and then assign the placeholders in a batch.
:(

vlad
Posts: 969
Joined: 01 Sep 08 15:20

Re: Custom XMP in Info vs Details Panel?

Post by vlad » 02 Jun 15 7:56

FYI, a script to swap version placeholders already exists and works well, as far as I remember. Unfortunately, I don't think it's going to help you at this point, because I understand you have not managed to get any placeholder consistently assigned.

Post Reply