How to purge labels from database?

[tstoddard]

tstoddard wrote:
Then I changed my preferences to NOT write hierarchical keywords but still to read hierarchical keywords.


Why did you change your preferences to NOT write hierarchical keywords? Did that make any difference? (Anyway, I somehow doubt that Stephen turned off the writing of hierarchical keywords.)

Interestingly, I had to do that to get this to work. My goal was to create a situation where I had an orphaned hierarchical keyword in my file. If I left that setting on, to write hierarchical keywords, when I deleted the flat keyword, the hierarchical keyword got deleted also. If I changed that setting, I could delete the flat keyword without deleting the hierarchical one.

I doubt that Stephen made the same changes I did, also, but he has admitted to experimenting quite a bit so it is possible that he had changed settings after some of these keywords and labels were created.

How did you configure Keyword processing in the global preferences for writing? My guess is that you specified: "Merge Catalog labels with Keywords". If you had "Replace keywords with Catalog labels" (my preference), I expect that the keyword would have been removed. (Furthermore: why would there be any difference between manual and automatic saving of metadata, regardless of the write preference?)

You're right about the Keyword processing, it was set to Merge Catalog labels with Keywords. I did change that but wasn't sure it made a difference. With my workflow, I have never noticed a difference in any behavior when changing that setting but for this example I didn't want PSU to mess with the keywords so I changed it.

I had to switch to manual saving of metadata because if I didn't, when I changed the metadata setting in the label to not write a keyword, the keyword was removed immediately. What I did was to change that setting and then delete the label before saving metadata to the file. That way, the label was unassigned from the file but the keywords remained.

Sure, but the labels created from hierarchical info should not be created under Miscellaneous. Was your label created under that category, Tom?

I created my test label under Miscellaneous to begin with so the hierarchical keyword was "Miscellaneous|TestLabel". Since Stephen's issue indicated his label was in that category, I figured it was an appropriate place to put it. I assume that his labels are in there because he either imported images with flat keywords in them and PSU created labels in Miscellaneous automatically or he entered the keywords using the details panel and when he did he was only entering them in the dc:subject field so PSU would have created labels in the Miscellaneous category to correspond with those keywords.

Like I said in my post, "it wasn't easy". I had to do some nonsensical things in order to make this happen so I doubt that Stephen did exactly what I did but it did prove to me that it is possible to have an orphaned hierarchical keyword written to a file that will result in a label and keyword being created or recreated automatically when metadata is written to a file.

[vlad]

You're right about the Keyword processing, it was set to Merge Catalog labels with Keywords. I did change that but wasn't sure it made a difference. With my workflow, I have never noticed a difference in any behavior when changing that setting but for this example I didn't want PSU to mess with the keywords so I changed it.

I had to switch to manual saving of metadata because if I didn't, when I changed the metadata setting in the label to not write a keyword, the keyword was removed immediately.

Tom, did that happen with the Merge labels/keywords setting? What's happening if you disable metadata auto-saving, then change the label details to not create a keyword and sometime later you manually save the metadata? Isn't the keyword removed in that case too?

What I did was to change that setting and then delete the label before saving metadata to the file. That way, the label was unassigned from the file but the keywords remained.

I understand. Indeed, it's a clever way to "break" things.

Like I said in my post, "it wasn't easy". I had to do some nonsensical things in order to make this happen

That's one thing about software engineering which is sometimes hard to grasp from the outside and could lead to frustration on both sides: in some not so uncommon cases, a software developer or support engineer could have a genuinely hard time trying to reproduce or debug some reported issue, while the customer experiences that frustrating issue (which may or may not be a bug) on a regular basis. In such cases, the key to alleviating mutual frustration and improving the customer experience, as well as the product quality, is communication.

Btw, that's why a system like Mantis is almost a necessity for a software product: it allows highly technical reports and targeted exchanges of data and info between customers and developers (as well as among different developers, if needed); such exchanges are not necessarily suitable, productive or easy to track in the product discussion forum, although that fact may not be intuitive to customers. (Perhaps it helps if we think for a moment to this very thread, where we all try to help Stephen based on incomplete info, but at the same time we naturally explore and discuss adjacent product functions and aspects too. In contrast, a tracking system like Mantis is designed to operate with higher ratios between hard data and noise.)

...Anyway, I'll stop here, lest I turn into a (much less talented) Marcel Proust myself...

[Stephen]

Many thanks Tom for doing the test and others for their ideas. PSu is frustratingly unforgiving for user errors, even if they are noticed immediately.

PreviewScreenSnapz037.jpg (245.97 KiB) Viewed 5056 times

I have sometimes (even recently) seen recommendations about settings, some of which I have followed immediately, only to discover a few days later that there were differing opinions and thus no consensus about the best practice. At that stage I usually left the setting until a rainy day. So it doesn't mean that I have not been listening, but that it is very time consuming to follow and act on discussions which are often not conclusive.

@Vlad
In this case my re-emerging misspelt labels are indeed location fields. However, as everything is synchronized (manually) why should that make a difference if I have changed them from writing to not writing keywords (or vice versa)?

[Mike Buckley]

Stephen,

This is OFF-TOPIC but very important.

There are two reasons you should disable reading of the ICS scheme. The first reason applies to everyone in all situations: reading the scheme slows down performance and, thus, should only be used to reconstruct a catalog. The second reason applies to you in your situation: considering that you have disabled writing the ICS scheme, it makes no sense to enable reading it because there is no information to read.

I feel very strongly that writing to the ICS scheme should be enabled. That's because if everything that could go wrong does go wrong with your catalog, you can always reconstruct it by temporarily enabling the reading of your ICS scheme. A few people over the years have reported in the forum that doing so was a life-saver for them, so it makes absolutely no sense to me to disable that setting. It never made any sense to me when Hert began shipping Supreme with that setting disabled by default. Soon after I wrote my belief about that in this forum, he changed the default setting back to the enabled setting, which had been the default setting at least as far back as IDimager version 4. Perhaps that timing was a coincidence, perhaps not.

So, I strongly advise you to switch the settings of the two parameters pertaining to the ICS scheme.

[tstoddard]

Stephen,

I agree with Mike that you should reverse your ICS settings so that you are writing to ICS but not reading it. That may also be resulting in your situation with re-emerging keywords. If, at the time your "bad" miscellaneous labels were created, you were writing to ICS, and then later you changed that setting, you could now have "orphaned" ICS labels. In other words, your label was written to ICS orginally but when you later removed the label, that change wasn't written to ICS so the label is still listed in your file's ICS section. Since you are now reading ICS, when you apply changes using the details panel, PSU may be reading the ICS section and recreating those labels. When PSU creates or recreates a label, it configures the label to write a keyword by default, and, voila!, you have your unwanted keywords back again. I haven't tested this but it stands to reason that it would happen.

[Mike Buckley]

Tom's reasoning about the ICS causing Stephen's issue makes sense to me at least on a first reading, though I also have not tested it.

His thinking makes me realize that if I ever changed my ICS parameter to the enabled setting (it shipped enabled on my system and has never been disabled on my system in the 8 years I've been using IDimager cataloging products), I would temporarily disable the setting that would then force all images, not just the out-of-sync images, to be updated. I would then select all images in my catalog (probably one section at a time) and I would then save metadata to the file. For me, that would be the safe way of ensuring that all ICS data is stored in all files.

So, I would recommend, Stephen, that you do the same. Otherwise, I can imagine problems with your metadata surfacing from time to time. As an example, I've never experienced your issue in the entire time I've been using my catalog. One reason is that I've never used the Places catalog category but I think consistent use of writing to the ICS scheme and never changing that setting might be another reason.

EDIT: See Tom's and my clarification of this post in the next post written by him and the immediate follow-up post written by me.

[tstoddard]

His thinking makes me realize that if I ever changed my ICS parameter to the enabled setting (it shipped enabled on my system and has never been disabled on my system in the 8 years I've been using IDimager cataloging products), I would temporarily disable the setting that would then force all images, not just the out-of-sync images, to be updated.

Mike,

I find this statement confusing. First of all, you need to say which setting you are referring to. I am assuming that you mean the setting to write catalog data to ICS. If so, then why would disabling it and updating all images have any effect at all? If anything, it might cause all data to be removed from the ICS section, which doesn't seem to be what you are suggesting.

It seems to me that what you meant to say is:

If you had ever disabled writing to ICS and now you want to make sure that ICS data is stored in all of your files you would enable writing to ICS and then force all images to be updated.

If, on the other hand, you are talking about changing the setting to read ICS then I don't follow that logic either. If you had it disabled for some period of time and then enabled it again, I don't see how that would accomplish anything when writing metadata to files.

Have I misunderstood you in some way?

[Mike Buckley]

It seems to me that what you meant to say is:

If you had ever disabled writing to ICS and now you want to make sure that ICS data is stored in all of your files you would enable writing to ICS and then force all images to be updated.

Correct. Apologies for the confusion!

[Mike Buckley]

Now that I have had a chance to digest rather than just review the Preferences settings, I think there are two types of settings at work that make Stephen's issues happen. They are the two ICS settings and all of the ramifications explained by Tom and the Read setting that merges the keywords with the catalog labels mentioned by Vlad. There is nothing wrong with enabling the setting that merges keywords with catalog labels when reading the files, but that setting combined with changes over time with the ICS settings could very well cause these problems.

So, once Stephen changes both ICS settings and saves the catalog data to all image/sidecar files including those that are indicated to be in synch, my hunch is that his problem discussed in this thread will go away.

[Mike Buckley]

@Vlad
In this case my re-emerging misspelt labels are indeed location fields. However, as everything is synchronized (manually) why should that make a difference if I have changed them from writing to not writing keywords (or vice versa)?

I'm not Vlad, but I believe the answer to that question is provided by Tom in his explanation of the ramifications of having changed ICS settings in the past.

[snowman1]

Additional to the excellent explanations from Mike and Tom, I am also wondering if the merge of labels with keywords (in both directons but especially the write direction) may be adding to the problem, as the "bad keywords" would not be removed by the synch as they would be with the replace option, but rather the new ones would just be added and the "bad" ones remain?

[vlad]

Great advices from Mike, Tom and Snowman. I concur that ICS reading could be the cause (or a factor) for Stephen's re-emerging labels.

Just a couple of side remarks:

If you had ever disabled writing to ICS and now you want to make sure that ICS data is stored in all of your files you would enable writing to ICS and then force all images to be updated.

Excellent point. An additional situation for re-writing the metadata (including ICS) to all image files would be after updating to a PSU release which has fixed one or more bugs related to ICS writing. IIRC, a bug with the ICS portfolio data was reported and fixed several months ago. So, if someone has images in portfolios and their metadata was last saved before the fix, then the ICS fields may not hold the right metadata for correctly recreating portfolio collections. (There was another bug with private labels leaking to the ICS metadata; also, Frank reported an issue with ICS and versioning, although there has been no follow-up on that, afaik.)

People who want to always keep their metadata up-to-date could uncheck the dedicated setting: "Only update out-of-sync images". I am not sure how practical it is to uncheck it on a permanent basis, unless someone has a small catalog or a very fast system. In addition, there is also the risk that any new bug in metadata writing will affect all image files.

Stephen:

Since we are discussing both ICS and the creation of keywords, I would also like to make you aware that there is also a difference between private labels vs. "do not create keyword" labels. The former are not stored anywhere in the metadata, while the later are stored only in the ICS fields. If you assign a "ugly woman" label to images with your client's wife, you may want to configure that label (or top level category) as private, lest the vigilant client discovers the label inside the ICS metadata.

Regarding keyword processing: if you decide to use only PSU to tag your images, then I would recommend using "Replace" for writing. You could still leave "Merge" for reading, just in case you ever import images which were previously tagged (outside PSU).

Please let us know if our suggestions have helped you on the issue with re-emerging labels.

[vlad]

P.S. To be honest, I don't recall whether the ICS portfolio bug was actually related to metadata storage or only to ICS reading. If the later, then there is no need (related to portfolio collections) to force the updating of metadata for all images.

[Mike Buckley]

IIRC, a bug with the ICS portfolio data was reported and fixed several months ago.

Good point. I don't remember if I force saved my Portfolio images, so I will attend to that this weekend. Thanks for the reminder!

"Only update out-of-sync images". I am not sure how practical it is to uncheck it on a permanent basis, unless someone has a small catalog or a very fast system.

I would be especially concerned for those like me who enable the auto synch.

if you decide to use only PSU to tag your images, then I would recommend using "Replace" for writing. You could still leave "Merge" for reading, just in case you ever import images which were previously tagged (outside PSU).

I concur on both points. On the second point, using "merge" or "replace" has the same effect if the images are never tagged outside Supreme, which is my case. I configure "merge" for reading just in case something unimaginable goes wrong resulting in some of the embedded keywords to be missing or corrupt. If I ever have a need to read the information from the file to the catalog, "merge" is actually safer than "replace" if I want to ensure that all of my catalog labels remain assigned to the image.

[Stephen]

Many thanks. Is this the consensus of the debate then?

Identical screen shot as before added again for clarity:

PreviewScreenSnapz037.jpg (245.97 KiB) Viewed 4884 times

Should now be changed to:

* Do NOT read hierarchical keywords

* Set "Replace keywords with Catalog labels"

* Disable reading but enable writing to the ICS scheme

* Uncheck the dedicated setting: "Only update out-of-sync images"

* "Replace catalog label with existing keywords"
You could still leave "Merge" for reading???
(BTW, PSu's pop-up help tips are identical in every case!)

This would help anybody with Attention Deficit Disorder.