Script-based filters for images with orphan keywords

[vlad]

I had some free time and used it to write two script-based filters for identifying (filtering in) images with orphan keywords in their metadata (xmp:dc:subject). By orphan keywords I mean here unknown or unassigned keywords; more precisely:

Unknown keyword = a keyword without any corresponding label in the catalog structure
Unassigned keyword = a keyword without any corresponding label assigned to the image

The "Filter With Unknown Keywords" displays only images with one or more unknown keywords in their metadata. The "Filter With Unassigned Keywords" is more inclusive: it filters in the images with one or more orphan keywords. (Such a keyword either doesn't have a corresponding label defined in the catalog, or that label has not yet been assigned to the image).

Just for the record, I wrote these two scripts mostly thinking they might be useful to Harald in his effort to clear up unwanted keywords - but thought they might be useful to others (including myself) too. Feel free to provide any feedback, including encountered problems or oddities. (My own testing of the scripts has been fairly light, especially as I couldn't easily create orphan keywords in PSU.)

Hope that helps,
Vlad

P.S.1) Instructions for script-based filters:

To make PSU recognize the filter scripts, first save them in the "Scripts\FilterScripts" sub folder of the application data folder (e.g. C:\Users\<yourname>\AppData\Local\IDimager Systems, Inc\Scripts\FilterScripts). If that folder doesn't exist yet then create it yourself. The filters will then appear in the FILTER dropdown within the filter bar.

P.S. 2) Please note a minor (and subtle) technical point about "Filter With Unassigned Keywords":

If there is an assigned private label that matches an existing keyword (while there is no non-private label that matches it), I still treat the keyword as a non-assigned keyword. (I should actually test for labels configured to not create keywords - whether private or not - but I haven't found in the API a way to do it; I'm waiting for a hint from Hert.) If, for some reason, that's not what you want, you could simply remove the following line:

Code: Select all

if not Catalog.IsPrivateProp(AProps.Items[j]) then

[HaraldE]

Thanks a lot Vlad for your help, here are my first comments

For testing I used a date range of 1990 - 1999 and in there I have 2118 images

Using the "unassigned" filter I get 17 selected
Using the "unknown" filter I get 16 selectde.

Of these two sets the 16 are same. The extra one in Unknown filter (its name is 313) has one keyword and no label. However among the other 16 there are a few that also have one keyword and no label. So I can not see what is special about image 313

Also with my limited knowledge of Keywords and Lables it is not clear to me the difference betwen your two scripts, as explained above. Could you describe the difference again , maybe using an example?

Also did I use the originally suggested filter criteria ( Filter By Macro Command and specify: %xmp:dc:subject + Doesn't Equal + leave the value empty) on these 2118 images and that gave me 105 selected ones. And filtering with at least one label gave me 88 selected

Regards, Harald

[Hert]

Vlad, nice scripts.
In the iteration loop of the "Unassigned" script, I think you should clean the props first. AProps.Clear

[vlad]

Hi Harald,

Thanks a lot Vlad for your help, here are my first comments

For testing I used a date range of 1990 - 1999 and in there I have 2118 images

Using the "unassigned" filter I get 17 selected
Using the "unknown" filter I get 16 selectde.

Of these two sets the 16 are same. The extra one in Unknown filter (its name is 313) has one keyword and no label. However among the other 16 there are a few that also have one keyword and no label. So I can not see what is special about image 313.

I think you meant to say that 313 is filtered in by "unassigned" but not by "unknown". Now, look at the keyword in 313. My guess is that the catalog contains a label with the same name as the keyword, but that label is not currently assigned to 313. Could you confirm that?

Also with my limited knowledge of Keywords and Lables it is not clear to me the difference betwen your two scripts, as explained above. Could you describe the difference again , maybe using an example?

Labels defined the catalog: Apple, Lemon

Image_1 has only label Apple assigned and its metadata has one keyword: Apple.
Image_2 has only label Apple assigned and its metadata has two keywords: Apple, Orange.
Image_3 has only label Apple assigned and its metadata has two keywords: Apple, Lemon.

The "unknown keywords" script will display only Image_2, because it has the "unknown" keyword Orange. (No Orange label exists in the catalog.) The "unassigned keywords " script will display both Image_2 (because the Orange label doesn't exist and is not assigned) and Image_3 (because label Lemon is not assigned, although it exists in the catalog). Does that clear up the difference?

Also did I use the originally suggested filter criteria ( Filter By Macro Command and specify: %xmp:dc:subject + Doesn't Equal + leave the value empty) on these 2118 images and that gave me 105 selected ones. And filtering with at least one label gave me 88 selected

That means you have at least 17 images with keywords but no labels assigned to them, which precisely matches your reported result of the "unassigned" script. That all makes sense to me - you just have to carefully look over those 17 images and clear up their keywords if you don't care abot them. (Pay special attention to image 313, because you should have a label defined but not assigned for one of the keywords - you may want to either assign that label to 313 or perhaps completely remove the label, if it is obsolete.)

Regards,
Vlad

P.S. Hert, thanks for the tip - I'll take a look.

[Hert]

Btw, you should also remove the line AParam.Free next the bottom of the unassigned script.

[HaraldE]

Vlad, I now understand your scrips, you explanation was brilliant

I think you meant to say that 313 is filtered in by "unassigned" but not by "unknown". Now, look at the keyword in 313. My guess is that the catalog contains a label with the same name as the keyword, but that label is not currently assigned to 313. Could you confirm that?

Correct and Confirmed

I know have the knowledge (feels good) and the tools (just splendid) to sit down and judge the private mess I have created.

As of today I have 80+ k images in the catalog. Of these 1155 have label(s). And examples of my mess are as follows:
> Some labels are not used at all and most have 1 or 2 or 3 images assigned
> There are labels which are from a folder path like 1991 and 1992 and 1993 and ...
> There are spelling errors, for example a label called WALK and another called WALKS
> Stupid labels like 287-326xxx and 327-380xxx
> ... and so on

What I now will do is a carefull study of the current labels and find out which should have a future life. I have realised that there actually are some valid labels in there.

Vlad, I assume you will tell us if/when you update your 2 scripts.

Regards, Harald

[HaraldE]

Vlad or Hert or anyone, just realised I had a further question.

Assume I have used the word "ARCHIVE" in a mixed way. For some images it is a simple keyword and for others it is a label.
Now if I enter "ARCHIVE" in the searh area (top right), will PSU now find both the keyword and the label photos?

Regards, Harald

[vlad]

Now if I enter "ARCHIVE" in the searh area (top right), will PSU now find both the keyword and the label photos?

Yes: it will find photos with only the label + photos with only the keyword + photos with both. However, you could restrict a search to specific domains, for example (off the top of my head):
ARCHIVE@PROP
ARCHIVE@XMP

The first query will search for 'ARCHIVE' labels, the second will search for 'ARCHIVE' anywhere in the metadata.

[HaraldE]

Thanks Vlad,

This will be an eficient way to find out how I have (miss-) used some of these words
I am aware that some further photos will also be found, like when ARCHIVE is used in the DESCRIPTION, but that is a smaller problem.

Regards, Harald

[vlad]

Hi Hert,

Vlad, nice scripts.
In the iteration loop of the "Unassigned" script, I think you should clean the props first. AProps.Clear

The iteration loop uses AProps, so that can't be cleared inside the loop. Am I wrong or have I misunderstood your suggestion?

Btw, you should also remove the line AParam.Free next the bottom of the unassigned script.

Removing that line doesn't seem to do any harm, but why is it necessary?

One more question, please: is there any way to implement a progress bar while filtering a collection? (I'm asking because filtering by macro commands or by scripts is rather slow on my old laptop.)

[Hert]

The iteration loop uses AProps, so that can't be cleared inside the loop. Am I wrong or have I misunderstood your suggestion?

Why wouldn't it be able to clear the props inside the loop?
If you're uncomfortable with that (and better in general) then create AProps in the loop and free it in the loop.

Removing that line doesn't seem to do any harm, but why is it necessary?

You're freeing something you didn't create

One more question, please: is there any way to implement a progress bar while filtering a collection? (I'm asking because filtering by macro commands or by scripts is rather slow on my old laptop.)

Filtering doesn't give a progress. That would be something that the application should add. You could make a feature request for that

[vlad]

Just to let you know: I have updated the filter for unassigned keywords to version 1.1 - you could find the updated version attached to the original post (instead of the old version). There are two (minor) changes:
- the memory freeing policy is now implemented as per Hert's advices
- the script now correctly identifies as unassigned/unknown a keyword that has a corresponding label defined and assigned, but that label had been configured by the user to not create a keyword. (Previously, only private labels were included in that special check.)

[LucLL]

Hi Vlad,
Thanks for the scripts. Just been playing around with them and I actually have a question.
I'm not sure if these scripts would also work for this or if there would be a way to make them to do this
I'm new to PS and have just imported over 100k images. I have found that images which had old people tags from MS WPG were not actually loaded as labels/tags/keywords although they are still shown as areas on the pictures with the name.
Would it be possible to identify those images? So iow would it be possible to filter the tags/areas which do not exist as labels?
Just asking, as I could not find another way to filter these at the moment and it could safe me a lot of time to re-assign those names. It does not have to be as a area. If they can just be linked to a tag that's sufficient.
Suggestions welcome, thanks in advance.

Fyi: I'm since years on a Mac, would never have thought to still have those areas around but they could safe time maybe.

Luc

[vlad]

Hi Luc,
Do I understand correctly that you would need a script (or method) for filtering images with orphan area tags, which do not have corresponding catalog labels (or metadata keywords) assigned to the image? Unfortunately, I haven't used and I'm not familiar with area tags - I don't even know if there's any scripting support for them. If Hert or someone else could point me to relevant info in the API, then I might be able to take a look (time permitting).

I could not find another way to filter these at the moment and it could safe me a lot of time to re-assign those names. It does not have to be as a area. If they can just be linked to a tag that's sufficient.

What do you mean by that?

Suggestions welcome, thanks in advance.

Just a shot in the dark: have you tried to collect all images with areas (see the corresponding item in Catalog->By Catalog State) and then examine the Label dropdown content in the Filter Bar? Perhaps some macro command filter could help too?

[LucLL]

Hi Vlád,

Do I understand correctly that you would need a script (or method) for filtering images with orphan area tags, which do not have corresponding catalog labels

Yes.

I could not find another way to filter these at the moment and it could safe me a lot of time to re-assign those names. It does not have to be as a area. If they can just be linked to a tag that's sufficient.

What do you mean by that?

I currently can't find a easy way to assign a catalog label to these

Just a shot in the dark: have you tried to collect all images with areas (see the corresponding item in Catalog->By Catalog State) and then examine the Label dropdown content in the Filter Bar? Perhaps some macro command filter could help too?

Just trying...... I didn't notice there was a state for areas. I see all of them in there but I can't seem to filter them. When I do a text search in that selection that works, so that might be a way to do it. Subset to a name, assign the label and then subset to someone else I can see in the range and assign that label. Each time starting from the full area catalog state.... And repeat that till li guess I had everyone on the list..... Not really automatic but at least it's a way out
And it's only about 1000 pictures or so so that's ok.

Thanks for the great suggestion and tip....buy you a beer one day

Cheers!