Keywords - Delimited vs Simple

tstoddard
Posts: 584
Joined: 07 Sep 12 12:51

Keywords - Delimited vs Simple

Post by tstoddard » 20 Sep 12 19:38

I am new to cataloging but have spent a lot of time getting up to speed over the last two weeks. One thing that I'm vacillating the most over is what set of options to select in the synchronization options. Here are some of my questions:

1. What do I gain by writing delimited keywords? I can't seem to find any functionality that will change in PS when I write delimited keyword versus simple keywords. If I click on a parent label in Catalog View, I see all photos that have child labels assigned regardless of whether or not they are stored as delimited keywords. So what's the difference?

2. What is the IDimager ICS Shema and what is the result of unchecking the option of writing the catalog label structure to it?

3. If I have delimited keywords stored already and I change my mind and want to switch back to simple keywords, is the way to do that other than to unassign all keywords and start all over? Can I just change the setting and then resync all of my files somehow? Can I access the data directly and modify it using SQL or regular expressions?

Any explanation or examples would be appreciated.
Tom Stoddard

gcoupe
Posts: 461
Joined: 16 Mar 05 19:29
Location: Heelweg, The Netherlands

Re: Keywords - Delimited vs Simple

Post by gcoupe » 20 Sep 12 20:41

My answer to question (1) is:

Using delimited keywords enables keyword hierarchies. Why are keyword hierarchies useful?

First, because they offer flexibility and grouping of similar items and concepts. For example, my keyword "cows" is actually part of a hierarchy that starts Nature/Animals/livestock/cattle/dairy cattle/cows. That way, when I search for photos (using either Windows Live Photo Gallery or IDimager) with the keyword cows, it will just show me those with cows in them. But if I search for photos with the keyword livestock, it will show me photos of cows, horses, pigs, sheep, and so on. When I use IDimager (or PS) to tag a image of a cow with the cows keyword, it will automatically add all the parent keywords in the structure: dairy cattle, cattle, livestock, Animals, and Nature.

Second, using a keyword hierarchy is more convenient for me for keyword navigation. Without a hierarchy, I'd have a keyword list that would be a flat list of thousands of entries. With the hierarchy, at the root level, I have just seven top level entries: Activites, Events, Nature, People, Places, Objects, Styles. Although the Catalog Labels of IDI and PS also help in navigation...

In answer to your question (2), the IDimager ICS Schema is a way of embedding the keyword hierarchy as XMP metadata into the image files directly. Then when the file is read by an application that understands this schema, the hierarchy can be reconstructed on the fly. Currently, the simplistic way of handling keyword hierarchies is to use a delimiter character such as . / \ or | in the keyword strings. This is not altogether satisfactory, and the IDimager ICS schema is a better way of handling it. The drawback is that it is a vendor-proprietary mechanism (albeit one that is open for others to implement read/write modules). There is an equivalent cross-industry standard for handling keyword hierarchies that has been proposed by the Metadata Working Group, but as far as I am aware, no application currently on the market has implemented it. See http://gcoupe.wordpress.com/2012/03/21/ ... -blessing/ for more details.
Geoff Coupe
--------------
Photo Supreme /Windows 10 Pro 64 bits + Windows Home Server 2011 = DAM

tstoddard
Posts: 584
Joined: 07 Sep 12 12:51

Re: Keywords - Delimited vs Simple

Post by tstoddard » 20 Sep 12 21:07

Geoff,

Thank you for that explanation. I completely understand the advantages of hierarchical labels but it appears that I can have that without writing delimited keywords out to the file's metadata. Am I mistaken? Photo Supreme allows me to create hierarchical category labels and assign them to photos without me selecting the option of writing the keywords out to the IPTC metadata field in a delimited format. My question is why do I need to write them in a delimited format if I can still have the functionality afforded by hierarchical category labels in Photo Supreme?

I guessed that the ICS schema was just a way to store the hierarchy's structure in an XML format but I'm still confused by what is meant in the preferences dialogue when it says, "Write the Catalog label structure to the IDimager ICS scheme". Does that mean that it will be written to the file? If so, why bother writing delimited keywords also? Will they only be used by programs that have the ability to interpret delimited keywords or does Photo Supreme use them somehow?
Tom Stoddard

Mike Buckley
Posts: 3965
Joined: 10 Jul 08 14:18

Re: Keywords - Delimited vs Simple

Post by Mike Buckley » 20 Sep 12 23:16

tstoddard wrote:I completely understand the advantages of hierarchical labels but it appears that I can have that without writing delimited keywords out to the file's metadata.
You are correct so long as you are using Supreme's database. If you use another program to search for keywords, that program will be searching the keywords embedded in the metadata fields, not Supreme's database. If you post your images on a website that allows users to search for keywords embedded in the metadata of your images, they won't have the advantage of the heirarchical keywords that Geoff explained if you don't embed delimited keywords in your metadata.

To take your logic one step farther, if you are always going to use Supreme to find your images and nobody is ever going to search the metadata fields of your image files, you might understandably think there is no reason to synch your image files with the database. However, that leads to my response to your next point.
when it says, "Write the Catalog label structure to the IDimager ICS scheme". Does that mean that it will be written to the file?
Yes. The reason to do that is purely a matter of safety. If everything that could possibly go wrong does go wrong including failure of your backups, you will always be able to recreate your database catalog by adding the information stored in the image files to your catalog database. That has happened to people who have reported here in the forum that the ability to do that has been their lifesaver, so to speak. If they hadn't written to the ICS scheme and if they hadn't synched their image files with the database, they wouldn't have been able to do that.
If I have delimited keywords stored already and I change my mind and want to switch back to simple keywords...Can I just change the setting and then resync all of my files somehow?
I think that's probably true. However, I have only glanced at Supreme (after being intimately familiar with IDImager V5). So, you ought to get confirmation from someone who knows it better than I do. I don't know if Supreme has the flexibility (and complexity) that makes it possible to do that and assuredly produce the results that you want.

george
Posts: 931
Joined: 24 Jun 07 15:57
Location: USA

Re: Keywords - Delimited vs Simple

Post by george » 21 Sep 12 0:34

Mike Buckley wrote:If everything that could possibly go wrong does go wrong
...such as your favorite digital asset management software being discontinued, you can migrate to another one as long as the new program can import the metadata into its own database.
George

tstoddard
Posts: 584
Joined: 07 Sep 12 12:51

Re: Keywords - Delimited vs Simple

Post by tstoddard » 21 Sep 12 1:58

Mike Buckley wrote:You are correct so long as you are using Supreme's database. If you use another program to search for keywords, that program will be searching the keywords embedded in the metadata fields, not Supreme's database. If you post your images on a website that allows users to search for keywords embedded in the metadata of your images, they won't have the advantage of the heirarchical keywords that Geoff explained if you don't embed delimited keywords in your metadata.
Thank you for confirming this for me. How likely is it that websites and other programs are going to understand what the delimited keywords mean? For example, if I use a period as my delimiter and I import my files into Windows Photo Gallery, they just treat each delimited set of keywords as a unique keyword. That result is no different than if I had written simple keywords to my metadata except it looks far more confusing with the delimited keywords. Now if I had used a forward slash as a delimiter then Windows Photo Gallery might be able to interpret that as a hierarchy (I haven't tested that). Since every program decides for itself what character to use for reading or writing delimited keywords, if they do it at all, then there is very unpredictable compatibility between programs.

What I'm getting at is that it seems, unless you have a specific target in mind, that using delimited keywords to maybe enable another program to recognize your label structure based on the delimited keywords alone would be a very haphazard way to do so. Maybe I'm wrong, though. I started out by stating that I'm new to this whole discipline of cataloging photos so I don't know what many other programs do or how common the ability to translate delimited keywords into a label structure is. All I do know is that when I look at my photos that I've written delimited keywords to in Windows Photo Gallery or Picasa, I end up looking at a long list of delimited keywords. Granted, when they are sorted alphabetically they do sort of represent the hierarchical structure, but, in Picasa I am unable to resize the tag panel to make it wide enough to display the entire delimited keyword, which makes them difficult to use. So in the end, my decision comes down to whether I am willing to sacrifice predictability for potential occasional advantage.
Tom Stoddard

Mike Buckley
Posts: 3965
Joined: 10 Jul 08 14:18

Re: Keywords - Delimited vs Simple

Post by Mike Buckley » 21 Sep 12 2:48

George,

IDimager Systems has always used an open architecture. If you or I want to migrate to another DAM product, we simply need to export our catalog database to a file and then import that file into the newly chosen software. This is one of the reasons I selected IDimager Version 4 (that's not a typo) as my first DAM tool. There are certain details of the catalog that the new software will not be able to import, but that's probably going to be because the new software doesn't have the capabilities of the IDimager Systems software, whether it's IDimager or Photo Supreme.

Mike Buckley
Posts: 3965
Joined: 10 Jul 08 14:18

Re: Keywords - Delimited vs Simple

Post by Mike Buckley » 21 Sep 12 3:05

tstoddard,

I don't know how most web sites handle metadata, so I also don't understand how they understand what keywords mean. I don't even know if most websites allow the user to search for a keyword as part of a larger keyword.

Considering your concern about this stuff, I recommend that you consider the following resources:

1) Peter Krogh's The DAM Book, which is in its second edition and is generally recognized throughout the industry as a thorough reference.
2) The American Society of Media Photographers publishes a website explaining best practices. It's at http://www.dpbestflow.org/
3) David Reicks, author of the Controlled Vocabulary Keyword Catalog, which I use and is available for use with Supreme, hosts a Yahoo! Group. Ask your questions there and you'll get very well informed answers. http://tech.groups.yahoo.com/group/cont ... ocabulary/

gcoupe
Posts: 461
Joined: 16 Mar 05 19:29
Location: Heelweg, The Netherlands

Re: Keywords - Delimited vs Simple

Post by gcoupe » 21 Sep 12 7:35

tstoddard wrote:
Mike Buckley wrote:You are correct so long as you are using Supreme's database. If you use another program to search for keywords, that program will be searching the keywords embedded in the metadata fields, not Supreme's database. If you post your images on a website that allows users to search for keywords embedded in the metadata of your images, they won't have the advantage of the heirarchical keywords that Geoff explained if you don't embed delimited keywords in your metadata.
Thank you for confirming this for me. How likely is it that websites and other programs are going to understand what the delimited keywords mean? For example, if I use a period as my delimiter and I import my files into Windows Photo Gallery, they just treat each delimited set of keywords as a unique keyword. That result is no different than if I had written simple keywords to my metadata except it looks far more confusing with the delimited keywords. Now if I had used a forward slash as a delimiter then Windows Photo Gallery might be able to interpret that as a hierarchy (I haven't tested that). Since every program decides for itself what character to use for reading or writing delimited keywords, if they do it at all, then there is very unpredictable compatibility between programs.

What I'm getting at is that it seems, unless you have a specific target in mind, that using delimited keywords to maybe enable another program to recognize your label structure based on the delimited keywords alone would be a very haphazard way to do so. Maybe I'm wrong, though. I started out by stating that I'm new to this whole discipline of cataloging photos so I don't know what many other programs do or how common the ability to translate delimited keywords into a label structure is. All I do know is that when I look at my photos that I've written delimited keywords to in Windows Photo Gallery or Picasa, I end up looking at a long list of delimited keywords. Granted, when they are sorted alphabetically they do sort of represent the hierarchical structure, but, in Picasa I am unable to resize the tag panel to make it wide enough to display the entire delimited keyword, which makes them difficult to use. So in the end, my decision comes down to whether I am willing to sacrifice predictability for potential occasional advantage.
For the specific case of Windows Photo Gallery, that software uses the "/" character as the delimiter, which is why I've selected the same character as the delimiter in IDimager/Photo Supreme. That means that:

1) I use IDimager as my primary DAM tool, assigning Catalog Labels to my images
2) IDimager writes out IPTC Core Keywords as hierarchical strings (e.g. Events/Seasons/Spring)
3) Windows Photo Gallery reads these strings and constructs the same Descriptive Tag hierarchy as IDimager has for Catalog Labels.
4) Windows Photo Gallery will show the Descriptive Tag on an image as the leaf node of the string. E.g. for an image with a keyword Events/Seasons/Spring, WPG will show the Descriptive Tag "Spring", but it also knows that there is a hierarchy, so searching for images with the term "Seasons", or looking for all images with the Tag "Seasons" will include that image in the results.

You've tried the delimiter character ".", and as you've found, neither WPG or Picasa treat that as a delimiter, so you just get a long string as an individual keyword.

WPG does handle hierarchies, but you have to use "/" as the delimiter. Picasa, so far, does not understand hierarchies at all, as far as I am aware, so I think you're out of luck there.

As has been said by others, using metadata in the image files themselves, rather than just in the application database is a form of future-proofing.

My groundrule:
I insist that any software used in the digital workflow (transfer from camera to computer, image selection, digital processing, cataloguing, publishing and asset management) will respect any EXIF, IPTC and XMP metadata that may be stored in the image file itself.
I am not interested in asset management software that stores image metadata away in a proprietary format in the software itself. That way lies painting oneself into a corner down the road… However, I will accept asset management software that copies metadata from image files into its own database for performance reasons, so long as the database and the image files metadata content are kept in sync transparently (i.e. it takes little or no effort on my part).
Geoff Coupe
--------------
Photo Supreme /Windows 10 Pro 64 bits + Windows Home Server 2011 = DAM

Mike Buckley
Posts: 3965
Joined: 10 Jul 08 14:18

Re: Keywords - Delimited vs Simple

Post by Mike Buckley » 21 Sep 12 12:01

Forward slash (/) as the delimiter: If you use that delimiter and the David Riecks Controlled Vocabulary Keyword Catalog, be sure to remove that character from the top-level categories. Otherwise, duplicate entries will be produced in your vocabulary.

tstoddard
Posts: 584
Joined: 07 Sep 12 12:51

Re: Keywords - Delimited vs Simple

Post by tstoddard » 21 Sep 12 15:17

gcoupe wrote:I am not interested in asset management software that stores image metadata away in a proprietary format in the software itself. That way lies painting oneself into a corner down the road… However, I will accept asset management software that copies metadata from image files into its own database for performance reasons, so long as the database and the image files metadata content are kept in sync transparently (i.e. it takes little or no effort on my part).
Geoff, I share your sentiment, which is why I'm asking these questions. However; just writing data to the files by itself does not necessarily prevent you from painting yourself into a corner. What you write to those files may have the same result. For instance, writing delimited keywords to the IPTC keyword field using a "/" as a delimiter does restrict you to some degree. It may enable compatibility with Window Photo Gallery but it could also hinder your ability to use some other programs like Picasa, for example. This is why I am struggling with deciding what conventions I want to use when writing my data out to my files.

This brings me back to my original question number 3. If I make a decision about delimited keywords and go ahead and catalog all of my photos and then decide to change my options after all of my keywords (whether delimited or not) are written out to the metadata embedded in my files, how difficult will it be to change those keywords. Will I be able to simply replace a "/" with a "." or "|" if I decide that I now want my keywords to be compatible with something other than Windows Photo Gallery? Can I flatten my delimited keywords or change flat keywords into delimited keywords as a batch operation? I think I will experiment a little more with this to see if I can figure it out on my own. I had hoped that someone here would have already come across this issue and could save me some time.
Tom Stoddard

gcoupe
Posts: 461
Joined: 16 Mar 05 19:29
Location: Heelweg, The Netherlands

Re: Keywords - Delimited vs Simple

Post by gcoupe » 21 Sep 12 16:01

tstoddard wrote: Geoff, I share your sentiment, which is why I'm asking these questions. However; just writing data to the files by itself does not necessarily prevent you from painting yourself into a corner. What you write to those files may have the same result. For instance, writing delimited keywords to the IPTC keyword field using a "/" as a delimiter does restrict you to some degree. It may enable compatibility with Window Photo Gallery but it could also hinder your ability to use some other programs like Picasa, for example. This is why I am struggling with deciding what conventions I want to use when writing my data out to my files.

This brings me back to my original question number 3. If I make a decision about delimited keywords and go ahead and catalog all of my photos and then decide to change my options after all of my keywords (whether delimited or not) are written out to the metadata embedded in my files, how difficult will it be to change those keywords. Will I be able to simply replace a "/" with a "." or "|" if I decide that I now want my keywords to be compatible with something other than Windows Photo Gallery? Can I flatten my delimited keywords or change flat keywords into delimited keywords as a batch operation? I think I will experiment a little more with this to see if I can figure it out on my own. I had hoped that someone here would have already come across this issue and could save me some time.
At the moment, there is no de facto standard for handling keyword hierarchies. WLPG uses "/", Lightroom uses "|", and Picasa hasn't got a clue. Other applications use "."

Some applications, like IDI and Photo Supreme, offer flexibility in enabling you to choose the delimiter character for both reading and writing keyword hierarchies stored in IPTC Core Keywords. Some applications, like Lightroom, offer a one-way trip - Lightroom will read keyword hierarchies stored in IPTC Core Keywords using your choice of delimiter, but won't give you a choice when writing them out (it's the "|", or nothing).

Some applications offer a vendor-proprietary mechanism for handling keyword hierarchies. If this uses an XMP schema, then other vendors can interface with it. An example is Lightroom - as well as offering the simple "|" delimiter in the IPTC Core Keywords to denote a hierarchy, it also uses XMP metadata to define the structure of your keyword hierarchy. IDimager and Photo Supreme can both use this Lightroom XMP schema to handle your keyword hierarchy.

IDimager has its own XMP schema, which is used within IDimager products.

Finally, there has been an attempt (in November 2010), by the Metadata Working Group, to set a cross-industry standard for handling keyword hierarchies using an XMP schema. Unfortunately, as far as I know, no applications currently implement this proposed standard. Personally I find this particularly galling since Microsoft and Adobe are both founding members of the Metadata Working Group...

OK, now let's turn to your question 3...

I suspect your starting point will be the tools you select for your digital workflow. They will define your choice of whether you go for keyword hierarchies or simple keywords, and if you go for keyword hierarchies, what the hierarchy mechanism will be.

Since IDI is my main tool, and other family members use WLPG for browsing the collection, then using "/" as the delimiter in keyword strings stored in the IPTC Core Keyword metadata was the logical choice.

Since I have IDI, I can use that to make changes down the line if I need to.

For example, let us say, for the sake of argument, that Picasa will support keyword hierarchies in the next version, but Picasa uses the "." as the delimiter in the IPTC Core Keywords. And I decide to drop WLPG, and use Picasa instead.

Then I would simply change the delimiter option in IDI to write out metadata to use "." instead of "/". Doing a once-off forced export of metadata would enable me to use Picasa and IDI going forward.

You could also use Phil Harvey's excellent Exiftool to do this sort of thing if you're comfortable with Perl scripts (I'm not).

Of course, you need to be aware of what is possible, you can't really have a range of tools that demand conflicting ways of handling keyword hierarchies. So in the scenario above, where Picasa and WLPG have no common ground in the way in which they handle keyword hierarchies, you can't use both simultaneously. But moving from one situation to another should be possible in most cases.
Geoff Coupe
--------------
Photo Supreme /Windows 10 Pro 64 bits + Windows Home Server 2011 = DAM

tstoddard
Posts: 584
Joined: 07 Sep 12 12:51

Re: Keywords - Delimited vs Simple

Post by tstoddard » 21 Sep 12 17:15

Then I would simply change the delimiter option in IDI to write out metadata to use "." instead of "/". Doing a once-off forced export of metadata would enable me to use Picasa and IDI going forward.
Thanks for that thorough answer. I have one question about your response. What is the easiest way to do a "once-off forced export of metadata" for your entire catalog with either IDimager or Photo Supreme? My version of IDimager is a trial that will expire in 13 days and Hert told me that I can't buy a license for it. That gives me 13 days to do this in IDimager if I choose to. I would prefer to know how to do it in Photo Supreme since that is what I will probably end up using.

Thanks!
Tom Stoddard

gcoupe
Posts: 461
Joined: 16 Mar 05 19:29
Location: Heelweg, The Netherlands

Re: Keywords - Delimited vs Simple

Post by gcoupe » 21 Sep 12 17:34

tstoddard wrote: Thanks for that thorough answer. I have one question about your response. What is the easiest way to do a "once-off forced export of metadata" for your entire catalog with either IDimager or Photo Supreme? My version of IDimager is a trial that will expire in 13 days and Hert told me that I can't buy a license for it. That gives me 13 days to do this in IDimager if I choose to. I would prefer to know how to do it in Photo Supreme since that is what I will probably end up using.

Thanks!
So, if you're using PS...

1) Let's say that you've been using "/" as the delimiter in your IPTC Core Keyword strings, and now you need to have "." as the delimiter.
2) Go to PS Preferences, and select "Synchronize Settings".
3) Under the "Write Settings" tab, make sure that you have the following:
- Keywords processing - replace keywords with Catalog Labels
- Write delimited keywords - with the new delimiter character chosen, i.e. "."
4) Apply the Preference settings.

Rather than apply the new settings to your entire collection in one go (I've found that PS is likely to stop working), apply them in chunks. The best way to eat an elephant is a bit at a time...

So, for example, switch to the Folder view, select a folder, and then select (CTRL-A) all the thumbnails in that folder. Then right-click and choose Metadata, Save metadata to file.

This will write out your Catalog labels, and the hierarchy, to the IPTC Core Keywords fields, using the new "." delimiter, and overwriting the old keyword strings that used the "/" delimiter character.

Repeat for the remainder of your folders.

Lastly, don't forget to change the PS Preference for the "Synchronize settings" / "Read Settings" tab to be the new "." delimiter.
Geoff Coupe
--------------
Photo Supreme /Windows 10 Pro 64 bits + Windows Home Server 2011 = DAM

george
Posts: 931
Joined: 24 Jun 07 15:57
Location: USA

Re: Keywords - Delimited vs Simple

Post by george » 21 Sep 12 18:36

Mike Buckley wrote:George,

IDimager Systems has always used an open architecture. If you or I want to migrate to another DAM product, we simply need to export our catalog database to a file and then import that file into the newly chosen software. This is one of the reasons I selected IDimager Version 4 (that's not a typo) as my first DAM tool. There are certain details of the catalog that the new software will not be able to import, but that's probably going to be because the new software doesn't have the capabilities of the IDimager Systems software, whether it's IDimager or Photo Supreme.
Thanks Mike. I'm aware of all this and that's why I'm here as well. I joined a year before you did and came in on version 3 IIRC.
George

Post Reply