IDI DarwinCoreXMP for biodiversity metadata released

News and Announcements
Locked
fbungarz
Posts: 3306
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

IDI DarwinCoreXMP for biodiversity metadata released

Post by fbungarz » 05 Feb 10 15:12

Hello everyone,
we just finalized some work on developing an XMP metadata implementation for the Darwin Core Metadata standard.

From the Darwin Core website:
The Darwin Core is body of standards. It includes a glossary of terms (in other contexts these might be called properties, elements, fields, columns, attributes, or concepts) intended to facilitate the sharing of information about biological diversity by providing reference definitions, examples, and commentaries. The Darwin Core is primarily based on taxa, their occurrence in nature as documented by observations, specimens, and samples, and related information.
We share this Darwin Core XMP schema in the hope that it may get widely established as the one standard how metadata on biodiversity will be handled in image files.

For more detailed information have a look at the IDI Wiki :
http://wiki.idimager.com/index.php?titl ... P.27.27.27

I have also submitted a feature request (http://forum.idimager.com/viewtopic.php?f=6&t=12713). At the moment IDI does not support loading of separate parts of Image Details Panels. In a way this defies the purpose of Image Details. It should be possible to only import a specific set of Image Details needed for a specific purpose. Currently it is only possible to load a complete set with all Image Panels and thus loose previous customizations.

Cheers,
Frank

shortia
Posts: 27
Joined: 18 Apr 10 4:34

Re: IDI DarwinCoreXMP for biodiversity metadata released

Post by shortia » 18 Apr 10 4:51

I am thinking of migrating from ThumbsPlus as a long time user of an apparently fading product, and IDimager is currently at the top of my list. I am planning to use it in the short term to organize years of field notes, and in the long run years of slides/digital images. Almost all are of nature, so this biodiversity metadata looks ideal for me. Do you have any updates on the XMP? Does it seem to be integrated as you'd like? Thanks.

fbungarz
Posts: 3306
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: IDI DarwinCoreXMP for biodiversity metadata released

Post by fbungarz » 18 Apr 10 5:51

Dear Shortia,
although my feature request (http://forum.idimager.com/viewtopic.php?f=6&t=12713) has not yet been implemented the metadata schema works well and you can easily adapt it to your necessities. I suggest you install IDI as a trial and simply try it out.
BTW - I have been a Thumbs+ user myself and never did regret that I made the move. IDI is certainly more complex but also much more powerful. Once you seriously consider switching you may be interested in this how to:
http://wiki.idimager.com/index.php?titl ... o_IDImager

Also - right now is a real good time since IDI is on special offer to T+ customers.

Cheers,
Frank

shortia
Posts: 27
Joined: 18 Apr 10 4:34

Re: IDI DarwinCoreXMP for biodiversity metadata released

Post by shortia » 18 Apr 10 14:55

Yes, I saw that wiki- which is ideal for me. My most important migration is the galleries, which is where I sorted my best images and placed them into groups. I am not sure that was the best way for my workflow, but somehow that is just what happened naturally when I first started using Thumbs+, many years ago. I made the transition to digital cameras in 1998 and never looked back, so I have a LOT of images to deal with. I was happy with Thumbs, since they also enabled me to begin the process of sorting my videos and audio recordings. But the past year there has been non-existent support, and IDI actually looks far more powerful.

Since you seem to be an expert, could I ask you a few questions? Currently I have a module in some proprietary software where I can automatically back up my images every day, with associated metadata, to Amazon S3. Is there anything like this in IDI, or the ability to attach software to it that would let me do this? Is it fairly straightforward to download, and develop management so images are not overwritten when, for example, they reach the 10K boundary? I see in the list of supported formats IDI includes things like WAV, and MPEG files. I would ideally like to expand my catalog to my audio recordings and video files. Can I adapt IDI to unsupported files like AVCHD, or is it possible it will eventually support those too? Finally, I am currently using some proprietary software to photograph and then annotate source material like my old field journals, since the information there is often correlated with my wildlife photos. Is there a field available in this software where I can type extensive notes, or is it limited to keywords? I would, for example, like to be able to enter the equivalent of a paragraph or two to associate with some images. There seems to be a field under the metadata- called description, and I am hoping this field is associated and searchable with the image. I tried it last night, having installed the trial a couple days ago to get a feel for it, and it seems to work, but not understanding exactly how IDI works I am curious where this information is stored.

Thanks so much for writing the IDI DarwinCoreXMP. This alone is one feature that will sway me toward IDI over other similar software I have been considering. Something that has always tripped me up in my own cataloging of my scientific content (recordings, video, photos) is exactly how to hierarchically express habitat location. Given you seem to be working on a similar dataset, do you happen to have such a hierarchy, or any thoughts on how to develop one? Most of the software and controlled vocabulary I have seen isn't adequate for scientific work. Thanks again.

best,
Ruth

fbungarz
Posts: 3306
Joined: 08 Dec 06 5:03
Location: Galapagos, Ecuador

Re: IDI DarwinCoreXMP for biodiversity metadata released

Post by fbungarz » 18 Apr 10 16:08

Hi Ruth,
Currently I have a module in some proprietary software where I can automatically back up my images every day, with associated metadata, to Amazon S3. Is there anything like this in IDI, or the ability to attach software to it that would let me do this? Is it fairly straightforward to download, and develop management so images are not overwritten when, for example, they reach the 10K boundary?
There are several options from within IDI to upload images to photo-sharing websites like Flickr, Zenfolio, etc. This functionality uses their APIs and it should in theory be possible to adapt any other API to do the same to a backup site like Amazon S3. I am no expert in scripting, but I am sure it would be worth to put a request in the IDI scripting forum and there might well be someone to help you. You could also putz in a feature request that something that this would be implemented in IDI as a hard coded feature. With scripts almost anything is possible, but having it as part of IDI would be more convenient.
I see in the list of supported formats IDI includes things like WAV, and MPEG files. I would ideally like to expand my catalog to my audio recordings and video files. Can I adapt IDI to unsupported files like AVCHD, or is it possible it will eventually support those too?
You can configure IDI to manage whatever digital format you want. The only problem is that it will not generate thumbs for such a wide range as Thumbs+. This is actually a drawback in IDI, its thumbnail viewer supports a smaller amount of image formats, videos, etc. That said - it will recognize any file if it is configured to recognize it. Do a search on the forum hear for using IDI to manage non-image files...
Finally, I am currently using some proprietary software to photograph and then annotate source material like my old field journals, since the information there is often correlated with my wildlife photos. Is there a field available in this software where I can type extensive notes, or is it limited to keywords? I would, for example, like to be able to enter the equivalent of a paragraph or two to associate with some images. There seems to be a field under the metadata- called description, and I am hoping this field is associated and searchable with the image. I tried it last night, having installed the trial a couple days ago to get a feel for it, and it seems to work, but not understanding exactly how IDI works I am curious where this information is stored.
Unlike Thumbs+ you can configure IDimager to store all information both inside the database and inside the file itself. IDI comes with a set of routine standard fields like the field "description", "title", "headline" etc. These are established metadata standards by the International Press and Telecommunications Council (IPTC), or by Adobe (XMP), or camera manufacturers (EXIF).

You can visualize this information if you take a software like Phil Harvey's ExifTool. Adobe Photoshop CS4 also has a file info pane that will show you the raw data that resides inside the image. All this metadata is essentially a flavour of XML written to your image file. XML is short for eXtensible Metadata Language, a kind of hypertext language that describes content. XMP is the dialect of XML that Adobe developed to allow embedding of image metadata into an image file.

This means, whatever information you enter into a field like description will be stored inside your image once you synchronize that information between the IDI database and your image.

XMP means eXtensible Metadata Platform and extensible means that anyone expand this basic standard to include whatever information imaginable. Today, there are quite a few flavours of XML metadata sets out there. It started with the Dublin Core, then Adobe added a few of their own fields, then there is a PLUS standard for copyright management etc.

IDI is currently the only Digital Asset Management tool that allows any software user to do just that and I have used it to translate the Darwin Core into XMP.
Thanks so much for writing the IDI DarwinCoreXMP. This alone is one feature that will sway me toward IDI over other similar software I have been considering. Something that has always tripped me up in my own cataloging of my scientific content (recordings, video, photos) is exactly how to hierarchically express habitat location. Given you seem to be working on a similar dataset, do you happen to have such a hierarchy, or any thoughts on how to develop one? Most of the software and controlled vocabulary I have seen isn't adequate for scientific work. Thanks again.
In IDI you can very easily map your hierarchic catalog labels to be written to any metadata field of your choice. THat way, if you assign a label, say the scientific name of an animal to your image, IDI automatically writes this label all parent labels into the mapped xmp fields of your image file.
IDI Controlled vocabularies are simple text files. When you import them they will show up as labels in your IDI tree. These labels are not mapped to any field but you can fairly easily do that with checking all labels of a particular hierarchy and then map them to the fields you want.
I am happy to share an example of my Controlled vocabulary of the taxonomy of lichen fungi that you can import and play with (send me a PM and I'll send you the file).

Finally - what is even better: you can use the CSV import script (download it at the resource repository) to import data directly into your XMP fields from other databases.

All this is a bit hard to explain without screenshots and I always wanted to expand the wiki site about the Darwin Core but have not had the time to do so. Read through the metadata section of IDI's help, look through the wiki and I can also highly recommend Mike Buckley'S IDI workbook. IDI is very powerful and it is all a steep learning curve.

Cheers,
Frank

Locked