How to purge labels from database?

[vlad]

Nevertheless, I do think expecting such an "undo-button" to ever be implemented is extremely unlikely!

Agreed.

Some settings potentially immediately affect thousands of files - both the metadata in these files and the data in the catalog! How could PSu possibly keep track of all these changes?

By keeping track of all metadata changes, for every image (or sidecar) file. That may sound like mission impossible, but there is an entire category of programs (version control systems) designed to track, record and revert file changes; granted, no program that I know is designed to work specifically with image metadata. In addition, a global "Undo" in Supreme would require to track - and revert (if possible) - internal catalog changes, such as assigning a label.

You change a setting, thousands of files need updating. You change it back, how many files have, during that time, been updated, which ones now need to be changed back? This is far too complex for a simple "undo button". PSu would need to keep a history of many, highly complex changes that generally all interfere with one another. This is not just deleting one adjustment layer in photoshop. It is more like having to take snapshots (of the database and files at a particular stage in time)...

Frank is right. If Hert could pull off such a feature, it may well be ground breaking, at least as far as publicly available DAM's are concerned. However, that doesn't mean such a significant undertaking would make business sense.

Rather than expecting an "undo" button to emerge any time soon, I would vote for implementing some sort of incremental backup feature. A feature that does not require me to backup the full database every time, but only the last (few) changes.

That seems useful, but I'm not sure how it could be implemented without going all the way into change tracking and all that...

There is a feature request that essentially asks for such incremental backups (originally I was most concerned to keep two catalogs on different computers in sync, but it would equally address more efficient backups in general): http://mantis.idimager.com/view.php?id=2860
Feel free to support it ...

Frank, I've just seen your latest note in Mantis. Binary patches? Interesting idea, but I'm not sure how it would work for the Supreme database. (There's one thing to generate a patch when you already have two versions of a binary file and another thing to quickly generate such a patch, without producing the end result first.)

[Mike Buckley]

There is an apparent need to state the obvious , so I am happy to clarify my thoughts about experimentation. Experimentation can be done safely if you:

• Use a separate catalog named to indicate that it is a test catalog;
  Import image files into that catalog that are not part of any other catalog;
  Store those image files in folders that are named to indicate that they are test folders and are not part of any other catalog;
  Store any files created during the testing in folders that are named to indicate that they are test folders and are not part of any other catalog;
  When practical, make one change at a time by conducting a test before making another change;
  Make accurate, informative notes of all changes made and make the notes before conducting the associated test;
  Make accurate, informative notes of all test results; and
  Exercise prudent care throughout all experimentation.

You are configuring your system to write synonyms as keywords and to merge keywords with catalog labels when reading the file. So, if you read metadata from the file, all of those synonyms will be added to the Miscellaneous category of your catalog because they don't exist as part of the hierarchy in your catalog.

Mike, are you sure about that?

Yes, I'm very sure. I just now conducted the test not using a separate test catalog.

are synonyms stored inside hierarchical keywords too?

No. The synonyms are stored only in dc:subject as simple keywords. The facts that they are stored there and are not already in the catalog's hierarchy explains at least to me why they are added to the Miscellaneous category upon reading metadata from the file. In other words, I would expect that behavior.

[tstoddard]

tstoddard wrote:
When I modified my test label's metadata settings to not write a keyword and then I deleted the label before I wrote metadata to files, the keyword that the label had written originally was left in the file. Had I been using automatic writing of metadata, the result would have been different. The keyword would have been removed automatically when I changed the label details.


Tom, is that your supposition or a verified fact? I am asking because that's an important (although peculiar) aspect and I would like to understand if it could affect me in any way.

Vlad, it is verified in the context of the steps I outlined in the experiment I performed earlier. It might not always be the case but with my preferences set as I normally have them, it does work as I've described. I normally have my preferences set to automatically write catalog changes to files and when I change the metadata settings for a label, any of the files I have assigned to that label are marked as OOS and then immediately synced (metadata is written to the files). The keywords for that label are removed from both the dc:subject field and the lr:hierarchicalsubject field. If I don't sync automatically, then changing the metadata first still marks the files as OOS but then deleting the labels before syncing and then syncing after the delete, leaves the keywords intact.

[Stephen]

Nevertheless, I do think expecting such an "undo-button" to ever be implemented is extremely unlikely!

Agreed.

Before this side theme gets out of hand I am going to start a new thread about it to explain more clearly what I mean. I'm busy with something else just now. Please hold off a while.

[PhilBurton]

Stepping back a bit, if you were starting from scratch, what would be your best practices to either avoid being in this situation, or fixing it after the fact?

Phil

[Stephen]

Stepping back a bit, if you were starting from scratch, what would be your best practices to either avoid being in this situation, or fixing it after the fact?

That is a tough call, but I will try…
Migrating to another program can cause problems, but the divorce from Aperture was painful despite considerable testing. Apple packed the application, the settings, the edits and all images into ONE huge package file (probably unknown in the Windows world).

In retrospect (if possible) I would not import images which had left Aperture into PSu until I had been comfortable using the interface on a regular basis.

Then I would rename the (original Aperture) folder structure to suit the form I had since discovered more workable under PSu. That is naturally individual. Also I would NEVER select the option "create catalog labels from folder names" on import!

The default allocation of Places to misc category would still be a problem because (besides other fields) I usually add Location, City, State and Country during import. Studying PSu labels system was left until another day, so I had only been using the keywords fields in the Details panel until very recently. Now I just use Assign, but the mess with multiple labels has occurred.

A new user needs to get their PhD in PSu first to avoid such problems. None of this is obvious or intuitive to somebody who has never used PSu or another DAM before. Wonderful if you have the time and even especially if you are a professional librarian or researcher making their money from cataloging. I just need it to keep track of files which a defunct program, used to do adequately. PSu has many more functions, but I don't really need them. How does the saying go, however, having tasted blood… ?

I am now using PSu on a daily basis and the train is moving, so switching is difficult but not impossible.

(This is my opinion on the fly, which I what Phil wanted. I hope it doesn't digress to another discussion as the original thread is still very valid.)

[Mike Buckley]

Studying PSu labels system was left until another day, so I had only been using the keywords fields in the Details panel until very recently.

The importance of using the Assign panel to manage catalog labels rather than the Details panel to manage keywords should be emphasized in the Quick Start manual and it's not. Considering IDimager's goal to keep the program simple, I don't even understand why the program makes it possible to change keywords using the Details panel.

[fbungarz]

Considering IDimager's goal to keep the program simple, I don't even understand why the program makes it possible to change keywords using the Details panel.

Agreed, though I would not want to completely loose the ability of editing that metadata field. Like technical metadata (which are locked for editing) this would be a good candidate for being locked per default with the option to unlock that field.

[vlad]

Mike, Tom: thanks for your previous explanations and clarifications.

Considering IDimager's goal to keep the program simple, I don't even understand why the program makes it possible to change keywords using the Details panel.

I tend to agree. Although I once used the Details panel to edit keywords (for script testing purposes), I did not find that process reliable and I have since read other accounts (including Stephen's account) of problems caused by direct editing of keywords. Furthermore, the potential for user confusion and incorrect metadata likely gets compounded for anyone who stores both simple and hierarchical keywords (as recommended).

[Stephen]

Yes and it continues, because I am having to import some missing old files. With the imports I am also getting the bad keywords

Vlad, do you remember the thread over a year ago about me complaining that I always import new images in small batches and that mostly 3 files were not imported at the start of every batch (1 JPG and 2 RAWs / or / 2 JPGs and 1 RAW)? Well I haven't got to the bottom of it but in the process of cleaning up I am finding duplicate "(001)" files, but WITHOUT the originals - all in groups of 3! At the time I manually imported the individual missing files.

It appears that the originals WERE imported to the hard drive but the thumbnails were not built, so I thought that they were missing.*

I don't always need to verify the folders, but by doing so, I am now about to locate them.

*That isn't quite logical, so I don't really know what happened, other than that they did not display. I'm not going to waste time trying to re-engineer what happened, but it was a pain. This import problem appeared to have been fixed in an update, but more work is now resulting.

[fbungarz]

I too have experienced that I the importer appears to be fairly unreliable, when not downloading files but importing existing folders. It seems verify folders does a much better job...
To me it seems although the importer looks and feels a lot better than IDI's separate downloader & import dialogs, it actually does a much less reliable and not even nearly such a powerful job. It misses files during import (not download), is less flexible when renaming, it does not recognize existing version sets (but destroys them!), it does not recognize file masks assigned to version placeholders, and it does no longer convert files to DNG (if the DNG converter is used, the original file is thrown away, not versioned). Unfortunate...

[snowman1]

Hi Frank,

I'm not clear how the importer can destroy existing version sets as by definition (as you are importing them) the images are not previously in the catalog?

It could be argued that the importer should not be renamng at all - that is the job of the downloader, whereas importing (into the catalog) implicitly refers to existing images?

FWIW whilst there are aspects that I think could be improved (best not discussed on this thread) I do think it's very good. In particular the ability to operate the import and download parts independently - and how to treat duplicates - is really powerful.

[edit - I shall now quietly butt out again ]

[Stephen]

I too have experienced that I the importer appears to be fairly unreliable, when not downloading files but importing existing folders. It seems verify folders does a much better job....

When you write 'downloading files', do you mean downloading (importing) from a memory card?
If so, then I understand what you mean, but in my case it was the opposite. I had the problems when downloading files "not known by PSu". I quote because I never download directly from a memory card. My original images are always cloned with double verification to a different medium before importing.

To me it seems although the importer looks and feels a lot better than IDI's separate downloader & import dialogs

I cannot comment as I have never used IDI and this is the PSu forum anyway

[fbungarz]

Hi Snowman,
I was actually not sure if I should "butt in" as you call it. I actually do much prefer the general idea of PSu to bundle downloader and importer into one packet. Working with IDI I never thought is was very intuitive to keep the two separate.

@Stephen: you are right, this is a PSu forum, and I will be told by others that I have no reason to complain. Just to clarify: when I use IDI's lingo, downloading means copying fresh new files from a memory card or folder, typically renaming them and moving them into a completely new folder (typically created via a folder renaming rule). Importing instead simply incorporates already existing folders into the catalog.

As for as destroying existing version sets during "imort": I have been migrating from IDI and for many reasons discussed in detail elsewhere PSu would not correctly read ICS embedded in my files. That means, existing versions are not detected. When I then tried importing these files using version detection I learned that my custom placeholders would not be assigned (being told by Hert that by design custom file masks are not recognized during import) and only JPG+NEF pairs were detected, all other versions ignored. This in effect means PSu "destroys" existing version sets (when trying to migrate those from IDI).

So, I do not necessarily disagree with you: the PSu importer might be very good, it generally seems to be a more streamlined, thought-through version combining IDI's downloader and importer. For most purposes it probably does an excellent job. Mike will tell me again about my "special needs". So, I too better shut up now...

Perhaps I should have simply stated that there is even a Mantis report, where other users complain it ever so often misses some images (It is not just me and Stephen who apparently experienced this - Stephen: I have only used it extensively for importing so far, only occasionally for downaloading; the very few times I downloaded images, it did not miss files so far).

Cheers,
Frank

[Stephen]

Just to clarify: when I use IDI's lingo, downloading means copying fresh new files from a memory card or folder, typically renaming them and moving them into a completely new folder (typically created via a folder renaming rule). Importing instead simply incorporates already existing folders into the catalog.

@Frank. OK, Marcel ... just to make things clear for everybody.

My use has been threefold:
1. Copying fresh files from a folder (and renaming them in the process) - which PSu calls importing
2. Copying old files from another folder (and renaming them in the process) - which PSu calls ???
3. and just this week, telling PSu about files in another folder (NOT renaming) - which you call importing[/quote]

The missed files anomaly has only occurred in case #1 AFAIK, but is multiplying the bad label problems, once these files are located and reinstated in the PSu catalog.