Importing / Numbering / Renumbering

[Stephen]

This is for Paul's question in another thread. Just to separate the subject matter.

I have not yet used PSU for tethered shooting, so this might not apply to you.

My process:
1. Ingest all card content using ChronoSync to a folder name chronologically after the card (each of which I number).

2. Import into PSU from that folder, renaming each file to
YYYY-MM-DD_hh_mm_ss_filenumber.*
Adding the time makes it very unlikely to get duplicate file names when using multiple cameras
Also, if you happen to import the same file twice, then second will get renamed: YYYY-MM-DD_hh_mm_ss_filenumber(001).*

3. I put them into a subfolder with the name of the "event" and the date taken
Duplicate handling in the PSU import box. I make the name unique
I add an "import details profile" where my name and details of the shoot are added

Unfortunately PSU does not remember which images have been imported, so I try to import chronologically. I messed things up recently when not doing this. I had changed time zones and my Mac was set to automatically set time zone using current location. All of the dates and times on saved images were changed and I had to download all the cards again to sort things out. Leave to automatic feature "OFF" on a Mac!

[vlad]

Duplicate handling in the PSU import box. I make the name unique

Hi Stephen, just curious: why do you choose this over "Skip file"? Is it a safety measure? Also, do I recall correctly that the duplicate handling works only based on filename rather than file content (signature)?

[tstoddard]

Stephen,

Your practice of creating folder names based on cards and events seems a little complicated to me. I'm sure you have your reasons but I have a different strategy.

I believe that the entire goal of my folder and file naming should be to guarantee unique file names and nothing more. If the names contain content that can help me identify them by name, that's just a bonus. My folders, subfolders, and file names are all based on metadata from the files themselves so that the folder structure and file names can be unique and reproduced reliably. I use a renaming rule that guarantees unique file names. My renaming rule uses my initials, the camera model, the date taken, and part of the original file number. Folders are created based on the date the images were shot. That is really for no other reason than to group them into manageable sized groups for working on them and to make folder verification efficient. I gave up on trying to name folders based on image content a long time ago. Also, having my folder structure and file names based on image dates sometimes comes in handy when I need to go into my file system directly for any reason. Looking at my file names instantly enables me to identify what folder I can find the file in.

Trying to name folders based on content has it's problems. One advantage of using a program like PSU is that files can be grouped into many different logical collections without needing to rely on the folder structure. Of course, I realize that doing this leaves me very dependent on PSU but I believe that there will always be some software available on modern operating systems that will allow me to search for images based on the content of their metadata so I'm not too worried about that.

One flaw in my approach presents itself if I need to change any of the metadata that the folders or file names are based on. This can happen if I need to redate files after I've imported them. Fortunately, this rarely happens and I've always been able to work around it.

[Stephen]

Duplicate handling in the PSU import box. I make the name unique

Hi Stephen, just curious: why do you choose this over "Skip file"? Is it a safety measure? Also, do I recall correctly that the duplicate handling works only based on filename rather than file content (signature)?

Safety measure.
Duplicate file names is what I wanted to avoid and set up this step before knowing exactly how PSU would react.

[Stephen]

Stephen,

Your practice of creating folder names based on cards and events seems a little complicated to me.

It's actually very easy, although surely not perfect. I physically number all memory cards and the content is ingested into a folder with the same number and using the original file name. As we know, some cameras have a limit and re-set their numbering clock when they pass 9,999 images or whatever. If using a second camera then its images are on another card, so even if the file names are the same, they go into a different folder. That is just the ingestion process and these folders are duplicated on different media as a backup long before the original memory cards are deleted. Usually I carry enough cards for any trip.

Then they are imported into PSU, go into an event & date folder and at that time the file names get changed and images from a second camera on the same shoot get chronologically merged. The original date and image name is retained in case I every need to access the backups in the safe.

I use a renaming rule that guarantees unique file names. My renaming rule uses my initials, the camera model, the date taken, and part of the original file number. Folders are created based on the date the images were shot. That is really for no other reason than to group them into manageable sized groups for working on them and to make folder verification efficient. I gave up on trying to name folders based on image content a long time ago. Also, having my folder structure and file names based on image dates sometimes comes in handy when I need to go into my file system directly for any reason. Looking at my file names instantly enables me to identify what folder I can find the file in.

I use the original file number and the date taken, but add the time to enable uniqueness.
I have the camera model in the EXIF data and if I need to add my name before distributing images, that can be done in batch. Customers invariably rename them anyway.
Image content or place is more useful for me if looking down a directory list on the computer. I am far more likely to remember an event or place than the date it was taken. PSU sorts them by date automatically anyway.

One flaw in my approach presents itself if I need to change any of the metadata that the folders or file names are based on. This can happen if I need to redate files after I've imported them. Fortunately, this rarely happens and I've always been able to work around it.

No system is perfect.
I will see how my method fans out once I have been using it for a time. For importing and old archive and 3 months travel on the road it has worked quite well. Now the editing starts, so we shall see. That is a different issue though.

[Mike Buckley]

Duplicate handling in the PSU import box. I make the name unique

Hi Stephen, just curious: why do you choose this over "Skip file"? Is it a safety measure?

I also configure it to make the file name unique. If I configure it to skip the file, I won't know whether the file is being skipped because it's the same file that has already been downloaded or whether the file is being skipped is a different file but with the same file name. If it's the latter, I want to know that I messed up and I want to know where to look to determine how to fix whatever I messed up. That's because I never want two files of the same name on my system. Downloading the file and seeing that is has the appended file name indicates that I've got two file names that would have been the same if PSU had not made the file name unique.

[vlad]

Good points, Mike. Thanks.

I never want two files of the same name on my system.

I agree that's an excellent guideline.

Btw, it would seem to me that ramping up new PSU users - and, even more so, new DAM users - could become easier simply by providing a good, clear, fairly short document with guidelines and good practices ("avoid having two files named the same in the catalog" - use renaming, if needed; "don't assign two labels named the same" - periodically review the catalog structure), recommended and discouraged settings (turn on hierarchical keywords, turn off some other option) etc. As a matter of fact, I could see this being helpful even to longtime users. (While adavanced users certainly appreciate the flexibility offered by so many settings and supported workflows, it could be disconcerting without clear guidelines and pitfall warnings.)

(Sorry, I should perhaps have started a seperate thread for this suggestion but Mike's statement above seemed a case in point too good to skip.)

[tstoddard]

Stephen,

Thanks for sharing more details. Like I said before, I'm sure you have your reasons. You are a professional and as such you have very different needs than me. Most of the photos I take are taken in my home, my backyard or neighborhood, or in local parks and wildlife areas. I would have trouble coming up with event names for those instances and sometimes I will have images from multiple locations and/or events on the same memory card. I do travel a few times during the year but those photos are not a majority of the files I import. I think it's great to read about how you and others use PSU and the reasons that you do the things you do. It's one of the things that makes this forum so useful. Thanks again.

[Stephen]

Stephen,

Thanks for sharing more details. Thanks again.

I think we can all profit for each other here.

[Stephen]

Paul,

A better approach would be to create a renaming rule that will result in the file being renamed exactly the same any time it is imported.
Nikon Maker Notes include the ShutterCount which will be a unique number. Using the shutter count or at least part of it will also give you a unique (and reproducible) suffix for your file names. I'm sure others could provide additional ideas if you were to ask.

Shutter Count is a great idea. Thank you for an excellent suggestion.

I gave a lot of thought to numbering last year when preparing to start using PSU and import old archives and was cautious of manufacturer dependent factors, like i.e. Shutter Count. Sony does not provide a Shutter Count, if you should ever consider using their cameras, which could mess-up your future-proof archival system.

Even rental cameras or multiple cameras should be considered.

I settled on the time of exposure, which, as I am not undertaking ultra high-speed photography, should provide enough uniqueness and is a system usable on all cameras.

[tstoddard]

I gave a lot of thought to numbering last year when preparing to start using PSU and import old archives and was cautious of manufacturer dependent factors, like i.e. Shutter Count. Sony does not provide a Shutter Count, if you should ever consider using their cameras, which could mess-up your future-proof archival system.

I've thought about that also, Stephen. I use different naming rules for each camera that I own. This allows me to take some of these issues into account. I actually use the last three digits of the original file name right now. Every camera I've ever used appends sequential numbers to the end of the file names they create and this, in combination with a date and a unique abbreviation of the camera model, has been enough for my purposes. I don't think there is one perfect rule that you could use for every camera that would guarantee uniqueness and reproducibility but I think there is always a way to find one rule for each camera that will do it. I used to use hours, minutes, and seconds in my renaming but I tend to use a continuous shutter release fairly often, which results in multiple images within the same second. I suppose you could use fractions of seconds if you really wanted to but the resulting file names start to get pretty long and messy when you do that.

As usual, there is no right way for everyone. This discussion just helps us all to think through our strategies, which is a good thing.

[Mike Buckley]

I don't think there is one perfect rule that you could use for every camera that would guarantee uniqueness and reproducibility

Using a sequential number accomplishes that goal. If you're shooting with multiple cameras, the sequential number by itself will not accurately indicate the order the photos were taken but it will guarantee uniqueness and reproducibility. Most users would want to include other information in the file name as well but it's the sequential number, whether on its own or in combination with other information, that makes the file name unique.

[Paul C]

I don't think there is one perfect rule that you could use for every camera that would guarantee uniqueness and reproducibility

Using a sequential number accomplishes that goal. If you're shooting with multiple cameras, the sequential number by itself will not accurately indicate the order the photos were taken but it will guarantee uniqueness and reproducibility. Most users would want to include other information in the file name as well but it's the sequential number, whether on its own or in combination with other information, that makes the file name unique.

Using a sequential number will guarantee uniqueness, but not reproducibility, as I found in the issue that started this discussion. PSU will keep incrementing the sequence number, so if you import the same set of images twice, the second set will have new sequence numbers and so unique names - but they are duplicate images. Unless there is some way to make sure that the same image always gets the same sequence number you will not get reproducibility. I have been using "image file name-sequence number" for years and never thought about reproducibility. Now that I have experienced this issue in PSU, I am thinking about both uniqueness and reproducibility. I haven't decided exactly how, but I am going to change my renaming scheme.

[Stephen]

I made a mistake in my initial post and added to many hyphens. This is my system, but I will modify it.
yyyymmdd-hhnnss-9999.jpg
9999 is the file name and jpg or raw or whatever the extension is.

It is not too long, however I really do not need 20 of the year 2015, so in the future I might restart at 15. I would also add milliseconds and then I believe I would be perfectly safe in having uniqueness. (stoddard would also be happy). All files are chronologically sortable in any OS and the file name is only 20 digits long out of a maximum of 31. Therefore each can be extended by adding edit.jpg or approv.jpg or whatever one might need at the end of the string to further identify them without messing up the sorting.

Just my thoughts...

[Mike Buckley]

Using a sequential number will guarantee uniqueness, but not reproducibility, as I found in the issue that started this discussion. PSU will keep incrementing the sequence number, so if you import the same set of images twice, the second set will have new sequence numbers and so unique names - but they are duplicate images.

You can easily change the offset number when you want images that have already been imported to be imported a second time using the same file name as the first import. Having said that, I can't imagine why you would want to import any image more than once.