Batch renaming

Armitage Shanks
Posts: 9
Joined: 22 Feb 08 1:30

Batch renaming

Post by Armitage Shanks » 28 Jan 13 21:52

I'm a former IDimager V4 Pro user and recently upgraded my PC so I've made the switch to PS.

I don't use any of the cataloguing capability of the tool and simply wish to use it to rename and move files as I did successfully for several years with IDimager.

I am struggling to set up the same behaviour on PS and would be grateful for some assistance.

My wife regularly produces content (JPEGs, MOVs and AVIs) from a variety of cameras which I manually deposit into a temporary folder on my desktop. I then used to use IDimager to rename the files by getting it to inspect the metadata to, for example, 2013-01-28-19-43-12.JPG (year, month, day, hour, minute, second). It also created a directory hierarchy as required. For example, I have Photostore\2009\03\17 and 2009\03\19, and so on. When IDimager had several files which would have had identical filenames as they were taken within a second of one-another, it used to create, for example, 2013-01-28-19-43-12-001.JPG, 2013-01-28-19-43-12-003.JPG and 2013-01-28-19-43-12-003.JPG.

The attached screen capture illustrates the folder hierarchy and filename format.

Once the files had been renamed and placed into the correct folder, they were deleted from the temporary folder.

I've got some of this working with PS but the following isn't working for me:

1) Creating multiple files when multiple images were taken within a second of one-another.
2) Deleting the files from the temporary folder once they've been renamed and moved. What actually happens is that some of the files are deleted and a seemingly random selection of files are left behind. I then have to manually check to satisfy myself that a copy of the file has been dropped into the correct destination before I feel confident that I can manually delete it. (As an aside, I'm unclear what kind of 'verification' is taking place when I click the 'Import' button that might satisfy me that I can manually delete these files which the automatic operation is leaving behind).

Any help gratefully received.
Attachments
Photostore.jpg
Photostore.jpg (249.21 KiB) Viewed 5033 times

Armitage Shanks
Posts: 9
Joined: 22 Feb 08 1:30

Re: Batch renaming

Post by Armitage Shanks » 02 Feb 13 0:15

Is this not the correct method to raise a support case?

jstartin
Posts: 401
Joined: 23 Aug 06 13:47
Location: UK

Re: Batch renaming

Post by jstartin » 02 Feb 13 15:08

Armitage Shanks wrote:Is this not the correct method to raise a support case?
This is a user-to-user forum even if Hert, the developer, responds frequently. So, it's a good place to ask questions but perhaps not to raise a formal support case with Hert :wink: . Bugs and feature requests are best handled as explained here: http://forum.idimager.com/viewtopic.php?f=57&t=18353

For help working out whether you are running into one or more bugs in PSU, or trying to do something that it simply is not intended to do, you might have to give more detail of your settings and renaming rules etc.
Jim (Photo Supreme: AMD Quad-Core A8-5500 Accelerated Processor 3.2 GHz; internal AMD Radeon™ HD7560D; 4GB DDR3 SDRAM; Win10x64)

Armitage Shanks
Posts: 9
Joined: 22 Feb 08 1:30

Re: Batch renaming

Post by Armitage Shanks » 03 Feb 13 1:26

I'm very happy to provide whatever information may be required to give me back the functionality that I've lost!

Hert - are you able to help me with this, please?

jstartin
Posts: 401
Joined: 23 Aug 06 13:47
Location: UK

Re: Batch renaming

Post by jstartin » 03 Feb 13 9:40

I briefly tested a few things related to this. I edited (outside PSU) the Exif dates of a few "spare" files to make them identical, and set up a date&time rename rule in PSU. If I rename my altered files from batch or the context menu then PSU adds the extra number to ensure the new names are unique. If I use the same rule during import of these same files PSU only moves/renames the first one and leaves the others in the source location with original names. I don't know whether this is by design, or a bug.

You could possibly consider working around the limitation by importing files to an automatically created dated file structure but renaming them as a separate action.
Jim (Photo Supreme: AMD Quad-Core A8-5500 Accelerated Processor 3.2 GHz; internal AMD Radeon™ HD7560D; 4GB DDR3 SDRAM; Win10x64)

gcoupe
Posts: 244
Joined: 16 Mar 05 19:29
Location: Heelweg, The Netherlands

Re: Batch renaming

Post by gcoupe » 03 Feb 13 13:16

I have a feeling that there's a bug here. I recall that a few months back, when I did some action shots with continuous shooting, that only the first photo of a one-second time slice would get imported. The rest would remain on the camera memory card. And yes, I have a date&time renaming rule in the import routine.
Geoff Coupe
--------------
Photo Supreme /Windows 10 Pro 64 bits + Windows Home Server 2011 = DAM

jstartin
Posts: 401
Joined: 23 Aug 06 13:47
Location: UK

Re: Batch renaming

Post by jstartin » 03 Feb 13 15:03

I have a feeling Geoff is right. Actually there could be more than one bug, and perhaps a serious issue. I am fairly sure that with a particular combination of "Don't overwrite...", "Verify ingested image..." and "Delete from source..." settings, PSU Import has just deleted "rename rule conflicted" files from a card without copying them first!
Jim (Photo Supreme: AMD Quad-Core A8-5500 Accelerated Processor 3.2 GHz; internal AMD Radeon™ HD7560D; 4GB DDR3 SDRAM; Win10x64)

Armitage Shanks
Posts: 9
Joined: 22 Feb 08 1:30

Re: Batch renaming

Post by Armitage Shanks » 03 Feb 13 21:42

Yikes! Deleting files without copying them is probably about as serious as it gets, bug-wise, for image management software! Presumably, if such a bug exists, it has to be near the top of the list for getting fixed?

Hert, I'd be very grateful for your input here to confirm/dismiss the assertions made above and to clarify if you think I'm experiencing an issue or if my issues are simply user error?

Thank you.

Hert
Posts: 5873
Joined: 13 Sep 03 7:24

Re: Batch renaming

Post by Hert » 04 Feb 13 7:57

Jim, you make quite a statement there without fundaments. Please provide the exact details about that "combination" where you think files are lost without copying To me that sounds impossible because a delete can only take place if the target file is there. And even if a file name already exists due to a conflicting file name rule then the verification that takes place before delete would fail.
"I have a feeling" is a great start but not very helpful start for me to try to reproduce something. And please report bugs in Mantis so that it can be followed.

What I think can be the issue here is that the import module is multi threaded and 4 files are being handled concurrently (depending on your config even 8 or 16 files). And that the file naming rule generates duplicates and then multiple threads concurrently try to check for duplicates and they may get the same result.

First of all, I would recommend to use a naming rule that doesn't generate duplicates. Post your naming rule here so we can have a look at it.

Hert
This is a User-to-User forum which means that users post questions here for other users.
Feature requests, change suggestions, or bugs can be logged in the ticketing system

gcoupe
Posts: 244
Joined: 16 Mar 05 19:29
Location: Heelweg, The Netherlands

Re: Batch renaming

Post by gcoupe » 04 Feb 13 8:51

IDimager wrote: Post your naming rule here so we can have a look at it.
I use the built-in "Filename as Date-timestamp" rule.

When I had the error (and reported it here), I was using the built-in "Filename as Date-timestamp" rule copied from IDI and pasted into PSU. Now I just use the rule that is built-in to PSU. I admit that I haven't tried to reproduce the error since - I haven't been doing continuous shooting.
Geoff Coupe
--------------
Photo Supreme /Windows 10 Pro 64 bits + Windows Home Server 2011 = DAM

jstartin
Posts: 401
Joined: 23 Aug 06 13:47
Location: UK

Re: Batch renaming

Post by jstartin » 04 Feb 13 15:42

IDimager wrote:Jim, you make quite a statement there without fundaments. Hert
Hert

Sorry, I didn't express that very well and I should have done more research before posting at all. But...

I was "playing around" with the OP's problem in a few spare moments using different settings etc, but didn't note what they were. And then found my source files were nowhere to be found. (This was all, fortunately, on a test installation on a different PC to my real collection. I didn't have time to do more, and thought a rather vague forum post might prompt you or others to start testing. My poor judgement.

Today I have reconstructed the scenario and these are the details. I will also post on Mantis.

This is what I did:
Made 5 copies of a single Konica Minolta (MRW) raw file, using Windows (PSU not running).
Named them to 1.MRW, 2.MRW....5.MRW
In File "2" and "3" used Exiftool to change DateTimeOriginal, CreateDate and ModifyDate to 1 second later and 2 seconds later, respectively.
Copied files to a memory stick.
Started PSU.
Started Import dialog.
Set Source = memory stick
Set "Copy images to new location" ON
Set Folder to an existing catalogued folder (....\Import)
Set Subfolder to "Today" (Folder does not yet exist)
Set Filename to custom defined "%yyyy%mm%dd%hh%nn%ss.%FileExtension" saved as "DateTime" (with this rule target names for File4 and File5 conflict with File1 and each other)
Deselected "Don't overwrite existing images"
Deselected "Verify ingested image"
Selected "Delete image from source after ingesting"
Set "Import images into catalog" ON
Selected "Create a session catalog label"
Deselected all other options.
Ran Import

Outcome:
According to Windows Explorer three files with DateTime format names differing by 1 second appeared in .../Import/"Today"/.
No files remained on the source memory stick. I cannot find File 4 or 5 anywhere.
No files were catalogued.
I have repeated this several times, after cleaning up, verifying the cleaned catalog, compacting etc.

If "Don't overwrite existing images" and "Verify ingested image" are both enabled things are different - three files are renamed correctly and moved; File4 and File5 remain on the memory stick and are catalogued by PSU. I have not worked through every other possible combination systematically.
Jim (Photo Supreme: AMD Quad-Core A8-5500 Accelerated Processor 3.2 GHz; internal AMD Radeon™ HD7560D; 4GB DDR3 SDRAM; Win10x64)

Armitage Shanks
Posts: 9
Joined: 22 Feb 08 1:30

Re: Batch renaming

Post by Armitage Shanks » 08 Feb 13 23:02

My rules are as follows:

The "Custom defined..." Subfolder rule is: %yyyy \ %mm \ %dd \

The "Custom defined..." File name rule is: %yyyy - %mm - %dd - %hh - %nn - %ss . %FileExtension

Don't overwrite existing images: checked
Verify ingested image with original: checked
Auto rotate images: checked
Delete image from source after ingesting: checked
Mark ingested image as Read-Only: unchecked
Set date/time equal to the moment picture was taken: unchecked

Armitage Shanks
Posts: 9
Joined: 22 Feb 08 1:30

Re: Batch renaming

Post by Armitage Shanks » 18 Feb 13 1:07

Do you need anything more from me than I included in my previous post 10 days ago to look into this for me, Hert?

Thanks.

Hert
Posts: 5873
Joined: 13 Sep 03 7:24

Re: Batch renaming

Post by Hert » 18 Feb 13 9:34

I think your issue was recorded in Mantis. I made sure that multiple threads can't generate identical file names when the unique option is switched on. Still it was reported that that didn't fix the issue of bulk images. This is still an open issue.

I concur that there currently must be an issue with renaming files for images that were shot within the same second (burst). Best way to get around that is to use a rename rule that generates unique output, without depending on the "unique" settings.

Hert
This is a User-to-User forum which means that users post questions here for other users.
Feature requests, change suggestions, or bugs can be logged in the ticketing system

jstartin
Posts: 401
Joined: 23 Aug 06 13:47
Location: UK

Re: Batch renaming

Post by jstartin » 18 Feb 13 12:00

IDimager wrote:I made sure that multiple threads can't generate identical file names when the unique option is switched on.
I can't find a "unique option" anywhere; I thought PSU was designed to always enforce unique names with no option.
IDimager wrote:I concur that there currently must be an issue with renaming files for images that were shot within the same second (burst).
To keep the record clear, this only seems to be a problem when renaming is carried out during import; renaming files that are already in the catalog adds "(nnn)" as necessary. If that also happened during import there would be no problem.
IDimager wrote:Best way to get around that is to use a rename rule that generates unique output, without depending on the "unique" settings.
I agree with this. But import should be fail-safe and foolproof. At the moment a badly thought out rename rule and the settings I documented before can lead to data loss. Perhaps ticking the delete option should force "Don't overwrite" and "Verify" options to be on, with no choice. And perhaps a final import report should appear stating that "Some images were not imported/copied and remain in the source folder" if that is what has happened.

Armitage Shanks: My suggestion for you, to get a result as close as possible to the one you are used to from IDI, is to experiment with importing into dated folders without renaming, and then, as a separate action, running your rename rule on the imported files.
Jim (Photo Supreme: AMD Quad-Core A8-5500 Accelerated Processor 3.2 GHz; internal AMD Radeon™ HD7560D; 4GB DDR3 SDRAM; Win10x64)

Post Reply