File moving slowdown

Mke
Posts: 505
Joined: 15 Jun 14 15:39

File moving slowdown

Post by Mke » 07 May 20 17:10

One of the recent releases seems to have introduced a bug that slows down the moving of file from one directory to another.

When I try to move 100 RAW files (no stacks or versions) between folders, it now takes 3' 30":
  • For 8 seconds nothing seems to happen;
  • for 1' 12" my 4 lines of 'custom thumb info' are removed, one image at a time;
  • then 10 seconds to move the images;
  • then 50 seconds to rebuild the thumbnails.
If the initial folder contains more than 100 images, but only100 images are moved, this process takes even longer. Moving the actual folder, rather than a batch of images, only takes a couple of seconds.

This is repeatable under build 2828, and since updating to build 2872, on more than one folder, after database compaction, and after rebooting the PC (Windows 8.1).

It didn't used to be a problem, but it may have been been a few weeks since I last had to move files.

Not yet reported on Mantis in the hope that there's an easy fix...

[update Hert; changed title and removed “bug” qualification]

Hert
Posts: 6255
Joined: 13 Sep 03 7:24

Re: File moving slowdown

Post by Hert » 08 May 20 7:33

Though I haven’t tried it yet, I don’t think any changes were made that could impact this. Also check the what’s new page to see all changes in every build.

How you describe it your custom thumb info lines are slowing down a lot. This shouldn’t be the case, but is the only thing I can relate to “recent changes”. Recent changes were mainly about speeding up thumbnail scrolling and custom thumb info is part of thumbs.

How are you moving files. And try to disable your custom thumb lines when moving files to see if that makes a difference.
This is a User-to-User forum which means that users post questions here for other users.
Feature requests, change suggestions, or bugs can be logged in the ticketing system

Mke
Posts: 505
Joined: 15 Jun 14 15:39

Re: File moving slowdown

Post by Mke » 08 May 20 15:11

Hert wrote:
08 May 20 7:33
How are you moving files. And try to disable your custom thumb lines when moving files to see if that makes a difference.
I'm moving files by dragging and dropping in the 'folders' view.

Disabling custom thumbnails makes no difference to the timing; it's just that then nothing appears to happen for the first 2' 20".

I've also tried it with the default thumbnail lines, and that also takes 3' 20" to complete.

Is the clue in the 'building thumbnails' part? Why would it be necessary to do this (isn't it only necessary to change the reference to the existing thumbnails, rather than rebuilding them from scratch)?

From the What's New list, would there be any connection to the new "quick" verification method?

Hert
Posts: 6255
Joined: 13 Sep 03 7:24

Re: File moving slowdown

Post by Hert » 08 May 20 15:48

I just tried it here and it's rather fast. But your performance could be impacted if you have a lot of images in the active collection. Because after the drop, the active collection will be re-read from the database. Try it with a small collection and check if there's a performance difference
This is a User-to-User forum which means that users post questions here for other users.
Feature requests, change suggestions, or bugs can be logged in the ticketing system

Mke
Posts: 505
Joined: 15 Jun 14 15:39

Re: File moving slowdown

Post by Mke » 09 May 20 2:01

The times above are for moving 100 files from one directory to another empty one, so no big collection involved.

Moving 24 files (from the camera, with no additional metadata added) from one directory to another empty directory takes about 43 seconds (about 27" apparently doing nothing other than clearing thumbnail info image by image, 3" moving the files, then 13" building thumbnails).

Moving the files outside PSU takes a fraction of a second (as it should - the OS and files are all on a 1TB SSD).
Last edited by Mke on 09 May 20 2:18, edited 1 time in total.

Mke
Posts: 505
Joined: 15 Jun 14 15:39

Re: File moving slowdown

Post by Mke » 09 May 20 2:17

Just done another test on the 100 files:

Moving them within PSU still takes 3' 20";
Moving them via the OS, returning to PSU, running 'Verify Folder All', accepting and processing the results (deleting the old images from the catalogs, importing the new, including validation, synchronization and building thumbnails), takes 1' 4"

Hert
Posts: 6255
Joined: 13 Sep 03 7:24

Re: File moving slowdown

Post by Hert » 09 May 20 6:20

My point was the size of the open collection.
Moving 100 files from a set of 200 or a set of 10000 can make a difference.
Moving a file in PSU is three parts: 1. Telling the OS to move the file. 2. Updating the reference in the database to the new location (that’s milliseconds). 3. Reloading the collection
This is a User-to-User forum which means that users post questions here for other users.
Feature requests, change suggestions, or bugs can be logged in the ticketing system

Mke
Posts: 505
Joined: 15 Jun 14 15:39

Re: File moving slowdown

Post by Mke » 09 May 20 14:32

Hert wrote:
09 May 20 6:20
My point was the size of the open collection.
Moving 100 files from a set of 200 or a set of 10000 can make a difference.
Yes, I understood that; I'm moving all the files - a selection of 100 images from a folder of 100 images (and in my other example, 24 files out of a folder of 24) to a folder containing no images; that is, all the images being displayed are being moved. Or, to put is visually:

|____
......|____parent folder (0 images)
................|______source subfolder (100 images)
................|______destination subfolder (0 images)

That's why I mentioned, in my original post, that 'If the initial folder contains more than 100 images, but only 100 images are moved, this process takes even longer.'

Unless you meant something different by 'open collection'? For the avoidance of doubt, none of the images are in a 'portfolio collection'

I've also had another idea of the possible cause; could this be linked to the new algorithm that determines the need for compaction on startup? That only seems a couple of months ago(?) which could be the right kind of timescale, and I am using the SQLite version.

Mke
Posts: 505
Joined: 15 Jun 14 15:39

Re: File moving slowdown

Post by Mke » 09 May 20 15:10

Another possible clue while just running another test. The Windows 'file move popup' popped up for each of the first few images, separately for each image, one after the other, starting right from dropping the files on the new directory, and roughly corresponding to the timing of the removal of the 'custom thumb info'. I managed to catch a screenshot of one, below.
.
MovingFileDialog.png
MovingFileDialog.png (7.21 KiB) Viewed 2200 times
.
And another thought: could this be related to the change in thumbnail size introduced in build 2834? For example is it deleting and rebuilding thumbnails because of this change, rather than moving them?

Hert
Posts: 6255
Joined: 13 Sep 03 7:24

Re: File moving slowdown

Post by Hert » 11 May 20 10:44

I was able to reproduce this. And I think it's a Windows issues. The slow down is in "Calculating" as your screenshot also shows, which takes forever. If you didn't have this before then it got introduced in a recent Windows update. PSU had no changes in this area.
In build 2883, the Windows progress dialog is no longer shown and then the speed is back to normal. Please update.
This is a User-to-User forum which means that users post questions here for other users.
Feature requests, change suggestions, or bugs can be logged in the ticketing system

Mke
Posts: 505
Joined: 15 Jun 14 15:39

Re: File moving slowdown

Post by Mke » 12 May 20 16:19

Thanks Hert.

I've upgraded to v2886 and can confirm that it is now significantly quicker, at 2' 32", which is a worthwhile improvement. Though I'm sure it didn't use to take so long. I'll see if there are any other tests I can run.

Mke
Posts: 505
Joined: 15 Jun 14 15:39

Re: File moving slowdown

Post by Mke » 12 May 20 17:20

I found my copy of Version 5 build 2378, have been trying that out on the same laptop. Although with only the test folders in the catalogue.

Moving the same 100 files in the same way takes about 11 seconds in that build (about 10 seconds doing nothing obvious, and about a second to move all the files, and no apparent thumbnail building), so it doesn't seem to be a Windows issue.

I haven't kept other builds, but if you'd like to point me in their direction I can see if I can identify the build concerned.

I've arranged to borrow another laptop later, so will run a test on that too.

Hert
Posts: 6255
Joined: 13 Sep 03 7:24

Re: File moving slowdown

Post by Hert » 12 May 20 18:15

Great. Now upgrade the laptop to 2886 a see if there’s a difference.
This is a User-to-User forum which means that users post questions here for other users.
Feature requests, change suggestions, or bugs can be logged in the ticketing system

Mke
Posts: 505
Joined: 15 Jun 14 15:39

Re: File moving slowdown

Post by Mke » 12 May 20 22:40

I now have some timings for the borrowed laptop, also running Windows 8.1 with the same Windows patches installed, and moving a similar set of 100 images:

Build 2378: 11 seconds
Build 2886: 17 seconds

There's no sign of thumbnails being built on this machine in either case, which may indicate that the change of thumbnail size in build 2834 might be causing the slow-down on mine (an obvious difference between the machines is screen resolution - 1920 x 1080 on mine; 1366 x 768 on the borrowed). I'm not sure what else could explain why my machine spends so much time building thumbnails, when the lower spec machine apparently spends none?

BTW, the borrowed machine also has a lower spec slower CPU, half the RAM, and doesn't have an SSD.

Hert
Posts: 6255
Joined: 13 Sep 03 7:24

Re: File moving slowdown

Post by Hert » 13 May 20 7:13

Maybe you have a virus scanner on the main machine which is interfering?
This is a User-to-User forum which means that users post questions here for other users.
Feature requests, change suggestions, or bugs can be logged in the ticketing system

Post Reply