Two or more metadata edits in quick succession possibly causing OneDrive sync erors

[barneagle]

The photos in my Catalog are stored in a OneDrive folder on my PC that is automatically backed up in the cloud. For several years OneDrive has reported sync errors from time to time, always on photo files. So far as I'm aware there have never been sync errors on any other kind of file, so I wonder whether the problem might relate to the way I use Photo Supreme.

Yesterday, I noticed a sync error on a photo shortly after I had been working on it. I had first applied a Rating and then, shortly after, an additional Label. The photo folder now has two copies of the photo: File A (which retains the original file name) reflects the rating, and File B (with my computer's name appended by OneDrive to the file name) reflecting both the rating and the additional label.

The "XMP:MetaData Date" tag in the two files indicates that the rating was applied about two and half minutes before the additional label:
File A: MetaData Date: 2022:06:20 16:11:00.906Z
File B: MetaData Date: 2022:06:20 16:13:37.902Z

However, Windows file system data suggests that File A was modified only about 4 seconds before File B
File A: File Modification Date/Time: 2022:06:20 16:13:33+01:00
File B: File Modification Date/Time: 2022:06:20 16:13:37+01:00

I wonder whether the OneDrive sync error might have resulted because OneDrive picked up the two file system changes before attempting to sync either of them with the online copy, and perhaps it then tried to sync them in the wrong order.

While thinking about this I've remembered that there is a Preference for "Automatically write out Catalog changes to the image file". I've always had this on. I wonder if the sync problem might be solved if I turn this off and then, in future, use the "Save Metadata to File" (Ctrl-S) option manually after I've finished working on the photo.

[barneagle]

Unfortunately, after changing the Preference so that the image file is only saved when I manually select the Save Metadata to File option, I'm still getting OneDrive sync conflicts. So, my theory that edits in quick succession might be causing the problem is wrong.

I'm stumped as to what's causing these OneDrive errors.

[Hert]

Best to ask OneDrive because if there's one file you would expect OneDrive to sync one file.
PSU has no control over how OneDrive operates.

[gcoupe]

I realise this is an old thread, but after doing some major updating of my photo metadata in Photo Supreme, I've run into the same problem.

I think the only workaround is to pause OneDrive's automatic sync (you get a choice of 2/8/24 hours) while using Photo Supreme, and then restarting OneDrive sync once you've finished with using Photo Supreme.

[gcoupe]

Hmm - I tried the workaround of pausing OneDrive whilst I used Photo Supreme - and it didn't work. I added a label to 121 images, and 33 of them got created as a copy with the PC name appended...

Microsoft strikes again...

[Username]

I believe that Onedrive locks the file locally before it does it syncing to ensure they done change during.
This was the behavior of one drive a few years ago and I don't think they have changed their way of doing it.

[barneagle]

I posted the original question. My workaround was to move the photos folder used by PSu out of OneDrive (to a subfolder of the C drive) and to use separate tools to sync to a copy on OneDrive.

[Off topic: gcoupe, it was in one of your blogs that I first saw mention of Photo Supreme. Thank you for recommending it!]

[gcoupe]

This issue is still giving me problems. I've now tried @barneagle 's workaround by moving the folder hierarchy containing my photos to a staging folder outside of OneDrive, and then will sync this with the copy in OneDrive.

However, I'm not convinced that this will solve the problem. I suspect that the issue is being caused by too many changes in the filesystem being made at once, and that this overwhelms the OneDrive synchronisation process. My last metadata change before setting up the staging folder has caused changes to about 30,000 photos. OneDrive is now trying to sync these photos to the cloud and choking with two things:
1. The sync process frequently stops working. I have to pause OneDrive and then restart it to kick it back into action
2. OneDrive frequently can't merge the changed file with its twin in the cloud, so creates a new copy of the file with the PC name appended to the filename. This then gets downloaded to my PC by the sync process. My photo library is being polluted with copies (containing the changes to the metadata), meanwhile the originals do not seem to have the latest metadata changes (is OneDrive overwriting these with older files?).

I've reported this to OneDrive Support, and I have a telephone call (and presumably an online sharing system) booked with them next week. We'll see what happens then, but I don't have much confidence that anything will be done.

There's another issue with OneDrive that I recently discovered; OneDrive no longer searches photo tags, although it continues to claim that it does. See https://gcoupe.wordpress.com/2024/10/05 ... ps-back-3/

[gcoupe]

Well, the telephone support was a success inasmuch that I was able to demonstrate the problems to the support person. He first pointed me to this troubleshooting page: https://support.microsoft.com/en-us/top ... 1e1a3cf613 However, it was not relevant because my PC has no “OneDrive Cached Credentials” listed.

The next step is to delete the sign in settings and re-create them:

• Open Start > Run > and paste %localappdata%\Microsoft\OneDrive\settings
o Delete the file PreSignInSettingsConfig.json in %localappdata%\Microsoft\OneDrive\settings
• Restart your PC and try to sign into OneDrive again that will recreate the credentials from scratch.

Will try this, but not convinced that the issues will go away. Watch this space.

[gcoupe]

Just an update to this saga. I followed the suggestions above, but synchronising a folder with multiple changed files still caused the creation of false duplicate files. As a result, I've repeated the scenario and sent OneDrive and Fiddler logs to OneDrive Support at their request for further analysis. Fiddler is an application that records the OneDrive HTTPS traffic.

In this last synchronisation session, 1,050 files were changed and synchronised with the OneDrive cloud. There were 168 failures, where instead of the cloud file being updated to the new version, OneDrive created a new duplicate file alongside of the old version of the cloud file.

[Username]

Thanks for your updates on the matter.

Wonder if Onedrive looks at some different hashes or dates/times regarding modification and creation of files which is causing this?

[gcoupe]

I strongly believe that there is some sort of race condition occurring in the sync process when files are replaced, since the failures are random in nature and number. I've seen it happen with a batch of as few as 37 replacement files in a batch, and had successful syncs with batches of 100+ replacement files.

There is supposed to be an investigation of my case being carried out in Microsoft's Tier 3 Support, but then again I was promised a contact from them on November 28th with any updates. It's now December 2nd and I've not heard anything from them.

[noramila]

Hello,
I’ve faced this issue too. Using a tool like FreeFileSync for manual syncing helped me avoid duplicates with OneDrive.

Regards,
Noramila.