transfer large amount of files

[Zack Ibraham]

I have about 3 million files and folders sized (240G) on a local PC . Once the main job is synced only a thousand or so files change every day.

What I am trying to do is to sync the entire contents to a webhost via ftp and keep it updated as the files change on my local PC.

So what I need , is to transfer only the changes that happens on the files in the same time it changed on the destination .

I tried using GoodSync for this but it takes 15 hrs to analyses and by that time a number of files have changed.

Is there any software or string of batch files that would do this for me and utilize multiple connections for a ftp username?

- Greetings
Zack

 

[Zack Ibraham]

any ideas?

 

[HelloNNNewman]

Hi Zack,

I've started replying to this a few times, but everything I look at isn't what you are describing. Most have the time issues you are experiencing with GoodSync.

Comparitech had an interesting article a while back about syncing using Box services. It covers a few different areas and may give you an idea or two (hopefully). Software like what you've tried has limitations as you have found. Dunno... there are a few things out there where people claim to do it with scripts, but those are something I can't point you to directly as they simply seem unsafe and/or not on point with what you are looking to achieve.

 

[Phillip Remaker]

Is there a way you can use something else other than FTP? I know that it'll always be fairly slow via FTP as it's hard to check so many files to see if they have changed.
Clouds like Onedrive, Google Drive , Azure blob or any of the big name offerings will be a good destinations .
And as per my experince with Onedrive , be careful using OneDrive on active files because it's designed for storage not realtime sync... If a user tries to access/change files that OneDrive is currently syncing then the file gets locked or corrupted , OneDrive should not be used in Lu of network devices as active file sharing between multiple users but if you want to sync between multiple stations for a single user i.e from desktop to laptop and back for taking work home it's very good .

Then depending on the cloud destination system you could use something like rsync , Carbonite , GsRichcopy360 or you can check again with GoodSync ,all work fine in Windows 11

 

[Zack Ibraham]

Thanks, HelloNNNewman and Phillip Remaker , Your helpful info led me to end up transferring the data with the backup tool Gs Richcopy 360 and using OneDrive as a destination. This seems to have worked

 

[zocster]

Thanks, HelloNNNewman and Phillip Remaker , Your helpful info led me to end up transferring the data with the backup tool Gs Richcopy 360 and using OneDrive as a destination. This seems to have worked

Good, thanks for keeping us updated

 

[Zack Ibraham]

Good, thanks for keeping us updated

You are welcome