Syncing files between desktop and laptop
greentooth
I'm having some trouble keeping my desktop and laptop files in sync, especially for large projects, or just large files. I like to keep a lot of undo levels in zbrush, which balloons the file size.
I'd like to find some solution that is fast and simple, even automated. Something where I get home, connect my laptop to my local network, hit a button and it syncs. I'm even willing to get a home NAS, or a router with a built in storage port.
Google Drive and dropbox work for small files, and I use those a lot for general use. But our kind of work can generate files that quickly overwhelm these services, or net connections.
My home network is a gigabit connection, so I figure transferring files locally should be easy and fast, but trying to get windows 7 to get a solid local connection for file transfer has been nothing but frustrating.
I'm technically able enough to do things like install DD-WRT on routers, but networking tech eludes me.
Right now I'm just throwing everything on a high end USB 3.0 thumbdrive, and sync using MS Synctoy. It works, but I'd like to leverage all this network tech around the house.
Any help?
I'd like to find some solution that is fast and simple, even automated. Something where I get home, connect my laptop to my local network, hit a button and it syncs. I'm even willing to get a home NAS, or a router with a built in storage port.
Google Drive and dropbox work for small files, and I use those a lot for general use. But our kind of work can generate files that quickly overwhelm these services, or net connections.
My home network is a gigabit connection, so I figure transferring files locally should be easy and fast, but trying to get windows 7 to get a solid local connection for file transfer has been nothing but frustrating.
I'm technically able enough to do things like install DD-WRT on routers, but networking tech eludes me.
Right now I'm just throwing everything on a high end USB 3.0 thumbdrive, and sync using MS Synctoy. It works, but I'd like to leverage all this network tech around the house.
Any help?
Replies
Lots of people tell me rsync, or a commercial application of it like Syncrify.
Something sorta cheap you might try that would be somewhat painless (though you still have to pay for Dropbox or Box or GDrive or whatever) is using hard symbolic links on your desktop and laptop. http://www.2brightsparks.com/freeware/freeware-hub.html
Basically, you can map a specific directory on your HD (for example, /Dropbox/work/whateverproject/) to its own drive letter. Let's say you pick a Dropbox folder you like, and map that to your J: drive on both PCs, then use that as a working directory. As you're working and saving, it's always automatically syncing to Dropbox and then back to whichever PC you're not currently using. All the drive letters and directories stay intact as well. You don't have to run any extra software or do anything special, because it's all built into Windows. It'll be invisible to you and basically just work.
If your storage starts getting full, though, you could use something like SyncBack Free (http://www.2brightsparks.com/freeware/freeware-hub.html) to automatically either sync or move items in those folders to a non-cloud-storage location on a specific timetable. I haven't checked to see if you can set up conditional rules (if total X directory size > 25gb then move to Y directory), but it's worth a look.
Hope that helps, even though it's more of a hack than a solution.
Right now I'm using GoodSync to keep my PC and Laptop in sync. Each computer has a similar setup of jobs, which are essentially rules and directions for syncing, and they also use a Synology DS213 as the NAS that they sync to, but you can use an external drive if you want. GoodSync also has options for backing up to FTP, so if you wanted to you could also sync certain files to your own online storage. This solution is pretty automated if you set the jobs/rules right. I don't really have to interact with it at all other than occasionally updating it and clearing errors that are stopping it from syncing, for instance if you set a rule that states to not sync a file that has changed more than 30%, and to wait on you to specify what to do, you're going to have to remember to go into GoodSync to clear this if a file changes more than 30%.
Also if you are experiencing slow transfers on a gigabit network, you might want to check that the ports on the router, the computers' network cards, and any hubs are all capable of gigabit speeds.
Basically I use a command line like
rsync -avz /from_dir/ /to_dir
to incrementally sync the contents of from_dir and to_dir. You can use a bash script / bat file to run that line. No servers or version control systems are needed for this. It simply synchs the files between two directories.
Best thing about rsync: It copies only files that have changed and hence is very fast.
https://en.wikipedia.org/wiki/Rsync
Rsync can also do its thing across hosts in a LAN or even over the internet:
rsync -avz /from_dir/ user@host:/to_dir
etc.
It is fast and, once you have the right command lines (script!), easy to use.
1. Keeps all my computers in sync with no tech knowledge.
2. You can access the files from the web, or with the app. (I've shown my animation reel on my iPad dozens of times.)
3. It keeps old versions of files, that you can restore or download seperatly.
4. You can share a folder with a client, so they get work delivered as you create content. (Assuming you want to do that.) And vice versa.
5. I sync my 3ds Max and Maya hotkeys between my computers auto-magically. (image below)
6. You can share images easily on Polycount, and just save the file locally for it to update on the web. (again, image below.)
I suppose you can do most of this with any syncing service. But everyone I've worked with has Dropbox.
But, it looks like people use all sorts of other options so keep the recommendations coming, it could help out others who have more specific file sync needs.
And thank you very much for the tip about syncing max and maya hotkeys. I probably could sync stuff like scripts, tools, etc. Wonder if it works for zbrush.
I mean the caveat is that your internet connection will be the bottleneck if you are off your network, but it still works.
I still use dropbox in tandem, for smaller stuff like settings, scripts etc... but I use AeroFS to sync my work onto my home fileserver.
http://aerofs.com/features
Aero looks really cool in that.. you're not paying for space? the space you have is just whatever your HD is, yeah? How does it handle if I update a file offline, but so did my friend on the other side of my team folder, once I regain internet connectivity? It honestly sounds really good for working between 1-3 PO's, where as dropbox I can access my stuff from anywhere, and invite people to share, but I'm limited in space behind a paywall.
I'm not sure how it handles that particular case, but this issue would be there with dropbox also. Once the internet at your house or wherever is working, you can access files remotely.
I even think it uses a peer to peer system, where if some other computer is connected to the internet using your same aerofs account, or part of a team it will sync via the quickest route.
You can also share folders with other people.
With the constraints mentioned by the OP, I thought AeroFS would be a good fit, and cheaper than dropbox.
http://lifehacker.com/cloudup-shares-file-collections-both-local-and-online-1029323138
nm, I didn't read far enough. Private beta, plus it doesn't appear to share locally
https://owncloud.com/
Its not as popular as DropBox (for some reason). The main reason I use this over other syncing services is that you can sync specific folders and match them to the other PCs.
5 gigs free I think
The only limiting factor should be your upload connection.
My uploads from work are SUPER fast ...
Check out AeroFS (At this point it might seem like I have something to do with the company, but I don't)
I made a mistake. I thought I had use dropbox, instead I was using Google Drive.
thanks, but I did this the moment you posted first about this. But I have to sign up with them, which kinda irks me given that everything should stay in my local lan anyway - i.e. what's the reason for the signup?
I was wondering if there's maybe a clever open source solution or something that doesn't throw me at the mercy of a company (and the Chinese government who loves to block useful sites like Dropbox for no reason). I really just want this for easy home syncing.
They do upload some stuff
https://support.aerofs.com/entries/23728686-What-information-is-stored-on-the-AeroFS-servers-
I know there are things like what you are asking for, but they tend not to be for windows thus typically over my head.
Awhile ago I had a unix box which I'd turn on and I'd backup stuff to it using rsync, which is great because it only sends changes, not the whole file again. If you could set something like that up easily you'd be set. Dunno how well any windows ports would work though. Grsync might be the first one to check.