File server replication with Robocopy

Now it was time to get my data replicated from the old server to the new.  I considered setting up DFS and letting it copy over that way, but I’ve run into issues with that in the past and decided to go with the old reliable method: RoboCopy.  For those who may not know, it is Robust File Copy and is a replacement for Xcopy.  It has been a standard feature available in the OS since Windows Vista/Server 2008, and works a lot like rsync does in Linux.  Offering features such as the ability to copy permissions, resume interrupted transfers, mirror the source to the destination (including removing files that don’t exist in the source, but are present in the destination) and a myriad of other possibilities via command line flags.  I won’t get into everything the utility can do, but instead just point out which flags I used and why.

• /MIR = Mirror mode: Mirrors the source onto the destination (This comes in handy when your source is still in use, you can make an initial copy without the mirror flag and then perform a second copy with the mirror flag to true it up and copy any changes, deletions, files that were in use the first time, etc.
• /COPY:DATSOU = The copy flags, D for data, A for attributes, T for time stamps, S for NTFS ACLs, O for owner information, and U for audit information.  This will ensure that the file is exactly as it was in the source location.
• /MT = Multi-thread: By default it will copy 8 files at once, you can specify your own value with a number, I.e. /MT:12 or /MT:4
• /R:1 = Number of retries for failed copies… by default this is 1,000,000, so it’s a good idea to specify a value or your copy could get stuck at the first in use file or failure and just sit forever.
• /W:10 = The amount of time to wait between retries in seconds
• /TEE = Tees the output to both the terminal and the log file that you specify
• /log = Path to a log file, if you use this switch without /TEE you will get no output in the terminal.

Initially I was also using the /ZB switch, this is restart mode combined with backup mode.  If an access denied message is received it will automatically try again in backup mode with allows it to leverage the “backup right” permission to copy files that the account may not have read permissions on.  However, this was causing my initial transfer to take forever, so I opted to leave it out.  In a production environment, I would use this just to make sure there was no funny business with a failed transfer and truncated files.  At the very least you could do an initial copy without it, and then perform your true up copy with it to make sure everything is golden.

I had around 100 GB of data on the old server to move over, which didn’t take too long.  Then I also had around 1 TB of media files to move from one of my desktop PCs that had previously been running my Plex server.  I think all in all it took around 2-3 hours to get everything copied and verified.

As far as setup on the file server VM, I dedicated a 2 TB vhdx and have it mounted as the E:\ drive.  On that drive I have a Share folder which contains various documents, pictures, software install files, OS installation media, etc.  I went through and organized the structure after my copies were done… things were a little haphazard before, so I decided to break it down in a way that made more sense.  Also added a temp folder and put together a quick PowerShell script that runs as a scheduled task and cleans out any files older than 30 days. I then created a folder for Plex and copied all my media into it.  The last one is a backup folder for my Proxmox servers…. I’ll get into how my backups are setup in a later post.

The permissions aren’t anything too crazy, since it’s only myself and my wife using it.  The Plex folder is read only for standard users, the Share folder has modify permissions for authenticated users, and the VMBackups folder is only accessible to the service account I setup for backup purposes.

With the file copy done, it was time to turn off the dinosaur for good.  I saw a post from a sysadmin on a forum some time ago that said whenever he was decommissioning Windows servers, on that final shutdown when the reason dialog came up he would always type in “Thank you for your service”.  I found that rather fitting when I finally put my old server out to pasture, it certainly served me well for all those years.