Sync files remotely to a QNAP NAS

Sync your files remotely to a QNAP drive, wherever you are.

I work a lot with QNAP NAS drives and I enjoy the stable remote access that they provide, be it remote administration and management, or data access.
Here I will be talking about setting up your very own cloud service on your QNAP NAS drive, that will function just like Google Drive or MS Onedrive. It does take some major setting up to do, but after that - things should run smoothly.

In this scenario user data has to be available at all times and synced whenever there's an internet connection available, all data goes to the QNAP drive remotely (this provides backup and data administration by others).
We need our remote folders to be able to let us use Offline Files. So we need to set up a VPN service on the QNAP. If not the use of Offline Files (and the whole point of syncing files that are always available) - it would have been easier to simply run an FTP service from the QNAP drive, which requires less effort. But the Offline Files feature would demand that the QNAP folders be available as if we're accessing them through a LAN - hence the VPN service.

Official QNAP VPN setup instructions are found HERE.

QNAP server-side settings:

• Inside the QNAP web console, head over to Applications > VPN Service.
• You should first forward the necessary VPN ports using the Auto Router Configuration.
• QNAP provides a kind of DDNS called MyQnapCloud which gives you a free dynamic DNS if your server's public IP isn't static. You can also configure the VPN in an easy way using the myQNAPcloud feature inside the Qsync under Network Services.
• Inside Applications>VPN Service> VPN Server Settings - Check the "Enable PPTP VPN server"
• Important! Under the "VPN Client IP Pool" set the IP range, make sure you use a unique IP range not commonly used because you wouldn't want the VPN IP to clash with the local network IP of the client computer.

Client-side (Windows 7/Windows 8/Windows 8.1) settings:

• You can install the myQNAPcloud Connect utility which will build the VPN dialup connection for you, or you can manually set it up yourself. I simply installed the utility to let it setup the VPN connection and then removed the utility from start up. The utility is simple, however it will not automatically connect your VPN if you're on a wifi network, and it will not automatically reconnect if the internet connection was disrupted. We are going to fix this flaw by doing some manual settings and powershell scripting.
• In case you decide to build the VPN dialup connection yourself - here are the settings:

1. Inside Network and Sharing Center click "Set up a new connection or Network",

2. Choose "Connect to a workplace"

3. Choose "Use my Internet connection (VPN)"

4. Inside Internet address - insert the public IP of the network on which your NAS is located. Or if you have a DDNS by QNAP or third party - put it here. 

5. Check "Don't connect now..."

6. Insert your VPN username and password (usually it's the main NAS admin by default, but you can add VPN users inside the QNAP VPN Service page inside the web console)

7. Click Create but don't connect yet. Click Close.

8. Inside Network and Sharing center click on "Change Adapter settings".

9. Right click your newly created VPN connection icon and choose Properties.

10. Under the Security tab - make sure Type of VPN is PPTP, Data Encrytpion is set to Optional, and protocols PAP, CHAP and MS-CHAP v2 are enabled. 

11. Under the Networking tab - make sure that only IPv4 is checked, double click on it, choose Advanced, and uncheck the "Use the Default Gateway on remote network" to not use the gateway of the VPN for your internet traffic. 

12. OK all of the windows of the connection properties. 

Now we will write a Powershell script that will run in the background and check for connection drops and reconnect if necessary. 

• First let's create a folder on the C: drive and call it Script.
• Let's create a file in that folder and call it vpn.ps1
• Inside the file let's put the following code (using Notepad or another text editor):

$ip = "10.0.20.10"
$result = gwmi -query "SELECT * FROM Win32_PingStatus WHERE Address = '$ip'"
if ($result.StatusCode -eq 0) {
 Write-Host "$ip is up."
}
else{
 Write-Host "$ip is down."
 Write-Host "Disconnecting..."
 rasdial.exe YourVPN /DISCONNECT Write-Host "Connecting..."
 rasdial.exe YourVPN vpnUsername vpnPassword12345 
}

Make sure you put in the correct IP of the NAS (first line of the script) while it's inside the VPN. If your VPN IP pool starts from say 10.0.10.2 that means the server address is 10.0.10.1

Also make sure that after rasdial.exe you put the name of your VPN dial-up connection, its username and password (where it's stated).

Now we fire up powershell (just type powershell inside the Start menu search and run it in elevated mode), and we need to set it to accept running scripts. First, lets run this command:

get-executionpolicy

If the answer is "Restricted", run this command:

set-executionpolicy unrestricted

This will ask you to confirm, press Y.

After this is done, all that's left is setting up the task scheduler to run this script.

Inside Task Scheduler create a New Task, make sure that it runs with highest privileges and with the user logged on or not. The Trigger should be at Startup, Repeat the task every 5 minutes. Under Actions leave it as Start a program, point the path to C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
add an argument to run the script:
-File C:\Script\vpn.ps1

For the method of a stable VPN connection I used the tutorial posted here.

Now, after we're done setting up a stable VPN connection - it's time to enable Offline Files. (Offline Files are available only on Professional, Ultimate and Enterprise versions of Windows)
Browse to your server using its VPN IP, find the folder you'd like to sync, right click it and choose "always available offline" . It is best to create a mapped drive for this folder, so it's easily accessible when offline. Also make sure no one else is using this shared folder because sync conflicts may happen.
In Windows 8 Pro you might first have to enable Offline Files by going to Control Panel > Sync Center, click on Manage Offline Files, and then Enable Offline Files.

Sync network folders to OneDrive automatically

A few weeks ago I have written a post about synchronizing your OneDrive local folder to a network folder, in this post, however, we will talk about synchronizing server folders to the OneDrive folder automatically using a Robocopy script.

First off let's create sub-folders inside your OneDrive sync folder that will represent your network shares. For example if we have a server named Server and shared folders on it named Folder1 and Folder2 - we shall create them as subfolders inside our OneDrive sync folder, as:
(assuming the system partition is C:)

C:\Users\username\OneDrive\Folder1 (and other shared folders)
or
C:\Users\username\OneDrive for Business\Folder1 (and other shared folders)

Then, after all of our subfolders are created, we create a batch script containing:

@echo off

 robocopy "Source network folder" "Destination OneDrive Sub-folder" /MIR /COPY:DT /MON:1

We will have to create a Robocopy command like this for each one of the network folders that needs to be synced with OneDrive.

If the server requires credentials, you will have to add a line before the Robocopy command:

net use \\servername /USER:username password

After the Robocopy command you might want the script to log out, using the following command:

net use \\servername /d

This is a syncing script, that's why we're using the /MIR switch, this means that if we delete the file in the source folder, the file will soon be deleted in the destination folder. If you wish to copy files and not sync them, you can use /E /COPY (or /COPY:DT) instead of /MIR.

I also should mention that /COPY:DT switch in the above script is optional, it instructs Robocopy to preserve the files' Date and Time stamps but not attributes. I just prefer using it in this situation for better stability.

Now we can save the script as a BAT or a CMD file and run it (preferably in elevated mode) to see how it works. The script will never close unless the command prompt is closed manually, so this may create an annoyance to the user. That's why it may be a good idea to automate it's execution and keep it hidden from the user.

To do that we go into Task Scheduler and create a task to run this script. I recommend the trigger being the user's logon. Because if the trigger is a certain time of day - there may be duplicates of the same process, because the process never stops anyway.

Make sure you set the SYSTEM account as the account running the task, and mark it Hidden. This will allow the script to run invisibly in the background. This starts the cmd.exe and robocopy.exe processes and they only take a few hundred kilobytes in memory.

Sync files to OneDrive and a local server together automatically.

I have always thought that keeping valuable data in more than one place is essential to productivity. So it's always best practice to sync your files to an external location, especially in the work environment. A good stable sync is what we will talk about in this post.

Recently, I migrated one of the companies that I work for to Office 365. As some of you may know - Office 365 packages come with the OneDrive service that provides 25GB of online storage.
So up until the migration - the users' data was automatically synchronized to the in-house file server (using Offline Files). There is a great many ways to synchronize data from a user's machine to a network location - from using Offline Files, to scripting, to third party software. In this scenario I will talk about synchronizing to both a local network location and a cloud service (OneDrive for Business in this case) without using any third party software.

First, let me point out that OneDrive currently provides one sync folder on the computer where everything that you want synced has to go. You cannot designate other folders to be synchronized to OneDrive unless you copy them to the sync folder and continue to work from there. So let's say you do that but you'd still want the same data synchronized to your local file server. And you don't want to use third party software that may create system instability, or load it up.
So, we're going to accomplish this task by using a Robocopy monitoring script that will be running invisibly in the background. All the time.

So we create a batch script that will contain the following:

@echo off
robocopy "Source OneDrive Folder" "Destination network folder" /MIR /COPY:DT /MON:1

Now (assuming your system partition is C:), your source OneDrive folder is usually located at

C:\Users\username\OneDrive
or
C:\Users\username\OneDrive for Business

Your destination should be a local folder or a network share. Don't forget the quotation marks.

You might also need your shared folder credentials if the share is password protected, so to log into the share correctly the script would have to present the correct credentials. To do that you will have to add a line that logs into the share (before the Robocopy command):

net use \\servername /USER:username password

After the Robocopy command you might want the script to log out, using the following command:

net use \\servername /d

This is a syncing script, that's why we're using the /MIR switch, this means that if we delete the file in the source folder, the file will soon be deleted in the destination folder. If you wish to copy files and not sync them, you can use /E /COPY (or /COPY:DT) instead of /MIR.

I also should mention that /COPY:DT switch in the above script is optional, it instructs Robocopy to preserve the files' Date and Time stamps but not attributes. I just prefer using it in this situation for better stability.

Now we can save the script as a BAT or a CMD file and run it (preferably in elevated mode) to see how it works. The script will never close unless the command prompt is closed manually, so this may create an annoyance to the user. That's why it may be a good idea to automate it's execution and keep it hidden from the user.

To do that we go into Task Scheduler and create a task to run this script. I recommend the trigger being the user's logon. Because if the trigger is a certain time of day - there may be duplicates of the same process, because the process never stops anyway.

Make sure you set the SYSTEM account as the account running the task, and mark it Hidden. This will allow the script to run invisibly in the background. This starts the cmd.exe and robocopy.exe processes and they only take a few hundred kilobytes in memory.

You can also check out my post on how to sync network shares to OneDrive automatically.

Use an automated backup script that will check your backup drive letter.

I have tried many different tools for automated backup, both third party and the ones built into windows, but the best one by far is Robocopy. It is the most robust, stable and reliable tool that you'll ever find to automate backups both in the enterprise environment and at home.

I use Robocopy scripts to mainly create backups onto an external USB drive that can be routinely taken offsite and switched with another external drive.
The biggest problem when using automated backup onto an external drive is the fact that the letter of the drive may change at some point if the drive was unplugged and then re-plugged. This causes your backup procedure to miss the target drive and backup will not run.

Here's an example of a script that overcomes the problem of the changing drive letter.

In this example we run a mirror backup of a server folder onto an external backup drive, and the script will find out whether the backup drive is using drive letter G: or H:

@echo off

IF exist G:\ident.txt (goto run1) ELSE goto run2

 :run1

 robocopy "\\Server\Administration" "G:\Backup\Administration" /MIR /W:0 /R:0 /XF:SHT /LOG:backuplogr.txt /NS /NC /NFL /NDL /NP

 :run2

 robocopy "\\Server\Administration" "H:\Backup\Administration" /MIR /W:0 /R:0 /XF:SHT /LOG:backuplogr.txt /NS /NC /NFL /NDL /NP

We start by creating a procedure that will check for a file called "ident.txt" (could have any other name or extention) - this is an empty file you should create and put in the root folder of your backup drive.
So if the file is present on drive G: - we point the script to a label called "run1" which runs the backup to drive G:. If the file is not present on drive G: (or if there IS no drive G:) we bounce the process to the label called "run2" which will then run the backup onto drive H:.

Robocopy then starts backing up a shared folder called "Administration" located on a server named "Server" onto the the external USB drive.