Lead Image © rawintanpin, 123RF.com

Lead Image © rawintanpin, 123RF.com

Integrate remote cloud storage

Easy Access

Article from ADMIN 83/2024
By
You don't need native clients for every single service just to back up or synchronize your data in the cloud; Rclone helps you handle these tasks for multiple cloud accounts at the command line or in a graphical front end.

Often, users fail to back up their data regularly and synchronize backups. Of course, any data you store locally should also be stored on a remote mass storage device. If the local server or network-attached storage (NAS) is stolen or destroyed by natural events (e.g., floods, fires), the backups they store will also be lost. Because most companies and many private individuals now use cloud storage, it makes sense to transfer backups to the cloud, and Rclone [1] is the ideal solution.

Strategy

The standard command-line tool for backing up data on remote computers on the intranet is Rsync, but Rclone is far better suited for backing up data to the cloud. The learning curve is manageable because the tool is based on Rsync syntax (Table 1). Rclone can be easily installed on most popular distributions with built-in package managers.

Table 1

Rsync vs. Rclone

Feature Rsync Rclone
License GPLv3 MIT
Intended use Data backup between two computers or servers Data backup to the cloud, treating cloud storage like local drives
Backup type Unidirectional Unidirectional
Threads Single-threaded Multithreaded
Transfer method Single file blocks Complete files
Integrity check Yes Yes
Support for cloud services No Yes
Mount target drives No Yes

The latest versions are also available for download on the project's website for other operating systems, including various BSD derivatives and Solaris. Rclone is controlled from the command line, but the developers have also provided a graphical front end [2], although it is still in an experimental phase, so you might want to avoid using it in production environments.

The application was largely developed in the Go programming language and is available under an MIT license. Rclone can back up and synchronize entire databases, as well as individual files and directories, in the cloud, and you can encrypt and synchronize backups between different cloud services. Rclone supports more than 70 cloud providers, as well as individual systems such as on-premises SFTP servers.

For each supported provider, the developers have detailed individual configuration instructions [3] so that even beginners can set up the tool for use with an existing cloud account. Rclone can also use a cron job to synchronize data automatically, making sure your backups are always up to date.

Installation

The easiest way to install Rclone is to use the software repositories provided by your chosen distribution. If you only find an older version there, simply retrieve Rclone from the project's GitHub page. You can also use the script to install it on your system, for which you need curl. To install curl, integrate Rclone into your system, and start the basic configuration in a wizard, enter:

sudo apt install curl
$ curl https://rclone.org/install.sh | sudo bash
$ rclone config

The first step in the wizard asks for the remote (i.e., the target system). Pressing n opens the dialog for configuring a new cloud service. You will see a list of storage services, each of which has its own number. Simply enter the number that matches your provider.

The script then requests the credentials for the service, which can be location information or authentication data. The wizard does not display the password, but you do need to type it in a second time to confirm. In the next step, the access credentials you entered are displayed and the wizard prompts you to confirm. If you make a mistake, you can edit the entry by pressing e or delete it completely by pressing d . If the data is correct, press y . The wizard now displays the external service along with a selection of options, including an option for setting up another service (Figure 1). When done, press q to quit the configuration wizard.

Figure 1: The configuration wizard runs in a terminal window.

Alternatively, you can install Rclone by downloading the package from the website and using your distribution's software manager to install. RPM and DEB packages for 32- and 64-bit Linux systems are available on the website, along with packages for various versions of the ARM architecture. A ZIP archive [4] for Linux is also available that contains a generic binary package of the software. Simply unzip the file in a directory of your choice, change to that directory, and call up the configuration wizard as described above to configure the basic settings. The software is then ready for use.

If you already have an older version of the program in place, you can update it by running the rclone selfupdate command. If required, you can use the rclone version command to query the currently installed version (Figure 2).

Figure 2: Update the program with just a few commands.

For an overview of the software's numerous functions and parameters, type rclone -h at the prompt. Detailed help is available for every option, which you can access with the

rclone <option> --help

command. You can also combine parameters.

Network Drive

To set up cloud storage for this scenario, begin by creating a new directory in your personal folder before mounting your cloud storage on your system like a conventional network drive:

$ rclone --vfs-cache-mode writes mount <service> ~/<folder>

The <service> parameter refers to the cloud service you defined in the Rclone configuration phase, and <folder> refers to the newly created target folder. Please note that the spelling of the service name when entering the command must be exactly the same as when configuring Rclone.

Depending on your Internet access speed and the volume of data to be transferred, the complete folder structure of the cloud storage will appear in the target folder on your system after a wait (Figure 3). You now have full access to all content stored in the cloud, provided you also granted all rights for this service in the Rclone configuration.

Figure 3: Rclone mounts cloud storage on your system like a network drive.

In your desktop's file manager, you can then use the cloud directory and its subfolders in the same way as local directories. Copying files from the cloud to the local system is also a painless experience, but note that content, particularly larger documents such as PDF files, will take far longer to load than local files.

Also note that you do need to re-enter the command for mounting your cloud storage as a network drive every time your restart the system. It makes sense to create a script to integrate cloud storage automatically after a restart.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy ADMIN Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • ownCloud

    ownCloud is an alternative to the many online storage services whose data protection promises could turn out to be as nebulous as their offerings.

  • Google Cloud Storage for backups
    We compare Google Cloud Storage for Internet-based backups with Amazon S3.
  • Nine home clouds compared
    Dropbox was the first of a number of cloud service providers. However, only services that promise full control over your own data can give users a feeling of security. We provide an overview of nine cloud projects and two BitTorrent tools.
  • Synchronize passwords in KeePass
    Usernames and passwords play an important role in security. In this article, we show you how to set up the KeePass password manager and keep it synchronized across multiple devices.
  • How to back up in the cloud
    In cloud computing practice, backups are important in several ways: Customers want to secure their data, and vendors want to secure the essential details of their platforms. Rescue yourself, if you can.
comments powered by Disqus