How to Use rclone Command
Rclone is a command-line tool for copying, syncing, managing, and migrating files and directories to a remote server, cloud storage, or other storage. It is available for Linux, Windows, macOS, Debian, OpenBSD, and several other operating systems. Rclone currently supports over 40 cloud storage providers, including AWS, Google Cloud, Google Drive, Dropbox, pCloud, Zoho, Mega, and more. It also supports the SSH and SFTP protocols to sync files between Linux servers.
Rclone uses server-side transfers to minimize local bandwidth usage and transfers data between providers without using the local disk. Additionally, Rclone allows you to mount any local or cloud filesystem as a disk on Linux and Windows.
This guide will show you how to copy and sync files and directories to a remote Linux server using Rclone over the SSH protocol.
Installing Rclone
Rclone is included in the default Ubuntu repository. You can install it using the following command:
apt install rclone -y
After installing Rclone, verify the Rclone version with the following command:
rclone --version
Sample output:
rclone v1.50.2 - os/arch: linux/amd64 - go version: go1.13.6
Run the following command to view all available options with Rclone:
rclone --help
Sample output:
Available Commands: about Get quota information from the remote. authorize Remote authorization. cachestats Print cache stats for a remote cat Concatenates any files and sends them to stdout. check Checks the files in the source and destination match. cleanup Clean up the remote if possible config Enter an interactive configuration session. copy Copy files from source to dest, skipping already copied copyto Copy files from source to dest, skipping already copied copyurl Copy url content to dest. cryptcheck Cryptcheck checks the integrity of a crypted remote. cryptdecode Cryptdecode returns unencrypted file names. dbhashsum Produces a Dropbox hash file for all the objects in the path. dedupe Interactively find duplicate files and delete/rename them. delete Remove the contents of path. deletefile Remove a single file from remote. genautocomplete Output completion script for a given shell. gendocs Output markdown docs for rclone to the directory supplied. hashsum Produces an hashsum file for all the objects in the path. help Show help for rclone commands, flags and backends. link Generate public link to file/folder. listremotes List all the remotes in the config file. ls List the objects in the path with size and path. lsd List all directories/containers/buckets in the path. lsf List directories and objects in remote:path formatted for parsing lsjson List directories and objects in the path in JSON format. lsl List the objects in path with modification time, size and path. md5sum Produces an md5sum file for all the objects in the path. mkdir Make the path if it doesn't already exist. mount Mount the remote as file system on a mountpoint. move Move files from source to dest. moveto Move file or directory from source to dest. ncdu Explore a remote with a text based user interface. obscure Obscure password for use in the rclone.conf purge Remove the path and all of its contents. rc Run a command against a running rclone. rcat Copies standard input to file on remote. rcd Run rclone listening to remote control commands only. rmdir Remove the path if empty. rmdirs Remove empty directories under the path. serve Serve a remote over a protocol. settier Changes storage class/tier of objects in remote. sha1sum Produces an sha1sum file for all the objects in the path. size Prints the total size and number of objects in remote:path. sync Make source and dest identical, modifying destination only. touch Create new file or change file modification time. tree List the contents of the remote in a tree like fashion. version Show the version number.
Configuring Rclone
To define a storage provider or remote SSH server, you will need to configure Rclone. In this section, we will configure Rclone to use a remote SSH server as a backup server.
Start the rclone configuration by running the following command:
rclone config
You should see the following options:
No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n
Type n and hit Enter to create a new remote server.
name> remote-ssh
Please enter the name of your remote backup server and press Enter. You will then be prompted to choose from a list of storage providers.
Type of storage to configure. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value 1 / 1Fichier \ "fichier" 2 / Alias for an existing remote \ "alias" 3 / Amazon Drive \ "amazon cloud drive" 4 / Amazon S3 Compliant Storage Provider (AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, etc) \ "s3" 5 / Backblaze B2 \ "b2" 6 / Box \ "box" 7 / Cache a remote \ "cache" 8 / Citrix Sharefile \ "sharefile" 9 / Dropbox \ "dropbox" 10 / Encrypt/Decrypt a remote \ "crypt" 11 / FTP Connection \ "ftp" 12 / Google Cloud Storage (this is not Google Drive) \ "google cloud storage" 13 / Google Drive \ "drive" 14 / Google Photos \ "google photos" 15 / Hubic \ "hubic" 16 / JottaCloud \ "jottacloud" 17 / Koofr \ "koofr" 18 / Local Disk \ "local" 19 / Mail.ru Cloud \ "mailru" 20 / Microsoft Azure Blob Storage \ "azureblob" 21 / Microsoft OneDrive \ "onedrive" 22 / OpenDrive \ "opendrive" 23 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH) \ "swift" 24 / Pcloud \ "pcloud" 25 / Put.io \ "putio" 26 / SSH/SFTP Connection \ "sftp" 27 / Transparently chunk/split large files \ "chunker" 28 / Union merges the contents of several remotes \ "union" 29 / Webdav \ "webdav" 30 / Yandex Disk \ "yandex" 31 / http Connection \ "http" 32 / premiumize.me \ "premiumizeme" Storage> 26
Type 26 and press Enter to connect to the remote SSH server. You should see the following output:
SSH host to connect to Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value 1 / Connect to example.com \ "example.com" host> remote-ssh-server-ip SSH username, leave blank for current username, root Enter a string value. Press Enter for the default (""). user> root SSH port, leave blank to use default (22) Enter a string value. Press Enter for the default (""). port> 22 SSH password, leave blank to use ssh-agent. y) Yes type in my own password g) Generate random password n) No leave this optional password blank y/g/n> y Enter the password: password: Confirm the password: password:
Please enter the IP address, port number, username, and password of the remote SSH server. You should see the following options:
Path to PEM-encoded private key file, leave blank or set key-use-agent to use ssh-agent. Enter a string value. Press Enter for the default (""). key_file>
Just press Enter to use the default option.
The passphrase to decrypt the PEM-encoded private key file. Only PEM encrypted key files (old OpenSSH format) are supported. Encrypted keys in the new OpenSSH format can't be used. y) Yes type in my own password g) Generate random password n) No leave this optional password blank y/g/n> y
Type ‘y’ and hit Enter. You will be asked to set the SSH passphrase.
Enter the password: password: Confirm the password: password:
Define your password and press Enter:
When set forces the usage of the ssh-agent. When key-file is also set, the ".pub" file of the specified key-file is read and only the associated key is requested from the ssh-agent. This allows to avoid `Too many authentication failures for *username*` errors when the ssh-agent contains many keys. Enter a boolean value (true or false). Press Enter for the default ("false"). key_use_agent>
Press Enter for the default:
Enable the use of insecure ciphers and key exchange methods. Those algorithms are insecure and may allow plaintext data to be recovered by an attacker. Enter a boolean value (true or false). Press Enter for the default ("false"). Choose a number from below, or type in your own value 1 / Use default Cipher list. \ "false" 2 / Enables the use of the aes128-cbc cipher and diffie-hellman-group-exchange-sha256, diffie-hellman-group-exchange-sha1 key exchange. \ "true" use_insecure_cipher> 1
Type 1 and hit Enter:
Disable the execution of SSH commands to determine if remote file hashing is available. Leave blank or set to false to enable hashing (recommended), set to true to disable hashing. Enter a boolean value (true or false). Press Enter for the default ("false"). disable_hashcheck> Edit advanced config? (y/n) y) Yes n) No y/n> n
Type n and hit Enter:
Remote config -------------------- [remote-ssh] host = 45.58.42.69 user = root port = 22 pass = *** ENCRYPTED *** key_file_pass = *** ENCRYPTED *** use_insecure_cipher = false -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y
If everything is fine, type y and hit Enter to finish the setup:
Current remotes: Name Type ==== ==== remote-ssh sftp e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config
To exit the configuration wizard, press “q”.
At this point, Rclone is configured to sync and copy files to a remote server named “remote-ssh” via SSH.
How to Use Rclone
First, check all directories on the remote filesystem with the following command:
rclone lsd remote-ssh:/
Sample output:
-1 2021-07-08 07:23:38 -1 bin -1 2020-05-14 03:35:45 -1 boot -1 2021-07-08 07:23:34 -1 dev -1 2021-07-08 07:23:18 -1 etc -1 2020-05-14 03:38:40 -1 home -1 2020-05-15 20:07:11 -1 lib -1 2020-04-23 07:34:14 -1 lib32 -1 2020-05-14 03:29:00 -1 lib64 -1 2020-04-23 07:34:14 -1 libx32 -1 2020-05-14 03:28:53 -1 lost+found -1 2020-05-14 03:29:01 -1 media -1 2020-04-23 07:34:19 -1 mnt -1 2020-04-23 07:34:19 -1 opt -1 2021-07-08 07:23:25 -1 proc -1 2020-05-15 20:12:47 -1 root -1 2021-07-08 07:43:21 -1 run -1 2020-05-14 03:36:23 -1 sbin -1 2020-04-23 07:34:19 -1 srv -1 2021-07-08 07:23:26 -1 sys -1 2021-07-08 07:23:38 -1 tmp -1 2020-05-14 03:29:00 -1 usr -1 2020-05-14 03:29:00 -1 var
To check the size of a specific remote directory, run the following command:
rclone size remote-ssh:/etc
Sample output:
Total objects: 759 Total size: 26.654 MBytes (27948477 Bytes)
To list files and directories on the remote server in a tree format, run the following command:
rclone tree remote-ssh:/etc/default/
Sample output:
/ ├── acpid ├── bsdmainutils ├── console-setup ├── cron ├── dbus ├── grub ├── grub.d │ └── init-select.cfg ├── keyboard ├── locale ├── motd-news ├── networkd-dispatcher ├── nss ├── ntp ├── ssh └── useradd
To create a backup directory on the remote system, run the following command:
rclone mkdir remote-ssh:/backup
To copy all files and directories from the /boot/grub directory on the local system to the backup directory on the remote system, run the following command:
rclone copy /boot/grub remote-ssh:/backup/
To copy all files and directories from the backup directory on the remote system to the mnt directory on the local system, run:
rclone copy remote-ssh:/backup/ /mnt/
To remove a single file on a remote ssh server, run:
rclone deletefile remote-ssh:/backup/grub.cfg
To remove all the content of the remote directory, run:
rclone delete remote-ssh:/backup/i386-pc
To remove an empty directory on the remote system, run:
rclone remove remote-ssh:/backup/directory
To move files and directories from the local system to the remote system, run:
rclone move /mnt remote-ssh:/backup/
To display the content of the remote file, run:
rclone cat remote-ssh:/etc/fstab
Sample output:
# /etc/fstab: static file system information. # Use 'blkid' to print the universally unique identifier for a # device; this may be used with UUID= as a more robust way to name devices # that works even if disks are added and removed. See fstab(5). # <file system> <mount point> <type> <options> <dump> <pass> # / was on /dev/sda1 during installation UUID=29a0b164-1ba1-45a7-b23a-cdb98f23edbc / ext4 noatime,errors=remount-ro 0 1 /swapfile none swap sw 0 0
To synchronize the local directory with the remote directory, run:
rclone sync /mnt remote-ssh:/backup
To synchronize the remote directory with the local directory, run:
rclone sync remote-ssh:/backup /mnt
Conclusion
The above guide explains how to copy, move, and synchronize files and directories to a remote server using Rclone over the SSH protocol. With Rclone, you can now sync your local files and directories with Google Drive, Amazon S3, or other storage providers.