scp works in pretty much the same way as normal
cp, only it allows you to copy files between different machines.
user123@home_PC:~$ scp -r my_dir firstname.lastname@example.org: # copy directory "my_dir" from user's PC to a home directory on skirit
user123@home_PC:~$ scp -r email@example.com:~/results . # from user123's home on skirit, copy to user123's PC folder "results"
Alternative way to download data is a
wget will work only if the file is available via ftp or http(s) protocols, typically when it is a downloadable file on some server.
wget is faster and less safe than
scp, so it may be a method of choice if you need to download larger amount of data from a server where privacy is not an issue.
ssh firstname.lastname@example.org # login to a frontend
email@example.com:~$ mkdir data && cd data
firstname.lastname@example.org:~/data$ wget https://www.someServer.org/someData.zip # download file someData.zip from server https://www.someServer.org
wget you can only transfer data to Metacentrum machines.
wget is of no use if you want to transfer from Metacentrum.
sftp is just another protocol for data transfer. Contrary to
scp it is interactive and apart from copying it also enables the user to manipulate files and directories on the remote side. We recommend to use
scp if you need only to copy the data.
Windows users alert
Windows users need an SFTP client, we recommend the WinSCP application. Keep in mind you have to fill in as target chosen NFS4 server instead of frontend in Step 1. Make sure you have selected SFTP file protocol, too.
sftp user123@target_NFS4_server# Login
helpList available commands
get target_fileDownload target file to your local system
get -r target_directoryDownload target directory to your local system
put target_fileUpload target file to server
put -r target_directoryUpload target directory to server
There is a bug affecting Ubuntu 14.04+ concerning the recursive copy command. If
put -r fails, create the target directory on the server first to work around this issue.
rsync command is a more advanced and versatile copying tool. It enables the user to synchronize the content of two directories in a more efficient way than scp, because
rsync copies only the differences between the directories. Therefore it is often used as a tool for regular backups.
Copy directory data to archive:
$ rsync -zvh /storage/brno2/home/user123/data /storage/du-cesnet/home/user123/VO_metacentrum-tape_tape
Copy only the content of directory data to archive:
$ rsync -zvh /storage/brno2/home/user123/data/ /storage/du-cesnet/home/user123/VO_metacentrum-tape_tape
tar (tape archiver) command enables to pack files and directories into one archive file.
tar by itself does not compress the size of the files, and the resulting volume of the packed archive is roughly the same as the sum of the volumes of individual files.
tar is often used together with commands for file compression like
/storage/du-cesnet/home/USER/VO_metacentrum-tape_tape-archive, create uncompressed archive of the directory
tar cvf /storage/du-cesnet/home/USER/VO_metacentrum-tape_tape-archive/my-archive.tgz ~/my-archive
/storage/du-cesnet/home/USER/VO_metacentrum-tape_tape-archive, create archive of the directory
~/my-archive and compress it by
tar czvf /storage/du-cesnet/home/USER/VO_metacentrum-tape_tape-archive/my-archive.tgz ~/my-archive
List the content of the existing archive:
tar tzf /storage/du-cesnet/home/USER/VO_metacentrum-tape_tape-archive/my-archive.tgz
Unpack the whole archive
my-archive.tgz residing in
storage/du-cesnet/home/USER/VO_metacentrum-tape_tape-archive/ into current directory:
tar xzvf /storage/du-cesnet/home/USER/VO_metacentrum-tape_tape-archive/my-archive.tgz
Unpack part of the archive:
tar tzf /storage/du-cesnet/home/USER/VO_metacentrum-tape_tape-archive/my-archive.tgz # list the content of the archive
# unpack only file PATH1/file1 and directory PATH2/dir2 into the current directory
tar xzvf /storage/du-cesnet/home/USER/VO_metacentrum-tape_tape-archive/my-archive.tgz "PATH1/file1" "PATH2/dir2"
There are many other options to customize the tar command. For the full description, read manual pages (