====== Accessing computing resources remotely ======
[ __[[vlsi:home|Home]]__ ]
[ __[[vlsi:workbook|Design WorkBook]]__ ]
[ __[[vlsi:workbook:computing:main#contents|Back to index]]__ ]
== Contents ==
* [[vlsi:workbook:computing:remote#introduction|Introduction]]
* [[vlsi:workbook:computing:remote#connecting_to_a_vlsi_machine|Connecting to a VLSI machine]]
* [[vlsi:workbook:computing:remote#remote_connections_from_windows|Remote connections from Windows]]
* [[vlsi:workbook:computing:remote#remote_desktop_access_with_nx_client|Remote desktop access with NX Client]]
* [[vlsi:workbook:computing:remote#accessing_vlsi_machines_from_outside_the_infn_domain|Accessing VLSI machines from outside the INFN domain]]
* [[vlsi:workbook:computing:remote#remote_file_transfers|Remote file transfers]]
* [[vlsi:workbook:computing:remote#web_browsing_through_ssh|Web browsing through SSH]]
* [[vlsi:workbook:computing:remote#tips_and_tricks|Tips and tricks]]
**Keywords:** ssh, X-server, scp, sftp, tunnelling
===== Introduction =====
Once you have obtained a VLSI computing account, you are not required to physically
work in front of a VLSI machine. As a matter of fact, you can always access your VLSI
machine remotely using the **[[http://en.wikipedia.org/wiki/Secure_Shell|Secure Shell (SSH)]]**
connection protocol and just work as you would be in front of it. For security issues,
VLSI and INFN machines allow remote access only through SSH. Connections
using unsecured protocols such as FTP and Telnet are refused.
This chapter describes how you can access the VLSI computing resources remotely (both from
Linux/Mac or Windows machines) and gives further information about some common tasks (e.g. remote
file transfers through SCP/SFTP, tunnelling over SSH)
necessary to successfully work remotely.
===== Connecting to a VLSI machine =====
All IP addresses of the VLSI machines are visible from within the INFN computing networks,
as well as from university computers. UNIX/Linux and Mac operating systems provide an SSH
client out of the box which comes with the **[[http://www.openssh.org/|OpenSSH]]** package, simply run the
''ssh'' command from any terminal application.
To remotely access your VLSI machine open a terminal and type
ssh -X username@eltxxxx.to.infn.it
\\
The first time you connected you should retrive a warning message in the form
The authenticity of host can't be established.
RSA key fingerprint is .
Are you sure you want to continue connecting (yes/no)?
\\
This is required by the ''ssh'' client to add the machine to the list of known hosts.
Simply enter ''yes'' and press the ''Return'' key. This stuff is required just once.
Since the VLSI tools extensively use graphical interfaces you must also
enable the **X forwarding** with the ''-X'' option passed to the ''ssh'' command.
This is required to open graphical windows remotely. If you are using a Mac computer
use ''ssh -Y'' indeed, because Mac's operating systems are stricter about X forwarding
compared to Linux. The ''-Y'' option enables a //trusted X forwarding// and will allow
you to work without any problem. If you want to connect from a Windows computer indeed,
please refer to [[vlsi:workbook:computing:remote#remote_connections_from_windows|Remote connections from Windows]]
istructions.
Be aware that X forwarding needs specific UNIX configurations and permissions to be set
on the remote machine you want to access (e.g. ''~/.Xauthority'' file).
Since these setups are not under your control, if the first time you connected you face
any problems with X forwarding due to permission issues,
please contact the VLSI system administrator ([[mazza@NOSPAMto.infn.it]], remove NOSPAM in the address).
[tmp]\\
You might need to set the ''DISPLAY'' environment variable. This variable must to be set to
the IP address of your computer followed by a screen address.
The latter can always be taken to be :0.0.
setenv DISPLAY machine:0.0
\\
For more details about the usage of the ''ssh'' command type ''man ssh''.
Further readings and useful links are
[[http://en.wikibooks.org/wiki/Guide_to_Unix/Explanations/Connecting_to_Remote_Unix]]
and [[http://plaza.ufl.edu/cmcglone/eel5322/remote_setup.htm]]
===== Remote connections from Windows =====
If you are working on a system running Windows (XP, Vista, Seven) some additional
software is required to deal with remote connections. In particular, you need
* an SSH client for Windows to establish the remote connection
* an X server for Windows to do X forwarding
Many different applications (both commercial and free) are available. For a free, simple and
fast setup a popular choice is to use the **[[http://en.wikipedia.org/wiki/PuTTY|PuTTY]]**
SSH client and the **[[http://en.wikipedia.org/wiki/Xming|Xming]]** display server.
Another (actually quite complicated) free option is to use **[[http://www.cygwin.com/|Cygwin]]**.
PuTTY is a free and open source implementation of Telnet and SSH for Windows
and UNIX platforms. It provides an ''xterm'' terminal emulator as well. If you don't need
X forwarding PuTTY is enough to establish the remote connection and then interact with your machine
through the command line only. It comes as a single executable (putty.exe, ~470 KB) which you can download from
[[http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html|this page]]. \\
No installation is required, just save the file somewhere on your system.
Since the ''C:\windows\'' folder is included in the default Windows search path,
this is the recommended place, but you can put the executable wherever you like (e.g. Desktop).
Beside this SSH client, if you want to run a VLSI design tool or any other
application which requires graphical interfaces you need an X server. Xming
is a free implementation of the UNIX/Linux X Window System for Windows operating systems.
Downloads are available at //[[http://www.straightrunning.com/XmingNotes/]]//.
Although donations must be made to download the latest releases, packages
listed as //Public Domain// are completely free. \\
You can download the Xming main installer (Xming-6-9-0-31-setup.exe, 2.2 MB)
from [[http://sourceforge.net/projects/xming/files/Xming/6.9.0.31/|this direct link]].
An additional Xming-fonts package (Xming-fonts-7-5-0-47-setup.exe, ~31 MB)
provides standard core X fonts which are required by the most common UNIX/Linux applications.
The installation is very fast and easy, simply run the executables and follow the setup
wizards. We can mention that along with the basic X server (Xming.exe) the
default Xming full installation includes an enhanced version of PuTTY named PuTTY Link (plink.exe),
which is a console SSH client similar to the ''ssh'' command on UNIX/Linux and
Mac operating systems. The Xming-fonts package should be installed in the same directory where
you installed Xming.
If you have successfully installed both PuTTY and Xming applications, follow the links
reported below. They provide step-by-step instructions to setup and make a remote connection.
* [[vlsi:workbook:computing:remote:xming|Starting the X server]]
* [[vlsi:workbook:computing:remote:putty|Configuring PuTTY and connecting]]
\\
Note that
Use can these instructions to connect through SSH
to any remote machine (INFN machines, CERN lxplus etc).
Secure remote file transfers on Windows are possible as well and will be discussed later.
Useful links:
* [[http://www.chiark.greenend.org.uk/~sgtatham/putty|Putty main page]]
* [[http://www.chiark.greenend.org.uk/~sgtatham/putty/docs.html|PuTTY official documentation]]
* [[http://www.straightrunning.com/XmingNotes/|Xming main page (with documentation)]]
\\
Different Web tutorials about the usage of PuTTY and Xming can be foud at:
* //[[http://kb.mediatemple.net/questions/1595/Using+SSH+in+PuTTY+%28Windows%29#gs]]//
* //[[http://www.math.umn.edu/systems_guide/putty_xwin32.html]]//
* //[[http://www.linux.ucla.edu/~phaethon/xming_cadence/xming_cadence.html]]//
====== Remote desktop access with NX Client ======
NX Client dovrebbe essere.... piu' veloce! e quindi si dovrebbe riuscire
ad aprire cadence anche con meno banda! Esiste per Mac, Linux e Windows
tranquillamente. Peccato che NX client funziona SOLO se sulla macchina
su cui voglio collegarmi eltxxx.to.infn e' installato e correttamente configurato
NX server !!!!!!
As an alternative to a SSH connection, you can try a remote desktop access with the
NX Client tool.
To set a remote desktop access, you must install the NX Client for Windows or NX Client
for Linux in your computer. In the case of Windows, you must also install the "nxfonts-75dpi"
* [[vlsi:workbook:computing:remote:nxclient:linux|Linux instructions]]
* [[vlsi:workbook:computing:remote:nxclient:windows|Windows instructions]]
====== Setting up a VNC connection ======
Page:
[[vlsi:workbook:computing:remote:vnc|VNC VLSI]]
===== Accessing VLSI machines from outside the INFN domain =====
VLSI machines are visible only from within INFN and university networks.
You can always verify if your VLSI account is accessible using the ''ping''
command,
ping username@eltxxxx.to.infn.it
If you retrive an ''unknown host'' issue, then the machine cannot be accessed
through a simple ''ssh''.
However, sometimes you might need to access your VLSI account at home or from another institute.
To do this, you must first connect through SSH to a machine of the INFN computing cluster.
If you don't have an INFN computing account read [[computing_accounts|here]] how to obtain it.
If you already have an INFN UNIX account, available machines for **public login** are
**bennu.to.infn.it** and **phoinix.to.infn.it**.
To connect to one of these remote machines, open a terminal and use
ssh [-X] username@bennu.to.infn.it
\\
or configure a remote connection with PuTTY if you are working on Windows.
Once logged into an INFN machine, you are back to the //to.infn.it// domain and you can access
your VLSI machine **eltxxxx.to.infn.it** simply doing a further ''ssh'' at the new command prompt:
ssh [-X] username@eltxxxx.to.infn.it
\\
A smarter way to do this stuff is called **tunneling** and is described [[vlsi:computing_remote#tips_and_tricks|here]].
Note that if you want to open graphical windows use the ''-X'' option in both ''ssh'' connections on configure PuTTY
to enable X forwarding before connecting to INFN central machines.
Accessing your VLSI machine from outside the INFN domain is useful if you need to access
Cadence simulation data or other files and documents placed in your local area. \\
On the other hand, be aware that //**running Cadence remotely**// on a machine not at the University
of Turin //**might be specifically forbidden by a license agreement or some other technology restrictions**//!
In particular, Cadence remote working is forbidden unless the remote machine is within a specified
distance from the license host machine. Furthermore, the contract for the usage of a particular
technology may specifically prevent to run Cadence attached to that PDK outside the company site.
Please, **//run the VLSI tools only at the University of Turin//** or ask the system administrator
([[mazza@NOSPAMto.infn.it]], remove NOSPAM in the address) if you can remotely access Cadence and
your technology from other sites.
===== Remote file transfers =====
You will be required quite often to deal with file transfers between
a remote machine and your local system. For instance, you may have a file
containing some Cadence simulation results located on your remote VLSI machine,
and you need to download it to your laptop at home for further analysis.
Since copying files //to// or //from// a remote system is a very common task,
we provide here more information and how-to's about this topic.
There are many remote file transfer applications for both UNIX/Linux and Windows
environments. As for security reasons VLSI and INFN machines allow remote access
only through SSH, as well only **encrypted file transfers** using
[[http://en.wikipedia.org/wiki/Secure_copy|Secure Copy (SCP)]] and
[[http://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol|SSH File Transfer Protocol (SFTP)]]
programs can be performed.
:!: //**Note**// \\
It's possible to disable ''scp'' and ''sftp'' and have only SSH access, quindi
tutto il seguito ha senso solo se la remote machine e' configurata per permetter scp/sftp!
Il permesso viene definito nel file di configurazione di ssh ''/etc/ssh/sshd_config''
# override default of no subsystems
Subsystem sftp /usr/libexec/openssh/sftp-server
Se questa linea e' commentata... allora niente scp/sfp sulla macchina!
===== =====
\\
**Secure Copy (SCP)**
The SCP protocol allows files to be transferred remotely //to//, //from// or //between//
different hosts through an encrypted connection.
It uses SSH for data transfer and provides the same authentication
and the same security level as SSH.
Similar to ''ssh'', on both UNIX/Linux and Mac operating systems an SCP client comes with
**[[http://www.openssh.org/|OpenSSH]]** out of the box. Simply run the ''scp'' command
in a terminal window. The usage of ''scp'' is similar to the well known ''cp'' command
and follows the same basic syntax,
scp [options]
\\
If you want to copy a file **//from a local system to a remote system//** use
cd /path/to/filename/directory/
scp filename username@hostname:~/path/to/destination
where ''hostname'' is the name of the remote machine on which you want to copy the file (e.g. ''eltxxxx.to.infn.it'').
You will be prompted to enter your remote login password to complete the operation. Don't forget
to include the colon '':'' before the destination path. \\
Giving an absolute destination path is optional, if you use
scp filename username@hostname:
the source file will be copied into your remote home directory ''~/''.\\
You can also copy multiple files at once, just provide a list
of them,
scp filename1 filename2 username@hostname:~/path/to/destination
Finally, if you want to copy an entire directory you must use the ''-r'' option
and make a recursive copy,
scp -r directory username@hostname:~/path/to/destination
\\
In the same vein you can use ''scp'' to copy files **//from a remote system to a local system//**.\\
As the source file is located on the remote machine you must
put the hostname followed by the absolute path of the filename to be copied in front of the local
destination path,
cd /where/you/want/to/copy/your/file
scp username@hostname:~/path/to/filename .
The last dot ''.'' in the command means that the destination path is the
current directory, but you can specify any absolute path. \\
This is actually the more usual case. As an example, you are working on a machine
connected to INFN computing networks and you want to download a file from your
remote VLSI machine or from your INFN UNIX account.
\\
The most general syntax of ''scp'' allows copying files **//from a remote system to another remote system//**
without actually having to log into either of them. This could be useful to share files with
a colleague. The basic syntax is
scp username1@hostname1:~/path/to/filename username2@hostname2:~/path/to/destination
where ''username1@hostname1'' is the source from which you want to copy the file and
''username2@hostname2'' is the destination where you want it to be copied.
At first you will be prompted to enter the login password for ''username1''
on ''hostname1'' and then a second time for ''username2'' on ''hostname2''.
To succesfully copying files remotely the remote machines must be visible in the network.
Thus, you cannot use ''scp'' to directly access files located on your VLSI machine
from ouside INFN and university networks. You must perform two consecutive ''scp'''s indeed.
That is, you must first connect through ''ssh'' to an INFN central machine and copying files
from the VLSI account to your INFN UNIX account,
ssh username@phoinix.to.infn.it
...
...
phoinix.to.infn.it> mkdir tmp
phoinix.to.infn.it> scp username@eltxxx.to.infn.it:~/path/to/filename ~/tmp/
then you can run a second ''scp'' on your local system (e.g. your laptop at home)
and retrive the file from the INFN UNIX account.
scp username@phoinix.to.infn.it:~/tmp/filename .
\\
For more detailed information about the ''scp'' usage and options run ''man scp''
to access the manual pages. \\
Further useful link about ''scp'' could be //[[http://www2.imperial.ac.uk/~andy/sysnews/scp/]]//
and //[[http://kb.iu.edu/data/agye.html]]//.
\\
If you are working on a system running Windows (XP, Vista, Seven) you need to install
an SCP client. both graphical and command line applications.
A free and easy to use option is **PSCP** (pscp.exe) provided by the PuTTY project.
It is a command line
the official documentation
[[http://the.earth.li/~sgtatham/putty/0.62/htmldoc/Chapter5.html#pscp|Using PSCP to transfer files securely]]
===== =====
\\
\\
**SSH File Transfer Protocol (SFTP)**
The main disadvantage of ''scp'' is that if you want to copy a file
from a remote system to a local system you must know //a priori//
the exact absolute path of the remote file. An easier way to perform
this operation is to use an SFTP program.
On both UNIX/Linux and Mac operating systems **[[http://www.openssh.org/|OpenSSH]]**
provides an SFTP client out of the box with the ''sftp'' command line application. \\
Unlike the standard unsecured ''ftp'' client, ''sftp'' performs all the operations
(commands and data transfers) over an encrypted SSH connection,
preventing passwords and sensitive information from being transmitted in the clear over a network.
To start an ''sftp'' session, open a terminal and run the command
sftp username@hostname
where ''hostname'' is the name of the remote machine you want to connect to
(e.g. eltxxxx.to.infn.it). You will be asked to enter your login password.
After you have successfully logged into the remote machine,
''sftp'' moves to your remote home directory. The UNIX prompt
changes into ''sftp>'' and you can start interacting with the remote system
using ''sftp'' commands. Besides downloading files, you can move between
directories and list directory contents, as well as creating and removing
directories or deleting files. \\
To get a list of all available commands, type ''help'' at the ''sftp>'' prompt:
sftp> help
Available commands:
bye Quit sftp
cd path Change remote directory to 'path'
chgrp grp path Change group of file 'path' to 'grp'
chmod mode path Change permissions of file 'path' to 'mode'
chown own path Change owner of file 'path' to 'own'
df [-hi] [path] Display statistics for current directory or
filesystem containing 'path'
exit Quit sftp
get [-P] remote-path [local-path] Download file
help Display this help text
lcd path Change local directory to 'path'
lls [ls-options [path]] Display local directory listing
lmkdir path Create local directory
ln oldpath newpath Symlink remote file
lpwd Print local working directory
ls [-1aflnrSt] [path] Display remote directory listing
lumask umask Set local umask to 'umask'
mkdir path Create remote directory
progress Toggle display of progress meter
put [-P] local-path [remote-path] Upload file
pwd Display remote working directory
quit Quit sftp
rename oldpath newpath Rename remote file
rm path Delete remote file
rmdir path Remove remote directory
symlink oldpath newpath Symlink remote file
version Show SFTP version
!command Execute 'command' in local shell
! Escape to local shell
? Synonym for help
\\
Commands for navigating remote files and directories (''cd'', ''pwd'', ''mkdir'' etc.)
follow the same syntax of standard UNIX shell counterparts. The most notable difference
is that there is a local and a remote version of each command, hence you can interact
with both the remote and the local systems during an ''sftp'' session.
In particular, commands prefixed by an ''l'' (''lcd'', ''lpwd'', ''lmkdir'' etc.) idicate
a local command.
The ''get'' command allows you to **download files** from the remote machine,
sftp> cd /path/to/filename/directory/
sftp> get filename [/local/path/where/you/want/to/put/the/file]
If you omit the destination path the file will be put in the local
directory where you started ''sftp''. You can also download multiple
files at once by using the ''mget'' command,
sftp> mget filename1 filename2
Since ''sftp'' does not support recursive copies you cannot retrive
an entire directory and all sub-directories. You can perform a multiple
download over single files with ''mget'',
sftp> cd /path/to/directory/
sftp> mget *
or use the ''scp -r'' command instead. Actually, the best work aroud
when you want to copy a remote directory is to open a standard ''ssh'' session work
and [[vlsi:unix_basics#working_with_compressed_files|create a compressed archive]]
of the directory with ''tar'' and ''gzip'' utilities. Then you can download the
''tar.gz'' file with ''scp'' or ''sftp''.
You can also **upload files** from the local system to the remote system
by using ''put'' and ''mput'' commands,
sftp> put /absolute/local/path/to/filename [/remote/path/where/you/want/to/put/the/file]
If you omit the destination path the file will be uploaded in the remote
current directory.
\\
To end the ''sftp'' session, simply type ''exit'', ''quit'' or ''bye'' at the prompt, e.g.
sftp> exit
\\
For more detailed information about ''sftp'', run ''man scp'' to access the manual pages.
With PuTTY **PSFTP** (psftp.exe)
the official documentation
[[http://the.earth.li/~sgtatham/putty/0.62/htmldoc/Chapter6.html#psftp|Using PSFTP to transfer files securely]]
Another free and open source application is **Windows Secure Copy (WinSCP)**, which comes
as a graphical interface tool. is an open source SFTP client for Windows.
===== =====
\\
**Copying large files and directories**
The best work aroud when you want to copy large files and directories is to
[[vlsi:unix_basics#working_with_compressed_files|create a compressed archive]]
with ''tar'' and ''gzip'' utilities,
tar -xczf fileName.tar.gz fileName
Then you can download or upload the ''tar.gz'' file with ''scp'' or ''sftp''.
===== Web browsing through SSH =====
Reading scientific literature play a central role in the research and design activity.
Very often you will need to get **online papers**, largely from
**[[http://ieeexplore.ieee.org/Xplore/|IEEE]]** or
**[[http://www.journals.elsevier.com/nuclear-instruments-and-methods-in-physics-research-section-a-accelerators-spectrometers-detectors-and-associated-equipment/|Nuclear Instruments and Methods in Physics Research]]** journals.
The majority of such documents is available after paying subscribers only and
cannot be read or downloaded for free. Actually, academic and research institutes
can purchase subscriptions for their users. That is, you can access online papers
without any restrictions, because they have been purchased for you by the institute! \\
Both INFN and University of Turin provide this support, as well as CERN.
Be aware that at present the subscription with IEEE is available through a CERN
computing account only.
You are allowed to read and download papers without any authentication
only by using a Web browser running on a machine which is
part of the institute computing networks. Thus, if your are working on a
machine connected to university or INFN networks you are ok, as well as
if you are personally at CERN. Otherwise, you cannot access restricted
pubblications for free from your home or another site, because you don't
have a username and a password to do it.
To get rid of this limitation, some further work around is required.
The quick-and-dirty way is to simply use X forwarding and open the Web browser
on a remote machine. This would be a pain, because the connection speed would
be saturated to display the the graphical window of the remote browser!
The most efficient solution is to create an **SSH tunnel** indeed,
then configuring your browser to retrive Web data through it (port forwarding).
Using an SSH tunnel is significantly faster than trying
to open a remote session of the browser, because the display rendering
is done on your local machine and not on the remote one.
The recommended browser for this purpose is **Firefox**. Step-by-step instructions
to setup Firefox and create an SSH tunnel on both UNIX/Linux and and Windows
operating systems can be found
__[[vlsi:workbook:computing:remote:ssh_firefox|here]]__.
===== Tips and tricks =====
//The Secure Shell: The Definitive Guide//
**Tunnelling**
Double-hop ssh tunnel (ssh on one machine, then on another)
Situation: A and B are remote hosts. Local machine can SSH into A, but not B. B ONLY accepts SSH connections from A.
one SSH from local to A that tunnels from a secondary local port (like 2121) to port 21 on B, and then you can SSH to localhost:2121 and login on B
Un modo ad esempio e':
ssh -l username -L 7777:eltXXX.to.infn.it:22 zoroastro.to.infn.it cat -
poi la shell rest ain hang, si apre un altro terminale e si mette
ssh -p 7777 username@localhost
e funziona!
See also //[[http://picobit.wordpress.com/2009/04/26/linux-ssh-forwarding-tunneling-duble-ssh-using-the-console/]]//
\\
\\
**Performing SSH login and SCP without password**
:!: //**Note**// \\
Tutto questo funziona SOLO se ''/etc/ssh/sshd_config'' sulla server machine e' stato configurato per
permettere il login con un certificato!
#PubkeyAuthentication yes
di default e' commentato!
Enter your password every time you want to ssh into you machine is quite annoying!
''~/.ssh'' directory in your home
cd ~/.ssh
ssh-keygen [options]
generate a key
generate a couple of keys, one public and one private with the ''ssh-keygen'' command
which come with OpenSSH
You will be prompted to specify a path (the default one ''~/.ssh/id_rsa'' works fine) and a **passphrase** (do not
use an empty passphrase)
Generating public/private rsa key pair.
Enter file in which to save the key (/home/username/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
The key fingerprint is:
53:f8:08:27:20:a1:82:d7:ad:ac:44:30:48:71:17:05 username@local.hostname
After keys has been succesfully generated you have to **copy the public key on the remote system**
you want to connect to through SSH
ls ~/.ssh
you can use ''scp'' itself,
cd ~/.ssh
scp id_rsa.pub username@remote.hostname:~/
login on the remote system,
ssh username@remote.hostname
move the ''id.rsa.pub'' as ''~/.ssh/authorized_keys''
mv ~/id.rsa.pub ~/.ssh/authorized_keys
E questo e' sufficiente, loggarsi e immettere il passphrase oppure nulla se non la si e'
inserita.
[[http://www.thegeekstuff.com/2008/06/perform-ssh-and-scp-without-entering-password-on-openssh/]]
====== ======
\\
----
Last update: [[ pacher@NOSPAMto.infn.it | Luca Pacher ]] - Mar 11, 2013
~~NOTOC~~