====== Accessing computing resources remotely ====== [ __[[vlsi:home|Home]]__ ] [ __[[vlsi:workbook|Design WorkBook]]__ ] [ __[[vlsi:workbook:computing:main#contents|Back to index]]__ ] == Contents == * [[vlsi:workbook:computing:remote#introduction|Introduction]] * [[vlsi:workbook:computing:remote#connecting_to_a_vlsi_machine|Connecting to a VLSI machine]] * [[vlsi:workbook:computing:remote#remote_connections_from_windows|Remote connections from Windows]] * [[vlsi:workbook:computing:remote#remote_desktop_access_with_nx_client|Remote desktop access with NX Client]] * [[vlsi:workbook:computing:remote#accessing_vlsi_machines_from_outside_the_infn_domain|Accessing VLSI machines from outside the INFN domain]] * [[vlsi:workbook:computing:remote#remote_file_transfers|Remote file transfers]] * [[vlsi:workbook:computing:remote#web_browsing_through_ssh|Web browsing through SSH]] * [[vlsi:workbook:computing:remote#tips_and_tricks|Tips and tricks]] **Keywords:** ssh, X-server, scp, sftp, tunnelling ===== Introduction ===== Once you have obtained a VLSI computing account, you are not required to physically work in front of a VLSI machine. As a matter of fact, you can always access your VLSI machine remotely using the **[[http://en.wikipedia.org/wiki/Secure_Shell|Secure Shell (SSH)]]** connection protocol and just work as you would be in front of it. For security issues, VLSI and INFN machines allow remote access only through SSH. Connections using unsecured protocols such as FTP and Telnet are refused. This chapter describes how you can access the VLSI computing resources remotely (both from Linux/Mac or Windows machines) and gives further information about some common tasks (e.g. remote file transfers through SCP/SFTP, tunnelling over SSH) necessary to successfully work remotely. ===== Connecting to a VLSI machine ===== All IP addresses of the VLSI machines are visible from within the INFN computing networks, as well as from university computers. UNIX/Linux and Mac operating systems provide an SSH client out of the box which comes with the **[[http://www.openssh.org/|OpenSSH]]** package, simply run the ''ssh'' command from any terminal application. To remotely access your VLSI machine open a terminal and type ssh -X username@eltxxxx.to.infn.it \\ The first time you connected you should retrive a warning message in the form The authenticity of host can't be established. RSA key fingerprint is . Are you sure you want to continue connecting (yes/no)? \\ This is required by the ''ssh'' client to add the machine to the list of known hosts. Simply enter ''yes'' and press the ''Return'' key. This stuff is required just once. Since the VLSI tools extensively use graphical interfaces you must also enable the **X forwarding** with the ''-X'' option passed to the ''ssh'' command. This is required to open graphical windows remotely. If you are using a Mac computer use ''ssh -Y'' indeed, because Mac's operating systems are stricter about X forwarding compared to Linux. The ''-Y'' option enables a //trusted X forwarding// and will allow you to work without any problem. If you want to connect from a Windows computer indeed, please refer to [[vlsi:workbook:computing:remote#remote_connections_from_windows|Remote connections from Windows]] istructions. Be aware that X forwarding needs specific UNIX configurations and permissions to be set on the remote machine you want to access (e.g. ''~/.Xauthority'' file). Since these setups are not under your control, if the first time you connected you face any problems with X forwarding due to permission issues, please contact the VLSI system administrator ([[mazza@NOSPAMto.infn.it]], remove NOSPAM in the address). [tmp]\\ You might need to set the ''DISPLAY'' environment variable. This variable must to be set to the IP address of your computer followed by a screen address. The latter can always be taken to be :0.0. setenv DISPLAY machine:0.0 \\ For more details about the usage of the ''ssh'' command type ''man ssh''. Further readings and useful links are [[http://en.wikibooks.org/wiki/Guide_to_Unix/Explanations/Connecting_to_Remote_Unix]] and [[http://plaza.ufl.edu/cmcglone/eel5322/remote_setup.htm]] ===== Remote connections from Windows ===== If you are working on a system running Windows (XP, Vista, Seven) some additional software is required to deal with remote connections. In particular, you need * an SSH client for Windows to establish the remote connection * an X server for Windows to do X forwarding Many different applications (both commercial and free) are available. For a free, simple and fast setup a popular choice is to use the **[[http://en.wikipedia.org/wiki/PuTTY|PuTTY]]** SSH client and the **[[http://en.wikipedia.org/wiki/Xming|Xming]]** display server. Another (actually quite complicated) free option is to use **[[http://www.cygwin.com/|Cygwin]]**. PuTTY is a free and open source implementation of Telnet and SSH for Windows and UNIX platforms. It provides an ''xterm'' terminal emulator as well. If you don't need X forwarding PuTTY is enough to establish the remote connection and then interact with your machine through the command line only. It comes as a single executable (putty.exe, ~470 KB) which you can download from [[http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html|this page]]. \\ No installation is required, just save the file somewhere on your system. Since the ''C:\windows\'' folder is included in the default Windows search path, this is the recommended place, but you can put the executable wherever you like (e.g. Desktop). Beside this SSH client, if you want to run a VLSI design tool or any other application which requires graphical interfaces you need an X server. Xming is a free implementation of the UNIX/Linux X Window System for Windows operating systems. Downloads are available at //[[http://www.straightrunning.com/XmingNotes/]]//. Although donations must be made to download the latest releases, packages listed as //Public Domain// are completely free. \\ You can download the Xming main installer (Xming-6-9-0-31-setup.exe, 2.2 MB) from [[http://sourceforge.net/projects/xming/files/Xming/6.9.0.31/|this direct link]]. An additional Xming-fonts package (Xming-fonts-7-5-0-47-setup.exe, ~31 MB) provides standard core X fonts which are required by the most common UNIX/Linux applications. The installation is very fast and easy, simply run the executables and follow the setup wizards. We can mention that along with the basic X server (Xming.exe) the default Xming full installation includes an enhanced version of PuTTY named PuTTY Link (plink.exe), which is a console SSH client similar to the ''ssh'' command on UNIX/Linux and Mac operating systems. The Xming-fonts package should be installed in the same directory where you installed Xming. If you have successfully installed both PuTTY and Xming applications, follow the links reported below. They provide step-by-step instructions to setup and make a remote connection. * [[vlsi:workbook:computing:remote:xming|Starting the X server]] * [[vlsi:workbook:computing:remote:putty|Configuring PuTTY and connecting]] \\ Note that Use can these instructions to connect through SSH to any remote machine (INFN machines, CERN lxplus etc). Secure remote file transfers on Windows are possible as well and will be discussed later. Useful links: * [[http://www.chiark.greenend.org.uk/~sgtatham/putty|Putty main page]] * [[http://www.chiark.greenend.org.uk/~sgtatham/putty/docs.html|PuTTY official documentation]] * [[http://www.straightrunning.com/XmingNotes/|Xming main page (with documentation)]] \\ Different Web tutorials about the usage of PuTTY and Xming can be foud at: * //[[http://kb.mediatemple.net/questions/1595/Using+SSH+in+PuTTY+%28Windows%29#gs]]// * //[[http://www.math.umn.edu/systems_guide/putty_xwin32.html]]// * //[[http://www.linux.ucla.edu/~phaethon/xming_cadence/xming_cadence.html]]// ====== Remote desktop access with NX Client ====== NX Client dovrebbe essere.... piu' veloce! e quindi si dovrebbe riuscire ad aprire cadence anche con meno banda! Esiste per Mac, Linux e Windows tranquillamente. Peccato che NX client funziona SOLO se sulla macchina su cui voglio collegarmi eltxxx.to.infn e' installato e correttamente configurato NX server !!!!!! As an alternative to a SSH connection, you can try a remote desktop access with the NX Client tool. To set a remote desktop access, you must install the NX Client for Windows or NX Client for Linux in your computer. In the case of Windows, you must also install the "nxfonts-75dpi" * [[vlsi:workbook:computing:remote:nxclient:linux|Linux instructions]] * [[vlsi:workbook:computing:remote:nxclient:windows|Windows instructions]] ====== Setting up a VNC connection ====== Page: [[vlsi:workbook:computing:remote:vnc|VNC VLSI]] ===== Accessing VLSI machines from outside the INFN domain ===== VLSI machines are visible only from within INFN and university networks. You can always verify if your VLSI account is accessible using the ''ping'' command, ping username@eltxxxx.to.infn.it If you retrive an ''unknown host'' issue, then the machine cannot be accessed through a simple ''ssh''. However, sometimes you might need to access your VLSI account at home or from another institute. To do this, you must first connect through SSH to a machine of the INFN computing cluster. If you don't have an INFN computing account read [[computing_accounts|here]] how to obtain it. If you already have an INFN UNIX account, available machines for **public login** are **bennu.to.infn.it** and **phoinix.to.infn.it**. To connect to one of these remote machines, open a terminal and use ssh [-X] username@bennu.to.infn.it \\ or configure a remote connection with PuTTY if you are working on Windows. Once logged into an INFN machine, you are back to the //to.infn.it// domain and you can access your VLSI machine **eltxxxx.to.infn.it** simply doing a further ''ssh'' at the new command prompt: ssh [-X] username@eltxxxx.to.infn.it \\ A smarter way to do this stuff is called **tunneling** and is described [[vlsi:computing_remote#tips_and_tricks|here]]. Note that if you want to open graphical windows use the ''-X'' option in both ''ssh'' connections on configure PuTTY to enable X forwarding before connecting to INFN central machines. Accessing your VLSI machine from outside the INFN domain is useful if you need to access Cadence simulation data or other files and documents placed in your local area. \\ On the other hand, be aware that //**running Cadence remotely**// on a machine not at the University of Turin //**might be specifically forbidden by a license agreement or some other technology restrictions**//! In particular, Cadence remote working is forbidden unless the remote machine is within a specified distance from the license host machine. Furthermore, the contract for the usage of a particular technology may specifically prevent to run Cadence attached to that PDK outside the company site. Please, **//run the VLSI tools only at the University of Turin//** or ask the system administrator ([[mazza@NOSPAMto.infn.it]], remove NOSPAM in the address) if you can remotely access Cadence and your technology from other sites. ===== Remote file transfers ===== You will be required quite often to deal with file transfers between a remote machine and your local system. For instance, you may have a file containing some Cadence simulation results located on your remote VLSI machine, and you need to download it to your laptop at home for further analysis. Since copying files //to// or //from// a remote system is a very common task, we provide here more information and how-to's about this topic. There are many remote file transfer applications for both UNIX/Linux and Windows environments. As for security reasons VLSI and INFN machines allow remote access only through SSH, as well only **encrypted file transfers** using [[http://en.wikipedia.org/wiki/Secure_copy|Secure Copy (SCP)]] and [[http://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol|SSH File Transfer Protocol (SFTP)]] programs can be performed. :!: //**Note**// \\ It's possible to disable ''scp'' and ''sftp'' and have only SSH access, quindi tutto il seguito ha senso solo se la remote machine e' configurata per permetter scp/sftp! Il permesso viene definito nel file di configurazione di ssh ''/etc/ssh/sshd_config'' # override default of no subsystems Subsystem sftp /usr/libexec/openssh/sftp-server Se questa linea e' commentata... allora niente scp/sfp sulla macchina! ===== ===== \\ **Secure Copy (SCP)** The SCP protocol allows files to be transferred remotely //to//, //from// or //between// different hosts through an encrypted connection. It uses SSH for data transfer and provides the same authentication and the same security level as SSH. Similar to ''ssh'', on both UNIX/Linux and Mac operating systems an SCP client comes with **[[http://www.openssh.org/|OpenSSH]]** out of the box. Simply run the ''scp'' command in a terminal window. The usage of ''scp'' is similar to the well known ''cp'' command and follows the same basic syntax, scp [options] \\ If you want to copy a file **//from a local system to a remote system//** use cd /path/to/filename/directory/ scp filename username@hostname:~/path/to/destination where ''hostname'' is the name of the remote machine on which you want to copy the file (e.g. ''eltxxxx.to.infn.it''). You will be prompted to enter your remote login password to complete the operation. Don't forget to include the colon '':'' before the destination path. \\ Giving an absolute destination path is optional, if you use scp filename username@hostname: the source file will be copied into your remote home directory ''~/''.\\ You can also copy multiple files at once, just provide a list of them, scp filename1 filename2 username@hostname:~/path/to/destination Finally, if you want to copy an entire directory you must use the ''-r'' option and make a recursive copy, scp -r directory username@hostname:~/path/to/destination \\ In the same vein you can use ''scp'' to copy files **//from a remote system to a local system//**.\\ As the source file is located on the remote machine you must put the hostname followed by the absolute path of the filename to be copied in front of the local destination path, cd /where/you/want/to/copy/your/file scp username@hostname:~/path/to/filename . The last dot ''.'' in the command means that the destination path is the current directory, but you can specify any absolute path. \\ This is actually the more usual case. As an example, you are working on a machine connected to INFN computing networks and you want to download a file from your remote VLSI machine or from your INFN UNIX account. \\ The most general syntax of ''scp'' allows copying files **//from a remote system to another remote system//** without actually having to log into either of them. This could be useful to share files with a colleague. The basic syntax is scp username1@hostname1:~/path/to/filename username2@hostname2:~/path/to/destination where ''username1@hostname1'' is the source from which you want to copy the file and ''username2@hostname2'' is the destination where you want it to be copied. At first you will be prompted to enter the login password for ''username1'' on ''hostname1'' and then a second time for ''username2'' on ''hostname2''. To succesfully copying files remotely the remote machines must be visible in the network. Thus, you cannot use ''scp'' to directly access files located on your VLSI machine from ouside INFN and university networks. You must perform two consecutive ''scp'''s indeed. That is, you must first connect through ''ssh'' to an INFN central machine and copying files from the VLSI account to your INFN UNIX account, ssh username@phoinix.to.infn.it ... ... phoinix.to.infn.it> mkdir tmp phoinix.to.infn.it> scp username@eltxxx.to.infn.it:~/path/to/filename ~/tmp/ then you can run a second ''scp'' on your local system (e.g. your laptop at home) and retrive the file from the INFN UNIX account. scp username@phoinix.to.infn.it:~/tmp/filename . \\ For more detailed information about the ''scp'' usage and options run ''man scp'' to access the manual pages. \\ Further useful link about ''scp'' could be //[[http://www2.imperial.ac.uk/~andy/sysnews/scp/]]// and //[[http://kb.iu.edu/data/agye.html]]//. \\ If you are working on a system running Windows (XP, Vista, Seven) you need to install an SCP client. both graphical and command line applications. A free and easy to use option is **PSCP** (pscp.exe) provided by the PuTTY project. It is a command line the official documentation [[http://the.earth.li/~sgtatham/putty/0.62/htmldoc/Chapter5.html#pscp|Using PSCP to transfer files securely]] ===== ===== \\ \\ **SSH File Transfer Protocol (SFTP)** The main disadvantage of ''scp'' is that if you want to copy a file from a remote system to a local system you must know //a priori// the exact absolute path of the remote file. An easier way to perform this operation is to use an SFTP program. On both UNIX/Linux and Mac operating systems **[[http://www.openssh.org/|OpenSSH]]** provides an SFTP client out of the box with the ''sftp'' command line application. \\ Unlike the standard unsecured ''ftp'' client, ''sftp'' performs all the operations (commands and data transfers) over an encrypted SSH connection, preventing passwords and sensitive information from being transmitted in the clear over a network. To start an ''sftp'' session, open a terminal and run the command sftp username@hostname where ''hostname'' is the name of the remote machine you want to connect to (e.g. eltxxxx.to.infn.it). You will be asked to enter your login password. After you have successfully logged into the remote machine, ''sftp'' moves to your remote home directory. The UNIX prompt changes into ''sftp>'' and you can start interacting with the remote system using ''sftp'' commands. Besides downloading files, you can move between directories and list directory contents, as well as creating and removing directories or deleting files. \\ To get a list of all available commands, type ''help'' at the ''sftp>'' prompt: sftp> help Available commands: bye Quit sftp cd path Change remote directory to 'path' chgrp grp path Change group of file 'path' to 'grp' chmod mode path Change permissions of file 'path' to 'mode' chown own path Change owner of file 'path' to 'own' df [-hi] [path] Display statistics for current directory or filesystem containing 'path' exit Quit sftp get [-P] remote-path [local-path] Download file help Display this help text lcd path Change local directory to 'path' lls [ls-options [path]] Display local directory listing lmkdir path Create local directory ln oldpath newpath Symlink remote file lpwd Print local working directory ls [-1aflnrSt] [path] Display remote directory listing lumask umask Set local umask to 'umask' mkdir path Create remote directory progress Toggle display of progress meter put [-P] local-path [remote-path] Upload file pwd Display remote working directory quit Quit sftp rename oldpath newpath Rename remote file rm path Delete remote file rmdir path Remove remote directory symlink oldpath newpath Symlink remote file version Show SFTP version !command Execute 'command' in local shell ! Escape to local shell ? Synonym for help \\ Commands for navigating remote files and directories (''cd'', ''pwd'', ''mkdir'' etc.) follow the same syntax of standard UNIX shell counterparts. The most notable difference is that there is a local and a remote version of each command, hence you can interact with both the remote and the local systems during an ''sftp'' session. In particular, commands prefixed by an ''l'' (''lcd'', ''lpwd'', ''lmkdir'' etc.) idicate a local command. The ''get'' command allows you to **download files** from the remote machine, sftp> cd /path/to/filename/directory/ sftp> get filename [/local/path/where/you/want/to/put/the/file] If you omit the destination path the file will be put in the local directory where you started ''sftp''. You can also download multiple files at once by using the ''mget'' command, sftp> mget filename1 filename2 Since ''sftp'' does not support recursive copies you cannot retrive an entire directory and all sub-directories. You can perform a multiple download over single files with ''mget'', sftp> cd /path/to/directory/ sftp> mget * or use the ''scp -r'' command instead. Actually, the best work aroud when you want to copy a remote directory is to open a standard ''ssh'' session work and [[vlsi:unix_basics#working_with_compressed_files|create a compressed archive]] of the directory with ''tar'' and ''gzip'' utilities. Then you can download the ''tar.gz'' file with ''scp'' or ''sftp''. You can also **upload files** from the local system to the remote system by using ''put'' and ''mput'' commands, sftp> put /absolute/local/path/to/filename [/remote/path/where/you/want/to/put/the/file] If you omit the destination path the file will be uploaded in the remote current directory. \\ To end the ''sftp'' session, simply type ''exit'', ''quit'' or ''bye'' at the prompt, e.g. sftp> exit \\ For more detailed information about ''sftp'', run ''man scp'' to access the manual pages. With PuTTY **PSFTP** (psftp.exe) the official documentation [[http://the.earth.li/~sgtatham/putty/0.62/htmldoc/Chapter6.html#psftp|Using PSFTP to transfer files securely]] Another free and open source application is **Windows Secure Copy (WinSCP)**, which comes as a graphical interface tool. is an open source SFTP client for Windows. ===== ===== \\ **Copying large files and directories** The best work aroud when you want to copy large files and directories is to [[vlsi:unix_basics#working_with_compressed_files|create a compressed archive]] with ''tar'' and ''gzip'' utilities, tar -xczf fileName.tar.gz fileName Then you can download or upload the ''tar.gz'' file with ''scp'' or ''sftp''. ===== Web browsing through SSH ===== Reading scientific literature play a central role in the research and design activity. Very often you will need to get **online papers**, largely from **[[http://ieeexplore.ieee.org/Xplore/|IEEE]]** or **[[http://www.journals.elsevier.com/nuclear-instruments-and-methods-in-physics-research-section-a-accelerators-spectrometers-detectors-and-associated-equipment/|Nuclear Instruments and Methods in Physics Research]]** journals. The majority of such documents is available after paying subscribers only and cannot be read or downloaded for free. Actually, academic and research institutes can purchase subscriptions for their users. That is, you can access online papers without any restrictions, because they have been purchased for you by the institute! \\ Both INFN and University of Turin provide this support, as well as CERN. Be aware that at present the subscription with IEEE is available through a CERN computing account only. You are allowed to read and download papers without any authentication only by using a Web browser running on a machine which is part of the institute computing networks. Thus, if your are working on a machine connected to university or INFN networks you are ok, as well as if you are personally at CERN. Otherwise, you cannot access restricted pubblications for free from your home or another site, because you don't have a username and a password to do it. To get rid of this limitation, some further work around is required. The quick-and-dirty way is to simply use X forwarding and open the Web browser on a remote machine. This would be a pain, because the connection speed would be saturated to display the the graphical window of the remote browser! The most efficient solution is to create an **SSH tunnel** indeed, then configuring your browser to retrive Web data through it (port forwarding). Using an SSH tunnel is significantly faster than trying to open a remote session of the browser, because the display rendering is done on your local machine and not on the remote one. The recommended browser for this purpose is **Firefox**. Step-by-step instructions to setup Firefox and create an SSH tunnel on both UNIX/Linux and and Windows operating systems can be found __[[vlsi:workbook:computing:remote:ssh_firefox|here]]__. ===== Tips and tricks ===== //The Secure Shell: The Definitive Guide// **Tunnelling** Double-hop ssh tunnel (ssh on one machine, then on another) Situation: A and B are remote hosts. Local machine can SSH into A, but not B. B ONLY accepts SSH connections from A. one SSH from local to A that tunnels from a secondary local port (like 2121) to port 21 on B, and then you can SSH to localhost:2121 and login on B Un modo ad esempio e': ssh -l username -L 7777:eltXXX.to.infn.it:22 zoroastro.to.infn.it cat - poi la shell rest ain hang, si apre un altro terminale e si mette ssh -p 7777 username@localhost e funziona! See also //[[http://picobit.wordpress.com/2009/04/26/linux-ssh-forwarding-tunneling-duble-ssh-using-the-console/]]// \\ \\ **Performing SSH login and SCP without password** :!: //**Note**// \\ Tutto questo funziona SOLO se ''/etc/ssh/sshd_config'' sulla server machine e' stato configurato per permettere il login con un certificato! #PubkeyAuthentication yes di default e' commentato! Enter your password every time you want to ssh into you machine is quite annoying! ''~/.ssh'' directory in your home cd ~/.ssh ssh-keygen [options] generate a key generate a couple of keys, one public and one private with the ''ssh-keygen'' command which come with OpenSSH You will be prompted to specify a path (the default one ''~/.ssh/id_rsa'' works fine) and a **passphrase** (do not use an empty passphrase) Generating public/private rsa key pair. Enter file in which to save the key (/home/username/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: The key fingerprint is: 53:f8:08:27:20:a1:82:d7:ad:ac:44:30:48:71:17:05 username@local.hostname After keys has been succesfully generated you have to **copy the public key on the remote system** you want to connect to through SSH ls ~/.ssh you can use ''scp'' itself, cd ~/.ssh scp id_rsa.pub username@remote.hostname:~/ login on the remote system, ssh username@remote.hostname move the ''id.rsa.pub'' as ''~/.ssh/authorized_keys'' mv ~/id.rsa.pub ~/.ssh/authorized_keys E questo e' sufficiente, loggarsi e immettere il passphrase oppure nulla se non la si e' inserita. [[http://www.thegeekstuff.com/2008/06/perform-ssh-and-scp-without-entering-password-on-openssh/]] ====== ====== \\ ---- Last update: [[ pacher@NOSPAMto.infn.it | Luca Pacher ]] - Mar 11, 2013 ~~NOTOC~~