# Introduction to the School of Earth Sciences UNIX computer ...

Introduction to the School of Earth Sciences UNIX computer network Kevin Keay February 6 2009 Outline

Course materials What is UNIX? Our UNIX network A brief discussion of why we use UNIX A taste of UNIX A basic overview A typical UNIX task The University of Surrey, UNIX Tutorial for Beginners: Selected topics An Introductory Tutorial on UNIX (a lab session) Course materials Handout: An Introductory Tutorial on UNIX which is also at:

http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/ An_Intro_Tute_on_UNIX_Feb-2009.pdf Notes: Introduction to the School of Earth Sciences UNIX computer network at: http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/ Intro_to_SES_UNIX_computer_network.pdf A useful online resource from the University of Surrey, UNIX Tutorial for Beginners, is available at: http:// www.ee.surrey.ac.uk /Teaching/Unix/ What is UNIX?

UNIX is a computer operating system Well suited to multi-tasking and multiple users Powerful command-line and scripting capabilities Graphic user interface: X Windows Easily accessed from Microsoft Windows (PC) or X Darwin (iMac) as well as a native UNIX machine Important: UNIX is case-sensitive e.g. nedit is different from NEDIT or Nedit

What is UNIX? (2) UNIX was developed initially by Bell Labs circa 1970 and has grown into an open software project with many world-wide collaborators. A history is given at: http://www.bell-labs.com/history/unix/ There are many variants e.g. Linux, Mac OS X.

Cygwin for Windows is a UNIX emulation i.e. looks and feels just like UNIX. What is UNIX? (3) After three decades of use, the UNIX computer operating system from Bell Labs is still regarded as one of the most powerful, versatile, and flexible operating systems (OS) in the computer world. Its popularity is due to many factors, including its ability to run on a wide variety of machines, from PCs to supercomputers, and its portability - all of which led to its adoption by many

manufacturers. Our UNIX network (1) Solaris OS: atlas, mawson, orthus, rossby (2) Red Hat Linux OS: cove, gulf, gyre, tide, wave (3) Debian Linux OS: vislab01, vislab02, vislab03, vislab04 (4) OSF1 OS: zeus (OS means operating system). Group 3 are the newest and fastest machines. Group 2 are recommended for most uses. Apart from atlas and orthus, the group 1

machines are relatively slow. Group 4 is for specialised uses e.g. GCM work. So, normally you would select a Linux machine in group 2 or 3 unless you need to use the Solaris OS. Also, see section 1.1 of the notes Introduction to the School of Earth Sciences UNIX computer network. A brief discussion of why we use UNIX UNIX machines are generally better at

multi-tasking than MS Windows Very stable platform from a user standpoint: difficult to crash! Compilers (Fortran, C) allow the user to development their own software or implement programs written by others There are many sophisticated commercial packages available e.g. Matlab A brief discussion of why we use UNIX (2)

Scripts can be used to automate complex sequences of tasks: leads to efficiency and reduction of user errors Command-line arguments add flexibility and are very useful in scripts e.g. C-shell scripts There is a global resource of software designed to work under UNIX, much of it is freely available: you dont have to develop everything from scratch A taste of UNIX Imagine that you have connected to a UNIX machine e.g. cove, via ssh from a Cygwin xterm window:

ssh X cove You will find yourself located in your UNIX home folder e.g. /home/kevin A taste of UNIX (2) Change to another folder cd /work9/kevin/Project1 /work9 is another disk and the user kevin may create folders and files within it Check that you are in the correct location (optional!) pwd /work9/kevin/Project1 A taste of UNIX (3)

List the contents (files) of this folder ls pmsl.cmp sortdata.f Test.dat test2.dat X.dat List the files with .dat as part of the filename ls *.dat Test.dat test2.dat X.dat A taste of UNIX (4) Edit the Fortran source file sortdata.f (we are making changes to the

existing file) nedit sortdata.f A taste of UNIX (5) Compile the Fortran source program to create an executable i.e. a piece of software g77 o sortdata sortdata.f Here the o option expects an argument, in this case the filename of the executable (an option could be standalone e.g. j ) Note: An option usually starts with a - List the files in the folder (sortdata is a new file) ls pmsl.cmp sortdata sortdata.f Test.dat test2.dat

X.dat A taste of UNIX (6) Run the executable (software) that you designed in Fortran sortdata You might design your program to create an output data file or the results of a computation, even an image A taste of UNIX (7)

Create a plot of a pressure file conmap k iconkk B pmsl.cmp < icon The items with - are called options and control the program called conmap. The k option has an argument a file called iconkk which contains some additional parameters to control the program Additional control comes from the instruction file called icon The output of this command is a file called gmeta that can be converted to a PNG

image A taste of UNIX (7) A basic overview When you connect to a remote UNIX machine on our network you will be working in the C-shell (actually tcsh which is aliased to csh) C-shell scripts will begin with # or #!/bin/csh -f

There are other kinds of script e.g. sh (Bourne shell bash under Cygwin is a relative), perl After connection you will be located in: /home/user where user is your UNIX username e.g. /home/kevin Remember: UNIX is case-sensitive A basic overview (2) Under MS Windows files and folders (directories) are usually relative to the C drive i.e. c:

and under UNIX are relative to / Also \ is used to denote the file tree structure under Windows and / under UNIX Hence: c:\home\kevin\test.dir\prog.f under Windows would be: /home/kevin/test.dir/prog.f under UNIX You will be working under /home or /worknn e.g. /home/kevin, /work5/kevin, /work18/kevin A basic overview (3)

ls is used to list files in a folder e.g. ls readme.txt ls l pmsl.*.dat Note that the ls command has many options. See: man ls for more information A basic overview (4)

mkdir is used to create folders (directories) e.g. mkdir test.dir rmdir is used to remove a folder e.g. rmdir test.dir Note: The folder must be empty i.e. use rm first, then rmdir chmod is used to change file access permissions e.g. chmod 755 readcmp or chmod u+rwx,g+rx,o+rx readcmp where u is user, g is group and o is others. A basic overview

(5) rm is used to remove (delete) files e.g. rm test.dat By default rm will remove without warning. Your initial setup will alias rm such that rm == rm i i.e. you will be prompted to continue with the removal Prefixing with a \ will override the alias e.g. \rm rf */* will delete all of your files starting with the current folder BEWARE! A basic overview

(6) cp is used to copy files to other locations e.g. cp readme.txt /work18/kevin/src mv is used to rename files e.g. mv readme.txt Readme.txt Remember: UNIX is case-sensitive pwd will tell you where you are i.e. your current location (folder) A basic overview

(7) cd is used to change folders (directories) Assume you are in: /home/kevin cd test.dir will change the folder (directory) to test.dir i.e. you are now in: /home/kevin/test.dir cd .. will place you in the folder one level above i.e. in this case: /home/kevin cd ~ will always take you home i.e. to: /home/user A basic overview

(8) cd /home/kevin/src will place you in the folder called src cd /work18/kevin/src/readgrib.dir will place you in the folder readgrib.dir under /work18/kevin/src Combine .. and / to change folders e.g. cd ../test2.dir A basic overview (9) Wildcard characters *,?,- and , in conjunction with various brackets {} and [] are very useful in commands to

represent complex sets of filenames e.g. cp pmsl.*.dat /work18/kevin/DATA This will copy all files of this format where the * stands for anything in that part of the filename. A basic overview (10) ? represents a single character e.g. cp pmsl.{19[6-9]?,2000,2001}.dat /work18/kevin/ DATA This will copy: pmsl.1960.dat,pmsl.1961.dat,, pmsl.1999.dat,pmsl.2000.dat, pmsl.2001.dat i.e. {19[6-9]?,2000,2001} is expanded to: 1960,1961,1962,,1999,2000,2001

We could also use: pmsl.{19[6-9]?,200[0,1]}.dat A basic overview (11) For text editing use nedit on atlas or orthus and nedit or emacs on the Linux machines e.g. nedit prog.f To transfer files to and from a PC you can

use the utility SSH Secure File Transfer Client on the Windows desktop You can also use scp under Cygwin see Appendix A of the Notes A basic overview (12) Some documentation is available with man or info, however it is often cryptic e.g. man csh Often a Google web search will find

more helpful information, examples and tutorials A basic overview (13) Summary The basic UNIX commands to understand are: ls cd mkdir rmdir rm chmod cp mv pwd csh Learn how to use wildcards: * ? - , [] {}

Ensure that you can use a text editor: nedit (Solaris OS) nedit emacs (Linux) SSH Secure File Transfer Client (Windows) is useful for transferring files from a PC to a remote UNIX machine Also see: scp A typical UNIX task

Task: Prepare average geographical maps of mean sea level pressure (MSLP) for the months of Jan and Aug using NCEP Reanalysis data and plot the results The data is available in a widely-used format called NetCDF from the NOAA web site this site has interactive data selection features I manually downloaded the relevant files using a web browser (Mozilla) to my UNIX account e.g. /work9/kevin/mean_MSLP.dir A typical UNIX task (2)

I have written a Fortran program (read_nc2cmp) to decode the NetCDF files to our internal conmap format This program was based on an example from NCAR so I didnt have to begin from scratch The conmap files are then input to my Fortran program statconmap which produces the average maps Note: The average is performed at each gridpoint of the map for the set of desired maps A typical UNIX task (3)

The average maps (one for Jan, the other for Aug) may be plotted with various software e.g. NCAR Graphics, GrADS, Matlab One method that is useful in scripts is: (1) run conmap (NCAR Graphics) on an average map; output is a file called gmeta (2) Translate gmeta to a Postscript file using the utility g2ps (3) Use the utility convert to transform the Postscript file to a PNG file this may be pasted into a Word document A typical UNIX task (4)

In a UNIX C-shell window you can type commands directly e.g. read_nc2cmp MSLP.*.Jan.nc out.cmp statconmap out.cmp ave.Jan.cmp sd.cmp var.cmp conmap G ave.Jan.cmp You can also place these commands in a C-shell script A typical UNIX task (5) #!/bin/csh f # Required first line of a C-shell script or simply # set mons = (Jan Aug) # Array of months

@ nmons = $#mons # No. of months, in this case it will be 2 @ i = 1 # @ is used to define numbers as opposed to text (set) while ($i <= $nmons) # Perform loop (to end) while i <= 2 set m =$mons[$i] # m is mons(i) i.e. mons(1) = Jan => m = Jan # Decode NetCDF files for this month (m) # Note: out.cmp contains multiple maps read_nc2cmp MSLP.*.$m.nc out.cmp # Perform average of multiple conmap file out.cmp # Note: Outputs sd.cmp and var.cmp are not used but need to be specified statconmap out.cmp ave.$m.cmp sd.cmp var.cmp # Create global geographical plot of average map e.g. ave.Jan.cmp conmap G ave.$m.cmp # Output is gmeta

# Convert gmeta file to Postscript; creates g.ps (a hardwired output) g2ps gmeta # Convert Postscript file to PNG e.g. MSLP.Jan.png convert trim density 120 g.ps MSLP.\$m.png # @ i ++ # i -> i + 1 end exit A typical UNIX task (6) We wont look in detail at scripts in this introduction but will re-visit them

later in the semester Remember: Practice makes perfect! Experiment play with the commands and see if things make sense The University of Surrey, UNIX Tutorial for Beginners: Selected topics See: http://www.ee.surrey.ac.uk/Teaching/U nix/ Note: Tutorial 7 is a bit advanced for this introduction An Introductory Tutorial on UNIX

Refer to the handout Also see: Appendix C An Introductory Tutorial on UNIX of the notes An Introductory Tutorial on UNIX (2) Direct connection to a UNIX machine From the console of a machine in the UNIX Lab or the VisLab: (1) Log on with your UNIX username and password (2) You should be in the KDE desktop. Left-click on the Shell-Konsole icon or

Terminal Program icon in the toolbar at the bottom of the screen. This will activate a terminal window (xterm). You may now enter UNIX commands. Remote connection to a UNIX machine via Cygwin From a PC in the PC Lab or elsewhere: (1) Log on with your University username and password (2) Click on the Cygwin desktop icon. This will open a bash shell window. (3) Within this window type: startx & This will open a terminal window (xterm). (4) Within the terminal window type: xterm & to open a new window. From this new window type: ssh X machine or ssh X [email protected] if your UNIX username differs from your University username e.g. ssh X [email protected] Note for machine choose from:

An Introductory Tutorial on UNIX (3) Initial setup of a UNIX account Connect to a remote UNIX machine e.g.: ssh X cove.earthsci.unimelb.edu.au After logging on: % cp /home/kevin/UNIX_Course/zunix_setup.zip - Note the dot % unzip zunix_setup.zip % ls l .cshrc.* % exit .

An Introductory Tutorial on UNIX (4) Owing to recent changes to atlas use a Linux machine like cove, tide or gyre: e.g. ssh X cove.earthsci.unimelb.edu.au In step 8 you will need to convert the SUN conmap file to a PC version with binswap i.e. binswap c pmsl.ncep.1996060106.cmp{,} Note: {,} replaces pmsl.ncep.1996060106.cmp as the output. You can use a different output name if you wish e.g. binswap c pmsl.ncep.1996060106.cmp pmsl.cmp

## Recently Viewed Presentations

• The mind is not a vessel to be filled, but a fire to be ignited. (Plutarch) Bloom's Revised Taxonomy Taxonomy of Cognitive Objectives 1950s- developed by Benjamin Bloom Means of expressing qualitatively different kinds of thinking Adapted for classroom use...
• Ce sont AND 2. A plural BANGS adjective, « des » becomes DE in most cases (for French I). Instead of using « des », you use « de. » Don't ask why! That's just the way it is with...
• Know how to use it in a sentence. Memorize the list because I am not giving you a words bank. BANAL. So lacking in originality as to be obvious and boring. (tiresome, worn-out, clichéd) ... EXTOL. To praise enthusiastically. When...
• Unterteilt einer Menge von Objekten in Klassen, so dass um die Klassengrenzen herum ein möglichst breiter Bereich frei von Objekten bleibt. Geeignet für Regression und Klassifikation. Nichtlineare Klassifikation durch Transformation des Eingaberaums. Frank Rosenblatt, The Perceptron--a perceiving and recognizing automaton.
• If the app crashes or freezes, close the Payfirma HQ app out. Double tap the home button of your iOS device. Tap and hold the Payfirma HQ icon until a (-) icon appears on the top left corner. Tap on...
• Step Up to the Plate: Sports PR. Bonnie Clark, Vice President of Communications, Philadelphia Phillies. Michael Preston, Director of Public Relations,
• "Get rid of all bitterness, rage and anger, brawling and slander, along with every form of malice. Be kind and compassionate to one another, forgiving each other, just as in Christ God forgave you."
• Aucours du voyage plusieurs marins sont victimes d'illusions d'optique ce qui inquiète de plus en plus les autres marins. Le 10 octobre 1492, les marins manquent de patience, et pensent être perdus au milieu de l'océan, de plus la nourriture...