AAnn IInnttrroodduuccttiioonn ttoo SShheellll PPrrooggrraammmmiinngg _L_a_s_t _E_d_i_t _M_a_r_c_h _2_3_, _1_9_9_2 Reg Quinton Computing and Communications Services The University of Western Ontario London, Ontario N6A 5B7 Canada 11.. BBoouurrnnee SShheellll A shell is a _c_o_m_m_a_n_d _l_i_n_e _i_n_t_e_r_p_r_e_t_o_r (cf. DCL on VAX/VMS). It takes commands and executes them. As such, it implements a programming language. The Bourne shell is used to create _s_h_e_l_l _s_c_r_i_p_t_s -- ie. programs that are interpreted/executed by the shell. You can write shell scripts with the C-shell; however, this is _n_o_t recommended. There are lots of shell scripts on the system. For example [10:45am julian] cd /bin [10:46am julian] file * | grep /bin/sh basename: executable script for /bin/sh dirname: executable script for /bin/sh echo: executable script for /bin/sh false: executable script for /bin/sh test: executable script for /bin/sh tplot: executable script for /bin/sh true: executable script for /bin/sh _e_t_c_._._. Other good examples in the boot sequence -- //eettcc//rrcc..** or //eettcc//iinniitt..dd//**. On CCS systems look at examples in //uussrr//llooccaall//sshhaarree//bbiinn. 11 UUWWOO//sshh 22.. CCrreeaattiinngg aa SSccrriipptt Suppose you often type the command find . -name _f_i_l_e -print and you'd rather type a simple command, say sfind _f_i_l_e Create a shell script [11:01am julian] cd ~/bin [11:01am julian] ed sfind _e_t_c_._._. [11:03am julian] page sfind find . -name $1 -print [11:03am julian] chmod a+x sfind [11:03am julian] rehash [11:04am julian] cd /usr/src/usr.local [11:04am julian] sfind tcsh ./shells/tcsh UUWWOO//sshh 22 22..11.. OObbsseerrvvaattiioonnss This quick example is far from adequate but some observa- tions: (1) Shell scripts are simple text files created with an editor. (2) Shell scripts are marked as eexxeeccuutteeaabbllee -- [11:14am julian] chmod a+x sfind (3) Should be located in your search path and ~~//bbiinn should be in your search path. (4) You likely need to rreehhaasshh if you're a Csh user (but not again when you login). (5) Arguments are passed from the command line and refer- enced. For example, as $$11. (6) Within a shell script -- any command! 33 UUWWOO//sshh 33.. AA RReeaall EExxaammppllee We have many shell scripts we've made. [11:45am julian] man day DAY(Local) EP/IX Reference Manual NAME day - convert date string to day of the week SYNOPSIS day date month year _e_t_c_._._. [11:44am julian] day 19 Feb 92 Wed [11:44am julian] which day /usr/local/share/bin/day [11:49am julian] cd /usr/local/share/bin [11:49am julian] file day day: executable script for /bin/sh NNoottee:: installed utilities should have a manual page, should conform to Unix conventions, etc. UUWWOO//sshh 44 33..11.. ##!!//bbiinn//sshh All Bourne Shell scripts should begin with the sequence [12:52pm julian] page day #!/bin/sh _e_t_c_._._. From eexxeecc((22)): "On the first line of an interpreter script, following the "#!", is the name of a program which should be used to interpret the contents of the file. For instance, if the first line contains "#! /bin/sh", then the con- tents of the file are executed as a shell script." You can get away without this, but you shouldn't. All good scripts state the interpretor explicitly. Long ago there was just one (the Bourne Shell) but these days there are many interpretors -- Cshell, Ksh, Bash, and others. 55 UUWWOO//sshh 33..22.. CCoommmmeennttss//RRCCSS hheeaaddeerr All good Bourne Shell scripts should begin with the sequence [12:58pm julian] page day #!/bin/sh # # $Author: reggers $ # $Date: 91/10/30 14:31:19 $ # $Header: /ccs/export/share/ftp/pub/unix/uti... # # Usage: day 19 Oct 91 # reports back the day of the week # # January 1, 1901 was a Tuesday; this is my ... # # Bugs: I'm not handling leap/noleap centur... _e_t_c_._._. The Revision Control System, RCS, is a good tool for manag- ing software projects. What version? When was it written? Where are the sources? This is recommended. Comment your code as you build it. This should be required. Anyone should be able to read a shell script. UUWWOO//sshh 66 33..33.. SSeeaarrcchh PPaatthh All shell scripts should include a search path specifica- tion: PATH=/usr/ucb:/usr/bin:/bin; export PATH A PATH specification is recommended -- often times a script will fail for some people because they have a different or incomplete search path. The Bourne Shell does nnoott eexxppoorrtt environment variables to children unless explicitly instructed to do so. BBeewwaarree:: of "." in the search path, this opens a big hole for trojan horses. PATH=:/usr/ucb:/usr/bin:/bin:; export PATH This example is a vveerryy bbiigg mmiissttaakkee! Watch out for the lead- ing and trailing colon, don't make the mistake. 77 UUWWOO//sshh 33..44.. AArrgguummeenntt CChheecckkiinngg A good shell script should verify that the arguments sup- plied (if any) are correct. if [ $# -ne 3 ]; then echo 1>&2 Usage: $0 19 Oct 91 exit 127 fi This script requires three arguments and gripes accordingly. Some more argument checking (note the caveat): # check range of day (this is quick, not perfect) case "$day" in [1-9]|[123][0-9]) ;; *) echo 1>&2 Day \"$day\" out of range ... exit 127 ;; esac UUWWOO//sshh 88 33..55.. EExxiitt ssttaattuuss All Unix utilities should return an exit status. # is the year out of range for me? if [ $year -lt 1901 -o $year -gt 2099 ]; then echo 1>&2 Year \"$year\" out of range exit 127 fi _e_t_c_._._. # All done, exit ok exit 0 A non-zero exit status indicates an error condition of some sort while a zero exit status indicates things worked as expected. On BSD systems there's been an attempt to categorize some of the more common exit status codes. See //uussrr//iinncclluuddee//ssyysseexxiittss..hh. 99 UUWWOO//sshh 33..66.. UUssiinngg eexxiitt ssttaattuuss Exit codes are important for those who use your code. Many constructs test on the exit status of a command. The conditional construct is: if _c_o_m_m_a_n_d; then _c_o_m_m_a_n_d fi For example, if tty -s; then echo Enter text end with \^D fi Your code should be written with the expectation that others will use it. Making sure you return a meaningful exit status will help. UUWWOO//sshh 1100 33..77.. SSttddiinn,, SSttddoouutt,, SSttddeerrrr Standard input, output, and error are file descriptors 0, 1, and 2. Each has a particular role and should be used accordingly: # is the year out of range for me? if [ $year -lt 1901 -o $year -gt 2099 ]; then echo 1>&2 Year \"$year\" out of my range exit 127 fi _e_t_c_._._. # ok, you have the number of days since Jan 1, ... case `expr $days % 7` in 0) echo Mon;; 1) echo Tue;; _e_t_c_._._. Error messages should appear on stderr not on stdout! 1111 UUWWOO//sshh Output should appear on stdout. As for input/output dialogue (from ppuurrggee((ll))): # give the fellow a chance to quit if tty -s ; then echo This will remove all files in $* since ... echo $n Ok to procede? $c; read ans case "$ans" in n*|N*) echo File purge abandoned; exit 0 ;; esac RM="rm -rfi" else RM="rm -rf" fi NNoottee:: this code behaves differently if there's a user to communicate with (ie. if the standard input is a tty rather than a pipe, or file, or etc. See ttttyy((11))). UUWWOO//sshh 1122 44.. LLaanngguuaaggee CCoonnssttrruuccttss 44..11.. FFoorr lloooopp iitteerraattiioonn Substitute values for variable and perform task: for _v_a_r_i_a_b_l_e in _w_o_r_d _._._. do _c_o_m_m_a_n_d done For example, from ssyyssllooggdd..ddaaiillyy: for i in `cat $LOGS` do mv $i $i.$TODAY cp /dev/null $i chmod 664 $i done Alternatively you may see: for _v_a_r_i_a_b_l_e in _w_o_r_d _._._.; do _c_o_m_m_a_n_d; done 1133 UUWWOO//sshh 44..22.. CCaassee sswwiittcchh Switch to statements depending on pattern match case _w_o_r_d in [ _p_a_t_t_e_r_n [ | _p_a_t_t_e_r_n _._._. ] ) _c_o_m_m_a_n_d ;; ] ... esac For example, from ddaayy((ll)): case "$year" in [0-9][0-9]) year=19${year} years=`expr $year - 1901` ;; [0-9][0-9][0-9][0-9]) years=`expr $year - 1901` ;; *) echo 1>&2 Year \"$year\" out of range ... exit 127 ;; esac UUWWOO//sshh 1144 44..33.. CCoonnddiittiioonnaall EExxeeccuuttiioonn Test exit status of command and branch if _c_o_m_m_a_n_d then _c_o_m_m_a_n_d [ else _c_o_m_m_a_n_d ] fi For example, from ddaayy((ll)): if [ $# -ne 3 ]; then echo 1>&2 Usage: $0 19 Oct 91 exit 127 fi Alternatively you may see: if _c_o_m_m_a_n_d; then _c_o_m_m_a_n_d; [ else _c_o_m_m_a_n_d; ] fi 1155 UUWWOO//sshh 44..44.. WWhhiillee//UUnnttiill IItteerraattiioonn Repeat task while command returns good exit status. {while | until} _c_o_m_m_a_n_d do _c_o_m_m_a_n_d done For example, from ppuurrggee((ll)): # for each argument mentioned, purge that directory while [ $# -ge 1 ]; do _purge $1 shift done Alternatively you may see: while _c_o_m_m_a_n_d; do _c_o_m_m_a_n_d; done UUWWOO//sshh 1166 44..55.. VVaarriiaabblleess Variables are sequences of letters, digits, or underscores beginning with a letter or underscore. Numeric variables (eg. like $1, etc.) are positional vari- ables for argument communication. 44..55..11.. VVaarriiaabbllee AAssssiiggnnmmeenntt Assign a value to a variable by _v_a_r_i_a_b_l_e=_v_a_l_u_e. For example: PATH=/usr/ucb:/usr/bin:/bin; export PATH or TODAY=`(set \`date\`; echo $1)` 1177 UUWWOO//sshh 44..55..22.. EExxppoorrttiinngg VVaarriiaabblleess Variables are nnoott exported to children unless explicitly marked. From xxddmm//XXsseessssiioonn: # We MUST have a DISPLAY environment variable if [ "$DISPLAY" = "" ]; then if tty -s ; then echo "DISPLAY (`hostname`:0.0)? \c"; read DISPLAY fi if [ "$DISPLAY" = "" ]; then DISPLAY=`hostname`:0.0 fi export DISPLAY fi Likewise, for variables like the PPRRIINNTTEERR which you want hon- ored by llpprr((11)). From a users ..pprrooffiillee (which we don't sup- port): PRINTER=PostScript; export PRINTER NNoottee:: that the Cshell exports all environment variables. UUWWOO//sshh 1188 44..55..33.. RReeffeerreenncciinngg VVaarriiaabblleess Use $$vvaarriiaabbllee (or, if necessary, $${{vvaarriiaabbllee}}) to reference the value. # Most user's have a /bin of their own if [ "$USER" != "root" ]; then PATH=$HOME/bin:$PATH else PATH=/etc:/usr/etc:$PATH fi The braces are required for concatenation constructs. $$pp__0011 The value of the variable "p_01". $${{pp}}__0011 The value of the variable "p" with "_01" pasted onto the end. 1199 UUWWOO//sshh 44..55..44.. CCoonnddiittiioonnaall RReeffeerreennccee $${{_v_a_r_i_a_b_l_e--_w_o_r_d}} If the variable has been set, use it's value, else use _w_o_r_d. From xxddmm//XXsseessssiioonn: POSTSCRIPT=${POSTSCRIPT-PostScript}; export POSTSCRIPT $${{_v_a_r_i_a_b_l_e::--_w_o_r_d}} If the variable has been set and is not null, use it's value, else use _w_o_r_d. These are useful constructions for honoring the user envi- ronment. Ie. the user of the script can override variable assignments. Cf. programs like llpprr((11)) honor the PPRRIINNTTEERR environment variable, you can do the same trick with your shell scripts. $${{_v_a_r_i_a_b_l_e::??_w_o_r_d}} If variable is set use it's value, else print out _w_o_r_d and exit. Useful for bailing out. UUWWOO//sshh 2200 44..55..55.. AArrgguummeennttss Command line arguments to shell scripts are positional vari- ables: $$00,, $$11,, ...... The command and arguments. With $$00 the command and the rest the arguments. $$## The number of arguments. $$**,, $$@@ All the arguments as a blank separated string. Watch out for "$*" vs. "$@". And, some commands: sshhiifftt Shift the postional variables down one and decrement number of arguments. sseett _a_r_g _a_r_g _._._. Set the positional variables to the argument list. 2211 UUWWOO//sshh Command line parsing uses sshhiifftt (see also ggeettoopptt((11))): # parse argument list while [ $# -ge 1 ]; do case $1 in _p_r_o_c_e_s_s _a_r_g_u_m_e_n_t_s_._._. esac shift done A use of the sseett command (from ssyyssllooggdd..ddaaiillyy): # figure out what day it is TODAY=`(set \`date\`; echo $1)` cd $SPOOL for i in `cat $LOGS` do mv $i $i.$TODAY cp /dev/null $i chmod 664 $i done UUWWOO//sshh 2222 44..55..66.. SSppeecciiaall VVaarriiaabblleess $$$$ Current process id. This is very useful for construct- ing temporary files. From ccaalleennddaarr((11)): tmp=/tmp/cal0$$ trap "rm -f $tmp /tmp/cal1$$ /tmp/cal2$$" trap exit 1 2 13 15 /usr/lib/calprog >$tmp $$?? The exit status of the last command. From ccttcc((11)): $command # Run target file if no errors and ... if [ $? -eq 0 ] then _e_t_c_._._. fi 2233 UUWWOO//sshh 44..66.. QQuuootteess//SSppeecciiaall CChhaarraacctteerrss Special characters to terminate words: ; & ( ) | ^ < > new-line space tab These are for command sequences, background jobs, etc. To _q_u_o_t_e any of these use a backslash (\\) or bracket with quote marks ("""" or ''''). SSiinnggllee QQuuootteess Within single quotes _a_l_l characters are quoted -- including the backslash. The result is one word. grep :${gid}: /etc/group | awk -F: '{print $1}' DDoouubbllee QQuuootteess Within double quotes you have variable subsitution (ie. the dollar sign is interpreted) but no file name gener- ation (ie. ** and ?? are quoted). The result is one word. if [ ! "${parent}" ]; then parent=${people}/${group}/${user} fi UUWWOO//sshh 2244 BBaacckk QQuuootteess Back quotes mean run the command and substitute the output. if [ "`echo -n`" = "-n" ]; then n="" c="\c" else n="-n" c="" fi and TODAY=`(set \`date\`; echo $1)` 2255 UUWWOO//sshh 44..77.. FFuunnccttiioonnss Functions are a powerful feature that aren't used often enough. Syntax is _n_a_m_e () { _c_o_m_m_a_n_d_s } For example, from ppuurrggee((ll)): # Purge a directory _purge() { # there had better be a directory if [ ! -d $1 ]; then echo $1: No such directory 1>&2 return fi _e_t_c_._._. } UUWWOO//sshh 2266 Within a function the positional parmeters $0, $1, etc. are the arguments to the function (not the arguments to the script). Within a function use rreettuurrnn instead of eexxiitt. Functions are good for encapsulations. You can pipe, redi- rect input, etc. to functions. For example, from aadddduusseerrss((ll)) # deal with a file, add people one at a time do_file() { while parse_one _e_t_c_._._. } _e_t_c_._._. # take standard input (or a specified file) and do it. if [ "$1" != "" ]; then cat $1 | do_file else do_file fi 2277 UUWWOO//sshh 44..88.. SSoouurrcciinngg ccoommmmaannddss You can execute shell scripts from within shell scripts. A couple of choices: sshh _c_o_m_m_a_n_d This runs the shell script as a separate shell. For example, on Sun machines in //eettcc//rrcc sh /etc/rc.local .. _c_o_m_m_a_n_d This runs the shell script from within the current shell script. For example, on NeXT machine in //eettcc//rrcc # Read in configuration information . /etc/hostconfig What are the virtues of each? What's the difference? UUWWOO//sshh 2288 The second form is useful for configuration files where environment variable are set for the script. For example, from bbaacckkuupp((ll)) for HOST in $HOSTS; do # is there a config file for this host? if [ -r ${BACKUPHOME}/${HOST} ]; then . ${BACKUPHOME}/${HOST} fi _e_t_c_._._. Using configuration files in this manner makes it possible to write scripts that are automatically tailored for differ- ent situations. 2299 UUWWOO//sshh 55.. SSoommee TTrriicckkss 55..11.. TTeesstt The most powerful command is tteesstt((11)). if test _e_x_p_r_e_s_s_i_o_n; then _e_t_c_._._. and (note the matching bracket argument) if [ _e_x_p_r_e_s_s_i_o_n ]; then _e_t_c_._._. On System V machines this is a builtin (check out the com- mand //bbiinn//tteesstt). On BSD systems (like the Suns) compare the command //uussrr//bbiinn//tteesstt with //uussrr//bbiinn//[[. UUWWOO//sshh 3300 Useful expressions are: tteesstt {{ --ww,, --rr,, --xx,, --ss,, ...... }} _f_i_l_e_n_a_m_e is file writeable, readable, executeable, empty, etc? tteesstt _n_1 {{ --eeqq,, --nnee,, --ggtt,, ...... }} _n_2 are numbers equal, not equal, greater than, etc.? tteesstt _s_1 {{ ==,, !!== }} _s_2 Are strings the same or different? tteesstt _c_o_n_d_1 {{ --oo,, --aa }} _c_o_n_d_2 Binary oorr; binary aanndd; use !! for unary negation. For example if [ $year -lt 1901 -o $year -gt 2099 ]; then echo 1>&2 Year \"$year\" out of range exit 127 fi Learn this command inside out! It does a lot for you. 3311 UUWWOO//sshh 55..22.. SSttrriinngg mmaattcchhiinngg The test command provides limited string matching tests. A more powerful trick is to match strings with the ccaassee switch. # parse argument list while [ $# -ge 1 ]; do case $1 in -c*) rate=`echo $1 | cut -c3-`;; -c) shift; rate=$1 ;; -p*) prefix=`echo $1 | cut -c3-`;; -p) shift; prefix=$1 ;; -*) echo $Usage; exit 1 ;; *) disks=$*; break ;; esac shift done Of course ggeettoopptt would work much better. UUWWOO//sshh 3322 55..33.. SSyyssVV vvss BBSSDD eecchhoo On BSD systems to get a prompt you'd say: echo -n Ok to procede?; read ans On SysV systems you'd say: echo Ok to procede? \c; read ans In an effort to produce portable code we've been using: # figure out what kind of echo to use if [ "`echo -n`" = "-n" ]; then n=""; c="\c" else n="-n"; c="" fi _e_t_c_._._. echo $n Ok to procede? $c; read ans 3333 UUWWOO//sshh 55..44.. IIss tthheerree aa ppeerrssoonn?? The Unix tradition is that programs should execute as qui- etly as possible. Especially for pipelines, cron jobs, etc. User prompts aren't required if there's no user. # If there's a person out there, prod him a bit. if tty -s; then echo Enter text end with \^D fi The tradition also extends to output. # If the output is to a terminal, be verbose if tty -s <&1; then verbose=true else verbose=false fi UUWWOO//sshh 3344 BBeewwaarree:: just because stdin is a tty that doesn't mean that stdout is too. User prompts should be directed to the user terminal. # If there's a person out there, prod him a bit. if tty -s; then echo Enter text end with \^D >&0 fi Have you ever had a program stop waiting for keyboard input when the output is directed elsewhere? 3355 UUWWOO//sshh 55..55.. CCrreeaattiinngg IInnppuutt We're familiar with redirecting input. For example (from aadddduusseerrss((ll))): # take standard input (or a specified file) and do it. if [ "$1" != "" ]; then cat $1 | do_file else do_file fi alternatively, redirection from a file: # take standard input (or a specified file) and do it. if [ "$1" != "" ]; then do_file < $1 else do_file fi UUWWOO//sshh 3366 You can also construct files on the fly. From ssiiggnnoonn((ll)): rmail bsmtp < rcpt to: data from: <$1@newshost.uwo.ca> to: Subject: Signon $2 subscribe $2 Usenet Feeder at UWO . quit EOF NNoottee:: that variables are expanded in the input. 3377 UUWWOO//sshh 55..66.. SSttrriinngg MMaanniippuullaattiioonnss One of the more common things you'll need to do is parse strings. Some tricks TIME=`date | cut -c12-19` TIME=`date | sed 's/.* .* .* \(.*\) .* .*/\1/'` TIME=`date | awk '{print $4}'` TIME=`set \`date\`; echo $4` TIME=`date | (read u v w x y z; echo $x)` UUWWOO//sshh 3388 With some care, redefining the input field separators can help. #!/bin/sh # convert IP number to in-addr.arpa name name() { set `IFS=".";echo $1` echo $4.$3.$2.$1.in-addr.arpa } if [ $# -ne 1 ]; then echo 1>&2 Usage: bynum IP-address exit 127 fi add=`name $1` nslookup <