UNIX Shell-Scripting Basics

(1)

UNIX Shell-Scripting Basics

(2)

Agenda

 What is a shell? A shell script?

 Introduction to bash

 Running Commands

 Applied Shell Programming

(3)

What is a shell?

%▌

(4)

What is a shell?

/bin/bash

(5)

What is a shell?

#!/bin/bash

(6)

What is a shell?

INPUT

shell

OUTPUT ERROR

(7)

What is a shell?

 Any Program

 But there are a few popular shells…

(8)

Bourne Shells

 /bin/sh

 /bin/bash

“Bourne-Again Shell”

(9)

Other Common Shells

 C Shell (/bin/csh)

 Turbo C Shell (/bin/tcsh)

 Korn Shell (/bin/ksh)

(10)

An aside: What do I mean by /bin ?

 C Shell (/bin/csh)

 Turbo C Shell (/bin/tcsh)

 Korn Shell (/bin/ksh)

(11)

An aside: What do I mean by /bin ?

 /bin, /usr/bin, /usr/local/bin

 /sbin, /usr/sbin, /usr/local/sbin

 /tmp

 /dev

 /home/borwicjh

(12)

What is a Shell Script?

 A Text File

 With Instructions

 Executable

(13)

What is a Shell Script?

% cat > hello.sh <<MY_PROGRAM

#!/bin/sh

echo ‘Hello, world’

MY_PROGRAM

% chmod +x hello.sh

% ./hello.sh Hello, world

(14)

What is a Shell Script? A Text File

% cat > hello.sh <<MY_PROGRAM

#!/bin/sh

MY_PROGRAM

% chmod +x hello.sh

(15)

An aside: Redirection

 cat > /tmp/myfile

 cat >> /tmp/myfile

 cat 2> /tmp/myerr

 cat < /tmp/myinput

 cat <<INPUT Some input INPUT

 cat > /tmp/x 2>&1

INPUT

env

OUTPUT ERROR

0

1 2

(16)

What is a Shell Script? How To Run

#!/bin/sh

MY_PROGRAM

% chmod +x hello.sh

(17)

What is a Shell Script? What To Do

#!/bin/sh

echo ‘Hello, world’

MY_PROGRAM

% chmod +x hello.sh

(18)

What is a Shell Script? Executable

#!/bin/sh

MY_PROGRAM

% chmod +x hello.sh

(19)

What is a Shell Script? Running it

#!/bin/sh

MY_PROGRAM

% chmod +x hello.sh

% ./hello.sh Hello, world

(20)

Finding the program: PATH

 % ./hello.sh

 echo vs. /usr/bin/echo

 % echo $PATH

/bin:/usr/bin:/usr/local/bin:

/home/borwicjh/bin

 % which echo /usr/bin/echo

(21)

Variables and the Environment

% hello.sh

bash: hello.sh: Command not found

% PATH=“$PATH:.”

% hello.sh

Hello, world

(22)

An aside: Quoting

% echo ‘$USER’

$USER

% echo “$USER”

borwicjh

% echo “\””

”

% echo “deacnet\\sct”

deacnet\sct

% echo ‘\”’

\”

(23)

Variables and the Environment

% env

[…variables passed to sub-programs…]

% NEW_VAR=“Yes”

% echo $NEW_VAR Yes

% env

[…PATH but not NEW_VAR…]

% export NEW_VAR

% env

(24)

Welcome to Shell Scripting!

(25)

How to Learn

 man

 man bash

 man cat

 man man

 man –k

 man –k manual

 Learning the Bash Shell, 2^nd Ed.

 “Bash Reference” Cards

http://www.tldp.org/LDP/abs/html/

(26)

Introduction to bash

(27)

Continuing Lines: \

% echo This \ Is \

A \ Very \ Long \

Command Line

This Is A Very Long Command Line

(28)

Exit Status

 $?

 0 is True

% ls /does/not/exist

% echo $?

1

% echo $?

0

(29)

Exit Status: exit

% cat > test.sh <<_TEST_

exit 3 _TEST_

% chmod +x test.sh

% ./test.sh

% echo $?

3

(30)

Logic: test

% test 1 -lt 10

% echo $?

0

% test 1 == 10

% echo $?

1

(31)

Logic: test

 test

 [ ]

 [ 1 –lt 10 ]

 [[ ]]

 [[ “this string” =~ “this” ]]

 (( ))

 (( 1 < 10 ))

(32)

Logic: test

 [ -f /etc/passwd ]

 [ ! –f /etc/passwd ]

 [ -f /etc/passwd –a –f /etc/shadow ]

 [ -f /etc/passwd –o –f /etc/shadow ]

(33)

An aside: $(( )) for Math

% echo $(( 1 + 2 )) 3

% echo $(( 2 * 3 )) 6

% echo $(( 1 / 3 )) 0

(34)

Logic: if

if something then

:

# “elif” a contraction of “else if”:

elif something-else then

: else then : fi

(35)

Logic: if

if [ $USER –eq “borwicjh” ] then

:

# “elif” a contraction of “else if”:

elif ls /etc/oratab then

: else then :

(36)

Logic: if

# see if a file exists if [ -e /etc/passwd ] then

echo “/etc/passwd exists”

else

echo “/etc/passwd not found!”

fi

(37)

Logic: for

for i in 1 2 3 do

echo $i done

(38)

Logic: for

for i in /*

do

echo “Listing $i:”

ls -l $i read

done

(39)

Logic: for

for i in /*

do

ls -l $i read

done

(40)

Logic: for

for i in /*

do

ls -l $i read

done

(41)

Logic: C-style for

for (( expr1 ; expr2 ; expr3 )) do

list done

(42)

Logic: C-style for

LIMIT=10

for (( a=1 ; a<=LIMIT ; a++ )) do

echo –n “$a ” done

(43)

Logic: while

while something do

: done

(44)

Logic: while

a=0; LIMIT=10

while [ "$a" -lt "$LIMIT" ] do

echo -n "$a ” a=$(( a + 1 )) done

(45)

Counters

COUNTER=0

while [ -e “$FILE.COUNTER” ] do

COUNTER=$(( COUNTER + 1)) done

 Note: race condition

(46)

Reusing Code: “Sourcing”

% cat > /path/to/my/passwords <<_PW_

FTP_USER=“sct”

_PW_

% echo $FTP_USER

% . /path/to/my/passwords

% echo $FTP_USER sct

%

(47)

Variable Manipulation

% FILEPATH=/path/to/my/output.lis

% echo $FILEPATH

/path/to/my/output.lis

% echo ${FILEPATH%.lis}

/path/to/my/output

% echo ${FILEPATH#*/}

path/to/my/output.lis

% echo ${FILEPATH##*/}

(48)

It takes a long time to become a bash

guru…

(49)

Running Programs

(50)

Reasons for Running Programs

 Check Return Code

 $?

 Get Job Output

 OUTPUT=`echo “Hello”`

 OUTPUT=$(echo “Hello”)

 Send Output Somewhere

 Redirection: <, >

 Pipes

(51)

Pipes

 Lots of Little Tools echo “Hello” | \ wc -c

INPUT

echo

OUTPUT ERROR

0

1 2

INPUT

wc 0

A Pipe!

(52)

Email Notification

% echo “Message” | \

mail –s “Here’s your message” \ [email protected]

(53)

Dates

% DATESTRING=`date +%Y%m%d`

% echo $DATESTRING 20060125

% man date

(54)

FTP the Hard Way

ftp –n –u server.wfu.edu <<_FTP_

user username password put FILE

_FTP_

(55)

FTP with wget

 wget \

ftp://user:[email protected]/file

 wget –r \

ftp://user:[email protected]/dir/

(56)

FTP with curl

curl –T upload-file \

-u username:password \

ftp://server.wfu.edu/dir/file

(57)

Searching: grep

% grep rayra /etc/passwd

% grep –r rayra /etc

% grep –r RAYRA /etc

% grep –ri RAYRA /etc

% grep –rli rayra /etc

(58)

Searching: find

% find /home/borwicjh \ -name ‘*.lis’

[all files matching *.lis]

% find /home/borwicjh \

-mtime -1 –name ‘*.lis’

[*.lis, if modified within 24h]

% man find

(59)

Searching: locate

% locate .lis

[files with .lis in path]

% locate log

[also finds “/var/log/messages”]

(60)

Applied Shell Programming

(61)

Make Your Life Easier

 TAB completion

 Control+R

 history

 cd -

 Study a UNIX Editor

(62)

pushd/popd

% cd /tmp

% pushd /var/log /var/log /tmp

% cd ..

% pwd /var

% popd /tmp

(63)

Monitoring processes

 ps

 ps –ef

 ps –u oracle

 ps –C sshd

 man ps

(64)

“DOS” Mode Files

 #!/usr/bin/bash^M

 FTP transfer in ASCII, or

 dos2unix infile > outfile

(65)

sqlplus

JOB=“ZZZTEST”

PARAMS=“ZZZTEST_PARAMS”

PARAMS_USER=“BORWICJH”

sqlplus $BANNER_USER/$BANNER_PW << _EOF_

set serveroutput on set sqlprompt ""

EXECUTE WF_SATURN.FZ_Get_Parameters('$JOB', '$PARAMS', '$PARAMS_USER');

(66)

sqlplus

sqlplus $USER/$PASS @$FILE_SQL \ $ARG1 $ARG2 $ARG3

if [ $? –ne 0 ] then

exit 1 fi

if [ -e /file/sql/should/create ] then

[…use SQL-created file…]

fi

Ask Amy Lamy! 

(67)

Passing Arguments

% cat > test.sh <<_TEST_

echo “Your name is \$1 \$2”

_TEST_

% chmod +x test.sh

% ./test.sh John Borwick ignore- this

Your name is John Borwick

(68)

INB Job Submission Template

$1: user ID

$2: password

$3: one-up number

$4: process name

$5: printer name

% /path/to/your/script $UI $PW \ $ONE_UP $JOB $PRNT

(69)

Scheduling Jobs

% crontab -l

0 0 * * * daily-midnight-job.sh 0 * * * * hourly-job.sh

* * * * * every-minute.sh 0 1 * * 0 1AM-on-sunday.sh

% EDITOR=vi crontab –e

% man 5 crontab

(70)

(71)

UNIX Shell-Scripting Basics