A quick overview: What is shell scripting?
Essentially, a shell script is a series of Unix commands. The main benefit of a shell script is that, if you find yourself frequently executing the same commands repeatedly, you can write a shell script and run the commands that way.
All shell scripts should begin with the following line:
#!/bin/bash
This line tells the environment how to interpret the script. Here we have used bash, which indicates the bash shell. Other scripting languages such as awk, perl, and python use this declaration syntax as well.
To run the script, use ./ before the file name – this runs the script if it’s in your current working directory. For example, if your script is called myScript.sh, then you would execute it by running:
./myScript.sh
Getting started
To get started, I’m going to build a BASH script that will copy a file from it’s current directory to another and append the date to its file name.
First, open a new file and add the BASH line indicated above:
#!/bin/bash
Before proceeding, I’m going to make sure my script has execute permissions:
chmod +x myScript.sh
The easiest way for me to proceed is to break up the goal of my program into smaller steps (the “exercises” below) and then use those to earmark my progress.
Exercise 1
Use the shell script to print the current date.
If I am in terminal, I can use date and have it output the date, but if I use echo date then echo will simply print the word “date” and not execute the command. In order to execute the command, I need use the following syntax:
$(<command>)
For example:
> echo $(date)
So I need to update my script to:
#!/bin/bash echo $(date)
Exercise 2
Update script to print a formatted version of the current date.
Using man date I can see that I need to supply a string argument to the date command to control how I want the date formatted. Since I want to use the numeric representation of the year, month, and day and I want to use 24 hour time, I will need to specify %Y, %m, %d, %H, %M, and %S in the appropriate sections of the string. In this case I want the date to be formatted as YYYY-mm-dd_HHMMSS. At the prompt, I test:
date "+%Y-%m-%d_%H%M%S"
The updated shell script is now:
#!/bin/bash echo $(date "+%Y-%m-%d_%H%M%S")
Exercise 3
Print an argument (string) to screen.
In order to copy a file from one place to another, I will need to specify the file. In a shell script, the arguments can be referenced using the order they are specified. $0 is used to reference the script being executed, $1 is the first argument, $2 is the second, etc.
So if I update the script to just echo the arguments:
#!/bin/bash #echo $(date "+%Y-%m-%d_%H%M%S") echo $0 echo $1 echo $2
And then run it, it will print:
> ./myScript.sh hi.png ./myScript.sh hi.png
The date line no longer prints because I’ve commented it out, it echoes the script execution for $0, and “hi.png” as $1. A null line is printed for $2 since there was no second argument specified (note that it does not fail).
Since I’m only going to be using the file name and the date, I will update the script to:
#!/bin/bash echo $(date "+%Y-%m-%d_%H%M%S") echo $1
Exercise 4
Concat argument string and formatted date
In terminal, if I want to concat two strings and print them, I can do something like:
echo $USER::$PATH
This will print my user name (stored in the $USER environmental variable), two colons, and the contents of my $PATH environmental variable. (Note: exactly what these are is outside of the scope of this entry, but feel free to Google environmental variables in Unix.)
I want to tell BASH to execute the date command and append its output to what will be the file name, $1, after an underscore:
#!/bin/bash echo $1_$(date "+%Y-%m-%d_%H%M%S")
Which looks like this:
> ./myScript.sh hi hi_2014-09-30_153741
Exercise 5
Input string must now be a file name. Concat file name, date, and file extension.
This script will encounter a little hiccup, in my opinion, if I use a file name instead of a regular string:
> ./myScript.sh hi.png hi.png_2014-09-30_153741
Ideally, I would want this to look something like hi_2014-09-30_153741.png: i.e. I want the timestamp appended to the file name specifically, not just the end of the string. To do this I’ll need to separate the file name and extension. While I was searching for Unix commands to do this, I found basename, which returns the whole file name (including extension) after extracting it from a path:
> basename /some/path/hi.png hi.png
With some more searching, I found information about Shell Parameter Expansion. So if I use a variable, I can do this:
> FILE="example.tar.gz"
> echo "${FILE%%.*}"
example
> echo "${FILE#*.}"
tar.gz
So now all I need to do is update my script:
#!/bin/bash
fileWithoutPath=$(basename $1)
echo ${fileWithoutPath%%.*}\_$(date "+%Y-%m-%d_%H%M%S")\.${fileWithoutPath#*.}
(The “extra” backslashes are to escape the underscore and period.) Although accurate, let’s try to make that a *little* more readable:
#!/bin/bash
formattedDate=$(date "+%Y-%m-%d_%H%M%S")
inputFile=$1
fileWithoutPath=$(basename $inputFile)
filename=${fileWithoutPath%%.*}
extension=${fileWithoutPath#*.}
updatedFilename=$filename\_$formattedDate\.$extension
echo $updatedFilename
Why the “additional” variables? Well, I happen to know that I intend to use flags as a learning exercise below, so it is easy to change $inputFile and point it to the appropriate flag value instead of changing all the instances of $1 later. Similarly, naming the other items allows me to very clearly read what $updatedFilename is. Not a great concern in as short a script as this one, but could come in with something more complicated.
Exercise 6
Copy file with updated name in current directory
For this we just need to use the cp command:
#!/bin/bash
formattedDate=$(date "+%Y-%m-%d_%H%M%S")
inputFile=$1
fileWithoutPath=$(basename $inputFile)
filename=${fileWithoutPath%%.*}
extension=${fileWithoutPath#*.}
updatedFilename=$filename\_$formattedDate\.$extension
echo $updatedFilename
cp $inputFile $updatedFilename
Exercise 7
Copy file to specified directory
The behavior I’ve decided on for this script is to copy the file to the current working directory if none is specified or to use a specified directory. To accomplish this, I’m going to use a simple switch statement (and remove the echo lines):
#!/bin/bash
formattedDate=$(date "+%Y-%m-%d_%H%M%S")
inputFile=$1
outputDir=$2
fileWithoutPath=$(basename $inputFile)
filename=${fileWithoutPath%%.*}
extension=${fileWithoutPath#*.}
updatedFilename=$filename\_$formattedDate\.$extension
if [ -z "$outputDir" ]; then
cp $inputFile $updatedFilename
else
cp $inputFile $outputDir/$updatedFilename
fi
Again, since I know I will be using flags I set $2 to the variable $outputDir. The -z flag in the if statement checks if the specified variable has a zero length string.
Exercise 8
Use flags instead of $1, $2, etc.
To keep it simple, I’m going to use single letter flags which will allow me to use getops. I am going to use -f for the input file, -d for the output directory, and -h for “help” (but I’m not going to write the help info yet).
In order to tell OPTARG that -f and -d need arguments I trail them with a colon. The loop goes through the arguments provided and matches them in the case statement. Since I haven’t written any help information, I just have it printing that the option was called:
#!/bin/bash
while getopts "f:d:h" opt; do
case $opt in
f) inputFile="$OPTARG"
;;
d) outputDir="$OPTARG"
;;
h)
echo "User used h!"
;;
esac
done
formattedDate=$(date "+%Y-%m-%d_%H%M%S")
#inputFile=$1
#outputDir=$2
fileWithoutPath=$(basename $inputFile)
filename=${fileWithoutPath%%.*}
extension=${fileWithoutPath#*.}
updatedFilename=$filename\_$formattedDate\.$extension
if [ -z "$outputDir" ]; then
cp $inputFile $updatedFilename
else
cp $inputFile $outputDir/$updatedFilename
fi
Note that I had minimal changes to the logic I already wrote – I commented out (and will delete) the where I set $inputFile and $outputDir to $1 and $2, respectively, and set the variables in the while loop/case statement. I didn’t need to hunt down multiple uses of $1 and $2 and replace them because I had set them to variables. Handy :)
Exercise 9
Create help output
To create the help output, I looked around to see if there is any “canon” way to do this. Looks like one preferred method is to just store the help output to a string and then print that string when the help flag is used. When working out this bit of code, I noticed that the order of the flags matters – so if I have -h last, then the program will not execute in the way that I intend (it will look for an argument for -f, etc.). I also added exit 0 to the -h block. exit 0 means that the program exited without errors.
#!/bin/bash
# Help output
usage="$(basename "$0") [-h] [-f <string> -d <string>] -- script copies file and appends timestamp to file name using YYYYmmdd-HHMMSS format.
where:
-h show this help text
-f set the input file
-d set the output directory, if unset will copy to current working directory"
###
while getopts "f:d:h" opt; do
case $opt in
h)
echo "$usage"
exit 0
;;
f) inputFile="$OPTARG"
;;
d) outputDir="$OPTARG"
;;
esac
done
formattedDate=$(date "+%Y-%m-%d_%H%M%S")
fileWithoutPath=$(basename $inputFile)
filename=${fileWithoutPath%%.*}
extension=${fileWithoutPath#*.}
updatedFilename=$filename\_$formattedDate\.$extension
if [ -z "$outputDir" ]; then
cp $inputFile $updatedFilename
else
cp $inputFile $outputDir/$updatedFilename
fi
So now when I use the -h flag, I see:
> ./myScript.sh -h myScript.sh [-h] [-f <string> -d <string>] -- script copies file and appends timestamp to file name using YYYYmmdd-HHMMSS format. where: -h show this help text -f set the input file -d set the output directory, if unset will copy to current working directory
Exercise 10
Some basic error handling
The main ways this script will encounter a problem are if:
- There is no input file specified
- An invalid flag is provided
- A specified input file does not exist
- A specified output directory does not exist
Since getops goes through each of the options in sequence, before the rest of the program runs I am going to have it check that at least one argument has been provided and exit if there are none:
if [[ $# == 0 ]]; then echo "An input file is required." echo '' echo "$usage" exit 1 fi
Note: $# counts the number of arguments provided.
Next, to check that the flags provided are correct, I will add the following to the getops while loop:
\?)
echo "Invalid option: please reference help below" >&2
echo ''
echo "$usage"
exit 2
In order to check the input file and output directories, I’m going to use a similar if statement as I did when I used -z to see if the value provided was a non-empty string. To check if the file exists, I will use -f. For example, with the input file:
if [ ! -f "$inputFile" ]; then echo "Input file not found!" echo '' echo "$usage" exit 2 fi
Now to put it all together:
#!/bin/bash
# Help output
usage="$(basename "$0") [-h] [-f <string> -d <string>] -- script copies file and appends timestamp to file name using YYYYmmdd-HHMMSS format.
Options:
-h show this help text
-f set the input file
-d set the output directory, if unset will copy to current working directory"
###
if [[ $# == 0 ]]; then
echo "An input file is required."
echo ''
echo "$usage"
exit 1
fi
while getopts "f:d:h" opt; do
case $opt in
h)
echo "$usage"
exit 0
;;
f) inputFile="$OPTARG"
;;
d) outputDir="$OPTARG"
;;
\?)
echo "Invalid option: please reference help below" >&2
echo ''
echo "$usage"
exit 2
;;
esac
done
if [ ! -f "$inputFile" ]; then
echo "Input file not found!"
echo ''
echo "$usage"
exit 2
fi
formattedDate=$(date "+%Y-%m-%d_%H%M%S")
fileWithoutPath=$(basename $inputFile)
filename=${fileWithoutPath%%.*}
extension=${fileWithoutPath#*.}
updatedFilename=$filename\_$formattedDate\.$extension
if [ -z "$outputDir" ]; then
cp $inputFile $updatedFilename
else
if [ ! -f "$ouputDir" ]; then
echo "Output directory not found!"
echo ''
echo "$usage"
exit 2
fi
cp $inputFile $outputDir/$updatedFilename
fi
Success! I now have a shell script that will copy a file to another directory, append the timestamp, and do some basic error checking.
The Coding Mant.is 