In this second installment, I’ll discuss why I chose a shell (Bash) to script the Object Storage libraries, and I’ll discuss some of the more complex issues when handling errors in Bash.
Language wars: Some may say that Python would have been a better choice than Bash. The main reasons I chose Bash are:
- While Python is a first-class object oriented language, during my prototyping I quickly discovered that all I was doing was setting up strings to call the curl command (the Linux function used to transact with Object Storage), or doing something similar with the indigenous networking libraries.
- I guessed that more people would understand Bash than Python, and that running Bash scripts is a lot easier since no assumptions about libraries or library versions need to be made.
- Debugging is made easier by the fact that the CLI (command line interface) is by default Bash in Linux, and any command in a script file can be pasted into the CLI – and work.
- Functions written in Bash can easily be included (imported) into the context of the CLI, thereby making the low-level functions accessible to the CLI user. For example, one of the Object Storage functions I wrote is called SLObjectStorageDownloadFile which downloads a file (an object) from a particular Object Storage region. I could use this to manually download an object if I needed.
- Speed of execution is not really an issue either. When you’re transacting gigabytes of data, the milliseconds you save by using Python becomes meaningless; both get to curl pretty quickly and then the execution is stuck there. The aim should be to use curl successfully.
I did learn that in choosing Bash, I lost the try/catch paradigm of Python; catching errors in Bash can be difficult and tedious. With complex while loops, bash will “push” a new sub-shell context in which that complex code is run. Any errors coming from a shell command (including a sub-shell) are caught by interrogating the special shell variable $? in the outer (or parent) shell. Normally a running command or sub-shell will call the exit function to terminate, and that function will either have a value of 0 (zero) or some non-zero value depending on the error. Let’s step through what I’m talking about here:
From the CLI, type the following command (all commands in italics):
date
Tue Jan 2 16:33:11 CST 2018
echo $?
0
Here I typed the date command which printed the current date as terminal output. I then used the echo command to print the value of the special shell variable $? which had a value of zero. This means that the command executed successfully.
Now let’s try an error:
date blah
date: invalid date `blah’
echo $?
1
Same thing here, except I purposely misused the date command to force an error. Please note that the $? variable will change after the execution of each command, including the echo in this example.
A typical construct would look something like this:
date
if [ $? != 0 ]
then
exit 1
fi
Careful here, because if you change the date command to force an error, the exit command will terminate your current shell, and if this is the shell (CLI) you logged into, you will be logged out.
An even more succinct syntax is:
if ! date
then
exit 1
fi
Sub shells: You can easily push a new shell, by typing:
bash
Assuming you just logged in, and got your first CLI shell, this will “push” a new shell. To exit from the sub-shell, you use the exit command, which will terminate the sub-shell and bounce you back to the first shell (the one you logged in with). Typing exit again, will terminate the last shell and will send you back to the login prompt. So in this sense, bash is just another command like date that will run as a child process; the shell that runs the command being the parent process. When you start a process (by invoking a command), the parent will kindly wait until the child process has finished.
You can invoke commands where the parent will not wait for the child process to finish, indeed as soon as you do this the parent process will come right back to the CLI prompt. The child in this case is called a background process. Try the following:
sleep 15
This command will create a child process that itself will simply wait for 15 seconds, and then exit. The parent will wait for the child process to finish, and then go back to the CLI prompt. To run the command in the background, do the following:
sleep 15 &
[1] 27119
Notice that the child process number (process ID) will be echoed back to you on the terminal, and the parent (your shell) will end up back at the CLI prompt. The next time you hit enter at the terminal (for whatever reason), if 15 seconds has elapsed, you will be notified that the background process terminated.
[1]+ Done
If the child process has any output, it will also appear as terminal output – which may confuse you especially if you are running multiple processes.
You can push a shell yourself, and run multiple command in that sub-shell:
( sleep 5 ; date )
This will push a new shell, with its own child process ID, and the sleep and date command run in that shell, will themselves be children of this new shell. You can even put this entire sub-shell in the background:
( sleep 5 ; date ) &
[1] 5022
# Wed Jan 3 16:54:44 CST 2018
In this case, you should get the prompt immediately after the child process ID has been displayed, and then 5-seconds later you’ll have the date displayed (from the grand-child process for date).
The reason I took you through this is to explain what’s going on when you need to trap errors in complex loops. I’ll explain further by discussing a simple loop.
while [ $somecondition == false ]
do
echo “try: $somecondition”
‘ multiple lines of complex code here
done
The above is a typical construct using a while loop (while the condition is not met, the code between the do and done will be executed). When the shell encounters such a construct, and the code between the do/done is too complex to evaluate in the current shell, a new shell is pushed and the complex chunk of code is executed in that sub-shell (including in this example the echo command). So in effect, it’s like you did this:
while [ $somecondition == false ]
do
(
echo “try: $somecondition”
‘ multiple lines of complex code here
)
done
Notice how I explicitly pushed a child shell, to execute the complex code by surrounding a portion of the code with brackets. This is what the shell would do automatically for you. The problem is that any shell variables that are altered or come into existence in the child-shell will have no effect in the parent shell. All exported shell variables in the parent shell, will have exactly the same values after the sub-shell returns, as before – no change. Any variables instanced in the sub-shell will not exist in the parent shell. I highly recommend that you use the second form, and explicitly wrap complex code with the brackets. That way you can augment the above, like this:
while [ $somecondition == false ]
do
(
echo “try: $somecondition”
‘ multiple lines of complex code here
exit 0
)
if [ $? != 0 ]
then
break
fi
done
Notice that, here, I can test the special shell variable $? , which can be set with the correct use of the exit command inside the sub-shell, and tested in the parent shell.
