{"id":33913,"date":"2016-05-09T13:59:28","date_gmt":"2016-05-09T08:29:28","guid":{"rendered":"http:\/\/www.tothenew.com\/blog\/?p=33913"},"modified":"2024-01-02T17:47:22","modified_gmt":"2024-01-02T12:17:22","slug":"foolproof-your-bash-script-some-best-practices","status":"publish","type":"post","link":"https:\/\/www.tothenew.com\/blog\/foolproof-your-bash-script-some-best-practices\/","title":{"rendered":"“Foolproof Your Bash Script” – Some Best Practices"},"content":{"rendered":"

I am a DevOps practitioner<\/a> and a lazy one too, so whenever we come across any task that needs to be repeated, we create a bash script. I have been\u00a0doing this for a\u00a0long time and after doing a lot\u00a0of mistakes, I figured that if we follow some basic rules we can make our script more portable and less prone to failure when semantics change. These rules and practices have been discussed further in the blog.<\/p>\n

\"\"<\/p>\n

1. Always start with a shebang<\/h2>\n

The first rule of shell scripting is that you always start your scripts with a shebang also called sha-bang, hashbang, pound-bang, or hash-pling. While the name might sound funny the shebang line is very important; it tells the system which binary to use as the interpreter for the script. Without the shebang line, the system doesn’t know what language to use to process the script.<\/p>\n

[js]#!\/bin\/bash[\/js]<\/p>\n

A typical bash shebang line would look like the following:<\/p>\n

2. Use Built-in Shell Options<\/h2>\n

Use set -o errexit (a.k.a. set -e) to make your script exit when a command fails. \u00a0add || true to commands that you allow to fail.
\nUse set -o nounset (a.k.a. set -u) to exit when your script tries to use undeclared variables.
\nUse set -o xtrace (a.k.a set -x) to trace what gets executed. Useful for debugging (optional).
\nUse set -o pipefail in scripts to catch mysqldump fails in e.g. mysqldump |gzip. The exit status of the last command that threw a non-zero exit code is returned.<\/p>\n

[js]#!\/bin\/bash
\nset -o nounset
\nset -o errexit[\/js]<\/p>\n

3. Naming conventions for Variables<\/h2>\n
    \n
  1. Strings should have the form name=value.<\/strong>
    \nIdeally variable names should only consist uppercase letters, digits, and the ‘_’ (underscore).<\/li>\n
  2. Variable Names shall not contain the character ‘=’ or ‘-‘<\/strong>
    \n$ running-process-id=`ps -eopid`\\<\/code><\/span><\/li>\n
  3. No spaces before or after that equal\u00a0sign<\/strong>
    \n$ language = PHP<\/code>
    \n-bash: language: command not found<\/li>\n
  4. Double quotes around every parameter expansion<\/strong>
    \n$ song=\"My song.mp3\"<\/code>
    \n<\/span>$ rm $song<\/code>
    \n<\/span><\/p>\n

    [js]rm: My: No such file or directory
    \nrm: song.mp3: No such file or directory [\/js]<\/p>\n

    $ rm \"$song\"<\/code><\/li>\n

  5. Don’t start Variable Name with special characters or Numbers<\/strong>
    \n$ -song=\"My song.mp3\"<\/code>
    \n<\/span><\/p>\n

    [js]-song=my song.mp3: command not found [\/js]<\/p>\n

    $ 123song=\"My song.mp3\"<\/code>
    \n<\/span><\/p>\n

    [js]123song=my song.mp3: command not found[\/js]<\/p>\n<\/li>\n<\/ol>\n

    4. Variable Annotations<\/h2>\n

    Bash allows for a limited form of variable annotations. The most important ones are:
    \nlocal<\/strong> (for local variables inside a function)
    \nreadonly<\/strong> (for read-only variables)<\/p>\n

    Strive to annotate almost all variables in a bash script with either local or readonly.<\/p>\n

    5. Use $() over backticks<\/h2>\n

    Avoid using backticks ““”, they are hard to read and in some fonts easily confused with single quotes. A lot of quoting needed in nesting.<\/p>\n

    Example<\/strong>:<\/p>\n

    $ echo \"one-$(echo two-$(echo three-$(echo four)))\"<\/code><\/p>\n

    [js]one-two-three-four[\/js]<\/p>\n

    $ echo \"one-`echo two-\\`echo three-\\`\\`echo four\\`\\`\\``\"<\/code><\/p>\n

    [js]one-two-three-four [\/js]<\/p>\n

    6. Prefer using Double Brackets<\/h2>\n

    Let me illustrate how [[ can be used and how it can help you to avoid some of the common mistakes made by using [] or test:<\/p>\n

    $ var=''
    \n$ [ $var = ''\" ] && echo True<\/code><\/p>\n

    [js]-bash: [: =: unary operator expected[\/js]<\/p>\n

    $ [ \"$var\" = '' ] && echo True<\/code><\/p>\n

    [js]True[\/js]<\/p>\n

    $ [[ $var = '' ]] && echo True<\/code><\/p>\n

    [js]True[\/js]<\/p>\n

    Using quotes, [ “$var” = ” ] expands into [ “” = ” ] and test has no problem.<\/p>\n

    Now, [[ can see the whole command before it’s being expanded. It sees $var, and not the expansion of $var. As a result, there is no need for the quotes at all! [[ is safer.<\/p>\n

    $ var=
    \n$ [ \"$var\" < a ] && echo True<\/code><\/p>\n

    [js]-bash: a: No such file or directory[\/js]<\/p>\n

    $ [ \"$var\" \\< a ] && echo True
    \n<\/code><\/p>\n

    [js]True[\/js]<\/p>\n

    $ [[ $var < a ]] && echo True
    \n<\/code><\/p>\n

    [js]True[\/js]<\/p>\n

    In this example, we attempted a string comparison between an empty variable and ‘a’. We’re surprised to see the first attempt does not yield True even though we think it should. Instead, we get some weird error that implies Bash is trying to open a file called ‘a’.<\/p>\n

    We’ve been bitten by File Redirection. Since test is just an application, the < character in our command is interpreted (as it should) as a File Redirection operator instead of the string comparison operator of test. Bash is instructed to open a file ‘a’ and connect it to stdin for reading. To prevent this, we need to escape < so that test receives the operator rather than Bash. This makes our second attempt work.<\/p>\n

    Using [[ we can avoid the mess altogether. [[ sees the < operator before Bash gets to use it for Redirection — problem fixed. Once again, [[ is safer. Even more dangerous is using the > operator instead of our previous example with the < operator. Since > triggers output Redirection it will create a file called ‘a’. As a result, there will be no error message warning us that we’ve committed a sin! Instead, our script will just break. Even worse, we might overwrite some important file! It’s up to us to guess where the problem is:<\/p>\n

    $ var=a
    \n$ [ \"$var\" > b ] && echo True || echo False<\/code><\/p>\n

    [js]True[\/js]<\/p>\n

    $ [[ \"$var\" > b ]] && echo True || echo False
    \n<\/code><\/p>\n

    [js]False[\/js]<\/p>\n

    Two different results, not good. The lesson is to trust [[ more than [. [ “$var” > b ] is expanded into [ “a” ] and the output of that is being redirected into a new file called ‘b’. Since [ “a” ] is the same as [ -n “a” ] and that basically tests whether the “a” string is non-empty, the result is a success and the echo True is executed.<\/p>\n

    Using [[ we get our expected scenario where “a” is tested against “b” and since we all know “a” sorts before “b” this triggers the echo False statement. And this is how you can break your script without realizing it. You will however have a suspiciously empty file called ‘b’ in your current directory.<\/p>\n

    Yes it adds a few characters, but [[ is far safer than [. Everybody inevitably makes programming errors. Even if you try to use [ safely and carefully avoid these mistakes, I can assure you that you will make them. And if other people are reading your code, you can be sure that they’ll absolutely mess things up.<\/p>\n

    7. Regular Expressions\/Globbing<\/h2>\n

    [[<\/strong> provides the following features over [<\/strong> :-<\/p>\n

    t=”abc123″
    \n[[ “$t” == abc* ]] # true (globbing)
    \n[[ “$t” == “abc*” ]] # false (literal matching)
    \n[[ “$t” =~ [abc]+[123]+ ]] # true (regular expression)
    \n[[ “$t” =~ “abc*” ]] # false (literal matching)
    \nNote, that starting with bash version 3.2 the regular or globbing expression
    \nmust not be quoted. If your expression contains whitespace you can store it in a variable:
    \nr=”a b+”
    \n[[ “a bbb” =~ $r ]] # true<\/p>\n

    Globbing based string matching is also available via the case statement:<\/p>\n

    [js]case $t in
    \nabc*) ;;
    \nesac[\/js]<\/p>\n

    8. Avoiding Temporary Files<\/h2>\n

    Some commands expect filenames as parameters so straightforward pipelining does not work.
    \nThis is where <() operator comes in handy as it takes a command and transforms it into something
    \nwhich can be used as a filename:<\/p>\n

    diff <(wget -O - url1) <(wget -O - url2)<\/code><\/p>\n

    Also useful are “here documents” which allow arbitrary multi-line string to be passed
    \nin on stdin. The two occurrences of ‘EOF’ brackets the document.<\/p>\n

    # DELIMITER is an arbitrary string<\/p>\n

    [js]command << EOF
    \nSome importent text
    \n$(cmd)
    \nEOF[\/js]<\/p>\n

    ‘EOF’ can be any text.<\/p>\n

    9. Debugging<\/h2>\n

    To perform a syntax check\/dry run of your bash script run:<\/strong><\/p>\n

    bash -n myscript.sh<\/code><\/p>\n

    To produce a trace of every command executed run:<\/strong><\/p>\n

    bash -v myscripts.sh<\/code><\/p>\n

    To produce a trace of the expanded command use:<\/strong><\/p>\n

    bash -x myscript.sh<\/code><\/p>\n

    -v and -x can also be made permanent by adding
    \nset -o verbose and set -o xtrace to the script prolog.
    \nThis might be useful if the script is run on a remote machine, e.g.
    \na build-bot and you are logging the output for remote inspection.<\/p>\n

    10. Trap forced exit of script<\/h2>\n

    Don’t let your script exit unexpectedly, trap when someone update press ctrl+c and exit from your script gracefully.<\/p>\n

    [js]# trap ctrl-c and call ctrl_c()
    \ntrap ctrl_c INT<\/p>\n

    function ctrl_c() {
    \n echo "** Trapped CTRL-C"
    \n}<\/p>\n

    for i in `seq 1 5`; do
    \n sleep 1
    \n echo -n "."
    \ndone[\/js]<\/p>\n

    This is how simple it is to Foolproof\u00a0\u00a0your bash scripts. In my next blog, I will talk about more interesting features available in Linux OS.<\/p>\n","protected":false},"excerpt":{"rendered":"

    I am a DevOps practitioner and a lazy one too, so whenever we come across any task that needs to be repeated, we create a bash script. I have been\u00a0doing this for a\u00a0long time and after doing a lot\u00a0of mistakes, I figured that if we follow some basic rules we can make our script more […]<\/p>\n","protected":false},"author":747,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":256},"categories":[2348],"tags":[475,519,3315,3317,3318,3319,3313,260,474,3312,3316,448,3314],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/33913"}],"collection":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/users\/747"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/comments?post=33913"}],"version-history":[{"count":1,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/33913\/revisions"}],"predecessor-version":[{"id":59861,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/33913\/revisions\/59861"}],"wp:attachment":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/media?parent=33913"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/categories?post=33913"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/tags?post=33913"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}