# Bash Programming Guidelines and Personal Conventions

The source of most of this guidance comes from previous experience. Particularly [salt-bootstrap](https://github.com/saltstack/salt-bootstrap) and [Nvidia Jetson](https://developer.nvidia.com/embedded/linux-tegra-r3272).

## General Formatting Guidelines

### Be posix compliant

Avoid [bashisms](https://hackmd.io/@nucore/rk5IKhclC?utm_source=preview-mode&utm_medium=rec) when possible. Many systems execute system level code using posix compliant shells like DASH or Bourne Shell. Reference [https://mywiki.wooledge.org/Bashism](https://mywiki.wooledge.org/Bashism)

### Use ALLCAPS for variable names

### Prepend single underscore to variables that SHOULD NEVER change

These variables should be named in such a way that their value is obvious.

Example

```bash
_TRUE=1
_FALSE=0
```

### Prepend double underscores to calculated variables

These variables are set by code elsewhere in the script.

Example

```bash
#!/usr/bin/env bash

__SCRIPTARGS="$*"
__SCRIPTFULLNAME=$(realpath "$0")
__SCRIPTNAME=$(basename "${__SCRIPTFULLNAME}")
__SCRIPTFULLPATH=$(dirname "${__SCRIPTFULLNAME}")

__GIT_STATUS=$(git status --porcelain)
if [ $? -ne 128 ]; then
    __COMMIT_ID=$(git rev-parse HEAD)
    # If the git command succeeded, check for uncommitted changes
    if [ -n "${__GIT_STATUS}" ]; then
        echo "Warning: There are uncommitted changes in the repository. These changes will be included in the build."
        __SOURCE_MODIFIERS="m" # m for modified, since there are uncommitted changes
    fi
else
    echo "Warning: Not a git repository. Skipping git status check."
    __COMMIT_ID="${COMMIT_ID}"
    __SOURCE_MODIFIERS="u" # u for unverified, since we can't verify the commit ID without git
fi
__COMMIT_ID_SHORT=$(echo "${__COMMIT_ID}" | cut -c1-8)
```

### Use all lowercase for function names

This differentiates VARIABLES from functions.

### Prepend double underscores to functions

This avoids possible conflicts with shell built-ins and other shell programs and commands.

The exception is when intentionally over riding a known command. In these cases, put the main logic in a function with a double underscore and reference the double underscore function from the over riding function.

Example

```bash
#!/usr/bin/env bash

#===============================================================================
#=====                              FUNCTIONS                              =====
#===============================================================================
#---  FUNCTION  ----------------------------------------------------------------
#         NAME:  __realpath
#  DESCRIPTION:  Download if file doesn't exist.
#        USAGE:  __realpath [file]
#        NOTES:  [file] can be a file or a directory. [file] must exist.
#-------------------------------------------------------------------------------
__realpath() {
    perl -e 'use Cwd "abs_path";print abs_path(shift)' "$1"
}

#---  FUNCTION  ----------------------------------------------------------------
#         NAME:  realpath
#  DESCRIPTION:  Cross-platform realpath command. Because Mac.
#        USAGE:  realpath [file]
#        NOTES:  [file] can be a file or a directory. [file] must exist.
#                Overrides the systeam realpath command and calls __realpath.
#                This exists due to some historical incompatibilities with
#                realpath on MAC os.
#-------------------------------------------------------------------------------
realpath() {
    __realpath $@
}
```

## Separate Large Scripts

Break large scripts into multiple files. Use .insh as the file extension for script blocks that are intended to be included as part of a larger script. These .insh files will be included using the `source` command.

### Script Configuration Files

Create a file, such as CONFIG.insh, which will contain only values that are expected to be modified by down stream users. These files should only contain configuration variables to be set by the user and comments. The comments should give a description of the configuration option and provide guidance as to how the value should be formatted. For values that have a limited set of choices, provide the list of allowed values in the comments.

Use the constants `$_TRUE` and `$_FALSE` for boolean configuration options. Obviously, any special variables, constants, or functions need to be imported prior to importing the CONFIG file.

There should not be any functions or programming logic in the config file. Keep it as simple as possible.

Example

```bash
#!/usr/bin/env bash

#===============================================================================
#=====                           SCRIPT OPTIONS                            =====
#===============================================================================
# Modify these values to change how the script operates.

# STATIC_VARIABLE:
#     This is an example of a configuration variable set to a static value. It
#     cannot be overridden except by modifying this file.
STATIC_VARIABLE="My Static Setting"
# OVERRIDABLE_VARIABLE:
#     This is an example of a configuration variable that can be modified by
#     environment variables. For example:
#         OVERRIDABLE_VARIABLE="NewValue" script.sh
OVERRIDABLE_VARIABLE="${OVERRIDABLE_VARIABLE:-DefaultValue}"
# BOOLEAN_VARIABLE: A boolean variable. 1 = True, anything else is False.
BOOLEAN_VARIABLE="${BOOLEAN_VARIABLE:-$_TRUE}"
# COMMIT_ID: The full SHA1 of the target GIT commit to build, or "latest".
COMMIT_ID="${COMMIT_ID:-latest}"
# BRANCH: The git branch to pull. e.g. master, release, develop, ...
BRANCH="${BRANCH:-master}"
# CLONE_DEPTH: How much commit history to pull
CLONE_DEPTH="${CLONE_DEPTH:-50}"
# WORK_DIR: The working directory used by build script
WORK_DIR="${WORK_DIR:-.build}"
# BUILD_DIR: The build directory
BUILD_DIR="${BUILD_DIR:-build}"
# INSTALL_DIR: The directory the app will be installed to
INSTALL_DIR="${INSTALL_DIR:-AppDir}"
# DISTRO: The build distro
DISTRO="${DISTRO:-ubuntu}"
```

### Reusable Functions

EXAMPLE

```bash
#!/usr/bin/env bash
# Common variables and functions

_TRUE=1
_FALSE=0

#===============================================================================
#=====                              FUNCTIONS                              =====
#===============================================================================
#---  FUNCTION  ----------------------------------------------------------------
#          NAME:  __check_command_exists
#   DESCRIPTION:  Check if a command exists.
#         USAGE:  if [!] (__check_command_exists [command]); then
#-------------------------------------------------------------------------------
__check_command_exists() {
    command -v "$1" > /dev/null 2>&1
}
```

## General Guidelines

Use [ShellCheck](https://www.shellcheck.net/). [ShellCheck for VSCode](https://www.shellcheck.net/).

Limit line length to *approx* 80 characters, when possible.

Apply [PEP 8](https://peps.python.org/pep-0008/) style guidelines where practical.

Start bash files with

```bash
#!/usr/bin/env bash
```

Declare global variables at the beginning of the script. Include a short comment to add context to the variables.

```bash
# name of BUP file generated by Nvidia script
BUPFILE="${BUPFILE:-bl_update_payload}"
```

Create variables that can be overridden

```bash
# ORIG_DIR: starting directory where the script will drop the user on exit
ORIG_DIR="${ORIG_DIR:-$(pwd)}"
```

Properly configured variables can be overridden at execution. For example

```bash
wp@host:~/$ ORIG_DIR="My override value" ./my_script.sh
```

Allow easy debugging early in the script. The following code block will enable easy debugging, which will print each line of code before it is executed

```bash
if [ ! -z ${DEV+x} ] && [ "$DEV" -eq "1" ]; then
    # enable early debugging with `DEV=1 build.sh`
    set -x
    PS4='+(${BASH_SOURCE}:${LINENO}): ${FUNCNAME[0]:+${FUNCNAME[0]}(): }'
else
    set -eu
fi
```

The script can be debugged by prefixing the script with the correct parameter assignment.

```bash
wp@host:~/$ DEV=1 ./my_script.sh
```

Group functions together under the following banner

```bash
#===============================================================================
#=====                              FUNCTIONS                              =====
#===============================================================================
```

Create a header for each function using the following template

```bash
#---  FUNCTION  ----------------------------------------------------------------
#          NAME:  __wget
#   DESCRIPTION:  Download if file doesn't exist.
#         USAGE:  __wget [outfile] [url]
#         NOTES:  if [outfile] exists, it will be tested to verify it is a
#                 regular file
#-------------------------------------------------------------------------------
__wget() {
    # check usage
    if [ -z "${1}" ] || [ -z "${2}" ]; then
        echo "__wget function usage:"
        echo " \$ __wget [outfile] [url]"
        exit 1
    fi
    # Create the DOWNLOADS directory if it doesn't already exist
    if [ ! -d "${DOWNLOADS}" ]; then
        mkdir -p "${DOWNLOADS}" || ( echo "error creating \"${DOWNLOADS}\" directory" && exit 1 )
    fi
    # test if [outfile] exists but is not a regular file
    # ie: [outfile] is device, directory, link, etc
    if [ -e "${DOWNLOADS}/${1}" ] && [ ! -f "${DOWNLOADS}/${1}" ]; then
        echo "__wget: [outfile] \"${DOWNLOADS}/${1}\" exists, but is not a regular file"
        exit 1
    fi
    # download [outfile] if it doesn't already exist
    if [ ! -e "${DOWNLOADS}/${1}" ]; then
        if ! (__check_command_exists wget); then
            __sudo apt-get update || ( echo "Unable to update apt packages" && exit 1 )
            __sudo apt-get install wget || ( echo "Unable to install wget" && exit 1 )
        fi
        if [ -f "../${DOWNLOADS}/${1}" ]; then
            echo "Copying ${1}"
            cp -v "../${DOWNLOADS}/${1}" "${DOWNLOADS}/${1}" || ( echo "error copying ${1} from parent directory" && exit 1 )
        else
            echo "Downloading ${1}"
            wget -q --show-progress -O "${DOWNLOADS}/${1}" "${2}" || ( echo "error downloading ${1}" && exit 1 )
        fi
    fi
}
```

For large, complex scripts, create a banner marking the start of execution, such as

```bash
#===============================================================================
#=====                                BEGIN                                =====
#===============================================================================
```

Use sub-banners if needed

```bash
#----------------------------------------------------------------------------------------------------------------------
#-----                                            Command Line Args                                               -----
#----------------------------------------------------------------------------------------------------------------------
```

Include a useful help

```bash
#---  FUNCTION  -------------------------------------------------------------------------------------------------------
#         NAME:  __usage
#  DESCRIPTION:  Display program help
#        USAGE:  __usage
#----------------------------------------------------------------------------------------------------------------------
__usage() {
    cat << EOT

Usage: ${__SCRIPTNAME} [options] [action]...

Actions:
  - dtb         Build Devicetree binary files
  - blob        Build Mass Flash tool to flash Nano over USB
  - bup         Build Bootloader Update Package
  - tpm         Build the TPM kernel modules
  - tpm-rng     Build the TPM hardware random number generator module
  - pkg         Build the update package to distribute to Nano
  - dev         Enable development (debug) mode. Mainly, disables cleanup of
                temporary files created during program execution
  - all         Same as ${__SCRIPTNAME} dtb bup tpm pkg blob
                Does not build the optional tpm-rng module
  - help        Display usage

Options:
  -h Display this help
  -D Enable debugging mode

Examples:
  - ${__SCRIPTNAME} all
  - ${__SCRIPTNAME} all dev
  - ${__SCRIPTNAME} dtb tpm bup

EOT
}
```

Bash test. Returns True if test expression passes

```bash
    if [ -z "${MAKE_TARGETS}" ]; then
        # do stuff
    fi
```

Bash negative test. Returns True if test expression does not passes

```bash
    if [ ! -z "${MAKE_TARGETS}" ]; then
        # do stuff
    fi
```

[Common bash](https://www.gnu.org/software/bash/manual/html_node/Bash-Conditional-Expressions.html) [test expressions](https://stackoverflow.com/a/21164441)

```text
The following test operators return "True" if the file exists
-a file :: Any file type
-e file :: Check for file existence, regardless of type (node, directory, socket, etc.)
-f file :: Any regular file that is not a directory
-b file :: Block special file
-c file :: Character file
-d file :: Directory type
-h file :: Symbolic link
-L file :: Symbolic link
-p file :: Named pipe file (FIFO)
-s file :: Socket file

The following operators test existing files for specific properties
-g file :: Any file AND set-group-id bit set
-k file :: Any file AND sticky bit set
-r file :: Any file AND file is readable
-w file :: Any file AND file is writable
-x file :: Any file AND executable bit is set
-s file :: Any file AND file size is greater than 0

Other common test operators
-v varname :: True if 'varname' is set and has been assigned a value
-R varname :: True if varname is set and is a name reference
-n string :: True if length of string is not zero
-z string :: True if length of string is zero
string1 == string2 :: True if the strings are equal
string1 = string2 :: True if the strings are equal. POSIX conformant
string1 != string2 :: True if the strings are not equal
string1 < string2 :: True if string1 sorts before string2
string1 > string2 :: True if string1 sorts after string2

Arthimetic test operators
arg1 -eq arg2 :: True if arg1 is equal to arg2
arg1 -ne arg2 :: True if arg1 is not equal to arg2
arg1 -lt arg2 :: True if arg1 is less than arg2
arg1 -le arg2 :: True if arg1 is less than or equal to arg2
arg1 -gt arg2 :: True if arg1 is greater than arg2
arg1 -ge arg2 :: True if arg1 is greater than or equal to arg2
```

[Command substitution](https://www.gnu.org/software/bash/manual/html_node/Command-Substitution.html)

```bash
$(command)
`command`
```

[Shell Arthimetic](https://www.gnu.org/software/bash/manual/html_node/Shell-Arithmetic.html)

[Brace expansions](https://www.gnu.org/software/bash/manual/html_node/Brace-Expansion.html)

```bash
mkdir /usr/local/src/bash/{old,new,dist,bugs}
```

or

```bash
chown root /usr/{ucb/{ex,edit},lib/{ex?.?*,how_ex}}
```

[Parameter (variable) expansion](https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html) and [https://wiki.bash-hackers.org/syntax/pe](https://wiki.bash-hackers.org/syntax/pe)

[GNU BASH reference](https://www.gnu.org/software/bash/manual/html_node/index.html)

[BASH Hackers Wiki](https://wiki.bash-hackers.org/)

[BASH Test Operators](https://kapeli.com/cheat_sheets/Bash_Test_Operators.docset/Contents/Resources/Documents/index)

[Advanced BASH Scripting Guide](Bash Test Operators)