update bash doc

.vscode/extensions.json

@@ -0,0 +1,6 @@
+{
+  "recommendations": [
+    "davidanson.vscode-markdownlint",
+    "yzhang.markdown-all-in-one"
+  ]
+}

bash_programming.md

@@ -1,27 +1,208 @@
-Start bash files with 
+# Bash Programming Guidelines and Personal Conventions
+
+The source of most of this guidance comes from previous experience. Particularly [salt-bootstrap](https://github.com/saltstack/salt-bootstrap) and [Nvidia Jetson](https://developer.nvidia.com/embedded/linux-tegra-r3272).
+
+## General Formatting Guidelines
+
+### Be posix compliant
+
+Avoid [bashisms](https://hackmd.io/@nucore/rk5IKhclC?utm_source=preview-mode&utm_medium=rec) when possible. Many systems execute system level code using posix compliant shells like DASH or Bourne Shell. Reference [https://mywiki.wooledge.org/Bashism](https://mywiki.wooledge.org/Bashism)
+
+### Use ALLCAPS for variable names
+
+### Prepend single underscore to variables that SHOULD NEVER change
+
+These variables should be named in such a way that their value is obvious.
+
+Example
+
+```bash
+_TRUE=1
+_FALSE=0
 ```
+
+### Prepend double underscores to calculated variables
+
+These variables are set by code elsewhere in the script.
+
+Example
+
+```bash
+#!/usr/bin/env bash
+
+__SCRIPTARGS="$*"
+__SCRIPTFULLNAME=$(realpath "$0")
+__SCRIPTNAME=$(basename "${__SCRIPTFULLNAME}")
+__SCRIPTFULLPATH=$(dirname "${__SCRIPTFULLNAME}")
+
+__GIT_STATUS=$(git status --porcelain)
+if [ $? -ne 128 ]; then
+    __COMMIT_ID=$(git rev-parse HEAD)
+    # If the git command succeeded, check for uncommitted changes
+    if [ -n "${__GIT_STATUS}" ]; then
+        echo "Warning: There are uncommitted changes in the repository. These changes will be included in the build."
+        __SOURCE_MODIFIERS="m" # m for modified, since there are uncommitted changes
+    fi
+else
+    echo "Warning: Not a git repository. Skipping git status check."
+    __COMMIT_ID="${COMMIT_ID}"
+    __SOURCE_MODIFIERS="u" # u for unverified, since we can't verify the commit ID without git
+fi
+__COMMIT_ID_SHORT=$(echo "${__COMMIT_ID}" | cut -c1-8)
+```
+
+### Use all lowercase for function names
+
+This differentiates VARIABLES from functions.
+
+### Prepend double underscores to functions
+
+This avoids possible conflicts with shell built-ins and other shell programs and commands.
+
+The exception is when intentionally over riding a known command. In these cases, put the main logic in a function with a double underscore and reference the double underscore function from the over riding function.
+
+Example
+
+```bash
+#!/usr/bin/env bash
+
+#===============================================================================
+#=====                              FUNCTIONS                              =====
+#===============================================================================
+#---  FUNCTION  ----------------------------------------------------------------
+#         NAME:  __realpath
+#  DESCRIPTION:  Download if file doesn't exist.
+#        USAGE:  __realpath [file]
+#        NOTES:  [file] can be a file or a directory. [file] must exist.
+#-------------------------------------------------------------------------------
+__realpath() {
+    perl -e 'use Cwd "abs_path";print abs_path(shift)' "$1"
+}
+
+#---  FUNCTION  ----------------------------------------------------------------
+#         NAME:  realpath
+#  DESCRIPTION:  Cross-platform realpath command. Because Mac.
+#        USAGE:  realpath [file]
+#        NOTES:  [file] can be a file or a directory. [file] must exist.
+#                Overrides the systeam realpath command and calls __realpath.
+#                This exists due to some historical incompatibilities with
+#                realpath on MAC os.
+#-------------------------------------------------------------------------------
+realpath() {
+    __realpath $@
+}
+```
+
+## Separate Large Scripts
+
+Break large scripts into multiple files. Use .insh as the file extension for script blocks that are intended to be included as part of a larger script. These .insh files will be included using the `source` command.
+
+### Script Configuration Files
+
+Create a file, such as CONFIG.insh, which will contain only values that are expected to be modified by down stream users. These files should only contain configuration variables to be set by the user and comments. The comments should give a description of the configuration option and provide guidance as to how the value should be formatted. For values that have a limited set of choices, provide the list of allowed values in the comments.
+
+Use the constants `$_TRUE` and `$_FALSE` for boolean configuration options. Obviously, any special variables, constants, or functions need to be imported prior to importing the CONFIG file.
+
+There should not be any functions or programming logic in the config file. Keep it as simple as possible.
+
+Example
+
+```bash
+#!/usr/bin/env bash
+
+#===============================================================================
+#=====                           SCRIPT OPTIONS                            =====
+#===============================================================================
+# Modify these values to change how the script operates.
+
+# STATIC_VARIABLE:
+#     This is an example of a configuration variable set to a static value. It
+#     cannot be overridden except by modifying this file.
+STATIC_VARIABLE="My Static Setting"
+# OVERRIDABLE_VARIABLE:
+#     This is an example of a configuration variable that can be modified by
+#     environment variables. For example:
+#         OVERRIDABLE_VARIABLE="NewValue" script.sh
+OVERRIDABLE_VARIABLE="${OVERRIDABLE_VARIABLE:-DefaultValue}"
+# BOOLEAN_VARIABLE: A boolean variable. 1 = True, anything else is False.
+BOOLEAN_VARIABLE="${BOOLEAN_VARIABLE:-$_TRUE}"
+# COMMIT_ID: The full SHA1 of the target GIT commit to build, or "latest".
+COMMIT_ID="${COMMIT_ID:-latest}"
+# BRANCH: The git branch to pull. e.g. master, release, develop, ...
+BRANCH="${BRANCH:-master}"
+# CLONE_DEPTH: How much commit history to pull
+CLONE_DEPTH="${CLONE_DEPTH:-50}"
+# WORK_DIR: The working directory used by build script
+WORK_DIR="${WORK_DIR:-.build}"
+# BUILD_DIR: The build directory
+BUILD_DIR="${BUILD_DIR:-build}"
+# INSTALL_DIR: The directory the app will be installed to
+INSTALL_DIR="${INSTALL_DIR:-AppDir}"
+# DISTRO: The build distro
+DISTRO="${DISTRO:-ubuntu}"
+```
+
+### Reusable Functions
+
+EXAMPLE
+
+```bash
+#!/usr/bin/env bash
+# Common variables and functions
+
+_TRUE=1
+_FALSE=0
+
+#===============================================================================
+#=====                              FUNCTIONS                              =====
+#===============================================================================
+#---  FUNCTION  ----------------------------------------------------------------
+#          NAME:  __check_command_exists
+#   DESCRIPTION:  Check if a command exists.
+#         USAGE:  if [!] (__check_command_exists [command]); then
+#-------------------------------------------------------------------------------
+__check_command_exists() {
+    command -v "$1" > /dev/null 2>&1
+}
+```
+
+## General Guidelines
+
+Use [ShellCheck](https://www.shellcheck.net/). [ShellCheck for VSCode](https://www.shellcheck.net/).
+
+Limit line length to *approx* 80 characters, when possible.
+
+Apply [PEP 8](https://peps.python.org/pep-0008/) style guidelines where practical.
+
+Start bash files with
+
+```bash
 #!/usr/bin/env bash
 ```
 
 Declare global variables at the beginning of the script. Include a short comment to add context to the variables.
-```
+
+```bash
 # name of BUP file generated by Nvidia script
 BUPFILE="${BUPFILE:-bl_update_payload}"
 ```
 
 Create variables that can be overridden
-```
+
+```bash
 # ORIG_DIR: starting directory where the script will drop the user on exit
 ORIG_DIR="${ORIG_DIR:-$(pwd)}"
 ```
 
 Properly configured variables can be overridden at execution. For example
-```
+
+```bash
 wp@host:~/$ ORIG_DIR="My override value" ./my_script.sh
 ```
 
 Allow easy debugging early in the script. The following code block will enable easy debugging, which will print each line of code before it is executed
-```
+
+```bash
 if [ ! -z ${DEV+x} ] && [ "$DEV" -eq "1" ]; then
     # enable early debugging with `DEV=1 build.sh`
     set -x
@@ -32,46 +213,82 @@ fi
 ```
 
 The script can be debugged by prefixing the script with the correct parameter assignment.
-```
+
+```bash
 wp@host:~/$ DEV=1 ./my_script.sh
 ```
 
 Group functions together under the following banner
-```
+
-#======================================================================================================================
+```bash
-#=====                                                 FUNCTIONS                                                  =====
+#===============================================================================
-#======================================================================================================================
+#=====                              FUNCTIONS                              =====
+#===============================================================================
 ```
 
 Create a header for each function using the following template
-```
+
-#---  FUNCTION  -------------------------------------------------------------------------------------------------------
+```bash
+#---  FUNCTION  ----------------------------------------------------------------
 #          NAME:  __wget
 #   DESCRIPTION:  Download if file doesn't exist.
 #         USAGE:  __wget [outfile] [url]
-#         NOTES:  if [outfile] exists, it will be tested to verify it is a regular file
+#         NOTES:  if [outfile] exists, it will be tested to verify it is a
-#----------------------------------------------------------------------------------------------------------------------
+#                 regular file
+#-------------------------------------------------------------------------------
 __wget() {
-    # do stuff
+    # check usage
+    if [ -z "${1}" ] || [ -z "${2}" ]; then
+        echo "__wget function usage:"
+        echo " \$ __wget [outfile] [url]"
+        exit 1
+    fi
+    # Create the DOWNLOADS directory if it doesn't already exist
+    if [ ! -d "${DOWNLOADS}" ]; then
+        mkdir -p "${DOWNLOADS}" || ( echo "error creating \"${DOWNLOADS}\" directory" && exit 1 )
+    fi
+    # test if [outfile] exists but is not a regular file
+    # ie: [outfile] is device, directory, link, etc
+    if [ -e "${DOWNLOADS}/${1}" ] && [ ! -f "${DOWNLOADS}/${1}" ]; then
+        echo "__wget: [outfile] \"${DOWNLOADS}/${1}\" exists, but is not a regular file"
+        exit 1
+    fi
+    # download [outfile] if it doesn't already exist
+    if [ ! -e "${DOWNLOADS}/${1}" ]; then
+        if ! (__check_command_exists wget); then
+            __sudo apt-get update || ( echo "Unable to update apt packages" && exit 1 )
+            __sudo apt-get install wget || ( echo "Unable to install wget" && exit 1 )
+        fi
+        if [ -f "../${DOWNLOADS}/${1}" ]; then
+            echo "Copying ${1}"
+            cp -v "../${DOWNLOADS}/${1}" "${DOWNLOADS}/${1}" || ( echo "error copying ${1} from parent directory" && exit 1 )
+        else
+            echo "Downloading ${1}"
+            wget -q --show-progress -O "${DOWNLOADS}/${1}" "${2}" || ( echo "error downloading ${1}" && exit 1 )
+        fi
+    fi
 }
 ```
 
 For large, complex scripts, create a banner marking the start of execution, such as
-```
+
-#======================================================================================================================
+```bash
-#=====                                                   BEGIN                                                    =====
+#===============================================================================
-#======================================================================================================================
+#=====                                BEGIN                                =====
+#===============================================================================
 ```
 
 Use sub-banners if needed
-```
+
+```bash
 #----------------------------------------------------------------------------------------------------------------------
 #-----                                            Command Line Args                                               -----
 #----------------------------------------------------------------------------------------------------------------------
 ```
 
 Include a useful help
-```
+
+```bash
 #---  FUNCTION  -------------------------------------------------------------------------------------------------------
 #         NAME:  __usage
 #  DESCRIPTION:  Display program help
@@ -109,23 +326,24 @@ EOT
 ```
 
 Bash test. Returns True if test expression passes
-```
+
+```bash
     if [ -z "${MAKE_TARGETS}" ]; then
         # do stuff
     fi
 ```
 
-
 Bash negative test. Returns True if test expression does not passes
-```
+
+```bash
     if [ ! -z "${MAKE_TARGETS}" ]; then
         # do stuff
     fi
 ```
 
-Common bash test expressions
+[Common bash](https://www.gnu.org/software/bash/manual/html_node/Bash-Conditional-Expressions.html) [test expressions](https://stackoverflow.com/a/21164441)
-https://www.gnu.org/software/bash/manual/html_node/Bash-Conditional-Expressions.html and https://stackoverflow.com/a/21164441
+
-```
+```text
 The following test operators return "True" if the file exists
 -a file :: Any file type
 -e file :: Check for file existence, regardless of type (node, directory, socket, etc.)
@@ -166,27 +384,33 @@ arg1 -gt arg2 :: True if arg1 is greater than arg2
 arg1 -ge arg2 :: True if arg1 is greater than or equal to arg2
 ```
 
-Command substitution
+[Command substitution](https://www.gnu.org/software/bash/manual/html_node/Command-Substitution.html)
-https://www.gnu.org/software/bash/manual/html_node/Command-Substitution.html
+
-```
+```bash
 $(command)
 `command`
 ```
 
-Shell Arthimetic - https://www.gnu.org/software/bash/manual/html_node/Shell-Arithmetic.html
+[Shell Arthimetic](https://www.gnu.org/software/bash/manual/html_node/Shell-Arithmetic.html)
 
-Brace expansions
+[Brace expansions](https://www.gnu.org/software/bash/manual/html_node/Brace-Expansion.html)
-https://www.gnu.org/software/bash/manual/html_node/Brace-Expansion.html
+
-```
+```bash
 mkdir /usr/local/src/bash/{old,new,dist,bugs}
 ```
+
 or
-```
+
+```bash
 chown root /usr/{ucb/{ex,edit},lib/{ex?.?*,how_ex}}
 ```
 
-Parameter (variable) expansion - https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html and https://wiki.bash-hackers.org/syntax/pe
+[Parameter (variable) expansion](https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html) and [https://wiki.bash-hackers.org/syntax/pe](https://wiki.bash-hackers.org/syntax/pe)
 
-GNU BASH reference - https://www.gnu.org/software/bash/manual/html_node/index.html
+[GNU BASH reference](https://www.gnu.org/software/bash/manual/html_node/index.html)
 
-BASH Hackers Wiki - https://wiki.bash-hackers.org/
+[BASH Hackers Wiki](https://wiki.bash-hackers.org/)
+
+[BASH Test Operators](https://kapeli.com/cheat_sheets/Bash_Test_Operators.docset/Contents/Resources/Documents/index)
+
+[Advanced BASH Scripting Guide](Bash Test Operators)