🪤 Bash Gotchas

Bash gotchas: the traps that bite every script, reproduced
shellcheck script.sh

Shell scripts fail in the same dozen ways, over and over: a file name with a space, an empty variable in a delete command, a counter that stays at zero, a comparison that says 10 is less than 9. None of these are bugs in bash. They are the documented rules of a language that was designed for typing commands, and the rules surprise people who expect it to behave like Python or C.
This page lists those traps, grouped by cause, and every one was reproduced for real with GNU bash 5.2.21 on Ubuntu 24.04, so you can see the symptom as well as the cure. Anything dangerous (a recursive delete) was run with echo in front, so the shell showed what it WOULD have done. For the broader tour of the language see the Bash Programming Tutorial, and for how scripts should fail see the Error Handling page.

After the shell expands a variable or a command substitution, it splits the result into words at spaces, tabs and newlines, and then treats each word as a glob pattern (* ? [ ]). Quotes switch that off. Almost every gotcha in this part is the same mistake: leaving the quotes out.

$ f="my file.txt"; touch $f; ls
file.txt
my
touch got TWO arguments (my and file.txt) and created two files. With quotes there is one:
$ rm -f my file.txt; touch "$f"; ls
my file.txt
Rule: write "$var" and "$(command)" everywhere, unless you want splitting and can say why. Only the cases where you do want it are exceptions (a deliberate word list).

An unquoted variable that contains * or ? is expanded against the files in the current folder:
$ pattern="*.txt"; echo $pattern; echo "$pattern"; star="*"; echo "$star" $star
a.txt b.txt
*.txt
* a.txt b.txt
The unquoted forms were replaced by the file names in the folder. A value you read from a user or a file, such as a name or a message, can contain a star, so the result depends on whatever files are lying around. Quote it: "$pattern".

This is the famous one. A path built from a variable that is empty or unset becomes the root of the filesystem:
$ dir=""; echo rm -rf $dir/*
rm -rf /bin /bin.usr-is-merged /boot /cdrom /dev /etc /home /lib /lib64 /lib.usr-is-merged /lost+found /media /mnt /opt /proc /root /run /sbin ... /sys /tmp /usr /var
(shortened; the real line listed every top-level folder). Run with echo here, but without echo that was a command to delete the whole machine. Two defences, both of which stop it:
$ dir=""; echo rm -rf "${dir:?dir is empty}"/*
bash: line 1: dir: dir is empty
$ set -u; echo rm -rf $dir/*
bash: line 1: dir: unbound variable
${var:?message} aborts the script with the message when the variable is unset or empty; set -u aborts on an unset variable (an empty one still gets through, so the :? form is stronger for dangerous paths). Also: never delete with a pattern built from a variable if you can name the thing exactly (rm -rf -- "$tmpdir" where tmpdir came from mktemp -d), and use "${var:?}" every time the value is part of an rm, mv or chmod -R.

$ for f in $(ls *.txt); do echo "[$f]"; done
[a.txt]
[b.txt]
[two]
[words.txt]
$ for f in *.txt; do echo "[$f]"; done
[a.txt]
[b.txt]
[two words.txt]
The file "two words.txt" became two items in the first loop. Never parse the output of ls: it is for people. A glob (for f in *.txt) gives every name intact, even with spaces, tabs or newlines, and find ... -print0 handles recursion (see item 6). The same goes for for l in $(cat file): it splits on every space, not on lines.
$ for l in $(cat words.txt); do echo "[$l]"; done
[hello]
[world]
[second]
[line]
The file has two lines and the loop ran four times. Read lines with while IFS= read -r line; do ...; done < file (item 7).

$ arr=(a "b c" d); echo "$arr"; echo ${#arr[@]}; for x in ${arr[@]}; do echo "<$x>"; done; for x in "${arr[@]}"; do echo "<$x>"; done
a
3
<a>
<b>
<c>
<d>
<a>
<b c>
<d>
$arr is just the first element ("a"). ${#arr[@]} is the number of elements (3). Unquoted ${arr[@]} loses the element boundaries (four words, "b c" split in two); "${arr[@]}" keeps each element whole. Rule: "${arr[@]}" to use all elements, "${#arr[@]}" for the count, and quote them every time.

A file named -l looks like an option to every command, once a * expands to it:
$ touch -- -l; ls *
(a long listing of the other files: ls took the file -l as its option)
$ ls -- *
in.txt
-l
nums.txt
...
$ ls ./*
./in.txt
./-l
./nums.txt
...
The first command printed a long listing: the file named -l became the option -l. The two cures: put -- before the names (it ends option parsing), or give every name a path (./*). In scripts, write rm -- "$f", cp -- "$a" "$b", and for f in ./*; do. A file name can also contain a newline, which breaks every line-based tool. With a file called "with\nnewline.txt" in the folder:
$ find . -name "*.txt" | while read -r f; do echo "[$f]"; done
[./in.txt]
[./plain.txt]
[./with]
[newline.txt]
[./nums.txt]
The one file came out as two broken names. NUL-separated names are the cure, because a file name cannot contain NUL:
$ find . -name "*.txt" -print0 | while IFS= read -r -d "" f; do printf "[%s]\n" "$f"; done
[./in.txt]
[./plain.txt]
[./with
newline.txt]
[./nums.txt]
Use -print0 with read -d "" (as here), with xargs -0, and with sort -z. Also use printf '[%s]\n' "$f", not echo (item 19).

read treats backslashes as escapes and trims leading and trailing whitespace, silently. Given a file with two lines, one indented and one a Windows path:
$ while read line; do echo "[$line]"; done < in.txt
[leading and trailing]
[C:tempnew]
$ while IFS= read -r line; do echo "[$line]"; done < in.txt
[  leading and trailing  ]
[C:\temp\new]
The plain read removed the indentation and swallowed both backslashes. The pattern to memorise for reading lines is while IFS= read -r line; do ...; done < file: IFS= keeps the whitespace, -r keeps the backslashes. A last line without a final newline is skipped by this loop: add || [[ -n $line ]] to the while condition if the files can end that way.

Every command in a pipeline runs in its own subshell, so variables set inside the loop disappear when it ends:
$ count=0; cat nums.txt | while read n; do count=$((count+1)); done; echo "count after pipe-while: $count"
count after pipe-while: 0
The loop ran three times, and the count is still 0. Three fixes, all run:
$ count=0; while read n; do count=$((count+1)); done < nums.txt; echo "count with redirect: $count"
count with redirect: 3
$ count=0; while read n; do count=$((count+1)); done < <(cat nums.txt); echo "count with process substitution: $count"
count with process substitution: 3
$ shopt -s lastpipe; count=0; cat nums.txt | while read n; do count=$((count+1)); done; echo "count with lastpipe: $count"
count with lastpipe: 3
Redirect from a file when you have one; use process substitution < <(command) when the input is a command (the loop then runs in the main shell); or shopt -s lastpipe, which runs the last part of a pipeline in the current shell (it works in scripts, where job control is off, and not in an interactive shell). The same trap applies to (cd dir; ...) and to $( ): anything done in a subshell, including cd, stays there:
$ cd /nonexistent_dir; echo "still in: $(basename "$PWD")"; (cd /usr; pwd); echo "back: $(basename "$PWD")"
bash: line 1: cd: /nonexistent_dir: No such file or directory
still in: lab2
/usr
back: lab2
The (cd /usr; pwd) printed /usr but the script was still in its own folder afterwards: handy when you want it, a puzzle when you do not.

wc -l counts newline characters, not lines. A file whose last line has no newline at the end is reported one short:
$ printf "a\nb" | wc -l; printf "a\nb\n" | wc -l
1
2
Both have two lines of text. Use grep -c '' file to count lines including an unterminated last one, or awk 'END{print NR}' file.

[ is an ordinary command, so an unquoted empty variable disappears and the test is left with the wrong number of arguments:
$ x=""; [ $x = y ] && echo yes || echo no
bash: line 1: [: =: unary operator expected
no
$ v="a b"; [ -z $v ] && echo empty || echo notempty
bash: line 1: [: a: binary operator expected
notempty
The error went to stderr and the test returned "false", so the script carried on with the wrong answer. The double-bracket form [[ ]] is a shell keyword and does not split or expand patterns, so both cases work:
$ x=""; [[ $x = y ]] && echo yes || echo no
no
$ v="a b"; [[ -z $v ]] && echo empty || echo "not empty (double brackets are fine)"
not empty (double brackets are fine)
Use [[ ]] in bash scripts (it also adds =~ for regular expressions and && and || inside the test) and quote the variables in plain [ ] when you need a POSIX test: [ "$x" = y ]. [[ ]] is not available in sh/dash: see the POSIX page.

Inside [[ ]] the operators < and > compare STRINGS (alphabetically). Numbers need -lt, -gt and so on, or (( )):
$ [[ 10 < 9 ]] && echo "string: 10 < 9 is TRUE"; (( 10 < 9 )) || echo "numeric: 10 < 9 is false"; [ 10 -lt 9 ] || echo "[ -lt ]: false"
string: 10 < 9 is TRUE
numeric: 10 < 9 is false
[ -lt ]: false
"10" sorts before "9" as text, because "1" comes before "9". Use (( a < b )) or [[ a -lt b ]] for numbers. The reverse trap: == compares strings, so 010 and 10 differ even though they look equal:
$ a=010; b=10; [[ $a == $b ]] && echo same || echo "strings differ"; (( a == b )) && echo "numbers same" || echo "numbers differ (010 is octal 8)"
strings differ
numbers differ (010 is octal 8)
which leads to the next one.

In arithmetic, a number with a leading zero is read as octal. 08 and 09 are not valid octal numbers, so they fail, which is a classic when the number comes from date +%H or a file name:
$ n=08; echo $((n+1))
bash: line 1: 08: value too great for base (error token is "08")
$ n=08; echo $((10#$n+1))
9
Force base ten with 10#: $((10#$n + 1)). First make sure $n is all digits: 10# on a non-number is an error too.

$ true && { echo "then-part runs but fails"; false; } || echo "else-part ALSO runs"
then-part runs but fails
else-part ALSO runs
The "else" part runs whenever the "then" part fails, not only when the condition fails. It is fine for short, safe commands (a log message after a test), and wrong for anything where the middle part can fail. Write a real if/else: if cmd1; then cmd2; else cmd3; fi. ShellCheck warns about this as SC2015.

An assignment must have no spaces. With spaces the shell thinks the first word is a command:
$ x = 5
bash: line 1: x: command not found
$ x=5; echo $x; y= 6; echo "[$y]"
5
bash: line 1: 6: command not found
[]
"y= 6" is the command "6" run with y set to empty. The opposite is true inside [[ ]] and [ ]: there you NEED spaces around = and around the brackets.

Brace expansion happens before variables are expanded, so a variable in a range is left alone:
$ n=3; echo {1..$n}; echo {1..3}; for i in $(seq 1 $n); do printf "$i "; done; echo
{1..3}
1 2 3
1 2 3
Use seq 1 "$n", or a C-style loop: for ((i=1; i<=n; i++)); do ...; done.

Inside [[ ]] an unquoted right side is a PATTERN; a quoted one is a literal string:
$ f=notes.txt; [[ $f == *.txt ]] && echo "glob match"; [[ $f == "*.txt" ]] && echo "literal match" || echo "no literal match"
glob match
no literal match
Quote it when you mean the exact text ("$expected"), leave it bare when you mean a pattern. A pattern held in a variable works the same way: [[ $f == $pat ]] is a pattern match, [[ $f == "$pat" ]] is exact. For regular expressions use =~ with the pattern unquoted (or in a variable).

When cd fails, the script usually carries on in the WRONG folder, and the next rm or mv acts there:
$ cd /nonexistent_dir; echo "still in: $(basename "$PWD")"
bash: line 1: cd: /nonexistent_dir: No such file or directory
still in: lab2
$ set -e; cd /nonexistent_dir; echo not reached
bash: line 1: cd: /nonexistent_dir: No such file or directory
$ cd /nonexistent_dir || { echo "cannot cd, stopping"; exit 1; }
bash: line 1: cd: /nonexistent_dir: No such file or directory
cannot cd, stopping
The first one continued in the starting folder. With set -e the script stops (second), and the explicit cd dir || exit 1 (third) says why. Write cd "$dir" || exit 1 every time, and put a command that depends on the folder AFTER it with &&: cd "$dir" && rm -f -- *.tmp is safer than the two commands on separate lines. A subshell is another way: (cd "$dir" && do_things) leaves the current folder untouched.

$ x=$(printf "a\n\n\n"); echo "length: ${#x}"; y="$(printf "a\n\n\n"; echo end)"; echo "${#y}"
length: 1
7
The three newlines after "a" were dropped (length 1). When the trailing newlines matter, add a sentinel character and strip it: y=$(cmd; echo x); y=${y%x}. (The second line shows the sentinel idea: with "end" appended, the newlines are kept: 7 characters.) It also means a variable never ends with a newline unless you put one back: printf '%s\n' "$x".

$ v=-n; echo $v; echo "[done]"; printf "%s\n" "$v"; echo -e "a\tb"; sh -c "echo -e \"a\tb\""
[done]
-n
a	b
-e a	b
The first echo printed nothing at all: it took the value "-n" as its option (no newline) and had nothing left to print. printf "%s\n" "$v" printed it correctly. The last two lines show -e: bash's echo interprets it, but /bin/sh on Ubuntu is dash, whose echo printed a literal "-e" and interpreted the escapes anyway. Rule: use printf for anything that is not a fixed word: printf '%s\n' "$var". It behaves the same everywhere, and its format string is the only place a % or backslash has a meaning.

An unquoted delimiter (<<EOF) expands variables and command substitutions inside; a quoted one (<<'EOF') keeps everything literal:
$ name=World; cat <<EOF
Hello $name, today is $(echo expanded)
EOF
Hello World, today is expanded
$ cat <<'EOF'
Hello $name, nothing $(expanded)
EOF
Hello $name, nothing $(expanded)
Use the quoted form for text that contains $, backticks or backslashes (usage texts, generated scripts, SQL, config files for other tools) and the unquoted form only when you want the substitutions. With <<-EOF, leading TAB characters (not spaces) are removed, which lets you indent the block inside an if or a function; editors that turn tabs into spaces break it, and the closing EOF must be alone on its line, with no trailing space.

$ printf "b\nA\na\nB\n" | sort | tr "\n" " "; echo; printf "b\nA\na\nB\n" | LC_ALL=C sort | tr "\n" " "; echo; locale | head -1
a A b B
A B a b
LANG=en_US.UTF-8
The same input sorted two ways: in an English locale, case is mostly ignored (a A b B); in the C locale, all capitals come before all lower case (A B a b). The result also changes between machines and with the user's settings. For scripts that compare, merge or de-duplicate (join, comm, sort -u, uniq), set LC_ALL=C on every command that must agree, or the same script gives different answers in cron and in your terminal.

$ echo ~ "~"
/home/demo
~
Tilde expansion happens only on an unquoted ~ at the start of a word. "~/data" is the literal text ~/data, and a file by that name is not found. Write "$HOME/data" instead.

  rm -rf $dir/*            rm -rf -- "${dir:?}"/*             quote and refuse empty
  for f in $(ls)           for f in ./*                       a glob, not parsed output
  for l in $(cat file)     while IFS= read -r l; do ...; done < file
  cat file | while read    while read ...; done < file        (or < <(cmd), or lastpipe)
  if [ $a = b ]            if [[ $a = b ]]  or  [ "$a" = b ]
  [[ $a < $b ]] (numbers)  (( a < b ))  or  [[ $a -lt $b ]]
  $((n+1)) with 08         $((10#$n+1))
  cd dir; rm *             cd dir || exit 1; rm -f -- *       (or cd "$dir" && ...)
  echo $var                printf '%s\n' "$var"
  a && b || c              if a; then b; else c; fi
  `command`                $(command)                         nests and reads better
  ${arr[@]} / $arr         "${arr[@]}"
  [ -z $v ]                [[ -z $v ]]  or  [ -z "$v" ]
  local v=$(cmd)           local v; v=$(cmd)                  keeps the exit status
  $*                       "$@"
  find | while read        find -print0 | while IFS= read -r -d '' f
  {1..$n}                  seq 1 "$n"  or  for ((i=1;i<=n;i++))

ShellCheck is a static analyser for shell scripts. It finds almost every item on this page before the script runs: shellcheck script.sh (install with sudo apt install shellcheck, or dnf install ShellCheck, or brew install shellcheck). Each finding has a code you can look up. ShellCheck is not installed on the lab machine, so the codes below are taken from its documentation and were not produced in a run here:
  SC2086   double quote to prevent globbing and word splitting (items 1, 2, 3)
  SC2046   quote this to prevent word splitting of a command substitution (item 4)
  SC2164   use cd ... || exit in case cd fails (item 17)
  SC2162   read without -r will mangle backslashes (item 7)
  SC2012   use find instead of ls to better handle non-alphanumeric file names (item 4)
  SC2155   declare and assign separately to avoid masking return values (the local v=$(cmd) problem)
  SC2015   note that A && B || C is not if-then-else (item 13)
  SC2128   expanding an array without an index only gives the first element (item 5)
  SC2006   use $(...) notation instead of legacy backticks
  SC2181   check exit code directly with e.g. if mycmd;, not indirectly with $?
Run it in your editor (most editors have a plugin), in a pre-commit hook and in CI: shellcheck -x script.sh follows source'd files, and -s sh checks against POSIX sh rules instead of bash. Add a comment line  # shellcheck disable=SC2086  directly above a line when a warning is a deliberate choice, and say why. Two other cheap tools: bash -n script.sh (syntax only, runs nothing) and set -x (a trace).

Quote every expansion: "$var", "$(cmd)", "${arr[@]}", "$@". The exceptions are rare and deliberate.
Protect dangerous paths: "${var:?}" before rm, mv, chmod -R and chown -R.
Do not parse ls, and do not read lines with for. Use a glob, find -print0, or while IFS= read -r.
Know where the subshells are: pipes, $( ), ( ). Variables and cd inside them do not come back.
Use [[ ]] for tests in bash, (( )) for numbers, and force base ten with 10# for values that may start with zero.
Use printf instead of echo, and -- before names that might start with a dash.
Check cd and every command whose failure would make the next line dangerous.
Let ShellCheck read the script, then test the ugly inputs on purpose: a name with a space, an empty value, a file starting with a dash, a path with a newline, a last line without a final newline.