Scripting · XB Field Notes

Shell word splitting turns unquoted variables into several arguments

Fri, 31 Dec 2027 00:00:00 +0000

A shell variable holds a string, but an unquoted expansion can turn that string into several command arguments. In Bash, word splitting examines expansion results using IFS as the delimiter set. This is why a pathname that works when typed with quotes can fail after being stored in a variable. Debug the arguments the receiving command actually gets, including empty ones, before blaming the filesystem.

Splitting depends on the context

With the default IFS, runs of spaces, tabs, and newlines separate fields. Leading and trailing IFS whitespace is discarded. An unquoted value containing two words therefore becomes two arguments; an empty unquoted value normally contributes none. In contrast, "$value" supplies one argument even when the string is empty. Empty and absent arguments can have completely different meanings to a program.

Splitting applies to results of unquoted parameter expansion, command substitution, and arithmetic expansion in suitable contexts. It is not a second parse of the entire command line. An ordinary assignment such as copy=$value stores the string without splitting or filename expansion. Later passing $copy unquoted to a command reintroduces both hazards. Bash conditional expressions using double brackets also have different expansion rules from ordinary command arguments.

IFS is a set of delimiter characters, not a multi-character separator or a regular expression. Non-whitespace delimiters behave differently from IFS whitespace: adjacent colons, for example, can produce an empty field. Setting IFS to an empty string disables splitting, but it does not turn an empty unquoted expansion into an explicit empty argument. Use a parser suited to the format when quoting, escaping, or structured records matter.

When it breaks

Disabling splitting does not disable the filename expansion that follows it. A variable containing a star can still become matching filenames when IFS is empty. The result can change merely because a file appeared in the working directory. Quoting the variable prevents both transformations; changing IFS alone only addresses one stage. Shell options controlling globbing are independent as well.

A scalar string is also a poor representation of an argument list. Adding quote characters inside the string does not make Bash treat those characters as shell quotation on expansion. The quotation that controls parsing must be in the command’s source. Avoid eval as a repair: it introduces another shell parse and can execute text that was previously data. For Bash commands assembled incrementally, use an array and expand it as "${args[@]}".

Some data is deliberately delimited, so splitting is not always a mistake. Make the format and delimiter explicit, keep changes to IFS local to the operation, and decide what empty fields mean. Do not use whitespace splitting to process arbitrary filenames or CSV records. These examples specifically target Bash; other shells, including zsh in its normal configuration, do not necessarily split scalar expansions the same way.

For example, a semicolon obtained from a variable does not by itself become a command separator during ordinary expansion. It remains argument data, even though splitting can still damage the surrounding value. This distinction separates incorrect argument construction from deliberately reparsing a string as code.

Check it quickly

This self-contained Bash experiment checks argument counts rather than using echo, which hides the distinction between one spaced argument and several arguments. It creates its own two matching files, tests an empty value, and checks the difference between whitespace and colon delimiters. The temporary directory is removed at exit.

bash --noprofile --norc <<'BASH'
set -e
set +f
unset GLOBIGNORE
shopt -u nullglob failglob dotglob
work=$(mktemp -d)
trap 'rm -rf -- "$work"' EXIT
cd "$work"
IFS=$' \t\n'
value='report final.md'
set -- $value
[[ $# -eq 2 ]]
set -- "$value"
[[ $# -eq 1 && $1 == 'report final.md' ]]
value=''
set -- $value
[[ $# -eq 0 ]]
set -- "$value"
[[ $# -eq 1 && $1 == '' ]]
touch a.txt 'two words.txt'
IFS=''
pattern='*.txt'
set -- $pattern
[[ $# -eq 2 ]]
set -- "$pattern"
[[ $# -eq 1 && $1 == '*.txt' ]]
IFS=:
record='red::blue'
set -- $record
[[ $# -eq 3 && $2 == '' ]]
printf 'colon field <%s>\n' "$@"
args=('report final.md' '' '*.txt')
set -- "${args[@]}"
[[ $# -eq 3 && $2 == '' && $3 == '*.txt' ]]
printf 'PASS: boundaries, empty values, globbing, delimiters\n'
BASH

Expect three colon-field lines, with an empty middle field, followed by PASS. The assertions before those lines distinguish zero arguments from one empty argument and prove that an empty IFS still permits globbing. In a failing script, put a small argument-counting function at the call boundary and print each argument separately with Bash printf’s %q format. That diagnostic shows boundaries and escaping; it does not turn a human-readable log into a safe command to execute later.

Sources

GNU Bash: Word Splitting, IFS and empty-field rules, checked 2026-09-06.
GNU Bash: Shell Expansions, splitting before filename expansion, checked 2026-09-06.
GNU Bash: Arrays, preserving a list of arguments, checked 2026-09-06.

Shell positional arguments are a numbered list

Mon, 29 Nov 2027 00:00:00 +0000

A shell script receives an ordered argument list, not a single command-line string that it must split again. Positional parameters expose that list as $1, $2, and subsequent entries; $# reports its size. A small fixed interface can use these positions directly. Optional settings and flags benefit from explicit parsing, but either design still needs to define the permitted count, empty values, and failure behavior.

Preserve the list before interpreting it

The script name in $0 is separate from the positional arguments and does not contribute to $#. With bash -c, the first argument after the command string becomes $0, which can surprise wrappers expecting it to become $1. Positions above nine require braces: ${10} means the tenth argument, whereas $10 means the first argument followed by a literal zero.

"$@" preserves one word per supplied argument, including empty arguments and values containing spaces. An unquoted expansion can split values and expand wildcard characters, while "$*" joins the list into one string. Use the quoted list when forwarding or iterating. set -- replaces the current positional list, and shift removes leading entries; neither operation changes the caller’s original list outside the relevant script or function scope.

An empty argument still occupies a position. Checking $# distinguishes an absent argument from one explicitly passed as ''. An expression such as ${1:?message} rejects both missing and empty values, which is appropriate only when the interface requires a nonempty value. Validate the count before reading required positions, then apply any separate rules about allowed contents.

Let getopts report syntax, then validate meaning

The portable getopts interface handles short options. An option character followed by a colon in the option specification takes a value; it does not automatically provide support for long names such as --user. A leading colon selects quiet error reporting: unknown options produce ?, and missing option values produce :, with the offending option available in OPTARG. The script must handle those cases explicitly.

OPTIND tracks parsing progress. Reset it to one before parsing a new argument list in the same shell, and after parsing use shift "$((OPTIND - 1))" to expose the remaining operands. -- ends option processing so a leading-hyphen filename can be an operand. A non-option also ends the ordinary option scan; do not assume options placed after operands will be rearranged automatically.

Parsing an option value does not establish that it is meaningful. An explicitly supplied empty string can satisfy the parser’s requirement for a value while violating the application’s requirement for a name. Likewise, defaults should apply only to omitted optional settings, not hide malformed input. Keep diagnostics on standard error and return a documented nonzero status before performing the requested work.

When it breaks

A common parser accepts the happy path but silently drops surplus operands. Another forgets to reset OPTIND, so a function’s first invocation works while later invocations skip options. Unquoted forwarding can produce the same outward symptom by changing the argument count before the parser sees it. Test the received list and the parsing state separately rather than adding guessed shifts until one invocation works.

The example below deliberately defines a narrow contract: optional -n NAME, optional -v, and exactly one input operand. Its name must be nonempty. The function uses Bash’s local declaration, so the complete example is Bash-specific even though the option parser itself is portable. A script targeting another shell should use that shell’s supported scoping rules instead of assuming local is standardized.

Check it quickly

Run the complete block in Bash. The first part demonstrates the tenth-position boundary and preservation of empty arguments. The parser then exercises valid input and four distinct invalid forms. All operations are local prints; no files or external commands are invoked by the demonstration function.

BASH_ENV= bash --noprofile --norc <<'BASH'
set -- one 2 3 4 5 6 7 8 9 ten
printf 'without-braces=<%s> tenth=<%s>\n' "$10" "${10}"
set -- '' 'two words'
printf 'count=%s\n' "$#"
for demo_arg in "$@"; do printf 'argument=<%s>\n' "$demo_arg"; done
demo_cli() {
 local OPTIND=1 demo_option demo_name=guest demo_verbose=no
 while getopts ':n:v' demo_option; do
 case "$demo_option" in
 n) demo_name=$OPTARG ;;
 v) demo_verbose=yes ;;
 :) printf 'missing value for -%s\n' "$OPTARG" >&2; return 2 ;;
 \?) printf 'unknown option -%s\n' "$OPTARG" >&2; return 2 ;;
 esac
 done
 shift "$((OPTIND - 1))"
 if [ "$#" -ne 1 ] || [ -z "$demo_name" ]; then
 printf 'need a nonempty name and exactly one input\n' >&2
 return 2
 fi
 printf 'name=<%s> verbose=%s input=<%s>\n' "$demo_name" "$demo_verbose" "$1"
}
demo_cli -v -n 'Jane Doe' -- '-input file'
if demo_cli -n 2>/dev/null; then exit 1; else printf 'missing=%s\n' "$?"; fi
if demo_cli -x 2>/dev/null; then exit 1; else printf 'unknown=%s\n' "$?"; fi
if demo_cli -n '' input 2>/dev/null; then exit 1; else printf 'empty=%s\n' "$?"; fi
if demo_cli one two 2>/dev/null; then exit 1; else printf 'extra=%s\n' "$?"; fi
demo_cli second
BASH

Expect one0 versus ten, followed by a count of two and two separately printed arguments. The valid invocation keeps Jane Doe and -input file intact. Missing values, unknown options, an empty name, and extra operands each return status two. The final call restores the defaults and accepts second, proving that repeated parsing starts with fresh state.

These observations verify the interface’s boundaries, not merely that a usage message exists. When extending a real command, add checks for the new option’s meaning and preserve the established treatment of quoting, empty values, and the end-of-options marker.

Sources

Bash positional parameters, numbered parameters and their function scope, checked 2026-09-07.
POSIX getopts, option syntax errors and resetting parsing state, checked 2026-09-07.

Shell parameter expansion rewrites values

Tue, 16 Nov 2027 00:00:00 +0000

Parameter expansion derives a value from a shell parameter before a command receives its arguments. It can provide a default, reject missing configuration, or remove part of a string without launching another process. This is expansion during execution, not an instruction to evaluate every variable when the script is first parsed: a function reads the current value when it runs.

Defaults distinguish missing from empty

${setting:-default} uses the default when setting is unset or empty. ${setting-default} substitutes only when it is unset, preserving an intentional empty value. The colon therefore changes configuration semantics; it is not optional punctuation. ${setting:+enabled} produces enabled only for a set, nonempty value, whereas ${setting+enabled} also accepts an empty one.

Ordinary default expansion does not assign anything. ${setting:=default} additionally stores the selected default in the variable, so subsequent reads see it. ${setting:?message} reports an unset or empty value on standard error and exits a noninteractive shell unsuccessfully. Without the colon, both operators test only whether the variable is unset. Choose according to the caller’s contract, rather than silently replacing required configuration.

Braces also separate a variable name from adjacent text. "${prefix}_suffix" appends a suffix to prefix, while "$prefix_suffix" reads a different variable altogether. This distinction matters when constructing filenames from several settings: an apparently missing value can be a name-boundary mistake rather than absent configuration. Print each component separately before investigating the resulting path, and remember that the final string can be syntactically valid even when one component came from the wrong variable.

Patterns transform strings, not files

${name#pattern} and ${name##pattern} remove the shortest and longest matching prefix. The corresponding suffix operators are % and %%. They use shell patterns, not regular expressions, and leave the original variable unchanged. If nothing matches, the result is the original string. A matching slash is just text here; these operations neither inspect directories nor resolve symbolic links.

Double-quote the complete expansion when its result must remain one argument. Quoting the result does not disable the pattern inside the braces: "${name##*/}" still removes through the last slash. These portable operators differ from Bash extensions such as substring offsets and global replacement. A script invoked as sh must not assume those extensions exist merely because the author’s interactive shell supports them.

When it breaks

An unquoted result can undergo field splitting and filename expansion. Braces delimit the variable name but do not protect argument boundaries. A value containing spaces or an asterisk can consequently turn into unrelated arguments. Also, a default expression is expanded only when selected; placing a command substitution there can execute a command on the missing-value branch. Keep defaults simple when side effects would be surprising.

Parameter expansion is not a command templating language. Text obtained from a variable is not reparsed into shell operators or quoting syntax. Construct argument lists with arrays in Bash, or explicit positional parameters in portable shell, instead of storing a command line and applying eval. set -u catches some missing values but does not validate their meaning, and pipefail has no bearing on these string operations.

Check it quickly

Run this self-contained example with Bash. It changes only variables inside a child shell and prints both the substituted values and the resulting variable state. The brackets make empty output visible.

bash --noprofile --norc <<'BASH'
unset demo_setting
printf 'unset=[%s]\n' "${demo_setting-default}"
demo_setting=
printf 'empty=[%s] colon=[%s]\n' \
 "${demo_setting-default}" "${demo_setting:-default}"
printf 'substitute=[%s] stored=[%s]\n' \
 "${demo_setting:-fallback}" "$demo_setting"
: "${demo_setting:=saved}"
printf 'assigned=[%s]\n' "$demo_setting"
demo_name='reports/weekly report.tar.gz'
printf 'base=[%s] one=[%s] all=[%s]\n' \
 "${demo_name##*/}" "${demo_name%.*}" "${demo_name%%.*}"
printf 'original=[%s]\n' "$demo_name"
show_value() { printf 'current=[%s]\n' "$demo_setting"; }
demo_setting=later
show_value
if (unset demo_setting; : "${demo_setting:?required for demo}") 2>/dev/null; then
 printf 'unexpected success\n'
else
 printf 'missing value rejected\n'
fi
BASH

Expect unset=[default], then empty=[] colon=[default]. The substitution line leaves stored=[]; the assignment line reports saved. The basename is weekly report.tar.gz, the shortest suffix removal leaves reports/weekly report.tar, and the longest leaves reports/weekly report. The original still contains its complete value.

Finally, the function prints later, proving that its variable was not frozen when the function was defined. The last line confirms rejection without terminating the outer demonstration. These observations distinguish substitution, assignment, and error handling; a successful command alone would not prove that an empty configuration value was preserved correctly.

Sources

Bash manual parameter expansion, defines expansion operators, checked 2026-06-10.
POSIX shell parameter expansion, unset versus null values and portable pattern removal, checked 2026-09-07.

Bash globbing expands before the command runs

Wed, 08 Sep 2027 00:00:00 +0000

A wildcard in a Bash command usually becomes a list of pathnames before the target program starts. That makes *.txt convenient, but it also makes the command depend on the current directory and shell options. The program receives arguments, without information saying which ones came from a wildcard. Decide whether you intend to pass matching files or a pattern for the program to interpret.

Match first, then preserve the arguments

Bash performs word splitting before filename expansion. An unquoted *, ?, or bracket expression can therefore produce several arguments. A matching filename containing spaces remains one argument: Bash does not run another word-splitting pass over the filenames it just found. The common loss of spaces happens later when a loop expands its variable without quotes. Use "$file" when passing each matched pathname onward.

Quote a pattern when its recipient needs the pattern itself. For example, find . -name '*.txt' gives the matching expression to find; leaving it unquoted lets Bash expand it against the current directory first. A quoted variable containing an asterisk also stays literal. An unquoted variable containing an asterisk can trigger pathname expansion, even though the star was not written directly in the command.

A pattern operates on directory entries, not a promised collection of regular files. A directory named archive.txt also matches *.txt. Use a file-type check when that distinction matters. Ordinary stars do not cross slashes, and leading dots require explicit matching unless dotglob is enabled. Recursive traversal needs its own deliberate approach; plain * does not search the whole tree.

The environment can also change which names qualify. In Bash, setting GLOBIGNORE to a nonempty value filters matching names and enables dotglob as a side effect. A script inherited from an interactive session may therefore see hidden files unexpectedly. The fixture below clears that variable and selects its options explicitly so its assertions describe a reproducible configuration.

When it breaks

By default, an unmatched pattern stays as literal text. A loop over *.png can consequently run once with a nonexistent filename. With nullglob enabled, the pattern disappears instead. This avoids that literal but introduces another risk: a command receiving no file arguments may read standard input, or apply its own default target. Neither nounset nor pipefail checks that a glob found files.

For optional inputs, collect matches in a Bash array with nullglob and test its element count before invoking the consumer. For required matches, failglob rejects an unmatched expansion before the command runs. Keep option changes inside a subshell when callers should retain their original settings. Do not assume a script’s behavior matches an interactive shell whose startup files changed those options.

A filename beginning with a hyphen may be interpreted as an option by the receiving program. Expanding ./*.txt prefixes such names with a directory component; use the program’s documented -- separator where supported as well. Quoting preserves argument boundaries but does not change option parsing. Finally, enumeration and use are separate events: files can disappear between them, so a successful match is not proof that a later read will succeed.

Check it quickly

Run the following block with Bash and the usual temporary-directory tools. It creates an isolated directory, checks literal and expanded arguments, exercises both no-match policies, and removes only its own fixture. It does not require any preexisting text files.

bash --noprofile --norc <<'BASH'
set -e
export LC_ALL=C
set +f
unset GLOBIGNORE
shopt -u nullglob failglob dotglob
work=$(mktemp -d)
trap 'rm -rf -- "$work"' EXIT
cd "$work"
touch -- a.txt 'two words.txt' .hidden.txt -option.txt
files=(./*.txt)
[[ ${#files[@]} -eq 3 ]]
[[ ${files[2]} == './two words.txt' ]]
printf 'match <%s>\n' "${files[@]}"
literal=('*.txt')
[[ ${literal[0]} == '*.txt' ]]
missing=(./*.missing)
[[ ${missing[0]} == './*.missing' ]]
shopt -s nullglob
missing=(./*.missing)
[[ ${#missing[@]} -eq 0 ]]
if (shopt -s failglob; printf '%s\n' ./*.missing) 2>/dev/null; then
 printf 'unexpected match\n' >&2
 exit 1
fi
shopt -s dotglob
files=(./*.txt)
[[ ${#files[@]} -eq 4 ]]
printf 'PASS: spaces, literal, no-match policies, hidden file\n'
BASH

Expect three initial match lines, with the spaced filename on one line, followed by the PASS message. The hidden file appears only in the final array after dotglob is enabled. The failing subshell is intentional and must not print an expanded filename. These assertions verify Bash expansion and argument preservation; they do not establish how an arbitrary downstream tool handles missing operands, directories, or options. Check that tool’s contract before replacing the harmless printf with a real operation.

Sources

GNU Bash: Filename Expansion, matching rules and no-match options, checked 2026-09-06.
GNU Bash: Shell Expansions, ordering and changes to argument count, checked 2026-09-06.
GNU Bash: Arrays, collecting matches and preserving elements, checked 2026-09-06.

$@ and friends give a function its whole argument list

Fri, 16 Apr 2027 00:00:00 +0000

A wrapper should pass the arguments it received without changing their boundaries. In Bash, "$@" does that when used as command arguments: each positional parameter becomes a separate quoted word. This matters for empty values, spaces, newlines, and literal wildcard characters. A command that happens to work with two simple filenames can still be broken if its wrapper reconstructs the list as a single string.

A list is different from joined text

The positional parameters are the arguments of the current script or function. $# reports their count, including arguments whose value is an empty string. "$1" selects the first argument, while "${10}" selects the tenth; braces distinguish that reference from the first argument followed by a zero. $0 identifies the invoked shell or script and is not an extra element of "$@".

At a normal command boundary, "$@" preserves each parameter as its own word. With no positional parameters, it produces no arguments. Quoted "$*" instead produces one string, joining parameter values with the first character of IFS. If IFS is unset, the join uses spaces; if it is empty, the join inserts no separator. The original spaces inside values remain, but the original boundaries cannot generally be recovered.

Unquoted $@ is not an equivalent shorthand. Its values can undergo word splitting and filename expansion. A literal star supplied by the caller may suddenly match files in the wrapper’s directory. Adding quote characters to a joined string does not restore the lost structure. In Bash, save a list with args=("$@") and forward it with "${args[@]}" when it must survive later changes to the positional parameters.

When it breaks

A function receives its own positional parameters while it runs. Calling it with arguments replaces the visible list for that call, and returning restores the caller’s list. Using set or shift inside the function changes that function’s list. This does not mean every other shell setting is local to the function: changing IFS without a local declaration, for example, can affect the caller. Limit variable changes deliberately.

Shift removes leading positional parameters and renumbers the remainder. Check that enough parameters remain before consuming an option and its value; shifting beyond the available count fails. A parser that only checks for a nonzero count can still mishandle an option requiring a second argument. Preserve the original list in an array if diagnostics or later forwarding require values already consumed by parsing.

Quoting also cannot prevent the receiving command from interpreting an argument as an option. A filename equal to --help remains one intact argument under "$@", but a tool might treat it as a help request. Use the tool’s documented option terminator or a suitable pathname prefix when passing operands. The wrapper must decide where its own options end and where the downstream command’s options begin.

For diagnostics, print the count and each argument separately. Joining everything with "$*" can make zero arguments, an empty argument, and arguments containing separators look misleadingly similar. Bash printf’s %q is useful for showing otherwise invisible boundaries, but logging and forwarding serve different purposes. Forward the saved list itself; do not execute a reconstructed log line.

Check it quickly

Run this complete Bash block. It verifies a four-element list containing an empty value, a spaced pathname, a literal star, and an option-like value. It then checks function scope, the difference between the two quoted expansions with no arguments, and an attempted shift beyond the list length. No files or external services are needed.

bash --noprofile --norc <<'BASH'
set -e
IFS=$' \t\n'
set -- '' 'report final.md' '*.log' '--help'
saved=("$@")
verify() {
 [[ $# -eq 4 && $1 == '' && $2 == 'report final.md' ]]
 [[ $3 == '*.log' && $4 == '--help' ]]
 printf 'arg <%q>\n' "$@"
}
verify "${saved[@]}"
consume() {
 [[ $# -eq 2 ]]
 shift
 [[ $# -eq 1 && $1 == child-two ]]
 set -- replaced
 [[ $1 == replaced ]]
}
consume child-one child-two
[[ $# -eq 4 && $2 == 'report final.md' ]]
joined=$(IFS=,; printf '%s' "$*")
[[ $joined == ',report final.md,*.log,--help' ]]
count() { printf '%s' "$#"; }
set --
[[ $(count "$@") == 0 ]]
[[ $(count "$*") == 1 ]]
set -- sole
if shift 2; then
 printf 'unexpected shift success\n' >&2
 exit 1
fi
[[ $# -eq 1 && $1 == sole ]]
printf 'PASS: forwarding, function scope, empty list, shift limit\n'
BASH

Expect four escaped argument lines and the final PASS message. The first argument is explicitly empty, and the star remains literal. Returning from consume must leave the outer four-element list unchanged. A failed oversized shift must preserve the single remaining argument. These assertions test argument transport; a production wrapper still needs tests for its particular option grammar, required values, and downstream invocation.

Sources

GNU Bash: Special Parameters, positional-list expansions, joining, and counts, checked 2026-09-06.
GNU Bash: Arrays, saving and forwarding separate elements, checked 2026-09-06.
GNU Bash: Word Splitting, unquoted expansions and empty arguments, checked 2026-09-06.

Exit status is a compact report of what went wrong

Sat, 27 Mar 2027 00:00:00 +0000

An exit status is a small value that a shell uses to decide what runs next. Zero means success for shell control flow; a nonzero status takes the failure branch. That does not give every number a universal application meaning. A search with no matches, invalid input, and an unavailable server may deserve different handling even though all can produce nonzero results. Read the command’s contract before deciding what to retry or ignore.

Keep the status attached to its command

Bash exposes the most recent foreground command’s status through $?. Capture it before logging, testing another condition, or running cleanup. Those operations have statuses of their own and can replace the evidence you wanted. A function normally returns the status of its last executed command, so a successful final log message can accidentally turn an earlier failure into a successful function result.

Use if command; then ... else ... fi when the command’s success determines the next action. In the else branch, save $? immediately if the exact failure code matters. An explicit return or exit can then propagate that saved result after diagnostics. Avoid using if ! command and expecting $? inside the success branch to contain the original failure: the negation has already turned it into success.

Bash uses status 127 when command lookup fails and 126 when it finds a command that cannot be executed. It represents fatal signal termination as 128 plus the signal number. These conventions help interpret shell reports, but an application can explicitly exit with the same number. A numerical status alone is therefore not proof of which internal event happened. Keep stderr and the identity of the failed operation alongside it.

When it breaks

A pipeline defaults to its last stage’s status. An upstream generator can fail while a downstream reader successfully consumes an empty or partial stream. With pipefail enabled, Bash selects the rightmost nonzero stage status, or zero if every stage succeeds. PIPESTATUS exposes each stage separately, but subsequent commands replace that array too. Capture the complete observation before running another command.

Errexit, enabled by set -e, has exceptions determined by syntax and calling context. Failures used as if conditions, certain commands in AND or OR lists, and other documented contexts do not automatically exit. A function invoked as a condition can continue after a failed command in its body. An ERR trap follows similar exclusions and is not a report of every possible failure. Neither mechanism proves a script is safe.

Adding || true broadly discards useful distinctions. It may be appropriate when every failure from that exact operation is acceptable, but it can hide an unexpected permission error or an unavailable executable. Handle documented expected results specifically, and propagate unexpected ones. Likewise, first && second || recover runs recover when second fails too; it is not equivalent to an if statement that only tests first.

Background launch has another boundary: successfully starting a job does not mean the job completed successfully. Save its identifier from $! and inspect wait’s result for that job. If a script starts several jobs, retain each identifier and wait for each required result. Otherwise a final successful foreground command can mask work that failed after being launched.

Check it quickly

The following complete Bash experiment uses intentional failures and checks their exact propagation. It demonstrates immediate capture, negation, a function called in a conditional context, and a background job that fails after launch. It does not contact external services or require any files.

bash --noprofile --norc <<'BASH'
set -e
fail_seven() { return 7; }
if fail_seven; then
 exit 1
else
 saved=$?
fi
printf 'saved status %s\n' "$saved"
[[ $saved -eq 7 ]]
[[ $? -eq 0 ]]
if ! fail_seven; then
 negated=$?
fi
[[ $negated -eq 0 ]]
continued=no
conditional_function() {
 false
 continued=yes
}
if conditional_function; then
 [[ $continued == yes ]]
else
 exit 1
fi
(exit 6) &
job_pid=$!
if wait "$job_pid"; then
 exit 1
else
 job_status=$?
fi
[[ $job_status -eq 6 ]]
printf 'PASS: saved failure, negation, conditional context, wait\n'
BASH

Expect a saved status of seven and the final PASS message. The negated status is zero, and the conditional function reaches its assignment despite errexit being set. Wait must report the child’s six. These assertions expose control-flow boundaries; they do not assign meanings to an arbitrary tool’s codes. When adopting the pattern, test both expected negative results and unexpected operational failures, and verify that the wrapper exits with the result its caller needs.

Sources

GNU Bash: Exit Status, shell status conventions, checked 2026-09-06.
GNU Bash: The Set Builtin, errexit exclusions, checked 2026-09-06.
GNU Bash: Pipelines, aggregate pipeline results, checked 2026-09-06.

A shell function reads its arguments like a script

Tue, 16 Mar 2027 00:00:00 +0000

A shell function groups commands under a name and receives its own positional arguments. In an ordinary call, its body runs in the current shell rather than automatically starting a separate process. That makes functions useful for reusable operations, but it also means assignments, directory changes, and other shell state can affect the caller. A function needs an explicit contract for inputs, output, status, and any intentional side effects.

Arguments belong to each invocation

Calling demo_func first second makes those two values available as $1 and $2 inside that invocation, with $# equal to two. Another function call temporarily installs its own list. When it returns, the caller’s list is restored. Using shift within the callee therefore does not consume the caller’s positional arguments. In Bash, $0 does not change to the function name.

Forward the list with "$@" so empty arguments, spaces, and wildcard characters retain their boundaries. Validate the argument count before accessing required positions, and distinguish an omitted argument from a deliberately empty value. A function has no parameter declaration that automatically enforces those rules.

Local variables have dynamic scope

Bash’s local builtin creates a variable that shadows a name in an outer scope. When the function returns, that previous binding becomes visible again. Declare temporary values and loop counters locally when modifying the caller’s variable is not part of the interface. Portable shell code must not assume every interpreter implements Bash’s local-variable facilities.

However, Bash uses dynamic scope: a function called from another function can see the caller’s local variables. If it assigns to such a name without declaring its own local, it can modify the caller’s local binding. The outer global remains protected, but the calling function’s value changes. Local variables are not lexically private objects inaccessible to all other functions.

Local declarations also do not automatically restore the working directory, shell options, traps, or files modified by the body. Those effects need their own contract. Running a function inside ordinary command substitution instead gives it a subshell environment: its variable and directory changes do not propagate back, although file writes still can. Capturing output is not a general sandbox.

Status and data use different channels

A function’s status is the last executed command’s status unless an explicit return changes it. Use standard output for the documented payload and standard error for diagnostics. return 7 reports a status to the caller; it does not return the text seven as data. exit terminates the executing shell rather than merely returning from the function.

Separate a local declaration from a command-substitution assignment when the producer’s status matters. In local result=$(producer), the observed status belongs to the declaration builtin and can hide the producer’s failure. A subsequent assignment without the declaration has the substitution’s status. Capture that result before logging or cleanup changes it.

When it breaks

An unlocalized counter can corrupt a calling loop, including one using a local variable of its own. Tests should call helpers from another function as well as from top-level code. A global-only test misses the dynamic-scope interaction. An unexpected progress message on standard output similarly corrupts callers that capture a single promised value.

Do not rely on set -e to make a function fail predictably at every failed command. Its effect depends on calling context; a function used as an if condition can continue after failures inside its body. Explicitly check operations whose failure must stop subsequent work, and propagate the required status after any diagnostics.

Check it quickly

Run this complete block in Bash. It changes only variables and positional arguments inside a fresh shell. The worker intentionally modifies its caller’s local variable, while the second function demonstrates argument preservation and a nonzero return. The final pair contrasts combined declaration with separate assignment.

BASH_ENV= bash --noprofile --norc <<'BASH'
demo_value=global
demo_worker() { demo_value=changed; }
demo_scope() {
 local demo_value=inner
 printf 'local-before=%s\n' "$demo_value"
 demo_worker
 printf 'local-after=%s\n' "$demo_value"
}
demo_scope
printf 'global-after=%s\n' "$demo_value"
demo_args() {
 local demo_arg
 printf 'function-count=%s\n' "$#"
 for demo_arg in "$@"; do printf 'argument=<%s>\n' "$demo_arg"; done
 shift
 printf 'after-shift=%s\n' "$#"
 return 7
}
set -- outer-one outer-two
if demo_args '' 'two words'; then
 exit 99
else
 demo_status=$?
fi
printf 'function-status=%s outer-count=%s outer-first=%s\n' "$demo_status" "$#" "$1"
demo_masked() { local demo_result=$(return 9); }
demo_checked() {
 local demo_result
 demo_result=$(return 9)
}
demo_masked
printf 'combined-declaration=%s\n' "$?"
if demo_checked; then exit 99; else printf 'separate-assignment=%s\n' "$?"; fi
BASH

Expect the local value to change from inner to changed, but the global value to remain global. Both supplied arguments print separately, including the empty one. The function returns seven after shifting its list; the outer shell still has two arguments beginning with outer-one. Finally, the combined declaration reports zero while the separate assignment reports nine. These results test the actual calling boundaries rather than assuming every variable called local is isolated.

Sources

Bash shell functions, function argument lists and dynamic variable scope, checked 2026-09-07.
Bash-specific builtins, local declaration behavior and return status, checked 2026-09-07.
Bash set builtin, errexit behavior when a function is used as a condition, checked 2026-09-07.

Command substitution puts one command's output into another line

Wed, 03 Mar 2027 00:00:00 +0000

Command substitution captures a command’s standard output and inserts the resulting text into a surrounding expression. The familiar $(command) form is useful for a timestamp, a computed label, or another small textual value. It is not a transparent channel for arbitrary bytes or a reliable way to transport a list of filenames. Its newline handling, execution environment, and exit status are part of the interface you need to understand.

Capture text, then decide how to use it

For the standard substitution form discussed here, Bash executes the enclosed commands in a subshell environment. It captures standard output and removes all trailing newline characters. Newlines inside the output remain. Double quotes prevent later word splitting and filename expansion, but they do not undo the removal of trailing newlines. If the exact final bytes matter, use a file or a stream instead of a shell variable.

A variable assignment such as result=$(producer) stores the resulting text without splitting it into arguments. When using the value later, write "$result" to preserve its boundary. Similarly, consumer "$(producer)" supplies a single argument, even when the captured result is empty. Leaving the substitution unquoted can create several arguments or no arguments, and stars in the output may match filenames in the working directory.

Standard error is separate from standard output unless you explicitly redirect it. A diagnostic can remain visible on the terminal while the captured value contains only the result. Redirecting both streams together may contaminate a machine-readable value with warning text. Design the producer’s output contract before deciding whether those streams belong together; suppressing every diagnostic can hide the reason a value is missing.

When it breaks

The subshell has its own variable changes and working directory. Assigning a variable or running cd inside the substitution does not update the caller’s corresponding state. Filesystem and network side effects still happen, however: creating a file inside the substitution creates a real file. Process isolation does not make an operation reversible or remove its external effects.

Successful capture also says nothing about whether the producer succeeded. An assignment-only command gets the status of its last command substitution, so if result=$(producer); then can check the result. In contrast, the status of printf '%s' "$(producer)" normally describes printf. A producer can emit partial output and then fail; retain its status immediately and decide whether that output is usable. A later logging command overwrites the value represented by $?.

Avoid relying on an implicit failure somewhere inside a multi-command substitution. Its final command can return success even after an earlier command failed. Explicitly test important operations in the producer and return the intended status. Likewise, in a function, separate a local declaration from the capturing assignment when you need to inspect the producer’s status; a declaration builtin can mask that status.

Substitution also collects output before the surrounding command runs, so it is unsuitable for unbounded output or interactive streaming. Do not use an unquoted substitution over find output to construct an arbitrary pathname list: whitespace and wildcard characters lose their original meaning. Pass filenames through an interface that retains boundaries, such as find’s exec facility, or use a deliberately specified record protocol.

Check it quickly

Run this complete block with Bash. It uses only shell builtins and asserts the output and status independently. No files or services need to be prepared. The producer with status seven is an intentional failure, and the test must still reach the final PASS line.

bash --noprofile --norc <<'BASH'
set -e
outer=parent
captured=$(outer=child; printf 'first\nsecond\n\n')
[[ $outer == parent ]]
[[ $captured == $'first\nsecond' ]]
printf 'captured <%q>\n' "$captured"
if partial=$(printf 'incomplete'; exit 7); then
 printf 'unexpected producer success\n' >&2
 exit 1
else
 status=$?
fi
[[ $status -eq 7 && $partial == incomplete ]]
printf 'producer status %s; partial <%s>\n' "$status" "$partial"
printf '%s' "$(exit 9)"
[[ $? -eq 0 ]]
IFS=$' \t\n'
set -- "$(printf 'two words\n')"
[[ $# -eq 1 && $1 == 'two words' ]]
set -- "$(printf '\n\n')"
[[ $# -eq 1 && $1 == '' ]]
printf 'PASS: capture, subshell scope, failure status, empty result\n'
BASH

The escaped captured value should contain one internal newline and no trailing newline. The next line reports status seven alongside the partial text. The deliberately successful outer printf demonstrates why checking only the consumer’s status misses producer failure. These checks establish the behavior of the small synchronous producers shown here. A real producer still needs its own timeout, output-format validation, and explicit error contract where those requirements apply.

Sources

GNU Bash: Command Substitution, standard substitution scope and newline handling, checked 2026-09-06.
GNU Bash: Word Splitting, treatment of unquoted captured output, checked 2026-09-06.
GNU Bash: Special Parameters, the status exposed by the question-mark parameter, checked 2026-09-06.

A here-string pipes text into a command's stdin

Mon, 15 Feb 2027 00:00:00 +0000

A Bash here-string supplies the expanded value of one word to a command’s standard input, with a newline appended. It is convenient when a small textual value already exists in a variable and the receiving tool reads stdin. The syntax removes the need for a separate printing command, but it does not promise byte-for-byte identity with the variable or a particular implementation using memory, a pipe, or a file.

Expansion happens before the input is read

The word after the triple-less-than operator can contain parameter expansion, arithmetic expansion, or command substitution. It is therefore not necessarily a literal constant. Single quotes preserve literal source text, while double quotes permit the usual variable and command expansions. Text obtained by expanding a variable is not recursively executed as shell code merely because it contains a dollar sign and parentheses.

In current Bash, a here-string’s word does not undergo word splitting or filename expansion. It is delivered as a single string. Quoting a variable in this position still makes the intended boundary easy to read, but do not confuse this redirection context with passing an unquoted variable as a normal command argument. The latter can split the value and expand wildcard characters into filenames.

The appended newline is unconditional. A value without a final newline gains one; a value already ending in a newline gains another. An empty variable consequently produces a one-byte input containing a newline, rather than immediate end-of-file. This makes here-strings natural for many line-oriented tools and unsuitable when the receiving protocol requires an exact length or a specific final byte.

When it breaks

Read applies its own interpretation after the shell constructs the input. By default, its field splitting can remove surrounding IFS whitespace, and backslashes can act as escapes. For one line whose spaces and backslashes must be retained, use IFS= read -r name. This protects the line’s content, but a single read still stops at the first newline. It does not capture an arbitrary multiline value into that variable.

A redirected builtin or loop can run in the current shell, so its assignments remain visible. That is useful compared with a loop placed in a pipeline, where Bash normally runs the stage in a subshell. The input redirection itself does not create a subshell around the command. An explicit subshell or command substitution elsewhere in the expression still has its usual scope and effects.

The absence of an explicit filename in the script does not establish how Bash stores the redirection internally. Avoid code that depends on stdin being seekable, on a fixed buffering capacity, or on there being no temporary backing storage. For large existing files, direct file redirection avoids first collecting the whole input in a shell variable. For exact textual bytes, a carefully formatted printf or a file can avoid the added newline.

Here-strings are also a shell-language feature, not portable syntax for every interpreter called sh. A script using them should select an interpreter that supports them, and tests should invoke that interpreter explicitly. Conversely, changing an echo pipeline to a here-string can alter both newline behavior and variable scope. Compare the consumer’s observed bytes and state changes rather than treating the forms as interchangeable typography.

Check it quickly

This complete fixture needs Bash and Python 3. Python displays stdin as hexadecimal so appended newline bytes remain visible; capturing that hexadecimal output does not lose the input bytes being inspected. The remaining checks use read and a redirected loop to test whitespace preservation, literal text, and multiline consumption. No fixture files are required.

bash --noprofile --norc <<'BASH'
set -e
hex_input() {
 python3 -c 'import sys; print(sys.stdin.buffer.read().hex())'
}
[[ $(hex_input <<< X) == 580a ]]
[[ $(hex_input <<< '') == 0a ]]
[[ $(hex_input <<< $'X\n') == 580a0a ]]
[[ $(printf '%s' X | hex_input) == 58 ]]
value=' a\b *.txt '
IFS= read -r received <<< "$value"
[[ $received == "$value" ]]
literal='$(printf injected)'
IFS= read -r received <<< "$literal"
[[ $received == '$(printf injected)' ]]
value=$'one\ntwo'
IFS= read -r first <<< "$value"
[[ $first == one ]]
lines=0
while IFS= read -r line; do
 lines=$((lines + 1))
done <<< "$value"
[[ $lines -eq 2 ]]
printf 'PASS: newline bytes, preserved text, single read, loop scope\n'
BASH

Expect only the PASS line. The hexadecimal assertions distinguish one newline from two and compare a here-string with printf that adds none. The literal command-looking text must remain unchanged, and the loop count must survive in the outer shell. These results test the documented input behavior, not the internal storage mechanism. Before using the pattern with another reader, check how that reader handles empty lines, multiple records, and trailing whitespace of its own.

Sources

GNU Bash: Redirections, here-string expansion and appended newline, checked 2026-09-06.
GNU Bash: Bash Builtins, read delimiters and raw-input mode, checked 2026-09-06.
GNU Bash: Command Substitution, captured output and trailing-newline removal, checked 2026-09-06.

case runs one branch that matches a pattern

Sat, 30 Jan 2027 00:00:00 +0000

A case statement routes one value through an ordered list of shell patterns. It is useful for a command dispatcher, a small set of accepted answers, or filename-suffix classification. Each branch states the values it handles, and several patterns can share one body. The ordering and the default branch are part of the interface: they decide what happens when an input overlaps several categories or belongs to none.

Patterns match the whole value

Case uses shell patterns, not regular expressions. A star matches any string, a question mark matches one character, and brackets specify a character set or range. Thus [0-9]* means a digit followed by anything; it does not mean an entire nonempty string of digits. To validate a decimal identifier without arithmetic interpretation, first reject an empty string and any string containing a non-digit.

The alternatives in start|up|run) share an action. A broad suffix pattern such as *.log) can classify a string without opening a file or searching the directory. Unlike pathname expansion, this matching does not establish that the named resource exists. A slash or leading dot also does not have the special exclusion role it has during ordinary filename expansion. Classification and filesystem validation remain separate operations.

The word being tested undergoes the documented expansions, but Bash does not word-split or glob it as ordinary command arguments. An empty unquoted variable in that position does not delete the syntactic word and cause a case syntax error. Quoting the tested value is still clear style. With nounset enabled, separately handle a missing positional parameter or validate the argument count; quotation alone does not make an unset parameter available.

When it breaks

With the usual double-semicolon terminator, the first matching branch runs and case finishes. Put exact values before broader families and the catch-all star last. Otherwise a generic option branch can swallow a later exact status command. Syntax checking cannot establish that your ordering expresses the intended precedence; exercise an input matching both patterns and assert which action runs.

If no branch matches, case returns zero without running an action. A missing default is therefore not automatically an error. A dispatcher that rejects unknown commands must explicitly report and return failure in its default branch. Conversely, an optional classification may deliberately allow no match. Decide that policy from the caller’s contract, rather than assuming case supplies input validation on its own.

Quoting a pattern changes its meaning. A quoted "*.log" matches the literal characters, whereas the unquoted pattern matches a suffix family. The same distinction matters when a variable supplies a pattern: decide whether it is a trusted matching expression or literal text. Do not accidentally grant wildcard behavior to an identifier that should be compared exactly.

Bash options and versions introduce further boundaries. Nocasematch changes case sensitivity, while extended patterns need extglob enabled before the relevant code is parsed. Newer Bash also provides branch terminators that continue execution or resume matching; do not assume they behave like double semicolons or work in older shells. The experiment below deliberately uses ordinary patterns and double semicolons compatible with Bash 3.2.

Check it quickly

Run this complete Bash block. Its dispatcher includes exact commands, grouped aliases, a wildcard family, an explicit empty-input error, and an unknown-input error. A second function checks decimal text without accepting a digit followed by letters. The assertions test results and error codes rather than merely looking for a successful script exit.

bash --noprofile --norc <<'BASH'
set -e
export LC_ALL=C
shopt -u nocasematch
route() {
 case "${1-}" in
 status|--status) printf '%s' status ;;
 start|up|run) printf '%s' start ;;
 '') return 2 ;;
 --*) return 3 ;;
 *.log) printf '%s' log ;;
 *) return 4 ;;
 esac
}
[[ $(route --status) == status ]]
[[ $(route up) == start ]]
[[ $(route 'report final.log') == log ]]
expect_error() {
 local expected=$1
 shift
 if route "$@"; then
 return 1
 else
 local actual=$?
 [[ $actual -eq $expected ]]
 fi
}
expect_error 2 ''
expect_error 2
expect_error 3 --unknown
expect_error 4 STATUS
digits() {
 case "${1-}" in
 ''|*[!0-9]*) return 1 ;;
 *) return 0 ;;
 esac
}
digits 007
if digits 7abc; then exit 1; fi
if digits ''; then exit 1; fi
empty=''
matched=no
case $empty in '') matched=yes ;; esac
[[ $matched == yes ]]
case unmatched in known) exit 1 ;; esac
[[ $? -eq 0 ]]
printf 'PASS: precedence, aliases, empty input, validation, no match\n'
BASH

Expect the PASS line with no earlier output. The exact status option must bypass the generic option error, and uppercase STATUS must be rejected because case-insensitive matching is disabled. The final checks demonstrate that an empty expanded word is valid and that no match returns success. These tests cover routing decisions only; replace the harmless actions with real operations only after testing their arguments, side effects, and failure propagation separately.

Sources

GNU Bash: Conditional Constructs, case ordering, expansion, and return status, checked 2026-09-06.
GNU Bash: Pattern Matching, wildcard and extended-pattern semantics, checked 2026-09-06.
GNU Bash: Word Splitting, contexts that split expanded values, checked 2026-09-06.