You add set -euo pipefail to a deployment script, pipe a stream into a loop to update an array or counter, and the variables are empty when the loop finishes. Or your script quietly skips the error trap because process substitution runs asynchronously and exits after the parent shell has already moved past the command.
textProcessed 0 deployment targets.
Deploying artifact to cluster: prod-east
cat: /tmp/manifest_cluster_list.txt: No such file or directory
bash: line 42: targets[@]: unbound variable
Bash treats pipelines and process substitutions as subshells under rules that often conflict with strict error modes. When you run cat targets.txt | while read -r line; do ...; done, Bash forks the right-hand side of the pipe into a separate subshell process. Any variable modifications inside that loop exist only in that subshell. When the subshell terminates, those mutations vanish. Combining this behavior with set -euo pipefail introduces several subtle ways for scripts to silently lose data or fail abruptly.
The Pipeline Scope Trap and the Process Substitution Alternative
In standard Bash execution, each command in a pipeline runs in its own subshell. This breaks code designed to accumulate data, populate lookup tables, or track operation tallies.
bash#!/usr/bin/env bash
set -euo pipefail
total_processed=0
declare -a active_servers=()
# The pipe forces the while loop into a subshell
grep -v '^#' /etc/hosts | while read -r ip host rest; do
if [[ -n "${ip}" && -n "${host}" ]]; then
active_servers+=("${host}")
total_processed=$((total_processed + 1))
fi
done
# This will print 0, and the array will be empty
echo "Processed ${total_processed} hosts."
echo "First host: ${active_servers[0]}"
Because set -u is active, referencing ${active_servers[0]} aborts the script immediately with unbound variable if the array remained unpopulated. The loop ran, it modified the array, and the entire state was discarded the microsecond the subshell closed.
The clean solution is to feed the loop via standard input redirection using process substitution. When input redirection feeds the loop, the while loop remains in the current shell process, preserving all mutations.
bash#!/usr/bin/env bash
set -euo pipefail
total_processed=0
declare -a active_servers=()
while read -r ip host rest; do
if [[ -n "${ip}" && -n "${host}" ]]; then
active_servers+=("${host}")
total_processed=$((total_processed + 1))
fi
done < <(grep -v '^#' /etc/hosts)
echo "Processed ${total_processed} hosts."
echo "First host: ${active_servers[0]}"
This preserves the current shell environment, but it opens the door to the next problem: how process substitution interacts with error detection under set -e.
Process Substitution Bypasses set -e and pipefail
Process substitution <(...) connects a background command to a file descriptor (typically /dev/fd/63 or a named pipe). Bash runs the command inside the substitution asynchronously.
Here is the problem: the exit status of a process substitution is not checked by the outer command, and it does not trigger set -e.
bash#!/usr/bin/env bash
set -euo pipefail
# fake_command does not exist.
# The process substitution will fail with status 127.
while read -r line; do
echo "Processing: ${line}"
done < <(non_existent_command_that_errors --flag)
# The script keeps running despite set -e
echo "Script finished successfully!"
Run this, and you will see the error message emitted to stderr, yet the script does not terminate.
textbash: non_existent_command_that_errors: command not found
Script finished successfully!
set -e inspects the exit code of foreground commands. The foreground command here is while read -r line, which simply reads EOF from the broken pipe and exits with status 0. The error in non_existent_command_that_errors occurred in an unmonitored background subshell.
To catch failures inside process substitutions, capture the background command PID using $! or switch to explicit file descriptors with wait semantics.
bash#!/usr/bin/env bash
set -euo pipefail
# Open a dedicated file descriptor to the process substitution
exec {subshell_fd}< <(non_existent_command_that_errors --flag)
subshell_pid=$!
while read -r line <&"${subshell_fd}"; do
echo "Processing: ${line}"
done
# Close the file descriptor
exec {subshell_fd}<&-
# Wait for the background process and assert its exit status
wait "${subshell_pid}"
echo "Script finished successfully!"
Running this script yields the correct failure behavior.
textbash: non_existent_command_that_errors: command not found
wait reaps the child process, retrieves its exit status, and because the exit code is non-zero, set -e triggers instantly.
The lastpipe Alternative and Why It Fails in Non-Interactive Shells
A common suggestion to fix subshell scope in pipelines is shopt -s lastpipe. This built-in option instructs Bash to run the final command of a pipeline in the current shell process instead of a subshell.
bash#!/usr/bin/env bash
set -euo pipefail
shopt -s lastpipe
counter=0
printf "one\ntwo\nthree\n" | while read -r line; do
counter=$((counter + 1))
done
echo "Count: ${counter}"
In non-interactive scripts, this works because job control is disabled by default. In an interactive shell, job control is enabled by default, so lastpipe has no effect unless job control is explicitly disabled (set +m). If your script, an imported library, or your CI runner enables job control (set -m or set -o monitor), lastpipe is disabled silently without a warning.
bash#!/usr/bin/env bash
set -euo pipefail
set -m # Job control enabled, common in CI or process wrappers
shopt -s lastpipe
counter=0
printf "one\ntwo\nthree\n" | while read -r line; do
counter=$((counter + 1))
done
# Prints 0 because set -m silently disables lastpipe
echo "Count: ${counter}"
Do not rely on lastpipe for production shell scripts. Use process substitution with explicit input redirection, or write the intermediate stream to a secure temporary FIFO.
Edge Case: Arithmetic Expansion Triggers set -e
A subtle trap when writing counters inside set -e loops is the post-increment or zero-result behavior of (( )) and $(( )).
In Bash, (( expr )) returns an exit status based on the value of the arithmetic expression:
- If the expression evaluates to a non-zero value, the exit code is
0(success). - If the expression evaluates to
0, the exit code is1(failure).
Under set -e, an arithmetic expression that evaluates to 0 will terminate your script.
bash#!/usr/bin/env bash
set -euo pipefail
counter=0
# This fails immediately because counter is 0, returning exit status 1
(( counter++ ))
echo "This line will never be reached."
Execution stops at (( counter++ )). The post-increment operator evaluates the expression to the original value of counter (which is 0) before incrementing it. Because the evaluated result is 0, the command fails, and set -e kills the script.
text$ bash test.sh
# Exit status 1, no output
To safely manipulate counters under set -e, use regular variable assignments or force a truthy evaluation:
bash# Safe: variable assignment with $(( )) does not trigger set -e on zero
counter=$((counter + 1))
# Safe: prefix increment if value becomes non-zero
# But still fails if counter was negative and hits 0!
# Best approach:
(( counter += 1 )) || true
The counter=$((counter + 1)) form evaluates inside standard assignment syntax. Bash does not treat the numeric value of the assignment as a process exit code, making it completely safe under set -e.
Edge Case: Command Substitution Strips Trailing Newlines and Ignores Pipefail on read
When parsing output streams line by line with read, the termination conditions of read interact poorly with pipelines and missing final newlines.
If the input source lacks a final trailing newline, read consumes the final bytes, assigns them to the destination variable, but returns a non-zero exit code (status 1) because it encountered EOF before a \n.
bash#!/usr/bin/env bash
set -euo pipefail
# Notice no newline after bar
input=$(printf "foo\nbar")
while read -r line; do
echo "Read: ${line}"
done < <(printf "%s" "${input}")
Output:
textRead: foo
The loop aborted before printing bar. When read hit bar, it hit EOF without a trailing newline, returned exit status 1, and the while loop evaluated the non-zero status as false, immediately terminating the loop.
To ensure the loop processes the final record even if the input lacks a newline, test for variable content alongside the exit code of read:
bash#!/usr/bin/env bash
set -euo pipefail
input=$(printf "foo\nbar")
while read -r line || [[ -n "${line}" ]]; do
echo "Read: ${line}"
done < <(printf "%s" "${input}")
Output:
textRead: foo
Read: bar
When read encounters bar, it returns 1, but [[ -n "${line}" ]] evaluates to true. The loop body executes for the final string. On the subsequent iteration, read reads nothing, returns 1, [[ -n "${line}" ]] evaluates to false, and the loop terminates cleanly without dropping data.
Verification Checklist
To verify that your Bash loops, error handling, and process substitutions behave reliably under production edge cases:
- Confirm process substitution exit statuses are tracked:
bashexec {fd}< <(sh -c 'exit 42')
pid=$!
wait "${pid}" || echo "Captured expected exit code: $?"
exec {fd}<&-
Expected output:
textCaptured expected exit code: 42
- Confirm subshell isolation is avoided by inspecting variables after the loop:
bashrecords=0
while read -r item; do
records=$((records + 1))
done < <(printf "a\nb\nc\n")
[[ "${records}" -eq 3 ]] && echo "Scope preserved"
Expected output:
textScope preserved
- Ensure zero-value initializations do not trigger early exit:
bashset -e
val=0
val=$((val + 1))
echo "Step completed: ${val}"
Expected output:
textStep completed: 1