Bash

Fix Truncated Bash While Loops Drained by Child Processes

A bash script running a while read loop over a file list or network stream terminates abruptly after the first iteration, leaving the remaining items unparsed. This happens most often when a command inside the loop body, such as ssh, ffmpeg, or mail, silently reads from standard input and drains the pipe before the next iteration begins.

Here is what the broken state looks like in an interactive terminal or a log file:

text
Processing host-prod-01...
Linux host-prod-01 5.15.0-101-generic x86_64
Done with host-prod-01.

Process finished with exit code 0

The source file contains fifty hosts, but the script exits cleanly after host-prod-01. No errors get printed to standard error, and the exit status is zero.

The Underlying Mechanism

Consider this typical bash pattern:

bash
#!/usr/bin/env bash
set -euo pipefail

while read -r host; do
    echo "Processing ${host}..."
    ssh "${host}" "uname -sr"
    echo "Done with ${host}."
done < hosts.txt

In bash, the redirection < hosts.txt binds file descriptor 0 (standard input, or stdin) for the entire while block. Every command executed inside that loop inherits file descriptor 0 from the loop itself unless you explicitly redirect it.

The ssh client is designed to capture local standard input and forward it to the remote system. When ssh executes in the first loop iteration, it sees an open standard input stream connected to hosts.txt. It reads every byte remaining in that stream until it encounters an end-of-file condition. When ssh finishes, the file pointer for hosts.txt is at the end. The loop evaluates read -r host a second time, immediately encounters the end of the file, returns a non-zero exit code, and the loop terminates.

Commands that consume stdin aggressively include:

  • ssh (and scp, sftp)
  • ffmpeg (which listens for terminal keystrokes like q to quit)
  • mail and sendmail
  • mplayer
  • tar -x or gzip when called without input files
  • python or node running inline code that reads sys.stdin

The Immediate Fix

You have two primary ways to fix the problem. The first is to redirect standard input to /dev/null for the specific command that steals it. The second is to move the loop’s file descriptor away from stdin.

Method 1: Null Redirection

For ssh, you can pass the -n flag, which redirects stdin from /dev/null. Alternatively, you can use the standard shell redirection < /dev/null.

bash
#!/usr/bin/env bash
set -euo pipefail

while read -r host; do
    echo "Processing ${host}..."
    ssh -n "${host}" "uname -sr"
    echo "Done with ${host}."
done < hosts.txt

If you are using a tool like ffmpeg that does not feature an -n flag with identical semantics, redirect standard input manually:

bash
while read -r video_file; do
    ffmpeg -nostdin -i "${video_file}" -codec:v libx264 "${video_file%.*}.mp4" < /dev/null
done < videos.txt

Method 2: Custom File Descriptors

The cleaner, more reliable solution is to assign the loop read operation to an alternate file descriptor. This isolates your input stream from any command that defaults to reading stdin.

File descriptors 0, 1, and 2 are reserved for standard input, standard output, and standard error. File descriptor 3 through 9 are available for custom use in POSIX shells. In modern bash (version 4.1 and newer), you can also let bash allocate a descriptor dynamically.

Using an explicit descriptor:

bash
#!/usr/bin/env bash
set -euo pipefail

while read -u 3 -r host; do
    echo "Processing ${host}..."
    ssh "${host}" "uname -sr"
    echo "Done with ${host}."
done 3< hosts.txt

The flag -u 3 tells read to consume data from file descriptor 3. The loop redirects hosts.txt into descriptor 3 via 3< hosts.txt. Because standard input (file descriptor 0) remains untouched, ssh does not see hosts.txt and cannot drain it.

Verifying the Fix

You can verify the behavior locally without real remote servers by simulating a greedy command. The following test script demonstrates the failure and the fix.

Create a file named items.txt:

text
alpha
bravo
charlie
delta

Now run a script that simulates a greedy child process using cat:

bash
#!/usr/bin/env bash
set -euo pipefail

echo "=== Broken Loop ==="
while read -r line; do
    echo "Read: ${line}"
    # cat drains all remaining stdin from the loop
    cat > /dev/null
done < items.txt

echo "=== Fixed with Custom File Descriptor ==="
while read -u 3 -r line; do
    echo "Read: ${line}"
    # cat now reads from descriptor 0, which is empty/interactive, not descriptor 3
    cat < /dev/null > /dev/null
done 3< items.txt

Execute the script:

bash
bash test_loop.sh

The output confirms the isolated loop processes all four entries:

text
=== Broken Loop ===
Read: alpha
=== Fixed with Custom File Descriptor ===
Read: alpha
Read: bravo
Read: charlie
Read: delta

Edge Case 1: Trailing Lines Without Final Newlines

Standard POSIX text files require each line to terminate with a newline character (\n). If your input file was generated by an external API, a Windows machine, or an export tool, the final line might lack a trailing newline.

When read hits an EOF before a newline, it populates the variable with the remaining characters, but exits with a non-zero status code (1). In a while read loop, this causes the loop to discard the final line.

bash
# Generate a file without a trailing newline
printf "server-a\nserver-b" > no_trailing_newline.txt

# This loop only processes server-a
while read -r line; do
    echo "Found: ${line}"
done < no_trailing_newline.txt

Output:

text
Found: server-a

To prevent dropping the final record, check if the variable holds data even when read returns a non-zero code. Combine the read command with a logical OR:

bash
while read -u 3 -r line || [[ -n "${line}" ]]; do
    echo "Found: ${line}"
done 3< no_trailing_newline.txt

Output:

text
Found: server-a
Found: server-b

If the string is not empty, the loop executes one final time.

Edge Case 2: Subshell Variable Loss via Pipelines

Developers often pipe command output directly into a while read loop instead of redirecting from a file on disk:

bash
count=0
find . -type f -name "*.log" | while read -r file; do
    count=$((count + 1))
done
echo "Total logs: ${count}"

Output:

text
Total logs: 0

By default in POSIX shells, each command in a pipeline runs inside a subshell. The variable count is modified within the subshell process table. When the pipe finishes, that subshell exits, taking the modified memory space with it. The parent shell’s count remains 0.

To fix this while keeping your code clean and free of temporary files, use process substitution instead of a pipeline:

bash
#!/usr/bin/env bash
set -euo pipefail

count=0

while read -u 3 -r file; do
    count=$((count + 1))
done 3< <(find . -type f -name "*.log")

echo "Total logs: ${count}"

Output:

text
Total logs: 142

Process substitution creates a named FIFO pipe or a path under /dev/fd/ that bash opens cleanly. The while loop runs directly in the execution context of the main shell process. The internal counter updates persist after the loop exits.

Edge Case 3: Accidental Trimming of Leading and Trailing Whitespace

The default value of IFS (Internal Field Separator) in bash contains space, tab, and newline: IFS=

    \t\n'
    .

    When read processes a line, it uses IFS to split words and strip leading and trailing whitespace. If you are processing structured data like indentation levels in code files, log entries with exact padding, or paths that begin with spaces, standard read -r line will alter your input.

    Input data:

    text
        indented_block_level_one
            indented_block_level_two
    

    Testing the default loop:

    bash
    while read -r line; do
        printf "%s\n" "${line}"
    done < indent.txt
    

    Output:

    text
    indented_block_level_one
    indented_block_level_two
    

    The indentation is gone. To preserve exact byte layouts, clear IFS locally for the read command only:

    bash
    while IFS= read -u 3 -r line || [[ -n "${line}" ]]; do
        printf "%s\n" "${line}"
    done 3< indent.txt
    

    Output:

    text
        indented_block_level_one
            indented_block_level_two
    

    Setting IFS= immediately preceding read limits the variable change strictly to that single command execution. Subsequent commands inside the loop retain the normal global IFS settings.

    More in Bash

    Bash 18 November 2026

    Fix Argument List Too Long and Slow Bulk File Deletions

    bashclilinux
    Bash 18 November 2026

    Fix GitHub CLI Auto Merge Failures and Branch Protection Locks

    bashgithubgit
    Bash 18 November 2026

    Fix Stalled Command Line Pipes and Output Buffering Delays

    bashlinuxcli