You run a cleanup script or a batch rename loop on a directory with two million small files, and find chokes before it deletes a single one. Or worse: you pipe find into xargs, the terminal goes quiet for ten minutes, your root partition runs out of inodes, and your shell prints an argument list error anyway.
Here is the exact terminal output when a naive argument expansion breaks on an oversized directory:
text$ rm -rf ./tmp/cache/*
-bash: /usr/bin/rm: Argument list too long
$ find ./tmp/cache/ -name "*.sess" | xargs rm -f
xargs: unterminated quote
$ find ./tmp/cache/ -name "*.sess" -exec rm -f {} \;
^C
[Process killed after 45 minutes: only 12,000 files deleted]
Why standard tools choke on millions of files
Bash hits an operating system wall called ARG_MAX. This is the maximum byte limit for the command-line argument list plus environment variables passed into the execve system call. On modern Linux, ARG_MAX is typically 2,097,152 bytes (2 MB), which you can check by running getconf ARG_MAX. When you type rm -rf ./tmp/cache/*, the shell expands the wildcard glob before passing it to rm. If you have 300,000 files named with UUIDs, the expanded string exceeds 10 MB, and the kernel kills the call instantly with E2BIG.
The second failure happens with default xargs. By default, xargs splits incoming standard input on newlines and whitespace. If a filename contains a single quote, a backslash, or a space, standard xargs treats the quote as the start of an unmatched string and crashes with unterminated quote, stopping the entire pipeline.
The third failure, -exec rm -f {} \;, does not crash. It crawls. Every single file match causes find to run a fork and execve cycle. Spawning 2,000,000 separate processes to remove 2,000,000 files takes hours because process creation overhead dwarfs the filesystem unlinking work.
The fast, safe fix with null bytes and batched exec
If you must use find, do not use wildcards, and do not use -exec ... \;. You have two direct paths depending on whether you are filtering or doing a pure bulk purge.
Method 1: The optimized find pipeline
Run find using the -print0 option combined with xargs -0. The -0 flag tells both utilities to separate filenames using the null character (\0) instead of spaces or newlines. Null bytes cannot exist inside Linux file names or paths, so this cannot break on weird formatting.
bashfind /path/to/target -type f -name "*.sess" -print0 | xargs -0 -P 4 -n 5000 rm -f
Look at the flags:
-print0: prints the full path followed by a null character.-0: reads null-terminated inputs.-P 4: spawns 4 parallel worker processes to delete files concurrently. This keeps high-IOPS NVMe drives saturated.-n 5000: batches 5,000 arguments per single invocation ofrm, keeping argument size well belowARG_MAXwhile reducing process spawning by 99.98 percent.
If you want find to handle batching natively without piping into xargs, use the plus sign terminator instead of the semicolon:
bashfind /path/to/target -type f -name "*.sess" -exec rm -f +
The + tells find to build argument lists up to the OS limit and pass them to a single rm process, repeating until all files are handled.
Method 2: The rsync blank-directory swap
If your job is to empty out an entire directory that contains millions of files, skip find and rm completely. Directory entries in filesystems like ext4 are stored in directory index structures (like hashed B-trees). When you delete files one by one with rm, the OS updates directory nodes incrementally.
The rsync sync-to-empty trick bypasses those slow incremental updates:
bashmkdir -p /tmp/empty_dir
rsync -a --delete /tmp/empty_dir/ /path/to/target/
rmdir /tmp/empty_dir
The trailing slash on /tmp/empty_dir/ is critical. It tells rsync to sync the contents of the empty folder into the target folder, unlinking entries through bulk system calls with far less lock contention than single unlink() passes.
How to verify the fix works
Before and during execution, verify that inodes and dentries are dropping without stalling your process table.
First, check the starting inode count:
bashdf -i /path/to/target
Output:
textFilesystem Inodes IUsed IFree IUse% Mounted on
/dev/nvme0n1p2 13107200 4821034 8286166 37% /
Next, test your deletion command in dry-run mode before dropping data:
bashfind /path/to/target -type f -name "*.sess" -print0 | xargs -0 -n 5000 echo | head -n 2
Output:
text/path/to/target/0000a.sess /path/to/target/0000b.sess ... [thousands of entries printed]
/path/to/target/01a9f.sess /path/to/target/01b2c.sess ...
Run the deletion command with an elapsed time monitor:
bashtime find /path/to/target -type f -name "*.sess" -print0 | xargs -0 -P 4 -n 5000 rm -f
Output:
textreal 0m14.281s
user 0m0.412s
sys 0m12.890s
Run df -i again:
bashdf -i /path/to/target
Output:
textFilesystem Inodes IUsed IFree IUse% Mounted on
/dev/nvme0n1p2 13107200 2821034 10286166 22% /
Two million inodes freed in fourteen seconds.
Edge case 1: Inodes freed but disk space still allocated
You run the command, your inode count drops, but df -h shows disk space has not changed at all.
This happens when a running process has open file handles to the files you deleted. In Linux, unlinking a file from the directory tree marks the filename as removed, but the storage blocks remain allocated until every process holding an open file descriptor closes it.
Find out who is holding the deleted files:
bashlsof +L1 /path/to/target
Output:
textCOMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME
python3.11 8192 app 3w REG 259,2 10485760 0 4194305 /path/to/target/worker.sess (deleted)
python3.11 8193 app 3w REG 259,2 10485760 0 4194306 /path/to/target/worker2.sess (deleted)
The +L1 flag tells lsof to look specifically for open files that have a link count of less than 1 (meaning deleted). The output shows Python processes holding those files open.
To reclaim the space immediately without corrupting application data, locate the file descriptor under /proc and truncate it:
bash# Truncate the file through the open descriptor
: > "/proc/8192/fd/3"
Or, if the application allows safe restarts, reload the daemon:
bashsystemctl restart app-worker
Once the process exits or reloads, the kernel frees the blocks.
Edge case 2: The parent directory itself is bloated
On ext4 filesystems, when a directory grows to hold millions of entries, the directory itself becomes a multi-gigabyte file on disk. Deleting all the files inside it does not shrink the directory entry file. It remains massive, which makes future lookups and ls commands slow even when the folder contains only three files.
You can inspect the directory size directly:
bashls -ld /path/to/target
Output:
textdrwxr-xr-x 2 app app 402653184 Feb 18 10:15 /path/to/target
That size column reads roughly 402 megabytes just for the directory mapping table.
rm -rf inside the directory cannot shrink this file; ext4 directory files do not release their allocated tree nodes back to the free block list. The fix is to recreate the directory node from scratch.
Move the old directory, create an empty replacement with the original permissions, and delete the renamed directory:
bash# Capture permissions and ownership
PERMS=$(stat -c "%a" /path/to/target)
OWNER=$(stat -c "%U:%G" /path/to/target)
# Rename the bloated directory
mv /path/to/target /path/to/target_old
# Recreate the directory
mkdir -p /path/to/target
chmod "$PERMS" /path/to/target
chown "$OWNER" /path/to/target
# Now purge the old directory
rm -rf /path/to/target_old
Check the new directory:
bashls -ld /path/to/target
Output:
textdrwxr-xr-x 2 app app 4096 Feb 18 10:20 /path/to/target
The directory is back to 4096 bytes. Shell wildcards work again, memory consumption drops, and path resolutions take microseconds instead of seconds.