File management is one of the common tasks that a user undertakes on a Linux system, which includes creating, copying, moving, modifying, and deleting files and directories.
This article provides a few command-line tips on how you can delete a large directory that contains thousands of files in a Linux system.
Table of Contents
Delete Files in Linux
The most common way of deleting files on a Linux system is using the rm command, which takes the following syntax format:
$ rm [ options ] sample_file.txt
For example, to delete a text file called file1.txt, run the command:
$ rm file1.txt
To forcefully remove a file without being asked for permission, pass the -f
flag as follows.
$ rm -f file1.txt
Delete Directory in Linux
To remove or delete a directory called sample_directory, run the following command:
$ rm -rf sample_directory
The -r
option recursively deletes the directory alongside all the subdirectories and files contained therein.
To delete or remove an empty directory use the rmdir command, which comes in handy when you want to remove an empty directory called test_directory as shown:
$ rmdir test_directory
Delete a Large Directory with Tons of Files
When the rm command is executed, the filesystem only removes the link to the file, which makes the file unavailable to the user, but in the real sense, the file’s data itself remains intact on the disk.
Therefore, when the rm command is issued, only the reference to the files is removed, which frees up the storage blocks in the filesystem.
As such, there exist several avenues to delete files in Linux.
Delete Files With Inode Number in Linux
For example, you can delete a file using its inode number. You can find out a file’s inode number using the stat command as shown.
$ stat file1.txt File: file.txt Size: 4076 Blocks: 8 IO Block: 4096 regular file Device: 801h/2049d Inode: 1573697 Links: 1 Access: (0664/-rw-rw-r--) Uid: ( 1000/ tecmint) Gid: ( 1000/ tecmint) Access: 2023-05-08 12:10:55.656070248 +0530 Modify: 2023-05-08 12:10:55.656070248 +0530 Change: 2023-05-08 12:10:55.656070248 +0530
In addition, you can pass the -i
flag in the ls command when listing files inside a directory.
$ ls -li 1573697 .rw-rw-r-- tecmint tecmint 4.0 KB Mon May 8 12:10:55 2023 file1.txt
To remove the file using its inode, use the find command as shown in the syntax below.
$ find /path/to/file -inum INODE_NUM -exec rm -i {} +
In our example, to remove file file1.txt that sits in the current directory, the command will be:
$ find /path/to/file -inum 1573697 -exec rm -i {} +
Hit 'y'
to confirm the removal and press ENTER.
Let us now see how to delete large directories with thousands of files.
Create a Directory with Thousands of Files
The good old rm command is the fastest way of deleting a large directory with thousands of files. To demonstrate this, we will, first, create a sample directory and navigate into it.
$ mkdir test_dir $ cd test_dir
Next, we will create an insanely huge number of files, in this case, 500,000 text files using the following bash for a loop.
$ time for item in {1..500000}; do touch file_name$item.txt; done
Fastest Way to Delete Directory in Linux
The fastest way to delete a large directory is using the good old rm directory as shown below. Here, the time option displays the time taken to successfully execute the command.
$ time rm -rf /test_dir
From the output, you can see that it has taken roughly 6 seconds to delete the entire directory.
Delete Large Directory with Find Command
Another way to delete large directories is using the find command as shown in the following syntax.
$ time find /path/to/directory -delete
Although not as fast as the rm command it still gets the job done.
$ time find test_dir -delete
Delete Large Directory with Perl Command
Another approach is to use the Perl scripting language inside the directory to remove tons of files.
$ cd test_dir $ time perl -e 'for(<*>){((stat)[9]<(unlink))}'
From the output, you can this that it took much longer to delete all the files in the directory than the previous commands that we looked at earlier.
Conclusion
There you have it. In this guide, we have looked at how you can delete large directories that contain thousands of files on a Linux system.
If it took *84 minutes* to touch 500k 0-byte files, you’re doing something wrong.
On a 2011 core-i5 iMac with OSX High Sierra 10.13 running against a Firewire-800 spinning disk with APFS filesystem:
# Parallel process 2x / time
Begin:
Thu May 11 09:01:03 CDT 2023
real 1m47.971s
user 1m11.711s
sys 1m44.261s
Thu May 11 09:02:51 CDT 2023
End
xargs ~30% CPU,
Terminal ~20% cpu – and you can also preface the create command with ‘nice‘.
As for deleting a directory with thousands of files, I’d argue the *safest* way to do so would be to use Midnight Commander. Navigate to the directory, hit
F8
, and it asks for confirmation, no unexpected results.This will not work on directories with a large number of files, where the shell (like bash) runs out of memory. There the find/exec rm method is a valid solution.
@Andreas,
Could you share one example, of how the find/exec rm method is a valid solution for this? It would be a great help for all Linux users…
Really very good article for deleting lakhs of files in a directory in Linux.