How to Find and Remove Duplicate Files in Linux - Best Methods

Duplicates can take up a lot of space on your Linux device and cause your computer to slow down. Moreover, you may need to free up memory, and the best way to do so is to find file duplicates on Linux and get rid of them.

Finding individual files manually may take hours, if not days, so we'll present the most efficient and productive methods below. These will help you find and remove duplicate files on Linux within seconds and make your device as good as new.

Best Methods for Finding and Removing Duplicate Files in Linux

Below is a list of the best Linux duplicate file finders on the market. We'll introduce each tool and present detailed step-by-step guides to help you navigate the journey.

Remove Duplicate Files in Linux With Fdupes

1. Download the tool according to your Linux parameters by running the following commands:

   How to install	Distribution
   sudo apt install fdupes	[On Debian, Ubuntu, and Mint]
   sudo yum install fdupes	[On RHEL/CentOS/Fedora and Rocky/AlmaLinux]
   sudo emerge -a sys-apps/fdupes	[On Gentoo Linux]
   sudo apk add fdupes	[On Alpine Linux]
   sudo pacman -S fdupes	[On Arch Linux]
   sudo zypper install fdupes	[On OpenSUSE]

2. The general command is fdupes < dir >, and you can set options from the table below.

   Options	Description
   -r	Scan all subdirectories in the main directory
   -A	Exclude hidden files from the scan
   -s	Access directories with symbolic links
   -d	Prompts users for files to preserve while deleting all other files.
   -m	Compare duplicate files

3. You can also specify multiple directories, such as fdupes < dir1> -r < dir2> to cut the searching time.
4. To find all the duplicate files, run the -m command and delete them by running the -d command.
5. If you need assistance on your fdupes journey, run the fdupes -help command.

   

Find File Duplicates in Linux With Rdfind

Instead, you'll get a file filled with all the identical data you can review and delete manually. Moreover, the tool can distinguish between the original and the duplicate files according to their depths and construction. Here's how to use Rdfind:

1. Install Rdfind according to your Linux distribution:

   How to install	Distribution
   sudo apt install rdfind	[On Debian, Ubuntu and Mint]
   sudo yum install rdfind	[On RHEL/CentOS/Fedora and Rocky/AlmaLinux]
   sudo emerge -a sys-apps/rdfind	[On Gentoo Linux]
   sudo apk add rdfind	[On Alpine Linux]
   sudo pacman -S rdfind	[On Arch Linux]
   sudo zypper install rdfind	[On OpenSUSE]

2. Next, run the rdfind /home/user command.
3. After running the command, you'll get the results.txt file in the directory you scanned.

   

4. Open the file, review the data, and delete the duplicate files of your choice.
5. You can also type the -dryrun command to list all the duplicates without taking any actions.

Find Identical Files in Linux With FSlint

Onto something a little bit different than the two above mentioned duplicate finder solutions. FSlint is a tool that provides a graphical user interface, making the duplicate finding and deleting experience much more productive and efficient. You can also categorize your files according to their types and save time accordingly. You can download and use FSlint by following these steps:

1. Download FSlint according to your distribution:

   How to install	Distribution
   sudo apt install fslint	[On Debian, Ubuntu and Mint]
   sudo yum install fslint	[On RHEL/CentOS/Fedora and Rocky/AlmaLinux]
   sudo emerge -a sys-apps/fslint	[On Gentoo Linux]
   sudo apk add fslint	[On Alpine Linux]
   sudo pacman -S fslint	[On Arch Linux]
   sudo zypper install fslint	[On OpenSUSE]

2. Input the path you want to scan in the box at the top.

   

3. Click on the Find option to discover duplicates.

   

4. Head to the Advanced Search Parameters tab to categorize your files.

   

Remove Duplicate Files in Linux With dupeGuru

1. Install dupeGuru according to the Linux distribution:

   How to install	Distribution
   sudo apt install dupeguru	[On Debian, Ubuntu, and Mint]
   sudo yum install dupeguru	On RHEL/CentOS/Fedora and Rocky/AlmaLinux]
   sudo emerge -a sys-apps/dupeguru	[On Gentoo Linux]
   sudo apk add dupeguru	[On Alpine Linux]
   sudo pacman -S dupeguru	[On Arch Linux]
   sudo zypper install dupeguru	[On OpenSUSE]

2. Open dupeGuru, select the directory, scan type, and input the desired path you want to scan.

   

3. After the program scans the directory, select the duplicates and choose what to do with them (delete, move, copy, etc.).

   

Find File Duplicates in Linux With Snapckage Manager and FSlint

You can combine the Snap Package Manager and FSlint to get the best graphical user interface experience and turn your duplicate deletion journey into a piece of cake. Here's how to install and use Snap and FSlint on your Linux device:

1. Install Snap according to your Linux distribution:

   How to install	Distribution
   sudo apt update && sudo apt install snapd	[On Debian, Ubuntu and Mint]
   sudo yum install snapd	[On RHEL/CentOS/Fedora and Rocky/AlmaLinux]
   sudo pacman -S snapd	[On Arch Linux]

2. Next, install FSlint using the Janitor by running this command: sudo snap install fslint-unofficial.
3. Open FSlint Janitor.

   

4. Click the Add button.

   

5. Select the directory you want to scan and click the OK button.

   

6. Select Duplicates and click on the Find button.

   

7. Select the files you want to remove and click the Delete button.

   

8. Click Yes to finish the process.

   

Find File Duplicates in Linux Using Rmlint

As suggested by the name, Rmlint is a command-line tool that allows you to discover and remove lint files from your machine. The tool is capable of detecting files with identical content, discovering empty files, broken links, orphaned files, and more. You can use this command line by following the steps below:

1. Download Rmlint according to your Linux distribution:

   How to install	Distribution
   sudo apt install rmlint	[On Debian, Ubuntu and Mint]
   sudo yum install rmlint	[On RHEL/CentOS/Fedora and Rocky/AlmaLinux]
   sudo emerge -a sys-apps/rmlint	[On Gentoo Linux]
   sudo apk add rmlint	[On Alpine Linux]
   sudo pacman -S rmlint	[On Arch Linux]
   sudo zypper install rmlint	[On OpenSUSE]

2. After installing the tool, run the rmlint command in any directory.

   

The command line will create a shell script on the directory you can run to remove all duplicate files from your Linux device.

Bonus: How to Recover Deleted/Removed Linux Files

Dedupe files in Linux can be a pain in the neck, especially if you're running out of storage. However, one of the things that may be even more stressful is losing the original files while trying to delete duplicates. You may have accidentally deleted the important files in the process, and now you're wondering why you even started the journey in the first place.

We've got you covered if you're currently in this scenario or want to prepare for the worst. We've researched the market and discovered the best Linux recovery tool: Wondershare Recoverit. This tool has the strongest recovery mechanisms in the world, with over 35 patented technologies proving its reliability.

Due to the advanced algorithms, Wondershare Recoverit has a 98% success rate and can recover all file types from any data loss scenario. You can always rely on this solution if things go downhill.

You'll need another computer using Windows or Mac to perform a successful recovery. Head to the official Wondershare Recoverit website using this device, download the tool, and start the recovery process by following this guide:

1. Open Wondershare Recoverit and click on NAS and Linux from the main page.
2. Click on Linux Recovery.

   

3. Next, follow the onscreen instructions to connect your Windows/Mac computer to the Linux machine.

   

4. After connecting, the program will automatically start scanning for lost files. You can track the progress or pause the process in the next window.

   

5. Once the recovery is complete, preview the files and select the ones to save. Click the Recover button.

   

6. Choose where to save the files.

   

Conclusion

Duplicate files on your Linux can take up more space than needed and cause unnecessary issues. If you want to remove such files from your system, we're here to help. We've presented reliable duplicate file finders for you to find and remove identical files in Linux, including Fdupes, Rddins, FSling, dupeGuru, Rmlint, and Snap with FSling.

We've also prepared a bonus tip to help recover data if you accidentally delete important files from your Linux device. Wondershare Recoverit has the industry's strongest recovery systems – you'll extract all your valuables if you choose this option.