How to Remove Old Folders and Keep the Latest in Linux terminal

Managing files and folders is a common task, especially in environments where backups, logs, or daily data create a growing list of directories. In this guide, we'll cover how to efficiently delete folders except for the most recent ones. This script is particularly useful in maintaining a clean directory and saving storage.

Command Breakdown

The core command to remove all but the last three folders is:

ls -1 | sort | head -n -3 | xargs -I {} rm -rf "{}"

Explanation of Each Part

1. ls -1:
2. Lists the contents of the directory, one item per line.

3. -1 forces ls to display output in a single column, making it easier to process with other commands.

4. sort:

5. Sorts the output by default in lexicographical order.

6. If folders are named by date in a YYYY-MM-DD format, this naturally orders them chronologically, oldest to newest.

7. head -n -3:

8. Displays all lines except the last three (-n -3).

9. This keeps the latest three folders while outputting only older ones for deletion.

10. xargs -I {} rm -rf "{}":

11. xargs takes each line of input and applies it as an argument to rm -rf.
12. -I {} defines {} as a placeholder for each item.
13. rm -rf "{}" removes each specified directory (-r for recursive, -f for force).

Example Scenario

Suppose you have folders named by date:

2024-10-18  2024-10-19  2024-10-20  2024-10-21  2024-10-22  2024-11-10  2024-11-11  2024-11-12

Running the command above will remove all folders except for 2024-11-10, 2024-11-11, and 2024-11-12.

Customizing the Script

Adjusting the Number of Days to Keep

To change the number of folders to keep, adjust the -n argument in head. For example:

ls -1 | sort | head -n -5 | xargs -I {} rm -rf "{}"

This would keep the last five folders instead of three.

Sorting by Modification Date

If the folders aren’t named by date, but you still want to keep the most recently modified ones, use ls -lt to sort by modification time:

ls -lt | awk '{print $9}' | head -n -3 | xargs -I {} rm -rf "{}"

This command sorts by modification time and then removes all but the three most recently modified folders.

Automating with a Cron Job

For frequent cleanup, automate this command with cron.

1. Open your crontab editor:

crontab -e

1. Add an entry to run the script daily at a specified time (e.g., midnight):

0 0 * * * /path/to/cleanup_script.sh

Ensure the script file (cleanup_script.sh) contains:

#!/bin/zsh
cd /path/to/folder
ls -1 | sort | head -n -3 | xargs -I {} rm -rf "{}"

1. Make the script executable:

chmod +x /path/to/cleanup_script.sh

This cron job will execute the script daily at midnight, removing all but the latest three folders.

Advanced Commands

For scenarios that require more complex filtering or conditional removal, here are some advanced command variations to further enhance your folder management.

1. Keeping Folders by Specific Date Range

If you want to keep folders within a specific date range (e.g., only those from November 2024), use grep in combination with sort:

ls -1 | grep '^2024-11' | head -n -3 | xargs -I {} rm -rf "{}"

• Explanation: grep '^2024-11' filters folders that start with 2024-11, so only November 2024 folders are considered.
• Dry Run: Replace rm -rf with echo to preview which folders would be affected.

2. Removing Folders Older Than a Specified Number of Days

Use find to target folders based on their age rather than relying on names:

find . -maxdepth 1 -type d -mtime +3 -exec rm -rf {} +

• Explanation:
• find . -maxdepth 1: Limits the search to the current directory.
• -type d: Only selects directories.
• -mtime +3: Selects directories last modified more than three days ago.
• -exec rm -rf {} +: Deletes the selected directories.

3. Selective Deletion Based on Pattern Matching

To delete folders that follow a specific naming pattern, such as keeping all dates but excluding certain tagged folders (e.g., with “_backup” suffix):

ls -1 | grep -v "_backup" | head -n -3 | xargs -I {} rm -rf "{}"

• Explanation: grep -v "_backup" excludes any folder names containing "_backup", ensuring these aren’t deleted in the cleanup process.

4. Combining Multiple Conditions

Suppose you want to keep the last three folders but exclude folders that contain a specific string (like test) and only delete folders older than three days. Combine find, grep, and xargs as follows:

find . -maxdepth 1 -type d -mtime +3 | grep -v "test" | sort | head -n -3 | xargs -I {} rm -rf "{}"

• Explanation:
• find . -maxdepth 1 -type d -mtime +3: Finds folders modified more than three days ago.
• grep -v "test": Excludes folders containing "test" in the name.
• sort and head -n -3: Orders the folders and keeps only the latest three.

5. Deleting All Except Specified Days

To delete everything except specific days (e.g., keep only folders with dates 2024-10-25, 2024-11-01, and 2024-11-12), use grep -v with specific date patterns:

ls -1 | grep -v -E "2024-10-25|2024-11-01|2024-11-12" | xargs -I {} rm -rf "{}"

• Explanation: grep -v -E "2024-10-25|2024-11-01|2024-11-12" excludes the specified dates, deleting everything else.

6. Dry Run Reminder

For safety, always test your commands with a dry run before executing:

ls -1 | sort | head -n -3 | xargs -I {} echo "{}"

Or for find commands:

find . -maxdepth 1 -type d -mtime +3 -exec echo {} +

This outputs which directories would be deleted without actually removing them, helping to ensure the right folders are targeted.

Safety Tips

1. Dry Run: Before executing, test the output to confirm which folders would be deleted by replacing rm -rf with echo:

ls -1 | sort | head -n -3 | xargs -I {} echo "{}"

1. Backups: Always ensure any critical data is backed up, especially before automating deletion scripts.

2. Permissions: Running the script as a non-root user can help avoid accidental deletion of important system folders.

This script and guide provide an efficient solution to managing old directories while keeping only the most recent ones. Customize it as needed, and follow best practices to ensure safe and reliable usage.

Dan Duran