Skip to content
Bash scripts to clean directories like /scratch, /work, etc.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
tests
.dir-locals.el
README.md
dirclean
dirclean.conf

README.md

** TEARDOWN NOTICE: **

This bash project has been abandoned in favor of python based HPC/gpfsclean for several reasons:

  • Bash has a hard time dealing with null characters required to parse some of the more extravagant file names.

  • Bats unit tests are limited making them less useful as a troubleshooting tool.


dirclean

Clean directories and e-mail warnings to users.

In a shared filesystem, it's often necessary to delete old files, but one must:

  1. Warn users sufficiently in advance.

  2. Spare directories which have any recently modified files within them. This requirement is a balancing act as it's possible for many users in research groups to work within one directory and stale files will accumulate in those directories over time. Ideally one would account for this possibility by adding some sort of find -maxdepth configuration parameter, but that also can dangerously increase the script runtime if a user had millions of files or directores under their top level directory. Therefore current behavior follows the previous ruby version by only inspecting the first level of files and directories and deciding their fate by whether it contains a valid file or directory.

Installation

Runtime dependencies are:

  • bash 4
  • mailx
  • find
  • stat
  • date
  • sed
  • sort
  • uniq

As dirclean is meant to be called from a cron job, it stores all inputs in a configuration file, including warn intervals, delete time, grace time and mailing details.

Records of e-mailed users and deleted directories are written to stdout. Any errors or warnings are written to stderr.

An example cron entry is:

# min hr dom mon dow  user  command
   00  7   *   *   *  root  /usr/local/libexec/dirclean > /var/log/dirclean.log

Tests

Deleting user files, as you can imagine, is a sensitive operation and not something one would want to do wrong. Therefore, we use the bats unit test suite.

Run the unit tests with:

module load bats
tests/*.bats

If you have done any unit testing before, the format is similar and the documentation is simple and concise. Learn to use bats from reading the man pages, or GitHub The GitHub link includes links to projects using bats as well as setting up your favorite text editor for Syntax highlighting, etc.

man bats
man 7 bats
# Or see https://github.com/sstephenson/bats

Hacking

  1. Read the dirclean script. Many stylistic elements can be intuitively determined by reading and copying what's already there. Some of the style guides below are not strictly followed. Use your noodle ;)

  2. As far as possible follow Jeff Lindsay

  3. When in doubt, follow Greg Wooledge

  4. Only for documenting functions, follow Google

You can’t perform that action at this time.