dtree-mailing

Dtree-mailing is a Python script which is used to email HPC users to reques them to remove old directories on the scratch disk to save space.

Pre-requisites

1. The output file from the dtree script. This file list the users and their directories.
2. The script MailViaServer3. This is used to merge user data with an email template and send it to users' email. This script is avilable from http://github.uconn.edu

Files in this repository

1. Dtree-mailing.py.
2. email.template. This is used to construct the email sent to each users. Strings in this document named {{NAME}}, {{EMAIL}}, {{LIST-FILE}} are replaced with particular value for each user email.

How to use dtree-mailing.py

You can run dtree-mailing.py without any arguments to see usage information. Typically, you run tree-mailing.py by giving it the name of an output file from dtree.py. These output files have several sections, but the relevant section here is preceded by the line '#SCRATCH DIRECTORIES'. That section of the file lists the directories that are larger and older than specific threshhold values. In our case, we want the output file that gives folders large than 200GB and older than 180 days. These files are in /gpfs/scratchfs1/admin/gpfs-scan-files/output/YYYY-MM-DD/dtree-180-200G.out where YYYY-MM-DD is the date, like "2020-12-25".

The email template file

This repository includes the file email.template, which contains a text file formatted for HTML mail. You can edit this file and change the text and HTML to suit your circumstances. In particular, you need to edit the Subject line to insert the correct Helpspot Ticket number.

Output from dtree-mailing.py

Dtree-mailing.py writes a bash script named send-mailing.sh which you can run. This script will send email to every user from the dtree output file. It calls the command-line program MailViaServer3, which is available from it's own repository.

Dtree-mailing.py also writes a number of files in the directory list/ - one files for each user. These files list the directories that we reqeust the user to remove.

Testing and sending the mailing

The last step of this process is to run the script send-mailing.sh. However, before you run it, you should test email.template. Do this by copying the send-mailing.sh script to test-mailing.sh. In this script, remove all the users except the first (the users are listed between the lines with EOM).

For this first user, replace the email address with your own - leave the other fields as is. Then run test-mailing.sh. The script will send a sample email to your email address only.
Review the email, make any corrections to email.template as necessary, and retry. Once the result looks correct, you can run the original script send-mailing.sh.

The script sending-mail.sh will also write a line into the file mail.log in the current directory giving the time and recipient for every email sent, so you have a record of who was emailed.