Skip to content
IBM mmapplypolicy migration scripts from /scratch to gpfs2
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.gitignore
README.md
as_tbl.R
as_tbl.slurm
list_dir_size.sh
quota.R
report

README.md

Migrate from scratch to gpfs2

Raw data files

list.dir_size - list of all files on /scratch under user and group directories (about 15 GB, so not uploaded to this git repo). This file was generated using list_dir_size.sh.

report - list of all quotas as caprured by the GPFS command mmrepquota gpfs2 last year.

Data processing

as_tbl.slurm aggregates list.dir_size by user and group directories and saves the result as the dirs.dir_size tab separated file.

quota.R - aggregates report to compare disk usage against quotas. To run this R script interactively:

module load gcc/5.4.0-alt r/3.4.2-gcc540
Rscript quota.R

Output of quota.R

Top groups and users above the 50 GB typical quota:
# A tibble: 71 x 5
            name    type          KB       limit  percent
           <chr>   <chr>       <dbl>       <dbl>    <dbl>
 1   stormcenter FILESET 29492839520 31138512896 94.71499
 2 dongare-share FILESET 19524282240 21474836480 90.91702
 3        maylab FILESET 19508856544 21474836480 90.84519
 4         airmg FILESET 14533020896 16106127360 90.23287
 5      manoslab FILESET  5835524384  6442450944 90.57926
 6 healthinfolab FILESET  2939391424  5368709120 54.75043
 7    climatelab FILESET  5225455296  5368709120 97.33169
 8      jinbolab FILESET  5069552704  5368709120 94.42778
 9          pire FILESET  1864375744  3221225472 57.87784
10      amd12020 FILESET  1108269568  2147483648 51.60782
# ... with 61 more rows

Comparison of total usage in October, max quotas, and disk size.
# A tibble: 3 x 2
  quantity     size
     <chr>    <chr>
1    usage 118.5 TB
2    quota 183.0 TB
3     disk 115.0 TB
You can’t perform that action at this time.