diff --git a/README.md b/README.md index 2c5d8dc..535228e 100644 --- a/README.md +++ b/README.md @@ -97,27 +97,45 @@ last job 2339010 had nothing to do and completed immediately in 0 seconds. ``` -Started SLURM job 2339006 -Task 5 started (seed 2339006, random number 0) ... succeeded! -Task 1 started (seed 2339006, random number 1) ... failed! -Task 2 started (seed 2339006, random number 2) ... failed! -Task 3 started (seed 2339006, random number 3) ... failed! -Task 4 started (seed 2339006, random number 4) ... succeeded! -Completed SLURM job 2339006 in 00:00:05 -Started SLURM job 2339007 -Task 3 started (seed 2339007, random number 1) ... failed! -Task 1 started (seed 2339007, random number 2) ... failed! -Task 2 started (seed 2339007, random number 4) ... succeeded! -Completed SLURM job 2339007 in 00:00:05 -Started SLURM job 2339008 -Task 1 started (seed 2339008, random number 3) ... failed! -Task 3 started (seed 2339008, random number 4) ... succeeded! -Completed SLURM job 2339008 in 00:00:05 -Started SLURM job 2339009 -Task 1 started (seed 2339009, random number 4) ... succeeded! -Completed SLURM job 2339009 in 00:00:04 -Started SLURM job 2339010 -Completed SLURM job 2339010 in 00:00:00 +Started SLURM job 2346932 +Task 5 started (seed 2346932, random number 0) ... succeeded! +Task 3 started (seed 2346932, random number 1) ... failed! +Task 1 started (seed 2346932, random number 2) ... failed! +Task 4 started (seed 2346932, random number 3) ... failed! +Task 2 started (seed 2346932, random number 4) ... succeeded! +Completed SLURM job 2346932 in 00:00:05 +Started SLURM job 2346933 +Task 4 started (seed 2346933, random number 2) ... failed! +Task 1 started (seed 2346933, random number 3) ... failed! +Task 3 started (seed 2346933, random number 4) ... succeeded! +Completed SLURM job 2346933 in 00:00:05 +Started SLURM job 2346934 +Task 4 started (seed 2346934, random number 1) ... failed! +Task 1 started (seed 2346934, random number 4) ... succeeded! +Completed SLURM job 2346934 in 00:00:04 +Started SLURM job 2346935 +Task 4 started (seed 2346935, random number 0) ... succeeded! +Completed SLURM job 2346935 in 00:00:00 +Started SLURM job 2346936 +Completed SLURM job 2346936 in 00:00:01 +``` + +The `joblog` file shows the failing jobs with "Exitval" of 1: + +```console +$ cat joblog +Seq Host Starttime JobRuntime Send Receive Exitval Signal Command +5 cn332 1557788313.413 0.199 0 62 0 0 ./script_that_sometimes_fails.sh 5 +3 cn332 1557788313.187 1.162 0 59 1 0 ./script_that_sometimes_fails.sh 3 +1 cn332 1557788312.971 2.197 0 59 1 0 ./script_that_sometimes_fails.sh 1 +4 cn332 1557788313.296 3.175 0 59 1 0 ./script_that_sometimes_fails.sh 4 +2 cn332 1557788313.080 4.209 0 62 0 0 ./script_that_sometimes_fails.sh 2 +4 cn332 1557788318.093 2.180 0 59 1 0 ./script_that_sometimes_fails.sh 4 +1 cn332 1557788317.867 3.220 0 59 1 0 ./script_that_sometimes_fails.sh 1 +3 cn332 1557788317.976 4.207 0 62 0 0 ./script_that_sometimes_fails.sh 3 +4 cn332 1557788322.804 1.471 0 59 1 0 ./script_that_sometimes_fails.sh 4 +1 cn332 1557788322.695 4.200 0 62 0 0 ./script_that_sometimes_fails.sh 1 +4 cn332 1557788327.417 0.162 0 62 0 0 ./script_that_sometimes_fails.sh 4 ``` ### Example 3: Parameter Sweep