Difference between revisions of "How to Use Job Dependencies"
Line 15: | Line 15: | ||
Submitted batch job 19520729 | Submitted batch job 19520729 | ||
</pre> | </pre> | ||
Here, the second job depends on the first on, which is identified by its job id. The <tt>--depend</tt>- or <tt>-d</tt>-option can be followed by one or more dependencies in comma-separated list. Each dependency has the form <tt><type>:<jobid>[:jobid...]</tt> | Here, the second job depends on the first on, which is identified by its job id. The <tt>--depend</tt>- or <tt>-d</tt>-option can be followed by one or more dependencies in comma-separated list. Each dependency has the form <tt><type>:<jobid>[:jobid...]</tt>. After the two jobs are submitted, you can use <tt>squeue</tt> to see the job status: | ||
<pre> | |||
[ | [abcd1234@carl]$ squeue -u $USER | ||
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON) | JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON) | ||
19520729 carl.p print.sh lees4820 PD 0:00 1 (Dependency) | 19520729 carl.p print.sh lees4820 PD 0:00 1 (Dependency) | ||
19520727 carl.p print.sh lees4820 R 1:18 1 mpcs030 | |||
[ | [abcd1234@carl]$ squeue -u $USER | ||
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON) | JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON) | ||
19520729 | 19520729 carl.p print.sh lees4820 R 0:14 1 mpcl109 | ||
</pre> | |||
Initially, the first job is running while the second one is pending with the reason <tt>(Dependency)</tt>. Once the first job is completed, the second one starts and after it has also completed we can check the final result: | |||
<pre> | |||
[abcd1234@carl]$ cat terminal.txt | |||
[ | |||
Hello | Hello | ||
World | World | ||
</pre> | |||
== Examples == | == Examples == |
Revision as of 16:53, 1 October 2019
The Basics
SLURM offers the possibility to submit jobs depending on already submitted jobs. This functionality can be useful in some situations as illustrated in the examples below. Before, let look at the basics which are also explained in the man-pages of sbatch. Suppose, we have the script print.sh which reads
#!/bin/bash echo "$1" >> terminal.txt sleep 120
and we would like to run it in two jobs on the cluster, but the second job should wait for the first one to terminate. This can be achieved using a job dependency as in the following example:
[abcd1234@carl]$ sbatch -p carl.p print.sh "Hello" Submitted batch job 19520727 [abcd1234@carl]$ sbatch -p carl.p --depend afterok:19520727 print.sh "World" Submitted batch job 19520729
Here, the second job depends on the first on, which is identified by its job id. The --depend- or -d-option can be followed by one or more dependencies in comma-separated list. Each dependency has the form <type>:<jobid>[:jobid...]. After the two jobs are submitted, you can use squeue to see the job status:
[abcd1234@carl]$ squeue -u $USER JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON) 19520729 carl.p print.sh lees4820 PD 0:00 1 (Dependency) 19520727 carl.p print.sh lees4820 R 1:18 1 mpcs030 [abcd1234@carl]$ squeue -u $USER JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON) 19520729 carl.p print.sh lees4820 R 0:14 1 mpcl109
Initially, the first job is running while the second one is pending with the reason (Dependency). Once the first job is completed, the second one starts and after it has also completed we can check the final result:
[abcd1234@carl]$ cat terminal.txt Hello World