Difference between revisions of "How to Manage Many Jobs"

From HPC users
Jump to navigationJump to search
(Created page with "== Introduction == Often, you may need to run many nearly identical jobs. There are different approaches to achieve this goal with minimal effort and some of the approaches ...")
 
Line 3: Line 3:
Often, you may need to run many nearly identical jobs. There are different approaches to achieve this goal with minimal effort and some of the approaches will be described below. Which approach best suits your needs depends on the nature of your problem but some hints are given for making the choice.
Often, you may need to run many nearly identical jobs. There are different approaches to achieve this goal with minimal effort and some of the approaches will be described below. Which approach best suits your needs depends on the nature of your problem but some hints are given for making the choice.


In the examples below, a simple program to decide whether a number is a prime will be used. Basically, you can run the program, e.g. with
In the examples below, a simple program to decide whether a number is a prime will be used. The example program can found [media:ManyTasks.tgz here] and in order to use it, you can download it and then
$ tar -zxvf ManyTasks.tgz            # to unpack
$ cd ManyTasks                        # go to directory
$ make                                # build executable
After that, you can run the program, e.g. with
  $ ./isprime 73
  $ ./isprime 73
  yes
  yes
to see if, in this case, 73 is a prime (yes it is).  
to see if a number, in this case 73, is a prime (yes it is). The idea of the example is to run <tt>isprime</tt> on every number in <tt>parameter.dat</tt>. Therefore, if we want to run this as a job we can think of it as having many jobs that are identical except for one parameter. A single job can also be called a task in this context. 


=== Packaging many Tasks in a Single Job ===
=== Packaging many Tasks in a Single Job ===

Revision as of 14:46, 16 August 2019

Introduction

Often, you may need to run many nearly identical jobs. There are different approaches to achieve this goal with minimal effort and some of the approaches will be described below. Which approach best suits your needs depends on the nature of your problem but some hints are given for making the choice.

In the examples below, a simple program to decide whether a number is a prime will be used. The example program can found [media:ManyTasks.tgz here] and in order to use it, you can download it and then

$ tar -zxvf ManyTasks.tgz             # to unpack
$ cd ManyTasks                        # go to directory
$ make                                # build executable

After that, you can run the program, e.g. with

$ ./isprime 73
yes

to see if a number, in this case 73, is a prime (yes it is). The idea of the example is to run isprime on every number in parameter.dat. Therefore, if we want to run this as a job we can think of it as having many jobs that are identical except for one parameter. A single job can also be called a task in this context.

Packaging many Tasks in a Single Job