EGSnrc

Overview

The follwing description can also be found here:

NRC’s electron gamma shower (EGS) software tool can meet your specific requirements relating to modelling the passage of electrons and photons through matter. EGSnrc relies on Monte Carlo, which is the most accurate method to model the transport of radiation.

EGSnrc is used to address a broad range of questions about the propagation of radiation in materials. It is particularly well-suited for medical physics purposes, such as the research and development of devices that allow medical professionals to detect radiation, image a patient’s anatomy using x-rays, or deliver a prescribed radiation dose to a tumour while sparing healthy tissue. The software is also employed directly by medical physicists in cancer clinics for research and for verifying radiation treatment plans.

The software is freely available for research and other non-commercial purposes, see [1] for details.

Using EGSnrc on the cluster

Loading the module

In order to use EGSnrc you have to load the corresponding modules with

 module load gcc
 module load egsnrc

after which using

 module list

should show e.g.

 Currently Loaded Modulefiles:
 1) shared         2) gcc/4.8.1      3) sge/6.2u5p2    4) egsnrc/2.4.0

First time usage

EGSnrc requires a directory in user space to store the so-called user codes. If you are using EGSnrc for the first time on the cluster you need create a directory and to copy these user codes there.

This can be done conveniently by running the provided script

 finalize_egs_foruser

(after you have loaded the module, of course). The script will ask you for a directory name to store the user codes

 Input the directory where you want to keep your user codes 
      [/user/somegroup/abcd1234/egsnrc]:

It is strongly recommended that you use the provided default (which is $HOME/egsnrc) by just typing RETURN.

Then you will be asked which user codes should be installed where you can enter e.g. 1 for Compile all. After that you should a few messages about codes being successfully compiled. This may take a few minutes.

After the installation of the user codes you will see the following message:

 *******************************************************************************
 
 The configuration was completed succesfully for user abcd1234.
 
 Your default shell is /bin/bash
 Add the following 3 statements to your .bashrc file: 
 
 export EGS_HOME=/user/hrz/lees4820/egsnrc/
 export EGS_CONFIG=/cm/shared/apps/egsnrc/2.4.0/HEN_HOUSE/specs/x86_64-gcc.config
 . /cm/shared/apps/egsnrc/2.4.0/HEN_HOUSE/scripts/egsnrc_bashrc_additions
 
 ******************************************************************************

You do not need to modify your .bashrc-file as all the required settings are done by loading the module as described above.

Remember, this step only needs to done once.

Testing

You can quickly test if everything is installed correctly by the command

 tutor1 -p tutor_data

which should produce around 100 lines of output in one second.

Running jobs on the cluster

As with any application on the cluster you have to submit a job script to run EGSnrc (see below for an example). A detailed introduction to job submission can be found under SGE Job Management (Queueing) System.

A job submission script for one of the EGSnrc user codes could look like this:

#!/bin/bash

### the following lines are used by SGE to determine your job requirements
### you may want to modify the requested resources, e.g. h_vmem or h_rt,
### if you need more memory or run time (respectively)
#$ -S /bin/bash
#$ -cwd
#$ -l h_rt=1:00:0
#$ -l h_vmem=500M
#$ -l h_fsize=100M
#$ -N egsnrc
#$ -j n


### here the actual job script begins
### you may want to modify this part according to your needs
### the job script should placed in the directory of the user code
### you want to use
echo Job started on `date`   # put a time stamp in log file

# load module EGSnrc
module load gcc
module load egsnrc

# settings (here you need to make your own modifications)
# NOTE: the input file has to be in the directory $EGS_HOME/$USERCODE
#       the PEGS4DATA files can be located in different locations
#       one is the HEN_HOUSE which is automatically set
USERCODE=cavrznrc
INPUTFILE=cavrznrc_template
PEGS4DATA=521icru

# run code, e.g. the example from cavrznrc
$USERCODE -i $INPUTFILE -p $PEGS4DATA

echo Job finished on `date`   # put a time stamp in log file

exit

Copy the job script to a file named e.g. run_egsnrc.sge which should be located in the directory of the user code you want to run. In the example, the user code is cavrznrc, the script is placed in ~/egsnrc/cavrznrc. You will need to modify the job script to your needs. In the first part of the script, resources for the job are requested (e.g. with the line #$ -l h_rt=1:00:0 the maximum run time of the job is set to one hour). Below, the actual job is described. Here you may want to modify, which user code you want to use, which input file and so on. Note, that EGSnrc has a peculiar way of handling files, which means for example that the input file can only be placed in the directory of the user code.

Once the job script is prepared, you can submit it to the SGE by the command

 qsub run_egsnrc

which will respond with

 Your job 32083 ("egsnrc") has been submitted

After that you can check the status of your jobs using the

 qstat

command.

You can submit multiple jobs to the cluster by having multiple sge-scripts or by modifying the sge-script between each qsub-command. Note however, that while you can modify the sge-script, you should not modify the input file when submitting more than one job. Instead you need to make copies for each submitted job.

When you job is completed (i.e. when the job-ID is no longer listed by qstat), you can find the output files from the EGSnrc user code in the usual place. In addition, you will find two more output files, e.g.

 egsnrc.e32083
 egsnrc.o32083

which contain the output normally printed on the screen.

Job Arrays

If you need to run the same code many times with different input files you should consider using a job array. Suppose you have a numbered sequence of input files:

 cavrznrc_001.egsinp  
 cavrznrc_002.egsinp
       ....
 cavrznrc_010.egsinp

Then you can use a job array to run the same job script on each of the input files. The following lines need to be added to the job script from above:

 #$ -t 1-10
 #$ -tc 2

The first one tells SGE that this job is an array, and that tasks number from 1 to 10 should be generated (it is also possible to define different ranges and increments, e.g. #$ -t 1000-1500:2 will generate tasks for every even number between (and including) 1000 and 1500). The second line limits the number of tasks that can run at the same time to 2 (this line is optional, and 2 is just an example).

In addition, you also need to modify the line where the input file is defined:

 INPUTFILE=cavrznrc_$(printf "%3.3i" $SGE_TASK_ID)

The number of the current task is stored in the variable $SGE_TASK_ID and can be used to create a unique file name for the input file (the exact details for this depend on your file names, of course).

Once the modified job script is ready, you can submit as before with

 qsub run_egsnrc_array.sge

but now your script will run multiple times with as many output.

Current Issues

Currently for some unknown reason only some of the user codes are working. Some only return with segmentation fault, these are:

 cavity
 egs_cbct
 egs_chamber
 egs_fac
 smooth
 tutor2pp
 tutor7pp

Hopefully, this problem can be solved in the future.