EGSnrc 2016

Introduction

The follwing description can also be found here:

NRC’s electron gamma shower (EGS) software tool can meet your specific requirements relating to modelling the passage of electrons and photons through matter. EGSnrc relies on Monte Carlo, which is the most accurate method to model the transport of radiation.

EGSnrc is used to address a broad range of questions about the propagation of radiation in materials. It is particularly well-suited for medical physics purposes, such as the research and development of devices that allow medical professionals to detect radiation, image a patient’s anatomy using x-rays, or deliver a prescribed radiation dose to a tumour while sparing healthy tissue. The software is also employed directly by medical physicists in cancer clinics for research and for verifying radiation treatment plans.

The software is freely available for research and other non-commercial purposes, see [1] for details. The latest version is now available on GitHub[2] and uses a new license model (free GNU license). Please read the documentation and license information on GitHub. By using EGSnrc on the cluster you are accepting the terms of the license. Also cite EGSnrc in any work published, the relevant citation can be found on GitHub as well.

The modules also includes the package BEAMnrc, see [3] for more information. Since version 2016 BEAMnrc is included in EGSnrc, no extra package need installation.

Installed version

The currently installed versions of EGSnrc are

• 2018
• 2019a (D)

on environment hpc-env/6.4 and

• 2017-wb.

on environment hpc-uniol-env

The latter is a release of EGSnrc from [4] that includes the application egs_brachy and may include additional changes in EGSnrc compared to the default. EGSnrc 2019a, however, is the most recent version and is installed on the environmenthpc-uniol-env as well as on hpc-env/6.4.

Using EGSnrc

If you want to find out more about EGSnrc on the HPC cluster, you can use the command

module spider egsnrc

This will show you basic informations e.g. a short description and the currently installed version.

To load the module in version 2018, you need to use the command

module load hpc-env/6.4         #changing the environment
module load EGSnrc/2018         #loading the module

Always remember: this command is case sensitive!

After loading the module, you can check if its succesfully loaded with the command

module list

First time usage

EGSnrc requires a directory in user space to store the so-called user codes. If you are using any version of EGSnrc for the first time on the cluster, you need to create a directory where these user codes can be copied to. Before you can start the script for doing this, you will need to load the module for EGSnrc. Do this by using the following command

module load EGSnrc

The rest of the process can be started by running the provided scripts for EGSnrc. Simply type

 finalize_egs_foruser

The script will ask you for a directory name to store the user codes:

 Input the directory where you want to keep your user codes 
      [/user/abcd1234/egsnrc-2017]: 

It is strongly recommended that you use the following directories which are also the suggested defaults: for EGSnrc 2016 or later the default is $HOME/egsnrc-VERSION where version is replaced by the version number (2017 in our case) The default is accepted by simply typing Return. The default directory is also set by loading the module as can be seen with

 echo $EGS_HOME

after loading the module for the desired version. If you are not using the recommended setting here you will need to change $EGS_HOME after loading the module.

Then you will be asked which user codes should be installed where you can enter e.g. 1 for Compile all. After that you should a few messages about codes being successfully compiled. This may take a few minutes.

After the installation of the user codes you will see the following message:

#######################################################################################

The EGSnrc configuration for abcd1234 is ALMOST complete ...

EGSnrc requires various environment variables and aliases to be defined.
Typically you add these definitions to your shell resource file located in
your home directory, to ensure the EGSnrc environment is properly set up
every time you log in.

Your default shell is /bin/bash
Add the following statements to your .bashrc file: 

export EGS_HOME=/user/abcd1234/egsnrc-2017/
export EGS_CONFIG=/cm/shared/uniol/software/EGSnrc/2017/HEN_HOUSE/specs/linux.conf
source /cm/shared/uniol/software/EGSnrc/2017/HEN_HOUSE/scripts/egsnrc_bashrc_additions

#######################################################################################

You do not need to modify your .bashrc-file as all the required settings are done by loading the module as described above.

Testing the setup

You can quickly test if everything is setup correctly by using the command

tutor1 -p tutor_data

which should produce about 100 lines of output in ca. one second.

Using EGSnrc on the HPC cluster

Since there many people working with the HPC cluster, its important that everyone has an equal chance to do so. Therefore, every job should be processed by SLURM.

For this reason, you have to create a jobscript for your tasks.

Example:

#!/bin/bash
               
#SBATCH --ntasks=1                  
#SBATCH --mem=2G 
#SBATCH --partition=carl.p                 
#SBATCH --time=0-2:00  
#SBATCH --job-name EGSNRC-TEST              
#SBATCH --output=EGSnrc-test.%j.out        
#SBATCH --error=EGSnrc-test.%j.err

#load EGSnrc
module load EGSnrc

#settings (here you need to make your own modifitcations)
# NOTE: the input file has to be in the directory $EGS_HOME/$USERCODE
#       the PEGS4DATA files can be located in different locations
#       one is the HEN_HOUSE which is automatically set
USERCODE=cavrznrc
INPUTFILE=cavrznrc_template
PEGS4DATA=521icru

# run code, e.g. the example from cavrznrc
$USERCODE -i $INPUTFILE -p $PEGS4DATA

If you jobscript is ready, save it under an name of your choice. We will be using "egsnrc-test.job" for this exampl.e

sbatch -p carl.p egsnrc-test.job (if you named you jobscript differently, add the right jobname here)

This testjob will almost instantly finish (about 1-2 seconds). You will find 2 files in directory you ran the job from:

• EGSnrc-test.JOBID.err
• EGSnrc-test.JOBID.out

Possible erros will be listed in the *.err-file. The output can be found in the *.out-file.

Job Arrays

If you need to run the same code many times with different input files you should consider using a job array. Suppose you have an input file named setup.egsinp containing the following (or similar) lines:

 initial random no. seeds = 1, 6
 ncase = 10000
 nbatch  = 10

If you now make a copy of this file naming it setup_template.egsinp and modify the lines to

 initial random no. seeds = 1, %seed%
 ncase = 1000
 nbatch  = 1

Note how ncase and nbatch have been divided by 10, wheres %seed% is introduced as a placeholder.

Now you could write a short bash-script, e.g.

#!/bin/bash

NPAR=10
FIRSTSEED=6
TEMPLATE="setup_template.egsinp"

for i in `seq $NPAR`
do
   seed=`expr $i + $FIRSTSEED - 1`
   INPFILE=`echo $TEMPLATE | sed s/_template/_w$i/`
   cat $TEMPLATE | sed s/%seed%/$seed/ > $INPFILE
done

saving it as generate_egsinp and giving it execute permissions with chmod a+x generate_egsinp. Now you can run this script

 generate_egsinp

which will generate a numbered sequence of 10 input files:

 setup_w1.egsinp  
 setup_w2.egsinp
       ....
 setup_w10.egsinp

The input files are ready. We can now start with setting up the jobscript to handle the array. For this example we need to add the following lines to our jobscript:

#SBATCH --array=1-10%2

This means:

• job should be handled as an arry (--array)
• tasks from 1 to 10 should be generated (1-10)
• 2 tasks can run at the same time (%2)

Its not needed to limit the amount of tasks that can run at the same time to 2, this is just an example. Further, we need to modify the line where the input file is defined:

 INPUTFILE=setup_w$(printf "%i" $SLURM_ARRAY_TASK_ID)

The task ID is stored in die variable $SLURM_ARRAY_TASK_ID. We use this variable to modify the inputfilenames.

We should now be ready to submit our job to slurm:

sbatch egsnrc-test-array.job

Documentation

The documentation of EGSnrc can be found on here. (PDF viewer required!)