R 2016

Introduction

R is a programming language and environment commonly used in statistical computing, data analytics and scientific research.

Installed version

The currently installed versions of R are

on environment hpc-env/8.3:

R/4.0.2-foss-2019b

on environment hpc-env/6.4:

R/3.4.4-intel-2018a
R/3.5.2-intel-2018a
R/3.6.1-intel-2018a

on environment hpc-uniol-env:

R/3.3.1

Additional installed packages

The R release contains a lot of additional packages. After loading and starting R ("module load R" and simply "R" on the command line), you can generate a list of all of them by using the following commands

ip <- as.data.frame(installed.packages()[,c(1,3:4)])
rownames(ip) <- NULL
ip <- ip[is.na(ip$Priority),1:2,drop=FALSE]
print(ip, row.names=FALSE)

You will receive a list of every package and its related version. It should look like this:

       Package     Version
           abc         2.1
      abc.data         1.0
         abind       1.4-3
       acepack     1.3-3.3
        adabag         4.1

Installing your own packages

If your are missing an R-packages you can contact Scientific Computing or, alternatively install the package in your own HOME directory. To do so you should first create a directory on the cluster, e.g. with

$ mkdir -p $HOME/R/lib

This would create a directory 'R' with the subdirectory 'lib' in your HOME folder. Now, we need to create two files and in some lines. To make this as easy as possible, you can just copy the following lines into your console:

echo -en "R_LIBS=\"~/R/lib\"\n" >> $HOME/.Renviron
echo -en "# adding fixed CRAN mirror for downloading packages\n\
cat(\".Rprofile: Setting CRAN repositoryn to ftp.gwdg.de\\\n\")\nr={}\n\
r[\"CRAN\"] = \"https://ftp.gwdg.de/pub/misc/cran\"\noptions(repos = r)" >> $HOME/.Rprofile

Afterwards, the files .Renvrion and .Rprofile should look include the following lines (if the files did not exits before, they will only contain these lines):

$ cat .Renviron
R_LIBS="~/R/lib"

$ cat .Rprofile
# adding fixed CRAN mirror for downloading packages
cat(".Rprofile: Setting CRAN repositoryn to ftp.gwdg.de\n")
r={}
r["CRAN"] = "https://ftp.gwdg.de/pub/misc/cran"
options(repos = r)

You can choose a different location for installing your R-libraries if you wish (by setting R_LIBS in .Renviron differently). There are also alternative mirrors, set your preferred on in .Rprofile.

Once this is done, you start R on the login node and begin installing packages:

R
> install.packages("lme4")
> library(lme4)
> library(car)

In this example, the packages lme4 and car will be downloaded from the CRAN mirror and installed in the directory given by R_LIBS, i.e. in your $HOME folder. Please note that R will not check if a package is already installed and will always reinstall a package by overwriting the previous install (you can program a logic for that into your R programs). You may want to separate package installation from the execution of jobs.

The package 'lme4' is already installed in the global R-installation (version 1.1-12) whereas the installation above will install a newer version (1.1-13 or newer). When you load the package with library(lme4) the installation in your HOME folder will be used. You can verify this with the R-command

> sessionInfo()
  ... lme4_1.1-13 ...

The next call library(car) will load the globally installed package 'car'. So you do not need to install every package in your HOME, only those packages that are missing or when you require an updated version.

Using R on the HPC cluster

If you want to use R on the HPC cluster, you will have to load its module. You can do that by using the command

module load R

Since R is installed on multiple environments and in different versions, possibly you will have to change environment and specify the version

module load hpc-env/6.4
module load R/3.5.2-intel-2018a

Basic Job Script for R

Suppose you want to create 100 random numbers and calculate their mean and standard deviation. In R the commands for that would be:

x <- runif(100,0.0,1.0)
mean(x)
sd(x)

If you want to do the calculation on the cluster store the above commands in a file named e.g. 'Rtest.R'. Then create a job script 'Rtest.job' with the content:

#!/bin/bash

#SBATCH --job-name=Rtest
#SBATCH --partition=carl.p
#SBATCH --time=24:00:0
#SBATCH --mem=5000M
   
# load modules
module load R

# run R 
R -f ./Rtest.R

and submit a job with the command:

sbatch Rtest.job

The output of R will appear in a file called 'slurm-<jobid>.out' once the job has been completed.

Usage of R and MPI

For parallelization the package doMPI is installed. To launch an parallel R script inside a SLURM script please use command line

  mpirun R --slave -f SCRIPTNAME SCRIPT_CMDLINE_OPTIONS

to enable SLURM to control all processes of your script. Please do not use the batch starting sequence R CMD BATCH!

The corresponding parallel environment in the SLURM submission script is specified by

 #SBATCH --ntasks=NUMBER_OF_TASKS

Note for doMPI:

• Before you start R with the mpirun command you have to unset the environment variable R_PROFILE in your SLURM-Script. Otherwise the MPI processes were not spawned. Please add following line to your jobscript:

  unset R_PROFILE

• Please use mpi.quit() at the end of your script. Otherwise it will not end.
• Here a small example R script for doMPI (it writes the current rank of MPI in b):

#!/usr/bin/env Rscript
#
# file name: test_dompi.R
#

library("doMPI")

# doMPI start
cl <- startMPIcluster()
registerDoMPI(cl)

# parallel foreach due to %dopar% using the MPI cluster
# note that one MPI process is the master (rank 0)and 
# distributes the work (iterations) among the other 
# processes (ranks 1 to (mpi.comm.size(0)-1))
# rnorm returns a vector with three elements, the
# option .combine="rbind" makes a table with 10 rows 
b<-foreach(i=1:10, .combine="rbind") %dopar% {
   my_rank<-as.integer(mpi.comm.rank(0))
   rnorm(3, my_rank, 0.01)  # return three random values near my_rank
}
closeCluster(cl)
print(b)

mpi.quit()

and he corresponding SLURM-script

#!/bin/bash

#SBATCH --job-name=test_dompi
#SBATCH --time=24:00:0
#SBATCH --mem-per-cpu=2G
#SBATCH --output=dompi-test.%j.out        
#SBATCH --error=dompi-test.%j.err 
#SBATCH --ntasks=4
   
# load modules
module load hpc-env/8.3
module load R

# unset the environment variable which is needed for Rmpi
# but makes problems with doMPI
unset R_PROFILE
  
# run R in parallel (mpirun knows the number of tasks requested)
mpirun R --slave -f ./test_dompi.R

Usage of NetCDF and R

A package for NetCDF has been installed together with R. In order to use it, please add the command

module load netCDF

to your job script before starting R. Your R-script should include a line

library(ncdf)

to load the NetCDF library. Please refer to the documentations of NetCDF and R for more informations.

Documentation

You can look up anything about R on their

• documentation pages
• manual pages