R 2016
Introduction
R is a programming language and environment commonly used in statistical computing, data analytics and scientific research.
Installed version
The currently installed versions of R are
on environment hpc-env/8.3:
R/4.0.2-foss-2019b
on environment hpc-env/6.4:
R/3.4.4-intel-2018a R/3.5.2-intel-2018a R/3.6.1-intel-2018a
on environment hpc-uniol-env:
R/3.3.1
Additional installed packages
The R release contains a lot of additional packages. After loading and starting R ("module load R" and simply "R" on the command line), you can generate a list of all of them by using the following commands
ip <- as.data.frame(installed.packages()[,c(1,3:4)]) rownames(ip) <- NULL ip <- ip[is.na(ip$Priority),1:2,drop=FALSE] print(ip, row.names=FALSE)
You will receive a list of every package and its related version. It should look like this:
Package Version
abc 2.1
abc.data 1.0
abind 1.4-3
acepack 1.3-3.3
adabag 4.1
Installing your own packages
If your are missing an R-packages you can contact Scientific Computing or, alternatively install the package in your own HOME directory. To do so you should first create a directory on the cluster, e.g. with
$ mkdir -p $HOME/R/lib
This would create a directory 'R' with the subdirectory 'lib' in your HOME folder. Now, we need to create two files and in some lines. To make this as easy as possible, you can just copy the following lines into your console:
$ echo -en "R_LIBS=\"~/R/lib\"" >> $HOME/.Renviron
$ echo -en "# adding fixed CRAN mirror for downloading packages\n\
cat(\".Rprofile: Setting CRAN repositoryn to ftp.gwdg.de\\\n\")\nr={}\n\
r[\"CRAN\"] = \"https://ftp.gwdg.de/pub/misc/cran\"\noptions(repos = r)" >> $HOME/.Rprofile
Afterwards, the two new files .Rprofile should look like this:
$ cat .Renviron
R_LIBS="~/R/lib"
$ cat .Rprofile
# adding fixed CRAN mirror for downloading packages
cat(".Rprofile: Setting CRAN repositoryn to ftp.gwdg.de\n")
r={}
r["CRAN"] = "https://ftp.gwdg.de/pub/misc/cran"
options(repos = r)
# adding fixed CRAN mirror for downloading packages
cat(".Rprofile: Setting CRAN repositoryn to ftp.gwdg.de\n")
r={}
r["CRAN"] = "https://ftp.gwdg.de/pub/misc/cran"
options(repos = r)
You can choose a different location for installing your R-libraries if you wish (by setting R_LIBS in .Renviron differently). There also alternative mirrors, set your preferred on in .Rprofile.
Once this is done, you start R on the login node and begin installing packages:
R
> install.packages("lme4")
> library(lme4)
> library(car)
In this example the package 'lme4' will be downloaded from the CRAN mirror and installed in the directory given by R_LIBS, i.e. in your HOME folder. Please note that R will not check if a package is already installed and will alway reinstall a package by overwriting the previous install. You may want to separate package installation from the execution of jobs.
The package 'lme4' is already installed in the global R-installation (version 1.1-12) whereas the installation above will install a newer version (1.1-13 or newer). When you load the package with library(lme4) the installation in your HOME folder will be used. You can verify this with the R-command
> sessionInfo() ... lme4_1.1-13 ...
The next call library(car) will load the globally installed package 'car'. So you do not need to install every package in your HOME, only those packages that are missing or when you require an updated version.
Using R on the HPC cluster
If you want to use R on the HPC cluster, you will have to load its module. You can do that by using the command
module load R
Since R is installed on multiple environments and in different versions, possibly you will have to change environment and specify the version
module load hpc-env/6.4 module load R/3.5.2-intel-2018a
Basic Job Script for R
Suppose you want to create 100 random numbers and calculate their mean and standard deviation. In R the commands for that would be:
x <- runif(100,0.0,1.0) mean(x) sd(x)
If you want to do the calculation on the cluster store the above commands in a file named e.g. 'Rtest.R'. Then create a job script 'Rtest.job' with the content:
#!/bin/bash #SBATCH --job-name=Rtest #SBATCH --partition=carl.p #SBATCH --time=24:00:0 #SBATCH --mem=5000M # load modules module load R # run R R -f ./Rtest.R
and submit a job with the command:
sbatch Rtest.job
The output of R will appear in a file called 'slurm-<jobid>.out' once the job has been completed.
Usage of R and MPI
For parallelization the package doMPI is installed. To launch an parallel R script inside a SLURM script please use command line
mpirun R --slave -f SCRIPTNAME SCRIPT_CMDLINE_OPTIONS
to enable SLURM to control all processes of your script. Please do not use the batch starting sequence R CMD BATCH!
The corresponding parallel environment in the SLURM submission script is specified by
#SBATCH --ntasks=NUMBER_OF_TASKS
Note for doMPI:
- Before you start R with the mpirun command you have to unset the environment variable R_PROFILE in your SLURM-Script. Otherwise the MPI processes were not spawned. Please add following line to your jobscript:
unset R_PROFILE
- Please use mpi.quit() at the end of your script. Otherwise it will not end.
- Here a small example R script for doMPI (it writes the current rank of MPI in b):
#!/usr/bin/env Rscript
#
# file name: test_dompi.R
#
library("doMPI")
# doMPI start
cl <- startMPIcluster()
registerDoMPI(cl)
# parallel foreach due to %dopar% using the MPI cluster
# note that one MPI process is the master (rank 0)and
# distributes the work (iterations) among the other
# processes (ranks 1 to (mpi.comm.size(0)-1))
# rnorm returns a vector with three elements, the
# option .combine="rbind" makes a table with 10 rows
b<-foreach(i=1:10, .combine="rbind") %dopar% {
my_rank<-as.integer(mpi.comm.rank(0))
rnorm(3, my_rank, 0.01) # return three random values near my_rank
}
closeCluster(cl)
print(b)
mpi.quit()
and he corresponding SLURM-script
#!/bin/bash #SBATCH --job-name=test_dompi #SBATCH --time=24:00:0 #SBATCH --mem-per-cpu=2G #SBATCH --output=dompi-test.%j.out #SBATCH --error=dompi-test.%j.err #SBATCH --ntasks=4 # load modules module load hpc-env/8.3 module load R # unset the environment variable which is needed for Rmpi # but makes problems with doMPI unset R_PROFILE # run R in parallel (mpirun knows the number of tasks requested) mpirun R --slave -f ./test_dompi.R
Usage of NetCDF and R
A package for NetCDF has been installed together with R. In order to use it, please add the command
module load netCDF
to your job script before starting R. Your R-script should include a line
library(ncdf)
to load the NetCDF library. Please refer to the documentations of NetCDF and R for more informations.
Documentation
You can look up anything about R on their
