Difference between revisions of "HPC User Wiki 2016"

From HPC users
Jump to navigationJump to search
(Ghostscript)
 
(114 intermediate revisions by 3 users not shown)
Line 3: Line 3:
<!-- Table of Contents von der Startseite entfernen-->
<!-- Table of Contents von der Startseite entfernen-->
__NOTOC__
__NOTOC__
This is the HPC User Wiki for the HPC clusters CARL and EDDY. If you would like to contribute please contact {{sc}}!


{{warningbox|
{{warningbox|
This is the HPC User Wiki for the HPC clusters CARL and EDDY. If you would like to contribute please contact {{sc}}!
The current wiki for the HPC cluster ROSA and STORM/MOUSE can be found [https://hpcwiki.uol.de/ here]. Unfortunately, the new wiki is far from complete so  you may need to adapt some information from this wiki still. We are working on it :)
}}
}}


Line 11: Line 13:
<center>
<center>
<br/>
<br/>
  ---------------------- [[FAQ|FAQ - Frequently Asked Questions]] ----------------------
  ---------------- [[FAQ|FAQ - Frequently Asked Questions]] ----------------
 
  ------------- Want to stay in contact with the HPC support and other users? Join our [https://mattermost.uni-oldenburg.de/signup_user_complete/?id=byc9quwbjjgc7cy57rwp9aphua Mattermost group!] -------------


  ---------------------- Want to stay in contact with the HPC support and other users? Join our [https://mattermost.uni-oldenburg.de/signup_user_complete/?id=byc9quwbjjgc7cy57rwp9aphua Mattermost group!] ----------------------
------------------ Tired of reading long wiki articles? ------------------  
------------- Check out our new [[HPC_Video_Lectures_2016|HPC Video Lectures]]! -------------


{| style="background-color:#eeeeff;" cellpadding="10" border="1" cellspacing="0"  
{| style="background-color:#eeeeff;" cellpadding="10" border="1" cellspacing="0"  
Line 27: Line 32:
* [[HPC Facilities of the University of Oldenburg 2016#EDDY| EDDY]]
* [[HPC Facilities of the University of Oldenburg 2016#EDDY| EDDY]]
* [[HPC Policies 2016| HPC Policies]]
* [[HPC Policies 2016| HPC Policies]]
** [[Cost Sharing Scientific Computing]]
* [[Unix groups| Groups ]]
* [[Unix groups| Groups ]]
** [[Group Tools|Managing your Group]]
** [[Group Tools|Managing your Group]]
* [[Acknowledging_the_HPC_facilities 2016| Acknowledging CARL/EDDY]]
* [[Acknowledging_the_HPC_facilities 2016| Acknowledging CARL/EDDY]]
* [[User Meetings]]
* [[User Meetings | User Meetings / User Representatives]]
 
 
''Other HPC resources:''
* [[HPC Facilities of the University of Oldenburg 2016#GOLD| GOLD]]
* [[ HLRN 2016 | HLRN ]]
|  
|  
* [[Quickstart Guide | Quickstart Guide]]
* [[Quickstart Guide | Quickstart Guide]]
Line 42: Line 53:
|
|
* [[Login | Login to the HPC cluster]]
* [[Login | Login to the HPC cluster]]
** [[Login#HPC_Account | Getting an HPC account]]
** [[Login#Connect to CARL/EDDY using the Remote Desktop Protocol | Remote Desktop Connection]]
** [[Login#Connect to CARL/EDDY using the Remote Desktop Protocol | Remote Desktop Connection]]
** [[Login#Using SSH Keys for login | Using SSH Keys]]
** [[Login#VPN:_Internet_.2F_From_Home | VPN: Connect from Home]]
* [[User environment - The usage of module 2016| Usage of module]]
* [[User environment - The usage of module 2016| Usage of module]]
** [[Software Environments and their Differences]]
** [[Software Environments and their Differences]]
Line 51: Line 65:
** [[Local Mounting of File Systems#Shares under Windows | Shares under Windows]]
** [[Local Mounting of File Systems#Shares under Windows | Shares under Windows]]
** [[Local Mounting of File Systems#Shares under Linux | Shares under Linux]]
** [[Local Mounting of File Systems#Shares under Linux | Shares under Linux]]
** [[Local Mounting of File Systems#Shares under Linux | Mounting a Directory on Nextcloud]]
** [[Local Mounting of File Systems#Mounting a Directory on Nextcloud | Mounting a Directory on Nextcloud]]
* [[License servers]]
* [[License servers]]
|
|
Line 63: Line 77:
* [[Interactive Jobs]]
* [[Interactive Jobs]]
* [[GPU Usage]]
* [[GPU Usage]]
* [[Best Practice]]  
* [[Best Practice]]
** [[How to Manage Many Jobs]]
|-
|-
|}
|}
Line 77: Line 92:
! scope="col" width="200px" | Quantum Chemistry
! scope="col" width="200px" | Quantum Chemistry
! scope="col" width="400px" colspan="2" | Biology & Medicine
! scope="col" width="400px" colspan="2" | Biology & Medicine
! scope="col" width="200px" | Mathematics/Scripting
! scope="col" width="200px" | Mathematics, Statistics & Scripting
|- valign="top"
|- valign="top"
|
|
* [[ALE 2016 | ALE ]]
* [[ALE 2016 | ALE ]]
* [[CMake 2016 | CMake]]
* [[GNU Compiler 2016 | GNU Compiler]]
* [[GNU Compiler 2016 | GNU Compiler]]
* [[Go 2016 | Go]]
* [[Go 2016 | Go]]
* [[Intel Compiler 2016 | Intel Compiler]]
* [[Intel Compiler 2016 | Intel Compiler]]
* [[Intel Tools]]
** [[Threading Building Blocks | Threading Building Blocks (TBB)]]
** [[Inspector]]
** [[VTune]]
** [[Advisor]]
* [[Lua 2016 | Lua]]
* [[MPI Libraries | MPI Libraries ]]
* [[MPI Libraries | MPI Libraries ]]
** [[MPI Libraries#Compiling with OpenMPI | Compiling with OpenMPI]]
** [[MPI Libraries#Compiling with OpenMPI | Compiling with OpenMPI]]
Line 89: Line 111:
* [[PGI Compiler 2016 | PGI Compiler]]
* [[PGI Compiler 2016 | PGI Compiler]]
** [[OpenACC]]
** [[OpenACC]]
*[[Parallel 2016 | parallel]]
* [[Qt 2016 | Qt]]
* [[Qt 2016 | Qt]]
* [[ROS 2016 | ROS]]
* [[ROS 2016 | ROS]]
Line 95: Line 118:
* [[Toolchains | Toolchains]]
* [[Toolchains | Toolchains]]
|  
|  
* [[AIMAll 2016 | AIMAll]]
* [[CP2K 2016 | CP2K]]
* [[CRYSTAL]]
* [[DIAMOND 2016 | DIAMOND]]
* [[Gaussian 2016 | Gaussian]]
* [[Gaussian 2016 | Gaussian]]
** [[GaussView]]
** [[GaussView]]
* [[GROMACS]]
* [[MOLGW 2016 | MOLGW]]
* [[NAMD]]
* [[NBO 2016 | NBO]]
* [[ORCA 2016 | ORCA]]
* [[ORCA 2016 | ORCA]]
* [[CP2K 2016 | CP2K]]
* [[NBO 2016 | NBO]]
* [[pDynamo]]
* [[pDynamo]]
* [[GROMACS]]
* [[NAMD]]
* [[PLUMED]]
* [[PLUMED]]
* [[CRYSTAL]]
* [[QuantumESPRESSO 2016 | QuantumESPRESSO]]
|  
|
*[[angsd 2016 | angsd]]
*[[ANTs 2016 | ANTs]]
*[[ANTs 2016 | ANTs]]
*[[AUGUSTUS 2016 | AUGUSTUS]]
*[[AUGUSTUS 2016 | AUGUSTUS]]
*[[bam_to_mate_hist 2016 | bam_to_mate_hist]]
*[[bam_to_mate_hist 2016 | bam_to_mate_hist]]
*[[bwa-mem2 2016 | bwa-mem2]]
*[[BCFtools]]
*[[BCFtools]]
*[[Beast]]
*[[Beast]]
Line 116: Line 145:
*[[BLAT]]
*[[BLAT]]
*[[BamTools]]
*[[BamTools]]
*[[BBMap 2016 | BBMap]]
*[[BioPerl]]
*[[BioPerl]]
*[[Biopython]]
*[[Biopython]]
Line 122: Line 152:
*[[Burrows-Wheeler Aligner]]
*[[Burrows-Wheeler Aligner]]
*[[canu 2016 | canu]]
*[[canu 2016 | canu]]
*[[cdbfasta 2016 | cdbfasta]]
*[[CD-HIT]]
*[[CD-HIT]]
*[[CellRanger 2016 | CellRanger]]
*[[cutadapt]]
*[[cutadapt]]
*[[discovardenovo 2016 | discovardenovo]]
*[[Exonerate 2016 | Exonerate]]
*[[Exonerate 2016 | Exonerate]]
*[[FastANI 2016 | FastANI]]
*[[FASTX-Toolkit]]
*[[FASTX-Toolkit]]
*[[FastQC]]
*[[FastQC]]
*[[FreeSurfer 2016 | FreeSurfer]]
*[[FreeSurfer 2016 | FreeSurfer]]
*[[FSL 2016 | FSL]]
*[[FSL 2016 | FSL]]
*[[GATK]]
*[[GATE 2016 | GATE]]
*[[GenomeThreader 2016 | GenomeThreader]]
*[[GeneMark-EX 2016 | GeneMark-EX]]
*[[GeneMark-EX 2016 | GeneMark-EX]]
*[[GenomeTools 2016 | GenomeTools]]
*[[GffCompare 2016 | GffCompare]]
*[[GffCompare 2016 | GffCompare]]
*[[gffread 2016 | gffread]]
*[[gffread 2016 | gffread]]
*[[hapflk 2016 |hapflk]]
*[[HISAT2 2016 | HISAT2]]
*[[HISAT2 2016 | HISAT2]]
|
*[[IGV and IGVTools]]
*[[IGV and IGVTools]]
*[[IQ-TREE 2016 | IQ-TREE]]
*[[ipyrad]]
*[[ ITK 2016 | ITK ]]
*[[jcvi 2016 | jcvi]]
*[[jcvi 2016 | jcvi]]
*[[KNIME 2016 | KNIME]]
*[[Kraken 2016 | Kraken]]
*[[Kaldi 2016 | Kaldi]]
*[[Kaiju 2016 | Kaiju]]
|
*[[Lighter]]
*[[Lighter]]
*[[MAFFT 2016 | MAFFT]]
*[[minimap2 2016 | minimap2]]
*[[MUSCLE 2016 | MUSCLE]]
*[[lpsolve 2016 | lpsolve]]
*[[lpsolve 2016 | lpsolve]]
*[[Lumpy]]
*[[Lumpy]]
*[[MCScanX 2016 | MCScanX]]
*[[medaka 2016 | medaka]]
*[[MIRTK 2016 | MIRTK]]
*[[MrBayes 2016 | MrBayes]]
*[[MrBayes 2016 | MrBayes]]
*[[MultiQC 2016 | MultiQC]]
*[[NextGenMap]]
*[[NextGenMap]]
*[[NiBabel 2016 | NiBabel]]
*[[nQuire 2016 | nQuire]]
*[[OrthoFinder 2016 | OrthoFinder]]
*[[OMA 2016 | OMA]]
*[[PEST++ 2016 | PEST++]]
*[[Pychopper 2016 | Pychopper]]
*[[Pychopper 2016 | Pychopper]]
*[[picard 2016 | picard]]
*[[Pilon 2016 | Pilon]]
*[[Picard]]
*[[Picard]]
*[[QUAST 2016 | QUAST / QUAST-LG]]
*[[Racon 2016 | Racon]]
*[[RepeatModeler 2016 | RepeatModeler]]
*[[REPET 2016 | REPET]]
*[[REPET 2016 | REPET]]
*[[RSEM 2016 | RSEM]]
*[[RSEM 2016 | RSEM]]
*[[Salmon 2016 | Salmon]]
*[[Salmon 2016 | Salmon]]
*[[SAMtools]]
*[[SAMtools]]
*[[SHAPEIT4 2016 | SHAPEIT4]]
*[[Sickle 2016 | Sickle]]
*[[Sickle 2016 | Sickle]]
*[[SPAdes 2016 | SPAdes]]
*[[spaln 2016 | spaln]]
*[[Mindboggle 2016 | Mindboggle]]
*[[SNAP 2016 | SNAP]]
*[[SNAP 2016 | SNAP]]
*[[SNAP-HMM 2016 | SNAP-HMM]]
*[[SNAP-HMM 2016 | SNAP-HMM]]
Line 156: Line 221:
*[[StringTie 2016 | StringTie]]
*[[StringTie 2016 | StringTie]]
*[[TopHat 2016 | TopHat]]
*[[TopHat 2016 | TopHat]]
*[[TransDecoder 2016 | TransDecoder 2016]]
*[[Trinity]]
*[[Trinity]]
*[[TopHat]]
*[[Trimmomatic]]
*[[VCFTools]]
*[[VCFTools]]
*[[ViennaRNA]]
*[[ViennaRNA]]
*[[TopHat]]
*[[wtdbg 2016 | wtdbg2 ]]
*[[GATK]]
*[[Fiji/Unet 2016 | Fiji/Unet]]
*[[Trimmomatic]]
|  
|  
* [[dakota 2016 | dakota]]
* [[Gurobi 2016 | Gurobi]]
* [[Julia 2016 | Julia]]
* [[Julia 2016 | Julia]]
* [[Maple 2016 | Maple]]
* [[MATLAB 2016 | MATLAB]]   
* [[MATLAB 2016 | MATLAB]]   
** [[Configuration MDCS 2016 | Configuration MDCS]]
** [[Basic Examples MDCS 2016 | Basic Examples MDCS]]
** [[Advanced Examples MDCS 2016 | Advanced Examples MDCS]]
** [[Advanced Examples MDCS 2016 | Advanced Examples MDCS]]
** [[Basic Examples MDCS 2016 | Basic Examples MDCS]]
** [[MDCS Troubleshooting]]
** [[Configuration MDCS 2016 | Configuration MDCS]]
** [[CVX]]
** [[CVX]]
** [[MDCS with GPUs]]
** [[MDCS with GPUs]]
** [[IQMtools]]
** [[IQMtools]]
* [[Mmg 2016 | Mmg]]
* [[NEURON 2016 | NEURON]]
* [[NEURON 2016 | NEURON]]
*[[osrm-backend 2016 | osrm]]
*[[parallel 2016 | parallel]]
*[[parallel 2016 | parallel]]
* [[Perl 2016 | Perl]]
* [[Perl 2016 | Perl]]
* [[Python 2016 | Python]]
* [[Python 2016 | Python]]
** [[JupyterLab]]
* ScientificPython
* ScientificPython
** [[Anaconda 2016 | Anaconda]]
** [[Anaconda 2016 | Anaconda3 / Miniconda3]]
* [[TCL 2016 | TCL]]
* [[R 2016 | R]]
* [[R 2016 | R]]
** R-bundle-Bioconductor
** [[RStudio 2016 | RStudio]]
* [[RStan 2016 | RStan]]
*[[SALOME 2016 | SALOME ]]
* [[STATA 2016 | STATA]]
* [[STATA 2016 | STATA]]
* [[Gurobi 2016 | Gurobi]]
* [[TCL 2016 | TCL]]
* [[Maple 2016 | Maple]]
|-
|-
{| style="background-color:#eeeeff;" cellpadding="10" border="1" cellspacing="0"
{| style="background-color:#eeeeff;" cellpadding="10" border="1" cellspacing="0"
Line 193: Line 269:
|
|
* [[BLAS and LAPACK 2016 | BLAS and LAPACK]]
* [[BLAS and LAPACK 2016 | BLAS and LAPACK]]
* [[FLAC 2016 | FLAC]] 
* [[libsamplerate 2016 | libsamplerate]]
* [[MariaDB 2016 | MariaDB]]
* [[PyTables 2016 | PyTables]]
* [[PyTables 2016 | PyTables]]
* [[SuiteSparse 2016 | SuiteSparse]]
* [[SuiteSparse 2016 | SuiteSparse]]
* [[MariaDB 2016 | MariaDB]]
 
|
|
* [[EGSnrc 2016 | EGSnrc]]
* [[EGSnrc 2016 | EGSnrc]]
* [[ElmerFEM 2016 | ElmerFEM]]
* [[FLUKA 2016 | FLUKA]]
* [[FLUKA 2016 | FLUKA]]
* [[fluka.cern | fluka.cern]]
* [[FDTD_Solutions / Lumerical 2016 | FDTD Solutions / Lumerical]]
* [[GEANT4 2016| GEANT4]]
* [[GEANT4 2016| GEANT4]]
** [[GEANT4 2016#GRAS | GRAS]]
** [[GEANT4 2016#GRAS | GRAS]]
** [[GEANT4 2016#GATE | GATE]]
** [[GEANT4 2016#GATE | GATE]]
** [[GEANT4 2016#g4py| g4py]]
** [[GEANT4 2016#g4py| g4py]]
* [[FDTD_Solutions 2016 | FDTD Solutions]]
* [[HDF5 2016 | HDF5]]
* [[HDF5 2016 | HDF5]]
* [[LEDA  2016 | LEDA]]
* [[LEDA  2016 | LEDA]]
* [[mumax 2016 | mumax]]
* [[NetCDF 2016 | NetCDF]]
* [[NetCDF 2016 | NetCDF]]
* [[Octopus 2016 | Octopus ]]
* [[PVMOS 2016 | PVMOS]]
* [[PVMOS 2016 | PVMOS]]
* [[SCUFF-EM 2016 | SCUFF-EM]]
* [[libsamplerate 2016 | sndfile-tools]]
* [[VASP 2016 | VASP]]
** [[p4vasp 2016 | p4vasp]]
* [[Yambo 2016 | Yambo]]
|
|
* [[Caffe]]
* [[Caffe]]
* [[Keras 2016 | Keras]]
* [[Tensorflow 2016 | TensorFlow]]
* [[Tensorflow 2016 | TensorFlow]]
* [[Theano 2016 | Theano]]
* [[Theano 2016 | Theano]]
* [[Keras 2016 | Keras]]
* [[KNIME 2016 | KNIME]]
* [[PyTorch 2016 | PyTorch]]
|
|
* [[ESMF 2016 | ESMF]]
* [[ESMF 2016 | ESMF]]
Line 226: Line 316:
* [[ncview]]
* [[ncview]]
* [[paraview]]
* [[paraview]]
* [[PCL 2016 | PCL]]
|}
|}
|}
|}
Line 231: Line 322:
== Courses and Tutorials ==
== Courses and Tutorials ==


Below you can find a list of Courses and Tutorials for our HPC clusters. Currently, a short, two hour introduction for the new clusters CARL and EDDY is planned. During the summer semester a lecture "Introduction to HPC" is held (2h/week). If you are interested in attending or would like to request a course please contact {{sc}}.
Below you can find a list of Courses and Tutorials for our HPC clusters. Currently, a short, two-hour introduction for the new clusters CARL and EDDY is planned. During the summer semester a lecture "Introduction to HPC" is held (2h/week). If you are interested in attending or would like to request a course please contact {{sc}}.


{| style="background-color:#eeeeff;" cellpadding="10" border="1" cellspacing="0"  
{| style="background-color:#eeeeff;" cellpadding="10" border="1" cellspacing="0"  
Line 244: Line 335:
* Introduction for EDDY users for migrating from FLOW [[media:Eddy_Introduction.pdf|[pdf]]]<br>(01.03.2017 and 02.03.2017, 10-12am c.t.)
* Introduction for EDDY users for migrating from FLOW [[media:Eddy_Introduction.pdf|[pdf]]]<br>(01.03.2017 and 02.03.2017, 10-12am c.t.)
* [[OpenACC Workshop]]<br>(28./29.3.2017, 10am-6pm, W01 0-008)
* [[OpenACC Workshop]]<br>(28./29.3.2017, 10am-6pm, W01 0-008)
* [[Workshop Nvidia HPC SDK]]<br>(06.07.2021, 10am-1pm, online)
* [[Workshop How to AMD]]<br>(04./05.04.2023, 14pm-17pm, online)
* [[HPC Introduction | Introduction to HPC (three day course)]]
* [[HPC Introduction | Introduction to HPC (three day course)]]
** [[HPC Introduction 03/2019 | most recent course]]<br>(25.-28.03.2019)
** [[HPC Introduction 03/2023 | upcoming or most recent course]]<br>(27.03-29.03.2023)
** [[HPC Introduction | all course]]
** [[HPC Introduction | all course]]
|
|
* [[Lecture HPC Introduction | Introduction to High-Performance Computing]]
* [[Lecture HPC Introduction | Introduction to High-Performance Computing]]
<!-- [[HPC Video Lectures 2016 | HPC Video Lectures ]] -->
|
|
*[[HPC_Video_Lectures_2016 | HPC Video Lectures]]
*[[Tutorials MDCS 2016 | Matlab Tutorials]]
*[[Tutorials MDCS 2016 | Matlab Tutorials]]
:*[[Audio Data Processing]]
:*[[Audio Data Processing]]
:*[[Using the MEX Compiler]]
:*[[Using the MEX Compiler]]
*[[R Tutorial]]
|}
|}
</center>
</center>
Line 282: Line 378:
|}
|}
</center>
</center>
<br/>
---------------- [[Migration|Migration to the new HPC clusters]] ----------------
The old wiki for HERO and FLOW is found [[Welcome to the HPC User Wiki of the University of Oldenburg | here]].<br><br>
The old wiki for HERO and FLOW is found [[Welcome to the HPC User Wiki of the University of Oldenburg | here]].<br><br>
<span style="color: green">[[HPC Support Internal]]</span> <br><br>
<span style="color: green">[[HPC Support Internal]]</span> <br>
[[Intel CPU Bug]]<br>
[[Globus Data Transfer]]

Latest revision as of 08:06, 27 June 2024

Documentation for High-Performance Computing at the University Oldenburg

This is the HPC User Wiki for the HPC clusters CARL and EDDY. If you would like to contribute please contact Scientific Computing!

The current wiki for the HPC cluster ROSA and STORM/MOUSE can be found here. Unfortunately, the new wiki is far from complete so you may need to adapt some information from this wiki still. We are working on it :)

General Information


---------------- FAQ - Frequently Asked Questions ---------------- 

 ------------- Want to stay in contact with the HPC support and other users? Join our Mattermost group! -------------

------------------ Tired of reading long wiki articles? ------------------ 
------------- Check out our new HPC Video Lectures! -------------
HPC Facilities Quickstart Guide User environment Job Management (Queueing) System


Other HPC resources:

Application Software and Libraries

----------------------  What's installed and what's new? ----------------------
Compiler and Development Tools Quantum Chemistry Biology & Medicine Mathematics, Statistics & Scripting
Libraries Physics Machine Learning Computational Fluid Dynamics Visualisation

Courses and Tutorials

Below you can find a list of Courses and Tutorials for our HPC clusters. Currently, a short, two-hour introduction for the new clusters CARL and EDDY is planned. During the summer semester a lecture "Introduction to HPC" is held (2h/week). If you are interested in attending or would like to request a course please contact Scientific Computing.

Courses and Workshops Lectures Tutorials

Contact

In case you need HPC support you can use the contacts listed below. The servicedesk of IT services can be contacted for all questions regarding HPC, and are recommended in particular when someone is absent (e.g. due to a vacation).

HPC Resource Name E-Mail
CARL and EDDY General HPC Support
and Servicedesk 		hpcsupport@uni-oldenburg.de
servicedesk@uni-oldenburg.de
CARL Stefan Harfst
Fynn Schwietzer 		stefan.harfst@uni-oldenburg.de
fynn.schwietzer1@uni-oldenburg.de
EDDY Wilke Trei wilke.trei@forwind.de 


---------------- Migration to the new HPC clusters ----------------

The old wiki for HERO and FLOW is found here.

HPC Support Internal
Intel CPU Bug
Globus Data Transfer

Retrieved from "https://wiki.hpcuser.uni-oldenburg.de/index.php?title=HPC_User_Wiki_2016&oldid=7295"