Python

From HPC users
Jump to navigationJump to search

Basics

The available installations of Python can be listed using the command

 module av python

Currently only one Python module is available and it can be loaded using

 module load python/2.7.9

With this command the corresponding Intel compilers and Intel MPI will be automatically loaded.

Additional installed packages

The actual release contains (among others) the following additional packages:

  • numpy
  • scipy
  • nose_cov
  • distribute
  • ScientificPython
  • PyClimate

A complete list of packages can be obtained with the command

 pip list

(only after the module was loaded). Additional packages can be installed upon request, but it is also possible to install packages in your own $HOME-directory (see below).

Usage of Python and MPI

For parallel scripts the Python installtion contains the package mpi4py. To launch an parallel Python script inside an SGE script please use command line

  mpirun -np $NSLOTS python SCRIPTNAME SCRIPT_CMDLINE_OPTIONS

to enable SGE to control all processes of your script.

The corresponding parallel environment in the SGE submission script is specified by

 #$ -pe impi NUMBER_OF_CORES
 #$ -R y

Installing Packages in your HOME-Directory

Most Python packages can be installed easily using the pip-command. In that case, the --user-option allows a local user based installation. E.g., to install the package PYPACKAGE use the command (after loading the module):

 pip install --user PYPACKAGE

If the installation is successful, the corresponding files are installed in

 $HOME/.local/lib/pythonx.y/site-packages

where x.y corresponds to the Python version currently loaded. Also, pip list should show the new package in the list.


Profiling Python programs

Details on how to profile entire programs, particular functions and other parts of your code can be found here.

External links