Difference between revisions of "Python"

From HPC users
Jump to navigationJump to search
 
(5 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Up to now there only one Python releases as module available on FLOW and HERO. The relase is available by
== Basics ==


  module load testing/python
The available installations of Python can be listed using the command


With this command the corresponding Intel compilers and Intel MPI will be automatically loaded.
  module av python


'''Note:''' The release will be replaced by an official release soon.
Currently only one Python module is available and it can be loaded using


  module load python/2.7.9
With this command the corresponding Intel compilers and Intel MPI will be automatically loaded.


== Additional installed packages ==
== Additional installed packages ==
The actual release contains following additional packages:
The actual release contains (among others) the following additional packages:


* numpy
* numpy
Line 18: Line 21:
* PyClimate
* PyClimate


== Usage of Python adn MPI ==
A complete list of packages can be obtained with the command
 
  pip list
 
(only after the module was loaded). Additional packages can be installed upon request, but it is also possible to install packages in your own <tt>$HOME</tt>-directory (see below).
 
== Usage of Python and MPI ==


For parallel scripts the Python installtion contains the package mpi4py. To launch an parallel Python script inside an [[SGE Job Management (Queueing) System| SGE]] script please use command line
For parallel scripts the Python installtion contains the package mpi4py. To launch an parallel Python script inside an [[SGE Job Management (Queueing) System| SGE]] script please use command line


   mpirun -bootstrap sge -np $NSLOTS -env I_MPI_FABRICS shm:ofa python ''SCRIPTNAME'' ''SCRIPT_CMDLINE_OPTIONS''
   mpirun -np $NSLOTS python ''SCRIPTNAME'' ''SCRIPT_CMDLINE_OPTIONS''


to enable SGE to control all processes of your script.
to enable SGE to control all processes of your script.
The corresponding parallel environment in the SGE submission script is specified by
  #$ -pe impi NUMBER_OF_CORES
  #$ -R y
== Installing Packages in your HOME-Directory ==
Most Python packages can be installed easily using the <tt>pip</tt>-command. In that case, the <tt>--user</tt>-option allows a local user based installation. E.g., to install the package <tt>PYPACKAGE</tt> use the command (after loading the module):
  pip install --user PYPACKAGE
If the installation is successful, the corresponding files are installed in
  $HOME/.local/lib/pythonx.y/site-packages
where <tt>x.y</tt> corresponds to the Python version currently loaded. Also, <tt>pip list</tt> should show the new package in the list.
== Profiling Python programs ==
Details on how to profile entire programs, particular functions and other parts of your code can be found [[Profiling_using_gprof#Profiling_Python_programs| here]].


== External links ==
== External links ==

Latest revision as of 18:52, 22 January 2015

Basics

The available installations of Python can be listed using the command

 module av python

Currently only one Python module is available and it can be loaded using

 module load python/2.7.9

With this command the corresponding Intel compilers and Intel MPI will be automatically loaded.

Additional installed packages

The actual release contains (among others) the following additional packages:

  • numpy
  • scipy
  • nose_cov
  • distribute
  • ScientificPython
  • PyClimate

A complete list of packages can be obtained with the command

 pip list

(only after the module was loaded). Additional packages can be installed upon request, but it is also possible to install packages in your own $HOME-directory (see below).

Usage of Python and MPI

For parallel scripts the Python installtion contains the package mpi4py. To launch an parallel Python script inside an SGE script please use command line

  mpirun -np $NSLOTS python SCRIPTNAME SCRIPT_CMDLINE_OPTIONS

to enable SGE to control all processes of your script.

The corresponding parallel environment in the SGE submission script is specified by

 #$ -pe impi NUMBER_OF_CORES
 #$ -R y

Installing Packages in your HOME-Directory

Most Python packages can be installed easily using the pip-command. In that case, the --user-option allows a local user based installation. E.g., to install the package PYPACKAGE use the command (after loading the module):

 pip install --user PYPACKAGE

If the installation is successful, the corresponding files are installed in

 $HOME/.local/lib/pythonx.y/site-packages

where x.y corresponds to the Python version currently loaded. Also, pip list should show the new package in the list.


Profiling Python programs

Details on how to profile entire programs, particular functions and other parts of your code can be found here.

External links