Learn more about Platform products at http://www.platform.com

[ Platform Documentation ] [ Title ] [ Contents ] [ Previous ] [ Next ] [ Index ]



Using Platform LSF with MVAPICH


Contents

[ Top ]


About Platform LSF and MVAPICH

MVAPICH is an open-source product developed in the Department of Computer and Information Science, The Ohio State University. MVAPICH is MPI-1 over VAPI for InfiniBand. It is an MPI-1 implementation on Verbs Level Interface (VAPI), developed by Mellanox Technologies. The implementation is based on MPICH and MVICH.

The LSF MVAPICH MPI integration is based on the LSF generic PJL framework. It supports the following MVAPICH variations:

Requirements

Assumptions and limitations

Glossary

MPI

(Message Passing Interface) A message passing standard. It defines a message passing API useful for parallel and distributed applications.

MPICH

A portable implementation of the MPI standard.

PAM

(Parallel Application Manager) The supervisor of any parallel job.

PJL

(Parallel Job Launcher) Any executable script or binary capable of starting parallel tasks on all hosts assigned for a parallel job.

RES

(Remote Execution Server) An LSF daemon residing on each host. It monitors and manages all LSF tasks on the host.

TS

(TaskStarter) An executable responsible for starting a task on the local host and reporting the process ID and host name to the PAM.

For more information

Files installed by lsfinstall

During installation, lsfinstall copies these files to the following directories:

These files... Are installed to...
TaskStarter
LSF_BINDIR
pam
LSF_BINDIR
esub.mvapich--sets the mode: rsh ssh or mpd
LSF_SERVERDIR
mvapich_wrapper
LSF_BINDIR
mpirun.lsf
LSF_BINDIR
pjllib.sh
LSF_BINDIR

Resources and parameters configured by lsfinstall

[ Top ]


Configuring LSF to Work with MVAPICH

esub.mvapich script

Modify the esub.mvapich in LSF_SERVERDIR to set MVAPICH_START_CMD.to one of ssh, rsh, or mpd. The default value is ssh.

mvapich_wrapper script

Modify the mvapich_wrapper script in LSF_BINDIR to set MVAPICH_HOME. The defaults are:

mpd command location


Make sure the mpirun_rsh/ssh/mpd command is under MVAPICH_HOME/bin.

[ Top ]


Submitting MVAPICH Jobs

bsub command

Use bsub -a mvapich to submit jobs:

If the starting command is mpd, you must submit your MVAPICH jobs as exclusive jobs (bsub -x).

bsub -a mvapich -n number_cpus mpirun.lsf 
[-pam "pam_options"] [mpi_options] job [job_options]

For example:

% bsub -a mvapich -n 3 mpirun.lsf /examples/cpi

A job named cpi will be dispatched and run on 3 CPUs in parallel.

Task geometry with MVAPICH jobs

MVAPICH supports the LSF task geometry feature except in MPD mode. When running in MPD mode, the order of the hosts specified in the machine file is not honored:

Submitting a job with a job script

A wrapper script is often used to call MVAPICH. You can submit a job using a job script as an embedded script or directly as a job, for example:

% bsub -a mvapich -n 4 < embedded_jobscript
% bsub -a mvapich -n 4 jobscript

Your job script must use mpirun.lsf in place of the mpirun command.

For more information

[ Top ]


[ Platform Documentation ] [ Title ] [ Contents ] [ Previous ] [ Next ] [ Index ]


      Date Modified: January 10, 2011
Platform Computing: www.platform.com

Platform Support: support@platform.com
Platform Information Development: doc@platform.com

Copyright © 1994-2011 Platform Computing Corporation. All rights reserved.