NAME
pmdampi - MPI performance metrics domain agent (PMDA)
SYNOPSIS
$PCP_PMDAS_DIR/mpi/pmdampi [-d domain] [-l logfile] [-n
num] [-t interval]
DESCRIPTION
pmdampi is a Performance Metrics Domain Agent (PMDA) which
exports metrics from MPI applications linked with the
pcp_mpi shared library. The metrics include counts and
accumulated time for selected MPI functions and statistics
on MPI buffer usage.
A brief description of the pmdampi command line options
follows:
-d It is absolutely crucial that the performance metrics
domain number specified here is unique and consis
tent. That is, domain should be different for every
PMDA on the one host, and the same domain number
should be used for the same PMDA on all hosts.
-l Location of the log file. By default, a log file
named mpi.log is written in the current directory of
pmcd(1) when pmdampi is started, i.e.
$PCP_LOG_DIR/pmcd. If the log file cannot be created
or is not writable, output is written to the standard
error instead.
-n By default pmdampi checks for a change in the set of
mmaps on every fetch. It realizes the possible
change due to a change in the modification date on
the mmaps' directories. The -n option reduces the
number of checks to every nth fetch.
-t By default pmdampi checks for unused mmaps on every
fetch. The application's mmap becomes unused if the
mpi process is killed and it's not given a chance to
remove its mmap file. The -t option restricts this
checking on each fetch to be only after a specified
time interval.
INSTALLATION
If you want access to the names, help text and values for
the MPI performance metrics, do the following as root:
# cd $PCP_PMDAS_DIR/mpi
# ./Install
root:
# cd $PCP_PMDAS_DIR/mpi
# ./Remove
pmdampi is launched by pmcd(1) and should never be exe
cuted directly. The Install and Remove scripts notify
pmcd(1) when the agent is installed or removed.
METRICS
pmdampi has metrics for monitoring particular MPI func
tions and MPI buffer usage. The function metrics are pre
fixed by mpi.func. The two most interesting function met
rics are:
mpi.func.time
mpi.func.calls
mpi.func.time specifies the accumulated time of a particu
lar function for a particular process. mpi.func.calls
specifies the accumulated number of calls of a particular
function for a particular process.
The MPI buffer usage metrics are prefixed by mpi.retries
and mpi.sends. These metrics are based on the types of
data obtained from MPI_SGI_stat_get(3) and specified in
/usr/include/mpi_ext.h.
INSTANCE DOMAINS
There are 2 types of instance domains:
process
process-function
An mpi process is identified by its ASH number and rank
number. It is of the form: "ash-in-hex:rank". For exam
ple, "3ab5:0". An mpi process-function is identified by
the ASH and rank concatenated with the function name as
specified in the control file (e.g. $PCP_VAR_DIR/con
fig/mpi/control). It is of the form: "ash-in-
hex:rank:func". For example, "3ab5:0:bsend".
FILES
$PCP_VAR_DIR/config/mpi/control
is the control file where one can specify the
functions of interest and SGI buffer stats of
interest. See pmdampi(3) for more details on
the control file's format.
/var/tmp/mpi_pcp
is the top level directory for the mmap(1) files
containing the exported MPI data. This direc
tory is divided into subdirectories for each ASH
(array session handle). In the ASH directory are
the mmaps for each rank/process.
$PCP_PMDAS_DIR/mpi/Install
installation script for the pmdampi agent
undo installation script for the pmdampi agent
$PCP_LOG_DIR/pmcd/mpi.log
default log file for error messages and other
information from pmdampi
PCP ENVIRONMENT
Environment variables with the prefix PCP_ are used to
parameterize the file and directory names used by PCP. On
each installation, the file /etc/pcp.conf contains the
local values for these variables. The $PCP_CONF variable
may be used to specify an alternative configuration file,
as described in pcp.conf(4).
SEE ALSO
PCPIntro(1), pmcd(1) and pcp_mpi(3).
Relevant information is also available from the on-line
PCP for MPI Tutorial. Provided the pcp.man.tutorial sub
system has been installed, access the URL
file:$PCP_DOC_DIR/Tutorial/mpi.html from your web browser.
Man(1) output converted with
man2html