NAME
pmlogger - create archive log for performance metrics
SYNOPSIS
$PCP_BINADM_DIR/pmlogger [-c configfile] [-h host] [-l
logfile] [-L] [-n pmnsfile] [-P] [-r] [-s endsize] [-t
interval] [-T endtime] [-v volsize] [-V version] [-x fd]
archive
DESCRIPTION
pmlogger creates the archive logs of performance metric
values that may be ``played back'' by other Performance
Co-Pilot (see PCPIntro(1)) tools. These logs form the
basis of the VCR paradigm and retrospective performance
analysis services common to the PCP toolkit.
The mandatory argument archive is the base name for the
physical files that constitute an archive log.
The -V option specifies whether a version 1 or version 2
archive is generated. A version 2 archive also stores the
associated Performance Metrics Name Space (PMNS). By
default a version 2 archive is generated.
Unless directed to another host by the -h option, pmlogger
will contact the Performance Metrics Collector Daemon
(PMCD) on the local host and use that as the source of the
metric values to be logged.
To support the required flexibility and control over what
is logged and when, pmlogger maintains an independent two
level logging state for each instance of each performance
metric. At the first (mandatory) level, logging is
allowed to be on (with an associated interval between sam
ples), or off or maybe. In the latter case, the second
(advisory) level logging is allowed to be on (with an
associated interval between samples), or off.
The mandatory level allows universal specification that
some metrics must be logged, or must not be logged. The
default state for all instances of all metrics when pmlog
ger starts is mandatory maybe and advisory off.
Use pmlc(1) to interrogate and change the logging state
once pmlogger is running.
If a metric's state is mandatory (on or off) and a request
is made to change it to mandatory maybe, the new state is
mandatory maybe and advisory off. If a metric's state is
already advisory (on or off) and a request is made to
change it to mandatory maybe, the current state is
retained.
of a metric and all instances of the same metric concur
rently. If specific instances are being logged and a
request to log all instances is made, then all instances
of the metric will be logged according to the new request,
superseding any prior logging request for the metric. A
request to log all instances of a metric will supersede
any previous request to log all instances. A request to
log specific instances of a metric when all instances are
already being logged is refused. To do this one must turn
off logging for all instances of the metric first. In
each case, the validity of the request is checked first;
for example a request to change a metric's logging state
to advisory on when it is currently mandatory off is never
permitted (it is necessary to change the state to manda
tory maybe first).
Optionally, each system running pmcd(1) may also be con
figured to run a ``primary'' pmlogger instance. Like
pmcd(1), this pmlogger instance is launched by
$PCP_RC_DIR/pcp, and is affected by the files /etc/con
fig/pmlogger (use chkconfig(1M) to activate or disable the
primary pmlogger instance), /etc/config/pmlogger.options
(command line options passed to the primary pmlogger) and
$PCP_VAR_DIR/config/pmlogger/config.default (the default
initial configuration file for the primary pmlogger).
The primary pmlogger instance is identified by the -P
option. There may be at most one ``primary'' pmlogger
instance on each system with an active pmcd(1). The pri
mary pmlogger instance (if any) must be running on the
same host as the pmcd(1) to which it connects, so the -h
and -P options are mutually exclusive.
When launched as a non-primary instance, pmlogger will
exit immediately if the configuration file causes no met
ric logging to be scheduled. The -L option overrides this
behavior, and causes a non-primary pmlogger instance to
``linger'', presumably pending some future dynamic re-con
figuration and state change via pmlc(1). pmlogger will
also linger without the -L option being used if all the
metrics to be logged are logged as once only metrics. When
the once only metrics have been logged, a warning message
will be generated stating that the event queue is empty
and no more events will be scheduled.
By default all diagnostics and errors from pmlogger are
written to the file pmlogger.log in the directory where
pmlogger is launched. The -l option may be used to over
ride the default behavior. If the log file cannot be cre
ated or is not writable, output is written to standard
error instead.
If specified, the -s option instructs pmlogger to
units has been accumulated. If endsize is an integer then
endsize records will be written to the log. If endsize is
suffixed by b or bytes then endsize bytes of the archive
data will be written out (note, however, that archive log
record boundaries will not be broken and so this limit may
be slightly surpassed). Other viable file size units
include: K, Kb, Kilobyte for kilobytes and M, Mb, Megabyte
for megabytes. These units may be optionally suffixed by
an s and may be of mixed case. Alternatively endsize may
be suffixed using a time unit as described in PCPIntro(1)
for the interval argument (to the standard PCP tool option
-t).
Some examples of different formats:
-s 100
-s 100bytes
-s 100K
-s 100Mb
-s 10mins
-s 10hours
The default is for pmlogger to run forever.
The -r option causes the size of the physical record(s)
for each group of metrics and the expected contribution of
the group to the size of the PCP archive for one full day
of collection to be reported in the log file. This infor
mation is reported the first time each group is success
fully written to the archive.
The log file is potentially a multi-volume data set, and
the -v option causes pmlogger to start a new volume after
a certain size in records, bytes, or time units has been
accumulated for the current volume. The format of this
size specification is identical to that of the -s option
(see above). The default is for pmlogger to create a sin
gle volume log. Additional volume switches can also be
forced asynchronously by either using pmlc(1) or sending
pmlogger a SIGHUP signal (see below). Note, if a scheduled
volume switch is in operation due to the -v option, then
its counters will be reset after an asynchronous switch.
Normally pmlogger operates on the distributed Performance
Metrics Name Space (PMNS), however if the -n option is
specified an alternative local PMNS is loaded from the
file pmnsfile.
Under normal circumstances, pmlogger will run forever
(except for a -s option or a termination signal). The -T
option may be used to limit the execution time using the
format of time as prescribed by PCPIntro(1).
Some examples of different formats:
-T 10mins
-T '@ 11:30'
From this it can be seen that -T 10mins and -s 10mins
When pmlogger receives a SIGHUP signal, the current volume
of the log is closed, and a new volume is opened. This
mechanism (or the alternative mechanism via pmlc(1)) may
be used to manage the growth of the log files - once a log
volume is closed, that file may be archived without ill-
effect on the continued operation of pmlogger. See also
the -v option above.
The buffers for the current log may be flushed to disk
using the flush command of pmlc(1), or by sending pmlogger
a SIGUSR1 signal. This is useful when the log needs to be
read while pmlogger is still running.
When launched with the -x option, pmlogger will accept
asynchronous control requests on the file descriptor fd.
This option is only expected to be used internally by PCP
applications that support ``live record mode'' via
pmRecordControl(3).
CONFIGURATION FILE SYNTAX
The configuration file may be specified with the -c
option. If it is not, configuration specifications are
read from standard input.
If configfile does not exist, then a search is made in the
directory $PCP_VAR_DIR/config/pmlogger for a file of the
same name, and if found that file is used, e.g. if con
fig.mumble does not exist in the current directory and the
file $PCP_VAR_DIR/config/pmlogger/config.mumble does
exist, then -c config.mumble and -c $PCP_VAR_DIR/con
fig/pmlogger/config.mumble are equivalent.
The syntax for the configuration file is as follows.
1. Words are separated by white space (space, tab or
newline).
2. The symbol ``#'' (hash) introduces a comment, and
all text up to the next newline is ignored.
3. Keywords (shown in bold below) must appear liter
ally (i.e. in lower case).
4. Each specification begins with the optional keyword
log, followed by one of the states mandatory on,
mandatory off, mandatory maybe, advisory on or
advisory off.
5. For the on states, a logging interval must follow
using the syntax ``once'', or ``default'', or
``every N timeunits'', or simply ``N timeunits'' -
N is an unsigned integer, and timeunits is one of
minute, hour or the plural form of one of the
above.
Internal limitations require the interval to be
smaller than (approximately) 74 hours. An interval
value of zero is a synonym for once. An interval
of default means to use the default logging inter
val of 60 seconds; this default value may be
changed to interval with the -t command line
option.
The interval argument follows the syntax described
in PCPIntro(1), and in the simplest form may be an
unsigned integer (the implied units in this case
are seconds).
6. Following the state and possible interval specifi
cations comes a ``{'', followed by a list of one or
more metric specifications and a closing ``}''.
The list is white space (or comma) separated. If
there is only one metric specification in the list,
the braces are optional.
7. A metric specification consists of a metric name
optionally followed by a set of instance names.
The metric name follows the standard PCP naming
conventions, see pmns(4), and if the metric name is
a non-leaf node in the PMNS (see pmns(4)), then
pmlogger will recursively descend the PMNS and
apply the logging specification to all descendent
metric names that are leaf nodes in the PMNS. The
set of instance names is a ``['', followed by a
list of one or more space (or comma) separated
names, numbers or strings, and a closing ``]''.
Elements in the list that are numbers are assumed
to be internal instance identifiers, other elements
are assumed to be external instance identifiers -
see pmGetInDom(3) for more information.
If no instances are given, then the logging speci
fication is applied to all instances of the associ
ated metric.
8. There may be an arbitrary number of logging speci
fications.
9. Following all of the logging specifications, there
may be an optional access control section, intro
duced by the literal token [access]. Thereafter
come access control rules of the form ``allow
hostlist : operation ;'' and ``disallow hostlist :
operation ;''.
The base operations are advisory, mandatory and
trol statements follow the syntactic and semantic
rules defined for the access control mechanisms
used by PMCD and are fully documented in pmcd(1).
EXAMPLES
For each PCP utility, there is a sample pmlogger configu
ration file that could be used to create an archive log
suitable for replaying with that tool (i.e. includes all
of the performance metrics used by the tool). For a tool
named foo this configuration file is located in
$PCP_VAR_DIR/config/pmlogger/config.foo.
The following is a simple default configuration file for a
primary pmlogger instance, and demonstrates most of the
capabilities of the configuration specification language.
log mandatory on once { hinv.ncpu hinv.ndisk }
log mandatory on every 10 minutes {
disk.all.write
disk.all.read
network.interface.in.packets [ "et0" ]
network.interface.out.packets [ "et0" ]
nfs.server.reqs [ "lookup" "getattr" "read" "write" ]
}
log advisory on every 30 minutes {
environ.temp
pmcd.pdu_in.total
pmcd.pdu_out.total
}
[access]
disallow * : all except enquire;
allow localhost : mandatory, advisory;
AUTOMATIC RESTART
It is often useful for pmlogger processes (other than the
primary instance) to be started and stopped when the local
host is booted or shutdown. The script $PCP_RC_DIR/pcplo
cal and the necessary soft-links are provided, and can be
modified by root to run PCP tools automatically. If you
want to find out more before starting, read the manual
pages for rc2(1), rc0(1), shutdown(1) and the file
/etc/init.d/README.
For example, changing $PCP_RC_DIR/pcplocal so that it con
tains:
# Add startup actions here
($PCP_BINADM_DIR/pmlogger_check &)
;;
# Add shutdown actions here
;;
will start pmlogger instances at boot time and terminate
them in an orderly fashion at system shutdown.
This script runs as root, so any pmlogger instances it
launches are also run as root. To run some pmlogger
instances as a particular user, create your own archive
logger control file (see pmlogger_check(1)) and use the
su(1) command. e.g.
# Add startup actions here
(su tanya -c "$PCP_BINADM_DIR/pmlogger_check -c /usr/people/tanya/ctl" &)
;;
at boot time will start the pmlogger instances described
in /usr/people/tanya/ctl, running as user tanya.
FILES
archive.meta
metadata (metric descriptions, instance domains,
etc.) for the archive log
archive.0 initial volume of metrics values (subsequent
volumes have suffixes 1, 2, ...)
archive.index
temporal index to support rapid random access to
the other files in the archive log
$PCP_TMP_DIR/pmlogger
pmlogger maintains the files in this directory
as the map between the process id of the pmlog
ger instance and the IPC port that may be used
to control each pmlogger instance (as used by
pmlc(1))
/etc/config/pmlogger
chkconfig(1M) control flag, to control launching
of pmlogger from $PCP_RC_DIR/pcp
/etc/config/pmlogger.options
command line options to pmlogger when launched
from $PCP_RC_DIR/pcp
$PCP_VAR_DIR/config/pmlogger/config.default
default configuration file for the primary log
ger instance launched from $PCP_RC_DIR/pcp
$PCP_VAR_DIR/config/pmlogger/config.*
assorted configuration files suitable for creat
ing logs that may be subsequently replayed with
the PCP visualization and monitoring tools
$PCP_LOG_DIR/pmloggerhostname
Default directory for PCP archive files for per
formance metric values collected from the host
hostname.
./pmlogger.log
(or $PCP_LOG_DIR/pmlogger/hostname/pmlogger.log
when started automatically by either
toring scripts such as pmlogger_check(1))
all messages and diagnostics are directed here
$PCP_RC_DIR/pcplocal
contains ``hooks'' to enable automatic restart
at system boot time
ENVIRONMENT
Normally pmlogger creates a socket to receive control mes
sages from pmlc(1) on the first available TCP/IP port num
bered 4330 or higher. The environment variable PMLOG
GER_PORT may be used to specify an alternative starting
port number.
PCP ENVIRONMENT
Environment variables with the prefix PCP_ are used to
parameterize the file and directory names used by PCP. On
each installation, the file /etc/pcp.conf contains the
local values for these variables. The $PCP_CONF variable
may be used to specify an alternative configuration file,
as described in pcp.conf(4).
SEE ALSO
PCPIntro(1), pmcd(1), pmdumplog(1), pmlc(1), pmlog
ger_check(1), pcp.conf(4), pcp.env(4) and pmns(4).
If you have the PCP product, relevant information is also
available from the on-line PCP Tutorial. Provided the
pcp.man.tutorial subsystem from the PCP images has been
installed, access the URL file:$PCP_DOC_DIR/Tuto
rial/archive.html from your web browser.
DIAGNOSTICS
The archive logs are sufficiently precious that pmlogger
will not truncate an existing physical file. A message of
the form
__pmLogNewFile: "foo.index" already exists, not over-
written
__pmLogCreate: File exists
indicates this situation has arisen. You must explicitly
remove the files and launch pmlogger again.
There may be at most one primary pmlogger instance per
monitored host; attempting to bend this rule produces the
error:
pmlogger: there is already a primary pmlogger running
Various other messages relating to the creation and/or
deletion of files in $PCP_TMP_DIR/pmlogger suggest a per
mission problem on this directory, or some feral files
have appeared therein.
Man(1) output converted with
man2html