NAME

       dplace  -  a  tool  for controlling placement of processes
       onto cpus


SYNOPSIS

       dplace [-c cpu_numbers] [-s skip_count] [-n process_name] [-x skip_mask] [-p placement_file] command [command-args]
       dplace -q


DESCRIPTION

       The given program is executed after scheduling and  memory
       placement  policies  are  set up according to command line
       arguments,

       By default, memory is allocated to a process on  the  node
       that  the process is executing on. If a process moves from
       node to node while it running, a higher percentage of mem­
       ory  references  will  be to remote nodes. Remote accesses
       typically have higher access times.   Process  performance
       may suffer.

       Dplace  is used to bind a related set of processes to spe­
       cific cpus or nodes to prevent process migrations. In some
       cases,  this  will improve performance since a higher per­
       centage of memory accesses will to the local node.

       Processes alway execute within a cpumemset. The  cpumemset
       specifies  the  cpus  that  are available for a process to
       execute on. By default, processes  usually  execute  in  a
       cpumemset that contains all the cpus in the system.

       Dplace  invokes a kernel hook (PAGG) to create a placement
       container consisting of all (or a subset of) the  cpus  of
       the  cpumemset.  The dplace process is placed in this con­
       tainer & (by default) is bound to the  first  cpu  of  the
       cpumemset  associated  with  the  container.  Then  dplace
       "execs" the <command>.

       The command is executing within this placement container &
       continues  to  be bound to the first cpu of the container.
       As the command forks child  processes,  they  inherit  the
       container  &  are  bound  to the next available cpu of the
       container.

       dplace maintains a global count of the  number  of  active
       processes  that  have been placed (by dplace) on each cpu.
       If no placement file is specified, dplace will bind a pro­
       cess  to the cpu that has the lowest number of active pro­
       cesses & is also in the users cpu list.  For  example,  if
       the  current  cpumemset consists of physical cpus 2, 5, 8,
       and 9, and the user types:

               mpirun -np 2 dplace -s1 app2

       app1 will run on cpus 2 & 5, and app2 will run on cpus 8 &
       9.


OPTIONS

       -c     Cpu_number(s).  Specified  as a list of cpu ranges:
              Example: "-c1", "-c2-4", "-c1,4-8,3".  Cpu  numbers
              are  NOT physical cpu numbers. They are logical cpu
              number that are relative to the cpus  that  are  in
              the set of allowed cpus as specified by the current
              cpumemset or "runon" command.  Cpu numbers start at
              0. If this option is not specified, all cpus of the
              current cpumemset are available. Note that a previ­
              ous  "runon"  command  may  be used to restrict the
              available cpus.

       -s     Skip the first <skip_count> processes before start­
              ing  to  place  processes onto cpus. This option is
              useful if  the  first  <skip_count>  processes  are
              "shepherd" processes that are used only for launch­
              ing the application. If <skip_count> is not  speci­
              fied, a default value of 0 is used.

       -n     Only  processes  named  <process_name>  are placed.
              Other processes are ignored and are not  explicitly
              bound  to cpus.  Note: process_name is the basename
              of the executible.

       -x     Provides the ability  to  skip  placement  of  pro­
              cesses.  <skip_mask>  is  a  bitmask.  If  bit N of
              <skip_mask> is set, then the N+1th process that  is
              forked is not placed. For example, setting the mask
              to 6 will cause the 2nd & 3rd processes from  being
              placed. The first process (the process named by the
              <command>) will be assigned to the first  cpu.  The
              second & third processes are not placed. The fourth
              process is assigned to the second cpu, etc..   ohis
              option  is  useful  for certain classes of threaded
              apps that spawn a few helper processes that are not
              typically  do  not use much cpu time.  (Hint: Intel
              OpenMP  applications  currently  should  be  placed
              using  -x6. This could change in future versions of
              OpenMP).

              directives  that are used to control process place­
              ment. [TBD]

       -q     List the global count of the number of active  pro­
              cesses  that  have  been placed (by dplace) on each
              cpu in the current cpuset. Note  that  cpu  numbers
              are  logical  cpu  numbers  within  the cpuset, NOT
              physical cpu numbers.


EXAMPLES

       The following examples assume the command is executed from
       a shell running in a cpumemset consisting of physical cpus
       8-15.

       To execute a process on a specific logical cpu(s):

               dplace -c2 date         # date runs on physical cpu 10.

               dplace make linux       # gcc & related processes run on
                                       # physical cpus 8-15.

               dplace -c0-4,6 make linux       # gcc & related processes run on
                                               # physical cpus 8-12 or 14.

               runon 4-7 dplace app    # runon restricts execution to
                                       # physical cpus 12-15. dplace
                                       # binds processes to cpus 12-15.


MPI EXAMPLE

       Most MPI applications are launched by mpirun and  use  N+1
       threads.  The  first thread is mainly inactive and usually
       does not need to be placed.

       To launch an MPI application use the following syntax:

               mpirun -np <process_count> dplace [dplace_args] app [args]

       Example:

               mpirun -np 8 dplace -s1 lu.8


BUGS

       The  placement  file  is   not   implemented   yet.    The
       <skip_mask> is a kludge. A better solution is needed.

       Dplace  depends  on  a loadable kernel module named "numa­
       tools". If this modules is not loaded, dplace will fail  &
       print a message remind the user to load the numatools mod­
       ule.


SEE ALSO

       cpumemsets(1), dlook(1), runon(1)


Man(1) output converted with man2html