nhfsstone - Network File System benchmark program
SYNOPSIS
nhfsstone [ -v ] [[ -t secs ] | [ -c calls ]] [ -l load ]
[ -p nprocs ] [ -m mixfile ] [ dir ]...
DESCRIPTION
nhfsstone (pronounced n-f-s-stone, the "h" is silent) is
used on a NFS client to generate an artificial load with a
particular mix of NFS operations. It reports the average
response time of the server in milliseconds per call and
the load in calls per second. The program adjusts its
calling patterns based on the client's kernel NFS statis
tics and the elapsed time. Load can be generated over a
given time or number of NFS calls.
Because it uses the kernel NFS statistics to monitor its
progress, nhfsstone cannot be used to measure the perfor
mance of non-NFS filesystems.
The nhfsstone program uses file and directory manipulation
in an attempt to generate particular NFS operations in
response to particular system calls. To do this it uses
several tricks that are based on a knowledge of the imple
mentation of the NFS client side reference port. For
example, it uses long file names to circumvent the kernel
name lookup cache so that a stat(2) system call generates
an NFS lookup operation.
The mix of NFS operations can be set with a mix file,
which is the output of the nfsstat(8C) command (see the
"-m" option below). The percentages taken from the mix
file are calculated based on the number of NFS calls, not
on the percentages printed by nfsstat. Operations with 0%
in the mix will never get called by nhfsstone. In a real
server load mix, even though the percentage of call for a
particular NFS operation may be zero, the number of calls
is often nonzero. Nhfsstone makes the assumption that the
number of calls to these 0 percent operations will have an
insignificant effect on server response.
Normally nhfsstone should be given a list of two or more
test directories to use (default is to use the current
directory). The test directories used should be located
on different disks and partitions on the server to realis
tically simulate typical server loads. Each nhfsstone
process looks for a directory <dir>/testdir<n> (where <n>
is a number from 0 to nprocs - 1). If a process directory
name already exists, it is checked for the correct set of
test files. Otherwise the directory is created and popu
lated.
-v Verbose output.
-t secs Sets calls based on the given running time (in
seconds) and the load.
-c calls Total number of NFS calls to generate (default
is 5000).
-l load Load to generate in NFS calls per second
(default is 30).
-p nprocs Number of load generating sub-processes to
fork (default is 7). This can be used to max
imize the amount of load a single machine can
generate. On a small client machine (slow CPU
or small amount of memory) fewer processes
might be used to avoid swapping.
-m mixfile Mix of NFS operations to generate. The format
of mixfile is the same as the output of the
nfsstat(8C) program. A mix file can be cre
ated on a server by typing "nfsstat -s > mix
file". The default mix of operations is: null
0%, getattr 13%, setattr 1%, root 0%, lookup
34%, readlink 8%, read 22%, wrcache 0%, write
15%, create 2%, remove 1%, rename 0%, link 0%,
symlink 0%, mkdir 0%, rmdir 0%, readdir 3%,
fsstat 1%.
USING NHFSSTONE
As with all benchmarks, nhfsstone can only provide numbers
that are useful if experiments that use it are set up
carefully. Since it is measuring servers, it should be
run on a client that will not limit the generation of NFS
requests. This means it should have a fast CPU, a good
ethernet interface and the machine should not be used for
anything else during testing. A Sun-3/50 can generate
about 60 NFS calls per second before it runs out of CPU.
Nhfsstone assumes that all NFS calls generated on the
client are going to a single server, and that all of the
NFS load on that server is due to this client. To make
this assumption hold, both the client and server should be
as quiescent as possible during tests.
If the network is heavily utilized the delays due to col
lisions may hide any changes in server performance. High
error rates on either the client or server can also cause
delays due to retransmissions of lost or damaged packets.
netstat(8C) -i can be used to measure the error and colli
sion rates on the client and server.
To best simulate the effects of NFS clients on the server,
at least two of the disk partitions that the server
exports and the partitions should be as far apart as pos
sible. The dkinfo(8) command can be used to find the phys
ical geometry of disk on BSD based systems. NFS opera
tions tend to randomize access the whole disk so putting
all of the nhfsstone test directories on a single parti
tion or on two partitions that are close together will not
show realistic results.
On all tests it is a good idea to run the tests repeatedly
and compare results. The number of calls can be increased
(with the -c option) until the variance in milliseconds
per call is acceptably small. If increasing the number of
calls does not help there may be something wrong with the
experimental setup. One common problem is too much memory
on the client test machine. With too much memory, nhfs
stone is not able to defeat the client caches and the NFS
operations do not end up going to the server at all. If
you suspect that there is a caching problem you can use
the -p option to increase the number of processes.
The numbers generated by nhfsstone are most useful for
comparison if the test setup on the client machine is the
same between different server configurations. Changing
nhfsstone parameters between runs will produce numbers
that can not be meaningfully compared. For example,
changing the number of generator processes may affect the
measured response time due to context switching or other
delays on the client machine, while changing the mix of
NFS operations will change the whole nature of the experi
ment. Other changes to the client configuration may also
effect the comparability of results. While nhfsstone
tries to compensate for differences in client configura
tions by sampling the actual NFS statistics and adjusting
both the load and mix of operations, some changes are not
reflected in either the load or the mix. For example,
installing a faster CPU or mounting different NFS filesys
tems may effect the response time without changing either
the load or the mix.
To do a comparison of different server configurations,
first set up the client test directories and do nhfsstone
runs at different loads to be sure that the variability is
reasonably low. Second, run nhfsstone at different loads
of interest and save the results. Third, change the server
configuration (for example, add more memory, replace a
disk controller, etc.). Finally, run the same nhfsstone
loads again and compare the results.
SEE ALSO
The nhfsstone.c source file has comments that describe in
detail the operation of of the program.
illegal calls value
The calls argument following the -c flag on the
command line is not a positive number.
illegal load value
The load argument following the -l flag on the com
mand line is not a positive number.
illegal time value
The time argument following the -t flag on the com
mand line is not a positive number.
bad mix file
The mixfile file argument following the -m flag on
the command line could not be accessed.
can't find current directory
The parent process couldn't find the pathname of
the current directory. This usually indicates a
permission problem.
can't fork
The parent couldn't fork the child processes. This
usually results from lack of resources, such as
memory or swap space.
can't open log file
can't stat log
can't truncate log
can't write sync file
can't write log
can't read log
A problem occurred during the creation, truncation,
reading or writing of the synchronization log file.
The parent process creates the log file in /tmp and
uses it to synchronize and communicate with its
children.
can't open test directory
can't create test directory
can't cd to test directory
wrong permissions on test dir
can't stat testfile
wrong permissions on testfile
can't create rename file
can't create subdir
A child process had problems creating or checking
the contents of its test directory. This is usually
due to a permission problem (for example the test
directory was created by a different user) or a
full filesystem.
bad mix format: can't find 'calls' value
bad mix format: unexpected EOF after 'calls'
bad mix format: can't find %d op values
bad mix format: unexpected EOF
A problem occurred while parsing the mix file. The
expected format of the file is the same as the out
put of the nfsstat(8C) command when run with the
"-s" option.
op failed:
One of the internal pseudo-NFS operations failed.
The name of the operation, e.g. read, write,
lookup, will be printed along with an indication of
the nature of the failure.
select failed
The select system call returned an unexpected
error.
BUGS
Running nhfsstone on a non-NFS filesystem can cause the
program to run forever because it uses the kernel NFS
statistics to determine when enough calls have been made.
Nhfsstone uses many file descriptors. The kernel on the
client may have to be reconfigured to increase the number
of available file table entries.
Shell scripts that used nhfsstone will have to catch and
ignore SIGUSR1 (see signal(3)). This signal is used to
synchronize the test processes. If the signal is not
caught the shell that is running the script will be
killed.
FILES
/vmunix system namelist
/dev/kmem kernel virtual memory
./testdir* per process test directory
/tmp/nhfsstone%d process synchronization log file
Man(1) output converted with
man2html