IInnssttaalllliinngg IInntteerrNNeettNNeewwss _R_i_c_h _$_a_l_z _U_p_d_a_t_e_d _b_y_: _J_a_m_e_s _B_r_i_s_t_e_r Internet Software Consortium _p_l_e_a_s_e _s_e_n_d _e_l_e_c_t_r_o_n_i_c _m_a_i_l _t_o _<_i_n_n_@_i_s_c_._o_r_g_> _A_B_S_T_R_A_C_T This document discusses how to install and set up InterNetNews. You should be familiar with Usenet and networks; the first section gives ref- erences to documentation for these topics, and the last appendix gives a Usenet overview for novices. This document also describes what many of the programs do and how they should be used. Even if you are a world-class expert at building and main- taining public software, you should probably read this. This is revision 1.19, dated 1996/11/10. 11.. TThhiinnggss YYoouu SShhoouulldd KKnnooww BBeeffoorree YYoouu DDoo AAnnyytthhiinngg InterNetNews is abbreviated _I_N_N, which is pronounced as the three letters, _e_y_e _e_n _e_n. It is a Usenet transport and expiration system for larger UNIX|- systems where NNTP is used for most Usenet traffic. This document is not a tutorial on Usenet. If you do not have much Usenet experience, you should read _U_s_i_n_g _U_U_C_P _a_n_d _U_s_e_n_e_t, ISBN 0-937175-10-2. You might also find it use- ful to read _M_a_n_a_g_i_n_g _U_U_C_P _a_n_d _U_s_e_n_e_t (get the most recent edition available), ISBN 0-937175-48-X. Both books are pub- lished by O'Reilly & Associates; send inquiries to to . There is a chapter on INN in _T_h_e _I_n_t_e_r_n_e_t _C_o_n_n_e_c_t_i_o_n_: _A _G_u_i_d_e _t_o _C_o_n_n_e_c_t_i_v_i_t_y _a_n_d _C_o_n_f_i_g_u_r_a_t_i_o_n, ISBN 0-201-54237-4 by Smoot Carl-Mitchell and John S. Quarterman. It is being published by Addison-Wesley and will be avail- able in early 1994 or at the end of 1993. You should know BSD-derived TCP/IP -- at least be com- fortable with host names and dotted-quad addresses. If you ----------- |- UNIX is a registered trademark of Unix Systems Laboratories. February 14, 1992 -2- have installation problems, you should know about UNIX- domain stream and datagram sockets and the like. In addi- tion to any documentation available from your vendor, you might find it useful to read the two IPC tutorials in _U_N_I_X _P_r_o_g_r_a_m_m_e_r_'_s _M_a_n_u_a_l_: _S_u_p_p_l_e_m_e_n_t_a_r_y _D_o_c_u_m_e_n_t_s _1. Copies can be purchased from the Usenix Association; send inquiries to . There are two RFCs that are important to InterNetNews. RFC 1036 describes the format of Usenet articles. It is incomplete and has some errors, but it is the only formal document available. RFC 977 defines NNTP, the Network News Transfer Protocol. RFCs are available from several places, including anonymous FTP to nnsc.nsf.net, where they can be found in the directory _r_f_c. Both RFCs are currently being revised. The 1036 revision is most likely going to be a ``tightening-up''; since INN already has a strict interpre- tation of the RFC, this revision will probably not affect InterNetNews very much. The 977 revision is adding new fea- tures and facilities, and while INN will not provide all of them, they will have some impact. InterNetNews does things differently from other news software. The most common Usenet systems for UNIX are B2.11 and C News. Both of them require a separate NNTP implemen- tation. The one everyone uses is called ``NNTP.'' Because this is confusing (they don't call _s_e_n_d_m_a_i_l ``SMTP''), I will refer to it as the ``reference implementation.'' You generally do not need to know anything about these other systems, but if you are curious, the official sites are as follows: Package Host Directory C News ftp.cs.toronto.edu pub/c-news B2.11 ftp.uu.net news/bnews-2.11 nntp ftp.academ.com public/nntp You might find the files _d_o_c_/_b_i_b_l_i_o, _d_o_c_/_p_r_o_b_l_e_m_s, and _d_o_c_/_r_f_c_e_r_r_a_t_a in the C News distribution worthwhile reading. The first is a bibliography, the second discusses known C News porting problems (see the DBZ sections in particular, and ignore most of the shell comments), while the third lists some technical and philosophical errors in RFC 1036. The commands below assume that _$_i_n_n is an abbreviation for the top of the InterNetNews source tree. INN could not have been written without access to the freely-redistributable sources of B2.11, C News, and NNTP. In particular, I want to thank Rick Adams; Geoff Collyer and Henry Spencer; and Stan Barber, Erik Fair, Brian Kantor, and Phil Lapsley. The financial support of UUNET Technologies is also greatly appreciated. The beta-test sites gave invaluable feedback. February 14, 1992 -3- 22.. IIff YYoouu AArree IImmppaattiieenntt If you don't want to follow these directions, do the following: cd $inn/config cp config.dist config.data chmod 644 config.data vi config.data (or emacs, or whatever) cd .. make world mkdir -p /usr/news /var/news /var/news/spool make install Then go read the appendixes. If you have any problems, read the rest of this document. 33.. DDiissttrriibbuuttiioonn RRooaaddmmaapp The INN sources are divided into the following directo- ries: _f_r_o_n_t_e_n_d_s Programs to feed articles to the central server and control it. _i_n_n_d The central NNTP server. It accepts incoming connections, receives articles, and arranges for them to be sent to downstream newsfeeds. _b_a_c_k_e_n_d_s Programs to transmit articles to other sites. _e_x_p_i_r_e Programs to purge the article files and his- tory database. _n_n_r_p_d An NNTP server for on-campus clients that do newsreading (as opposed to bulk article transfer). _l_i_b Library routines used by all the above. _i_n_c_l_u_d_e Header files used by all the above. The distribution also includes these directories: _s_a_m_p_l_e_s Prototype scripts and configuration files that might have to be edited before they are installed. _s_i_t_e A place to store edited copies of the files in the _s_a_m_p_l_e_s directory. _d_o_c Manual pages for all the above. February 14, 1992 -4- _c_o_n_f_i_g Tools to configure the release for your site. Finally, there are a handful of files in the top-level directory: _R_E_A_D_M_E A basic introduction. _C_O_P_Y_R_I_G_H_T The distribution copyright. InterNetNews is freely redistributable, provided proper credit is given. _M_A_N_I_F_E_S_T A one-line description of every file in the distribution. _B_U_I_L_D An interactive script to configure, build, and install INN. _m_a_k_e_d_i_r_s_._s_h A script to build INN's directories except for the top levels: /usr/news /var/news and /var/news/spool. As long as you have write permission to install the programs, this is the only part of the installation that needs to be done as root, except that inndstart needs to be installed setuid root, and so doing the install as root will make life eas- ier. _M_a_k_e_f_i_l_e Rules to call the other Makefiles and make distributions. _I_n_s_t_a_l_l_._m_s This document. It requires the ``-ms'' nroff/troff macro package. _M_a_k_e_L_i_b Script to build a directory with a replace- ment of the reference implementation's ``clientlib'' routines needed by remote _r_n. _M_a_k_e_I_n_e_w_s Script to build an _i_n_e_w_s distribution direc- tory. _M_a_k_e_R_n_e_w_s Script to build an _r_n_e_w_s distribution direc- tory. _s_e_d_f_._x_x_x Various _s_e_d scripts to filter the output of _l_i_n_t. 44.. BBuuiillddiinngg tthhee SSyysstteemm INN is built in steps. First, the _s_u_b_s_t program is built. Next, a configuration file containing key/value pairs is created. _S_u_b_s_t reads this file and uses it to edit a specific set of files in the INN distribution. (Most of the files that get modified are Makefiles or header files.) The library is then built; _l_i_n_t is usually a good way to see February 14, 1992 -5- if some of the basic configuration parameters are set up right. The next step is to compile (and lint) all the pro- grams. The programs are then installed, and the INN data files are set up. The configuration process is deliberately not interac- tive. Configure scripts like the one in _r_n are fun to watch, but they spend too much effort on the wrong job, like whether _g_r_e_p returns an exit status. It is also difficult to change one parameter and rebuild the software. (C News has this same problem.) INN's method also has its flaws. Because almost all configuration data is in one header file, changing almost anything will force everything to be recompiled. 44..11.. BBuuiillddiinngg ssuubbsstt INN uses the C News _s_u_b_s_t program to automate the con- figuration. _S_u_b_s_t is a very clever way of safely editing a file under the control of a configuration file. For more details, see the documentation in _d_o_c_/_s_u_b_s_t_._1. Thanks to Henry Spencer and Geoff Collyer for their permission to use and redistribute _s_u_b_s_t. We will use _c_o_n_f_i_g_._d_i_s_t as the configuration file to test the version of _s_u_b_s_t that you build. (You can always replace your config file with the distribution file and do another _m_a_k_e to restore the original versions.) The C News _s_u_b_s_t program is a shell script that uses _s_e_d to do the editing. The INN configuration file is too large for some versions of _s_e_d. The first step is to see if your _s_e_d will work. To do this, type the following: cd $inn/config cp config.dist config.data make sedtest If you get any error messages from _s_e_d such as ``too much command text'' (or if it dumps core) you have two choices. (You should also complain to your vendor.) One choice is to use another version of _s_e_d, such as the one distributed by the Free Software Foundation. If you do this, edit _c_o_n_- _f_i_g_/_M_a_k_e_f_i_l_e and change the line that defines the SED vari- able. If you want to use the C News script, then do the following: cd $inn/config make sh The other choice is to use the C version of _s_u_b_s_t. You might want to do this anyway, since it can be much faster. February 14, 1992 -6- To do this, type the following: cd $inn/config cp config.dist config.data make c quiet If you get any compilation errors, you will have to edit the file _c_o_n_f_i_g_/_s_u_b_s_t_._c. If you are using an early version of AFS, you will have edit the file to enable the USE_RENAME macro. If you have to make any other changes, please let me know. Since _s_u_b_s_t changes source files, you might want to make a backup copy of all the files that will be modified. You can do this by typing ``make backup'' in the _c_o_n_f_i_g directory. This will create a local tar file that contains all the files that will be modified into it. Doing ``make restore'' will unpacks the tar file. (Since _s_u_b_s_t makes its changes safely, this step is optional.) 44..22.. EEddiittiinngg ccoonnffiigg..ddaattaa Once you have _s_u_b_s_t working, the next step is to set up your configuration parameters. This is the hardest part of installing INN. _D_o_n_'_t _p_a_n_i_c_! There are many configuration parameters, but it should be very easy for you to determine the answer for most of them. To do this, you should copy _c_o_n_f_i_g_/_c_o_n_f_i_g_._d_i_s_t, the distribution master, to _c_o_n_f_i_g_/_c_o_n_- _f_i_g_._d_a_t_a, your local copy. INN is distributed to compile and run under BSD/OS 2.1 by default. The configuration file is divided into the following sections: Make config parameters Logging levels Ownerships and file modes C library differences C library omissions Miscellaneous config data Paths to common programs Paths related to the spool directory Execution paths for innd and rnews Sockets created by innd or clients Log and config files Innwatch configuration Tcl filtering configuration PGP control-message verification configuration Local configuration Actsync configuration You should have a copy of _c_o_n_f_i_g_._d_a_t_a nearby as you read the next few sections. It is probably a good idea to write down your changes on paper before you edit the file. February 14, 1992 -7- The format of the file is very strict. A line starting with a poundsign is a comment line. All other lines must be in this format: parameter _<_o_n_e_-_o_r_-_m_o_r_e_-_t_a_b_s_> value If there is no ``value'' the ``'' is still required. Do not put quote marks around the values -- if you do, you will usually get a syntax error while compiling the system. The discussion below uses quotes only to show where the values start and end. 44..22..11.. MMaakkee ccoonnffiigg ppaarraammeetteerrss This section is used primarily to identify the path to your C compiler, and what extra libraries or command-line switches are needed. For example, you could put _g_c_c _-_W_a_l_l on the _C_C line. If you need extra _-_I flags put them on the _D_E_F_S line. INN uses the _r_e_g_i_s_t_e_r declaration a great deal. If your compiler is very good, you might want to add _-_D_r_e_g_- _i_s_t_e_r_= to the _D_E_F_S line so that INN's declarations are ignored. The DBZ package can be compiled so that the database is memory-mapped. If you want to do this and have the _m_m_a_p system call, then add ``-DMMAP'' to the _D_B_Z_C_F_L_A_G_S parameter. If you need to link in other libraries (e.g., _-_l_n_e_t) put them on the _L_I_B_S line. The Makefiles usually filter all _l_i_n_t output through a _s_e_d script. If you are very paranoid, set _L_I_N_T_F_I_L_T_E_R to _c_a_t. If your lint output is in the broken multi-line for- mat: value type declared inconsistently exit llib-lc(297) :: test.c(7) function returns value which is always ignored printf Then set _L_I_N_T_F_I_L_T_E_R to be the ``sedf.sysv'' line. The _l_i_b directory also builds a _l_i_n_t library, so that you can make sure the other programs are properly using the library routines. The _L_I_N_T_L_I_B_S_T_Y_L_E parameter (used in _l_i_b_/_M_a_k_e_f_i_l_e and _l_i_b_/_m_a_k_e_l_l_i_b_._s_h) controls how the _l_i_n_t library is built. If your _l_i_n_t understands the ``-C'' flag, then set it to ``BSD''. If you need the ``-o'' flag to build a library, then set it to ``SYSV''. If neither of these work, you can set it to ``NONE''; this will just cre- ate an empty file so that the other Makefiles don't break. If you come up with a fourth alternative, let me know. February 14, 1992 -8- Unfortunately, on some systems _l_i_n_t is all but useless, so complain to your vendor and take the output with a grain of salt. You might get some warnings about ``struct _DDHAN- DLE'' being undefined. You can ignore them and ask your vendor to support the BSD ``-z'' lint flag. If you set _H_A_V_E___U_N_I_S_T_D to ``DO'' then you might get warnings about pro- totype mismatches for various functions declared in _i_n_c_l_u_d_e_/_c_l_i_b_r_a_r_y_._h. You can ignore them or remove the lines from the INN header file. The _M_A_N_P_A_G_E_S_T_Y_L_E parameter (used in _d_o_c_/_M_a_k_e_f_i_l_e and _d_o_c_/_p_u_t_m_a_n_._s_h) controls how manual pages are installed into your public directory while the _M_A_N_x parameters specify the directories where they get installed. If you do not want to install any manpages, set _M_A_N_P_A_G_E_S_T_Y_L_E to _N_O_N_E. 44..22..22.. LLooggggiinngg lleevveellss INN uses the modern _s_y_s_l_o_g that separates messages into both levels and categories. Look in your _<_s_y_s_l_o_g_._h_> header file for a ``LOG_NEWS'' macro, and check your _s_y_s_l_o_g(3) man- page to make sure that _o_p_e_n_l_o_g takes three arguments. If it doesn't, then you will have to use the library routine and server provided in the _s_y_s_l_o_g directory. This is described later. The different levels that are described in the _s_y_s_- _l_o_g(3) manpage are confusing, so INN uses its own names for the four levels it uses: L_FATAL Fatal error, about to exit L_ERROR Error that might require attention L_NOTICE Informational notice, no action needed L_DEBUG Protocol tracing or other debugging messages Depending on how your _s_y_s_l_o_g_._c_o_n_f(5) file is set up, you might want to change the _L___x_x_x parameters in this section. The _s_c_a_n_l_o_g_s script assumes that the first three cate- gories above are each directed into separate files. See _d_o_c_/_n_e_w_s_l_o_g_._5, _d_o_c_/_n_e_w_s_l_o_g_._8, and _s_y_s_l_o_g_/_s_y_s_l_o_g_._c_o_n_f for details. Logging is also described in more detail later. 44..22..33.. OOwwnneerrsshhiippss aanndd ffiillee mmooddeess The NNTP server needs to open the NNTP port; it is port number 119, which requires root access. This is the only part of INN that needs this privilege: all other programs can run under the distinct user and group id specified by the _N_E_W_S_U_S_E_R and _N_E_W_S_G_R_O_U_P parameters. Most news adminis- tration tasks must be done as user _N_E_W_S_U_S_E_R (see the expla- nation of _c_t_l_i_n_n_d below). In addition, _i_n_e_w_s will only let the _N_E_W_S_U_S_E_R user or members of the _N_E_W_S_G_R_O_U_P group post control messages other than cancel. February 14, 1992 -9- Some INN scripts (primarily the control message scripts and the daily maintenance script) need to send email to the news maintainer. The _N_E_W_S_M_A_S_T_E_R parameter specifies the right address. This is most often the login name of the account which has _N_E_W_S_U_S_E_R as its user id; use an alias to forward it to the right people. Some Usenet sites still use the Path header line to generate their email reply messages. Using the Path has never been guaranteed to work, and INN tries to help stop this practice by refusing to generate valid Path addresses. The _P_A_T_H_M_A_S_T_E_R parameter specifies what _i_n_e_w_s should put at the tail end of the Path line. If your _N_E_W_S_M_A_S_T_E_R mailbox is getting cluttered, then you might want to change this to be an alias that rejects the message or drops it into the bit-bucket. The default value is ``not-for-mail'' which usually results in bounced email. The _x_x_x___M_O_D_E parameters specify the permissions for articles and directories created within the spool area, and the active file, all of which are owned by user id _N_E_W_S_U_S_E_R. 44..22..44.. CC lliibbrraarryy ddiiffffeerreenncceess Editing the parameters in this section will require you to look around at the files in your _/_u_s_r_/_i_n_c_l_u_d_e directory. The _S_I_Z_E___T parameter is the datatype of the ``size'' parameters in subroutine calls like _m_e_m_c_h_r and _f_r_e_a_d. The _L_O_C_K___S_T_Y_L_E parameter specifies how file-locking should be done. _I_n_n_x_m_i_t is the only program that locks files; if you use the provided scripts, this isn't even necessary, so you can set this to ``NONE'' if you have any compile problems. The _D_I_R___S_T_Y_L_E parameter specifies what is returned by your _r_e_a_d_d_i_r(3) routine. This will be either a ``struct direct'' or a ``struct dirent''; set the parameter to ``DIRECT'' or ``DIRENT'' as appropriate. If you do not have UNIX-domain sockets, set _H_A_V_E___U_N_I_X___D_O_M_A_I_N to ``DONT.'' This means that INN will use a named pipe for the communication between _i_n_n_d and _c_t_l_i_n_n_d. It also means that there will be no local ``private'' port for _r_n_e_w_s to use; this should not cause any problems, although it makes it easier for anyone to use _r_n_e_w_s and post fake news articles. (You might also have to modify the _s_y_s_- _l_o_g routines; see the end of the file _s_y_s_l_o_g_/_R_E_A_D_M_E for details on this.) INN needs to know how many descriptors are available to use for files and sockets. There are several ways to get this number; the _F_D_C_O_U_N_T___S_T_Y_L_E parameter specifies which method to use. On most systems, the _g_e_t_d_t_a_b_l_e_s_i_z_e routine will do this, so leave the default of ``GETDTAB.'' On other February 14, 1992 -10- systems you need to use the _g_e_t_r_l_i_m_i_t, _s_y_s_c_o_n_f or _u_l_i_m_i_t routine, so set the parameter to ``GETRLIMIT'', ``SYSCONF'', or ``ULIMIT'', respectively. If you do not have any of those calls then set the parameter to ``CONSTANT'' and edit the file _l_i_b_/_g_e_t_d_t_a_b_._c to return the right number. To get this number, look for an _O_P_E_N___M_A_X constant in your system header files, or write a program that repeatedly opens _/_d_e_v_/_n_u_l_l until it gets an error. The last few parameters in this section, _x_x_x_V_A_L, are used primarily to keep _l_i_n_t quiet. These functions are declared in _i_n_c_l_u_d_e_/_c_l_i_b_r_a_r_y_._h, and the return values are pretty much always ignored. You can usually determine what these values should be by examining your manpages or your _l_i_n_t libraries. 44..22..55.. CC lliibbrraarryy oommiissssiioonnss INN uses library routines that might not be present in all UNIX systems, although they should be. The _l_i_b direc- tory provides versions of some of these routines, including copies of the freely-redistributable BSD string routines. The _M_I_S_S_I_N_G___S_R_C and _M_I_S_S_I_N_G___M_A_N parameters can be set to list those routines that are missing from your C library. If you don't have _s_t_r_c_a_s_e_c_m_p and _s_t_r_n_c_a_s_e_c_m_p then you will need the _s_t_r_c_a_s_e_c_m_p module built into your INN library. Add the ``.c'' and ``.o'' names to _M_I_S_S_I_N_G___S_R_C and _M_I_S_S_I_N_G___O_B_J, respectively. The following routines are all found in the file of the same name. If they are missing from your system, add them the same way: memchr strchr getopt memcmp strrchr mkfifo memcpy strspn strerror memset strtok memmove If you are using version 1 of the GNU C compiler on a Sparc running SunOS, you should add _i_n_e_t___n_t_o_a as a missing function. This is because the first version of _g_c_c didn't properly pass structures into routines compiled with the Sun C compiler. If you have an older version of _s_y_s_l_o_g add _s_y_s_l_o_g_._c and _s_y_s_l_o_g_._o to the appropriate parameters. Pyramid machines running OSx have fast assembly-lan- guage versions of the string routines in the ATT library. To use these routines, add ``$(OSXATTOBJ)'' to the _M_I_S_S_- _I_N_G___O_B_J_S parameter. This will cause _l_i_b_/_M_a_k_e_f_i_l_e to extract the object files from the ATT library, and add them to the February 14, 1992 -11- INN library. 44..22..66.. MMiisscceellllaanneeoouuss ccoonnffiigg ddaattaa All the parameters in this section become macros in the file _i_n_c_l_u_d_e_/_c_o_n_f_i_g_d_a_t_a_._h. You should at least look through the parameters up to _V_E_R_I_F_Y___C_A_N_C_E_L_S. (If set to ``DO'', then _i_n_n_d will ignore cancel messages unless the From or Sender header match those of the original poster.) In gen- eral, however, you can leave this section pretty much alone until you have some experience running INN. Nevertheless, here are some comments on some of the more useful parame- ters. _I_n_n_d can memory-map the _a_c_t_i_v_e file if you set _A_C_T___S_T_Y_L_E to ``MMAP''. On some systems, however, when a mapped file is updated its mtime is not updated. Apparently some versions of System V Release 4 have this problem. This causes problems for programs like _n_n_m_a_s_t_e_r which look at the _s_t___m_t_i_m_e field of the _s_t_a_t structure in order to determine if any new news has come in. (_N_n_m_a_s_t_e_r is part of the _n_n newsreading program.) The best work-around is probably an hourly _c_r_o_n job that touches the _a_c_t_i_v_e file. There are a number of parameters that control the behavior of _r_n_e_w_s. If you set _R_N_E_W_S___S_A_V_E___B_A_D to ``DO'' then articles that _i_n_n_d rejects for reasons like bad headers will be saved in the ___P_A_T_H___B_A_D_N_E_W_S directory; you will have to periodically scan this directory and clean it up. You can also control how _r_n_e_w_s logs duplicates (those aren't saved regardless of the value of _R_N_E_W_S___S_A_V_E___B_A_D), logging them through _s_y_s_l_o_g, to a file, or not. Note that if you set _R_N_E_W_S___L_O_G___D_U_P_S to ``FILE'', then you will want to change ___P_A_T_H___R_N_E_W_S___D_U_P___L_O_G, which appears later in the file. If you receive news from several UUCP feeds, you might want to log duplicates so that you can cut down your phone bills by optimizing your feeds. The _R_N_E_W_S_P_R_O_G_S parameter says whether or not to look in ___P_A_T_H___N_E_W_S_P_R_O_G_S for commands named on the incoming ``#!'' line of news batches. You probably want to set this to ``DO''. Make sure that the full path- name of _r_n_e_w_s, ___P_A_T_H___R_N_E_W_S, does not conflict with the directory where your unpackers are put, ___P_A_T_H___N_E_W_S_P_R_O_G_S. If _I_P_A_D_D_R___L_O_G is set to ``DO'' then the news log will report the IP address of hosts that send articles, rather then what they put in the Path line. This can be useful if you run _i_n_n_d with the ``-a'' flag. (If you do this, you might want to pick up ``hf.tar.Z'' via anonymous FTP to ee.lbl.gov; it is a filter that turns IP addresses into host names.) The _x_x_x___T_I_M_E_O_U_T parameters control various timers within INN; you might want to change some of these depending on your system load. February 14, 1992 -12- 44..22..77.. PPaatthhss ttoo ccoommmmoonn pprrooggrraammss INN uses a few standard programs like _/_b_i_n_/_s_h and _s_e_n_d_- _m_a_i_l. If you don't have _s_e_n_d_m_a_i_l then you must have a pro- gram that accepts a full message -- including headers -- on its standard input, and delivers it. This program is speci- fied by the ___P_A_T_H___S_E_N_D_M_A_I_L parameter, and is normally ``/usr/lib/sendmail -t''. The parameter is actually a _s_p_r_i_n_t_f format string that will be given the destination address as its only argument. on some sites (e.g., those running MMDF) the ``-t'' could be replaced with a ``%s''. INN puts most of its executables in the directory spec- ified by the ___P_A_T_H___N_E_W_S_B_I_N parameter. Some programs expect _i_n_e_w_s and _r_n_e_w_s to be in certain places; for example, UUCP usually wants _r_n_e_w_s in _/_b_i_n. The default configuration puts these programs in only one spot; if you need to have multi- ple links to the same file, you will have to do it yourself after INN is installed. If you have additional scripts and programs that you use to maintain your system, you can put them in whatever directory you want. You will probably need to add ___P_A_T_H___N_E_W_S_B_I_N to the PATH of any such scripts. If you have an _/_e_t_c_/_r_c_._l_o_c_a_l file you should make sure that it invokes the script named by the ___P_A_T_H___N_E_W_S_B_O_O_T parameter. On other systems (mostly System V derivatives), the system boot procedure automatically runs all the scripts in a particular directory, such as _/_e_t_c_/_i_n_i_t_._2. In that case, you should pick a name like _/_e_t_c_/_i_n_i_t_._2_/_S_9_9_n_e_w_s and have the news boot script installed there, or install it in the default _/_e_t_c_/_r_c_._n_e_w_s and make the link yourself. The daily maintenance script, _n_e_w_s_._d_a_i_l_y calls _s_c_a_n_l_o_g_s to rotate and trim log files, as well as generating sum- maries using _e_g_r_e_p and _a_w_k. On some systems the log files are too big for these programs so you might have to complain to your vendor and install the versions from the Free Soft- ware Foundation. The _s_c_a_n_l_o_g_s script has a short test you can run to see if your _e_g_r_e_p will work. 44..22..88.. PPaatthhss rreellaatteedd ttoo tthhee ssppooooll ddiirreeccttoorryy By default, all news articles are stored in directories under _/_u_s_r_/_s_p_o_o_l_/_n_e_w_s. Be careful if you pick a different value -- many newsreaders know about this directory name. INN uses a trick (which I first saw in C News) that lets it use this same directory to store its incoming news (spooled by _r_n_e_w_s when _i_n_n_d is not available), and its out- going batch files. Since no newsgroup can ever have a dot in its name, a directory like _o_u_t_._g_o_i_n_g can never be a news- group name, and it is safe to put the news batchfiles in there. This is used by the ___P_A_T_H___S_P_O_O_L_N_E_W_S parameter, and the ___P_A_T_H___B_A_T_C_H_D_I_R parameter. February 14, 1992 -13- Do not make ___P_A_T_H___L_O_C_K_S be in the same filesystem as ___P_A_T_H___S_P_O_O_L_N_E_W_S. If you do this, then INN will not be able to create any lock files when your spool directory is full. This will probably mean that _n_e_w_s_._d_a_i_l_y will not be able to run and that it won't call _e_x_p_i_r_e to free up disk space. You should also put ___P_A_T_H___N_E_W_S_L_I_B on a separate partition if you can, but that is not as important because it tends to fill up less often. If you change parameters in this section a great deal, then there is a chance that the _m_a_k_e_d_i_r_s_._s_h script will fail because some needed intermediate directories will not exist. This should not be a problem, as you can just create the directories yourself -- make sure to set the ownership and modes right -- and re-run the script. 44..22..99.. EExxeeccuuttiioonn ppaatthhss ffoorr iinnnndd aanndd rrnneewwss All control messages (other then ``cancel'') are car- ried out by scripts. Your system must be able to _e_x_e_c shell scripts directly. All the scripts distributed with INN start with ``#! /bin/sh.'' The ___P_A_T_H___C_O_N_T_R_O_L_P_R_O_G_S parameter specifies the direc- tory where these scripts exist. Do not set this to a public directory like _/_b_i_n because some bozo might send out an ``rm'' control message. The ___P_A_T_H___R_N_E_W_S_P_R_O_G_S directory serves the same purpose for _r_n_e_w_s when it needs to unpack batches. The _R_N_E_W_S_P_R_O_G_S parameter specifies if the directory is really used. 44..22..1100.. SSoocckkeettss ccrreeaatteedd bbyy iinnnndd oorr cclliieennttss The _i_n_n_d server and its clients (most notably _c_t_l_i_n_n_d) create UNIX-domain sockets or named pipes. They are created inside a ``firewall'' directory that gives access permission to a limited set of users. For example, assume the direc- tory is _/_u_s_r_/_l_o_c_a_l_/_n_e_w_s_/_i_n_n_d and that it is owned by user news in group news and has mode 0770. Using these permis- sions, then only members of the news group can use _c_t_l_i_n_n_d to create new groups because only they will be able to send a message to the _i_n_n_d socket. This directory (which is specified by the ___P_A_T_H___I_N_N_D_D_I_R parameter) is also used to determine the user and group id of all sub-processes spawned by _i_n_n_d, as well as the owner of all news articles and files. The owner of this directory is set at installation time and specified in the ``Owner- ships and file modes'' section, above. February 14, 1992 -14- 44..22..1111.. LLoogg aanndd ccoonnffiigg ffiilleess INN keeps its databases, and some control files their own directory, typically named _/_u_s_r_/_l_o_c_a_l_/_n_e_w_s. Log files are kept in _/_v_a_r_/_l_o_g_/_n_e_w_s. There are many parameters in this section that refer to files within this directory. Some sites will want to globally replace ``/usr/local/news'' with something like ``/var/news'', and ``/usr/lib/newsbin'' with ``/var/newsbin'' There are two files that contain access passwords, ___P_A_T_H___N_N_T_P_P_A_S_S and ___P_A_T_H___N_N_R_P_A_C_C_E_S_S. The default location for these files is in _/_u_s_r_/_l_o_c_a_l_/_e_t_c, so that it is gener- ally safe to export _/_u_s_r_/_l_o_c_a_l_/_n_e_w_s (read-only is probably best). INN programs do extensive logging, and the daily main- tenance scripts do extensive summary reports and analysis of them. It might take you some time to learn your way around the INN logging system; start by reading the newslog man- pages in the _d_o_c directory. 44..22..1122.. IInnnnwwaattcchh ccoonnffiigguurraattiioonn The INN server, _i_n_n_d, does not contain any checks to see if there is enough free space on the disk or if the sys- tem load average is low enough to allow article reception. There are two reasons for this. The first reason is philo- sophical: it is a mistake to bury this kind of policy information inside a program. For example, you don't want to have to recompile the program just because you moved to a different filesystem. (Yes, this could be partially answered by moving the information to an external config file, but any compiled rules are still likely to be incom- plete.) The second reason is pragmatic: there is no portable way to get standard measurements for the informa- tion needed. For example, C News provides three different routines to get the filesystem statistics (with conditional compilation) while the ``get load average'' file in IDA sendmail has over 700 lines. Rather than get tangled up in such a mess of #ifdef's, INN uses an external program (shell script) that invokes _c_t_l_i_n_n_d to stop and start the server as necessary. The pro- gram, _i_n_n_w_a_t_c_h, reads the control file _i_n_n_w_a_t_c_h_._c_t_l. _I_n_n_w_a_t_c_h is documented in _d_o_c_/_n_e_w_s_._d_a_i_l_y_._8, while _i_n_n_w_a_t_c_h_._c_t_l is documented in _d_o_c_/_i_n_n_w_a_t_c_h_._c_t_l_._5. The parameters in this section control when the server should stop accepting articles, and when it should start again. You will have to examine _s_i_t_e_/_i_n_n_w_a_t_c_h_._c_t_l and prob- ably modify it, usually to check the amount of free space on the disks. For example, there is a line in the file that has this fragment in it: February 14, 1992 -15- !!! df . | awk 'NR == 2 { print $4 }' ! ... This is looking at the fourth field of the second line to get the amount of freespace. You will have to change the ``2'' and ``4'' here, and on other lines, as appropriate for your system. (Changing the output of _d_f seems to be one of the things vendors like to do most; it is not worth my time to have INN keep track of all of them.) The parameter _I_N_N_W_A_T_C_H___S_L_E_E_P_T_I_M_E specifies how fre- quently _i_n_n_w_a_t_c_h should check the system -- the other param- eters should be set with this in mind, eg: there needs to be enough free space on the filesystem to last the next _I_N_N_W_A_T_C_H___S_L_E_E_P_T_I_M_E seconds. The _I_N_N_W_A_T_C_H___x_x_x_L_O_A_D parameters specify the load aver- age at which different actions should be taken. They are integers, representing the load average multipled by 100. For example, if you want to throttle the server when your load reaches 7.5, set _I_N_N_W_A_T_C_H___H_I_L_O_A_D to ``750.'' The _I_N_N_W_A_T_C_H___x_x_x_S_P_A_C_E parameters specify the minimum amount of disk space needed for each of INN's three major filesystems. The numbers are in ``local units,'' equivalent to whatever your _d_f uses (512-byte units, 1K blocks, etc). The _I_N_N_W_A_T_C_H___S_P_O_O_L_N_O_D_E_S parameter specifies how many inodes must be available in your spool directory. 44..22..1133.. TTccll ffiilltteerriinngg ccoonnffiigguurraattiioonn.. The innd server can be configured to include TCL hooks to be run whenever a new article is received and when cer- tain other actions occur. See the file README.tcl_hook for more details. The _T_C_L___S_U_P_P_O_R_T parameter specifies whether you want to compile in the Tcl support code. Select DO or DONT. The _T_C_L___L_I_B parameter specifies the extra libraries you need to link against to include tcl support. Typically ``-ltcl -lm'' are required. If you defined _T_C_L___S_U_P_P_O_R_T to DONT, then this should be blank. The ___P_A_T_H___T_C_L___S_T_A_R_T_U_P parameter specifies the path of the tcl script that is to be loaded at process startup time. A sample version is included in samples/startup.tcl which will be installed in the location you define here. The ___P_A_T_H___T_C_L___F_I_L_T_E_R parameter specifies the path of the tcl script that is to be loaded upon command by ctlinnd. See the ctlinnd.8 manpage for more details (the ``filter'' and ``reload'' commands). A sample is included in sam- ples/filter.tcl which will be installed in the location February 14, 1992 -16- defined by this parameter. 44..22..1144.. PPGGPP ccoonnttrrooll mmeessssaaggee vveerriiffiiccaattiioonn.. Inn now has mecahnisms in place that will do PGP veri- fication of control messages. The _W_A_N_T___P_G_P_V_E_R_I_F_Y parameter defines whether PGP veri- fication of control messages should be done. Select DO or DONT. See the file README.pgp for more details on this sub- ject. 44..22..1155.. AAccttssyynncc ccoonnffiigguurraattiioonn.. Landon Curt Noll's actsync(8) program has been merged into INN. This section defines some variables for the default config file for actsync. See the man page for more details on actsync. The _A_C_T_S_Y_N_C___H_O_S_T variable defines which remote host you'll be using to synchronize your active file. 44..33.. TTyyppiiccaall ccoonnffiigg..ddaattaa cchhaannggeess The following sections show some of the changes that need to be made to _c_o_n_f_i_g_._d_a_t_a so that INN will compile. They are only samples; ``your mileage may vary.'' Note that if you are using the first release of _g_c_c_2, set _U_S_E___C_H_A_R___C_O_N_S_T to ``DONT''. -S-u-n-O-S--4-.-x- MISSING_SRC memmove.c MISSONG_OBJ memmove.o February 14, 1992 -17- -A-I-X- DEFS -I../include -D_NO_PROTO -U__STR__ FORK fork FREEVAL void FUNCTYPE int HAVE_ST_BLKSIZE DONT HAVE_TM_GMTOFF DONT LDFLAGS LINTFILTER | sed -n -f ../sedf.aix LINTFLAGS -wkD -b -h $(DEFS) LINTLIBSTYLE SYSV LOCK_STYLE FCNTL MISSING_MAN MISSING_SRC NEED_TIME DO POINTER void USE_UNION_WAIT DONT Under AIX 3.1, you must also use the _s_y_s_l_o_g that comes with INN. This is not necessary for 3.2. Some versions also need _U_S_E___U_N_I_O_N___W_A_I_T set to ``DONT''. You should also run _r_c_._n_e_w_s from _i_n_i_t not _/_e_t_c_/_r_c; add a line like the following near the end of the _i_n_i_t_t_a_b file, just before the ``cons'' line: rcnews:wait:2:/usr/local/etc/rc.news >/dev/console 2>&1 -A-/-U-X- LIBS -lbsd Make sure you don't use _g_c_c version 1; it miscompiles the socket calls in _i_n_n_d_/_c_c_._c. February 14, 1992 -18- -B-S-D-I- ABORTVAL void ALARMVAL u_int EXITVAL volatile void _EXITVAL volatile void FREEVAL void GETPIDVAL pid_t GID_T gid_t HAVE_UNISTD DO HAVE_VFORK DONT HAVE_WAITPID DO LSEEKVAL off_t MISSING_OBJ MISSING_SRC _PATH_COMPRESS /usr/bin/compress _PATH_EGREP /usr/bin/egrep _PATH_MAILCMD /usr/bin/Mail _PATH_SENDMAIL /usr/sbin/sendmail -t PID_T pid_t POINTER void QSORTVAL void SIZE_T size_t SLEEPVAL u_int UID_T uid_t USE_UNION_WAIT DONT VAR_STYLE STDARGS Change the _S_H_E_L_L variable in _c_o_n_f_i_g_/_M_a_k_e_f_i_l_e and _s_i_t_e_/_M_a_k_e_- _f_i_l_e to point to _/_u_s_r_/_c_o_n_t_r_i_b_/_b_i_n_/_b_a_s_h. Edit _l_i_b_/_M_a_k_e_f_i_l_e so that the _i_n_s_t_a_l_l target does not try to make _._._/_l_l_i_b_- _l_i_n_n_._l_n. You must also use the GNU _s_e_d; the version dis- tributed with BSDI 0.9.4.1 enters an infinite loop process- ing newgroup messages. February 14, 1992 -19- -H-P---U-X--8-.-0- ABORTVAL void ALARMVAL unsigned int CLX_STYLE FCNTL CTYPE isXXXXX((c)) DEFS -I../include -DHPUX FDCOUNT_STYLE SYSCONF FREEVAL void GETPIDVAL pid_t GID_T gid_t HAVE_SETBUFFER DONT HAVE_ST_BLKSIZE DONT HAVE_TM_GMTOFF DONT HAVE_UNISTD DO HAVE_WAITPID DO LINTFILTER | sed -n -f ../sedf.sysv LINTFLAGS -b -h $(DEFS) LINTLIBSTYLE SYSV LOCK_STYLE LOCKF LOG_INN_PROG LOG_LOCAL7 LOG_INN_SERVER LOG_LOCAL7 LSEEKVAL off_t _PATH_MAILCMD /usr/bin/mailx NOFILE_LIMIT 200 PID_T pid_t POINTER void PROF QSORTVAL void RANLIB echo RES_STYLE TIMES SIZE_T size_t SLEEPVAL unsigned int UID_T uid_t USE_UNION_WAIT DONT _EXITVAL void You will probably also need to use the _b_d_f command instead of _d_f. February 14, 1992 -20- -S-G-I--I-n-d-i-g-o--w-i-t-h--I-R-I-X--4-.-0-.-1- ABORTVAL void ALARMVAL uint ACT_STYLE MMAP CFLAGS $(DEFS) -g -w CLX_STYLE FCNTL _EXITVAL void FORK fork FREEVAL void GID_T gid_t HAVE_ST_BLKSIZE DONT HAVE_TM_GMTOFF DONT HAVE_UNISTD DO LDFLAGS LIBS -lmld LINTFILTER | sed -n -f ../sedf.sysv LINTFLAGS $(DEFS) LINTLIBSTYLE SYSV LSEEKVAL off_t POINTER void QSORTVAL void RANLIB echo SIZE_T size_t SLEEPVAL uint UID_T uid_t _PATH_COMPRESS /usr/bsd/compress Also, the _M_I_S_S_I_N_G___x_x_x parameters should be empty. February 14, 1992 -21- -S-o-l-a-r-i-s--2-.-X-/-S-u-n-O-S--5-.-X-,--u-s-i-n-g--S-P-A-R-C-o-m-p-i-l-e-r--C--2-.-X- DEFS -I../include -DSUNOS5 -DPOLL_BUG USE_CHAR_CONST DO CFLAGS -O -Xa $(DEFS) LDFLAGS LIBS -lnsl -lsocket -lelf LINTLIBSTYLE SYSV LINTFLAGS -b -h $(DEFS) LINTFILTER | sed -n -f ../sedf.sysv RANLIB echo VAR_STYLE STDARGS SIZE_T size_t UID_T uid_t GID_T gid_t PID_T pid_t POINTER void ALIGNPTR long LOCK_STYLE LOCKF HAVE_UNISTD DO HAVE_SETSID DO HAVE_TM_GMTOFF DONT HAVE_WAITPID DO USE_UNION_WAIT DONT HAVE_VFORK DONT HAVE_UNIX_DOMAIN DO CLX_STYLE FCNTL RES_STYLE TIMES FDCOUNT_STYLE SYSCONF ABORTVAL void ALARMVAL unsigned GETPIDVAL pid_t SLEEPVAL unsigned QSORTVAL void LSEEKVAL off_t FREEVAL void _EXITVAL void MISSING_SRC MISSING_OBJ PATH_COMPRESS /bin/compress Make sure you use the C version of subst. February 14, 1992 -22- -S-y-s-t-e-m--V--R-e-l-e-a-s-e--4- FREEVAL void GETPIDVAL long HAVE_TM_GMTOFF DONT HAVE_WAITPID DO LDFLAGS LIBS -lnsl -lsocket LINTFILTER | sed -n -f ../sedf.sysv LINTFLAGS -b -h $(DEFS) LINTLIBSTYLE NONE LOCK_STYLE FCNTL MANPAGESTYLE NONE MISSING_MAN strcasecmp.3 MISSING_OBJ strerror.o strcasecmp.o MISSING_SRC strerror.c strcasecmp.c _PATH_MAILCMD /usr/bin/mailx POINTER void QSORTVAL void RANLIB RES_STYLE TIMES SIZE_T unsigned int USE_CHAR_CONST DONT USE_UNION_WAIT DONT I was never able to get _l_i_n_t to be useful on the machine I used. Some versions of System V (for example, Esix 4.0.3) need the following LIBS value: LIBS -lresolv -lsocket -lnsl -L/usr/ccs/lib -lelf On a Dell System V machine, you have to set _H_A_V_E___U_N_I_X___D_O_M_A_I_N to ``DONT.'' -U-l-t-r-i-x--4-.-x--(-R-I-S-C-)- ALARMVAL unsigned int FREEVAL void LDFLAGS LINTFILTER | sed -n -f ../sedf.sysv LINTFLAGS -b -u -x $(DEFS) LSEEKVAL off_t MISSING_MAN MISSING_OBJ syslog.o strerror.o MISSING_SRC syslog.c strerror.c POINTER void PROF -p QSORTVAL void SIZE_T unsigned int SLEEPVAL unsigned int _EXITVAL void Ultrix also requires the new _s_y_s_l_o_g. You cannot use _m_m_a_p; the Ultrix version of the system call is for devices, not files. Some sites have reported problems with using the February 14, 1992 -23- _s_y_s_l_o_g that INN includes (output doesn't show up, or float- ing point numbers are garbled, etc.). The file _j_t_k_o_h_l_-_s_y_s_- _l_o_g_-_c_o_m_p_l_e_t_e_._t_a_r_._Z in the _/_p_u_b_/_D_E_C directory on gate- keeper.dec.com has a ``for-Ultrix'' package that handles both old and new _s_y_s_l_o_g calls. While Ultrix has symlinks, it does not have the ``-follow'' option in its _f_i_n_d command. This is used in _e_x_p_i_r_e_/_m_a_k_e_a_c_t_i_v_e_._c; you will have to either install the GNU _f_i_n_d or edit the source file. 55.. OOtthheerr SSoouurrccee PPrreeppaarraattiioonnss In addition to setting up the configuration file, it might be necessary to do some other setups. 55..11.. SSyysstteemmss wwiitthh oolldd ssyyssllooggss If you need to install the _s_y_s_l_o_g that is distributed with INN, go to the top of the distribution and type ``make syslogfix''. This will also compile _s_y_s_l_o_g_d, the logging daemon. You should install this to replace your existing daemon, usually in _/_e_t_c_/_s_y_s_l_o_g. You will also need to install the new-style _s_y_s_l_o_g_._c_o_n_f file. If you cannot replace _s_y_s_l_o_g_d on your machine, then see the file _s_y_s_l_o_g_/_R_E_A_D_M_E for information on how to set it up as an alternate daemon. Ignore any complaints from _l_i_n_t about the INN sources calling _o_p_e_n_l_o_g with the wrong argument count. In fact, if you ddoonn''tt get any complaints, then something is wrong with the way _s_y_s_l_o_g, _<_s_y_s_l_o_g_._h_>, or the _l_i_n_t libraries are set up on your system. 55..22.. TThhee DDBBZZ ppaacckkaaggee INN uses the DBZ database package. Thanks to Jon Zeeff for his permission to use and redistribute DBZ, as modified by Henry Spencer. INN has its own set of modifications to DBZ. The changes are made with the _p_a_t_c_h program and the context diff in _l_i_b_/_d_b_z_._p_c_h. If you don't have _p_a_t_c_h installed, then you can make the changes manually. (If you don't have Larry Wall's _p_a_t_c_h program get it from any _c_o_m_p_._s_o_u_r_c_e_s_._u_n_i_x archive as well as many FSF archives and other places -- you'll be glad you did.) Both _i_n_n_d and _n_n_r_p_d have the option of keeping the DBZ hash table in memory, under the control of the INND_DBZIN- CORE and NNRP_DBZINCORE_DELAY parameters, respectively. This can consume lots of RAM proportional to the size of your history database, but it can also avoid a great deal of disk I/O. You should probably see the DBZ manpage in the _d_o_c directory for some (brief) additional discussion of this issue. February 14, 1992 -24- If you are using _v_f_o_r_k (specified in the _F_O_R_K parame- ter), or you want to _m_m_a_p the database, then you mmuusstt apply the patch. The Makefile in _l_i_b will normally do it for you automatically, anyway. The beginning of the patch file describes the changes made in more detail. If you do not apply the patch, then you must add add ``dbzalt.c'' and ``dbzalt.o'' to the MISSING_SRC and MISSING_OBJ parameters. Apparently the System V 386 compiler can't optimize _d_b_z_._c (the GNU C compiler doesn't have this problem). If you have ``-O'' in your _D_B_Z_C_F_L_A_G_S configuration parameter, then take it out. 55..33.. UUssiinngg wwrriitteevv INN makes extensive use the _w_r_i_t_e_v system call to write several I/O buffers in a single call. If you do not have _w_r_i_t_e_v then you must copy _i_n_c_l_u_d_e_/_u_i_o_._h to your _/_u_s_r_/_i_n_c_l_u_d_e_/_s_y_s directory. You must also add ``writev.c'' and ``writev.o'' to the MISSING_SRC and MISSING_OBJ parame- ters. The ``fake'' _w_r_i_t_e_v found in the _l_i_b directory is not highly efficient. You might want to write a better one that tries to _m_a_l_l_o_c a new buffer and join all the elements. Be careful about doing this because _i_n_n_d can use very big buffers. 66.. CCoommppiilliinngg tthhee SSyysstteemm Once the INN sources have been configured, they are ready to be compiled. If you are very confident of your changes, type the following: cd $inn make all If you do not get any errors, skip to the section titled ``Installing the System.'' If you are confident, but careful, type: cd $inn make world cat */lint This will compile everything, then run _l_i_n_t in all directo- ries. Another option is to run the _B_U_I_L_D script found at the top of the source tree. This will interactively configure, compile, and install the system. After running that script, skip to the section titled ``Installing the System.'' February 14, 1992 -25- If you are more cautious, you should type the follow- ing: cd $inn/config make quiet cd .. This will use your already-tested _s_u_b_s_t program with your new _c_o_n_f_i_g_._d_a_t_a file. You should then follow the steps in the following sections. 66..11.. BBuuiillddiinngg tthhee LLiibbrraarryy The next step is to build the INN library. Do the fol- lowing cd $inn/lib make libinn.a lint This will build the library and run _l_i_n_t on the sources, putting the output into a file named _l_i_n_t. If any- thing fails to compile, you probably made a configuration error, most likely in the ``C library differences'' section. In particular, double-check the _S_I_G_H_A_N_D_L_E_R and _x_x_x___S_T_Y_L_E parameters. The _l_i_n_t output should be almost empty, except for a couple of ``possible pointer alignment problem'' warnings in _d_b_z_._c. If you get much more than this, then you probably did not define the _P_O_I_N_T_E_R or _S_I_Z_E___T parameters properly. The _N_E_W and _R_E_N_E_W macros in _i_n_c_l_u_d_e_/_m_a_c_r_o_s_._h try to capture all the alignment problems associated with dynamic memory allocation. Also double-check the _A_L_I_G_N_P_T_R parameter and the _C_A_S_T macro in _i_n_c_l_u_d_e_/_m_a_c_r_o_s_._h. If _l_i_n_t reports any other problems, you should take the time to investigate them. Note that many _l_i_n_t libraries have errors. Also, you may get some problems in _y_a_c_c_p_a_r in _p_a_r_s_e_d_a_t_e_._y; these are most likely in the _y_a_c_c-generated C code. If you get any of these, complain to your vendor. If you find a portability issue that I missed, please let me know. Once the library is built, you should install it in the top-level INN directory. To do this type ``make install'' while still in the _l_i_b directory. This will also compile a _l_i_n_t library for use in linting the programs in the other directories. Note that any time a change is made to the library you must do ``make install''; it is not enough to type ``make libinn.a''. This is a deliberate decision -- like a February 14, 1992 -26- program, compiling a library is different from making it available for others to use, and installing a library should make it possible to run _l_i_n_t against it. 66..22.. CCoommppiilliinngg tthhee PPrrooggrraammss INN's programs are separated into six areas, as detailed in the roadmap. You'll need to build each one before you can install and use INN. 66..22..11.. TThhee FFrroonntteenndd PPrrooggrraammss Frontends are those programs that talk to the main news server, either offering it articles or controlling its action. This includes the following programs: _i_n_e_w_s The program that validates and prepares news articles and gives them to _i_n_n_d. This is mostly used by users (usually indirectly, through programs like _P_n_e_w_s), but also through special facilities such as news/mail gateways. _r_n_e_w_s Unpacks news batches from UUCP sites and offers them to _i_n_n_d. _c_t_l_i_n_n_d This program controls _i_n_n_d, directing it to do most of the tasks a news administrator will have to do: create newsgroups, update newsfeeds, and the like. To build these programs, type the following: cd $inn/frontends make all 66..22..22.. IInnnndd The next program is the main news server, which includes the following programs: _i_n_n_d _I_n_n_d accepts all incoming NNTP connections and either processes their traffic or hands them off to the NNTP ``newsreader'' server. It accepts articles, files them, and queues them so that they can be sent to downstream feeds. _I_n_n_d listens on the official NNTP port. On most systems only root can do this. _I_n_n_d is careful to set the modes of any files it creates, as well as the privileges of any processes it spawns. February 14, 1992 -27- _i_n_n_d_s_t_a_r_t Sites that are concerned about large root- access programs may wish to install _i_n_n_d_- _s_t_a_r_t. This program opens the port, changes its user and group ID to be that of the news administrator, and then _e_x_e_c's _i_n_n_d with the open port. It also sets up a secure execu- tion environment. It is a small program (about 100 lines) that is easily understood. You should use it because _i_n_n_d will run faster because it won't have to make any _c_h_o_w_n system calls. If you make _i_n_n_d_s_t_a_r_t setuid root then no news maintenance has to be done as root. To build these, type the following: cd $inn/innd make all Note that _i_n_n_d handles the filing and distribution of certain messages differently from other systems. For exam- ple, you can have newsgroups within ``control'' for the dif- ferent types of control messages. See _i_n_n_d_._8, _n_e_w_s_f_e_e_d_s_._5, and _a_c_t_i_v_e_._5 in the _d_o_c directory for details. 66..22..33.. TThhee NNeettNNeewwss RReeaaddiinngg DDaaeemmoonn _I_n_n_d implements a subset of the NNTP protocol -- only those commands that are needed for peer sites to feed news articles. You must install _n_n_r_p_d to allow users to read news. If a connection comes in from a host that is not a specified feed, then an _n_n_r_p_d process is spawned to handle it. (You can debug _n_n_r_p_d by running it interactively; put an entry for the host named ``stdin'' in your _n_n_r_p_._a_c_c_e_s_s file.) Build the newsreader server by doing the following: cd $inn/nnrpd make all Note that if users on a peer machine (one that feeds you news) want to read news from your server, then you have two choices. You can use _n_n_t_p_d from the reference platform (See Appendix II) and make sure not to list the peer in your _n_n_t_p_._a_c_c_e_s_s file. The other choice is to relink the reading software on the other machine with the INN library so that it uses the ``mode reader'' NNTP command extension. 66..22..44.. TThhee BBaacckkeenndd PPrrooggrraammss The backend programs take articles that _i_n_n_d received and offer them to your news neighbors. This includes the February 14, 1992 -28- following programs: _a_r_c_h_i_v_e A simple program to archive news articles. _b_a_t_c_h_e_r Collects articles into batches for UUCP delivery. _b_u_f_f_c_h_a_n A program to split a single _i_n_n_d stream into separate files. It can buffer data, flushing files based on command-line switches. _c_r_o_s_s_p_o_s_t A program that does link creation for cross- posted articles. Runs as a channel feed and lets innd hand off some of its i/o activity. _c_v_t_b_a_t_c_h A program to turn a file list into an INN batchfile. A transition aide that is only documented in the source. _f_i_l_e_c_h_a_n Another program to split a single _i_n_n_d stream into separate files. It is system-call intensive, but requires no locking protocol. _i_n_n_x_m_i_t A replacement for _n_n_t_p_x_m_i_t from the reference implementation. It reads a file containing a list of articles, and sends them to a host. _n_n_t_p_g_e_t A program to retrieve articles from a remote site. _s_h_l_o_c_k A program to provide a locking protocol for shell scripts. _s_h_r_i_n_k_f_i_l_e A program to shrink a file by removing lines from the beginning. It is useful for purging backlogged batchfiles. _s_y_s_2_n_f A program to turn a B or C News _s_y_s file into an INN _n_e_w_s_f_e_e_d_s file. This is a transition aide that is only documented in the source. To build this set of programs, type the following: cd $inn/backends make all 66..22..55.. EExxppiirree This directory includes programs to modify the history database as well as some utilities that might be useful in this task. The database is called the _h_i_s_t_o_r_y file, and it contains one line for every article on the system, specify- ing when it was received and where it was filed. This file February 14, 1992 -29- is indexed by the Message-ID, and the DBZ package provides fast retrieval from it. _c_o_n_v_d_a_t_e Converts between user-readable dates and the format used in the history file. _e_x_p_i_r_e Scans the history database to purge old entries, and remove old articles from the spool area. You can specify how long to keep sets of newsgroups. _m_a_k_e_a_c_t_i_v_e This program can be used to rebuild the _a_c_t_i_v_e file if it is lost in a crash. _m_a_k_e_h_i_s_t_o_r_y This program scans through the spool area and rebuilds the history files. _n_e_w_s_r_e_q_u_e_u_e This program can be used after a crash to resend articles to your neighbors. _p_r_u_n_e_h_i_s_t_o_r_y This is a tool for other programs that expire news. It reads a list of Message-ID's and filenames, and updates the history file to mark that the files have been deleted. This directory also includes _e_x_p_i_r_e_._p_c_h and _r_e_a_p_._p_c_h. The first is a patch to the C News expire program that lets it cooperate better with _i_n_n_d, sending it messages when articles have been removed. The second is a set of patches to the _r_e_a_p program that lets it cooperate with _p_r_u_n_e_h_i_s_- _t_o_r_y; it also adds some other useful features. Both patch files have additional information in them. Both programs are unsupported, provided by members of the beta-test group. To build these programs, type the following: cd $inn/expire make all If you are currently running C News, note that it has a directory named _e_x_p_i_r_e that is often the same pathname as INN's _e_x_p_i_r_e program. You will have to move, or remove, the directory before you can intall the INN program. 66..22..66.. SSccrriipptt aanndd ddaattaa ffiilleess In addition to the programs, INN requires several scripts. For example, one script starts the server when the machine boots while another prunes the log files and runs _e_x_p_i_r_e every night. Many of these scripts can be used as-is until you get a feel for how INN works. February 14, 1992 -30- INN also requires several data files. One specifies what sites feed you news, another what sites you feed, and so on. INN cannot provide these, other than giving sample entries. You'll probably find that writing these files will be the hardest part of your installation. Prototypes for all these files are provided in the _s_a_m_- _p_l_e_s directory. Your modified copies should be maintained in the _s_i_t_e directory. By splitting things up this way, official updates will never wipe out any changes you have made. To create the initial set of files, do the following: cd $inn/site make all See below for an explanation of each file. 66..33.. MMaannuuaall ppaaggeess INN comes with an extensive set of manual pages. You might want to edit the Makefile to set up the right owner- ship of the installed manual pages. Or you might want to not bother installing them at all. When it comes to reading them, you should start with _i_n_n_d_._8 and _c_t_l_i_n_n_d_._8. From there follow the cross-refer- ences as you want. 77.. IInnssttaalllliinngg tthhee SSyysstteemm Although either _i_n_n_d or _i_n_n_d_s_t_a_r_t must be run by root, most of the installation does not have to be done as root. The _$_i_n_n_/_m_a_k_e_d_i_r_s_._s_h script creates all the necessary direc- tories used by INN, and sets up the right ownerships and modes: owned by _N_E_W_S_U_S_E_R in group _N_E_W_S_G_R_O_U_P with 0775 per- missions (the ``firewall'' directory, ___P_A_T_H___I_N_D_D_D_I_R, has mode 0770). You should review this script, then run it. The rest of the installation should be done as the news administrator or as root. The Makefiles are very strict about setting the modes on the files that get installed. To install the programs, do the following: cd $inn make update This target does a ``make install'' in all program directo- ries. It installs the programs and manpages, but does not update or install any configuration files or scripts. This is important: in any directory (including the top-level one), a ``make install'' will install everything in that February 14, 1992 -31- directory into the right place. A ``make update'' can only be done in the top-level directory or in the _s_i_t_e directory, and it only replaces scripts, not configuration files. When updating to a new INN release, you will probably want to do an ``update'' first, and then review the changed files by doing ``make diff'' in the _s_i_t_e directory, and integrate your local changes as appropriate. The Makefile also has other targets that you might find useful, so the comments for entries like ``most'' and ``installed-diff', for exam- ple. The next, and last, step is to build your INN configu- ration files and utility scripts. If you have not already done so, type the following: cd $inn/site make all This will get copies of the scripts and files from the _b_a_c_k_- _e_n_d_s and the _s_a_m_p_l_e_s directories and run _s_u_b_s_t over them. Whenever patches are issued, doing a _m_a_k_e in this directory will let you know what files have been updated, without destroying your local changes. The _g_e_t_s_a_f_e_._s_h script does this. If you have either an _S_C_C_S or an _R_C_S directory then _g_e_t_s_a_f_e_._s_h will use the appropriate source control system for the files in this directory. The first set of files are used to carry out the con- trol messages. You might want to look them over; in partic- ular, look at the table in _c_o_n_t_r_o_l_._c_t_l and the newslog man- pages in _d_o_c. The control files are: checkgroups rmgroup control.ctl sendme default sendsys docheckgroups senduuname ihave version newgroup writelog parsecontrol pgpverify The following scripts are normally invoked by _c_r_o_n or at system boot time, and should not require many changes: innlog.awk scanlogs innstat tally.control news.daily tally.unwanted rc.news _R_c_._n_e_w_s starts the server. In versions 1.4 and earlier, this script was run by user root. In this version, _r_c_._n_e_w_s must be run by the user defined as the _N_E_W_S_U_S_E_R in the con- fig.data file. _N_e_w_s_._d_a_i_l_y invokes _e_x_p_i_r_e and _s_c_a_n_l_o_g_s. _S_c_a_n_l_o_g_s calls the other scripts to process the logs. You February 14, 1992 -32- might want to review these scripts just to see what they do. Do not get bogged down in the details, just read the com- ments. They are documented in the manpages news.daily(8) newslog(5), and newslog(8). There are some utility scripts to send news to your news feeds: nntpsend send-nntp nntpsend.ctl send-uucp send-ihave sendbatch They flush and lock the batch file for the specified site(s) and then call _i_n_n_x_m_i_t to send the articles to your down- stream feeds. _S_e_n_d_-_i_h_a_v_e is used for ``ihave/sendme'' feeds and is described in an appendix. _S_e_n_d_b_a_t_c_h and _s_e_n_d_-_u_u_c_p flush and lock batchfiles and call _b_a_t_c_h_e_r to queue up UUCP jobs. You might want to modify these files to change the flags given to _u_u_x; the default is to queue jobs up as grade ``d.'' You will almost definitely have to edit them to make sure that they properly parse the output of _d_f so that your spool area is not overrun! _N_n_t_p_s_e_n_d and _s_e_n_d_-_n_n_t_p do the same thing for NNTP feeds. You must determine how you want to propagate your articles -- the scripts give common ways of getting the job done. The following files will have to be edited to contain your local information. They all have manual pages in the _d_o_c directory that describe them: expire.ctl newsfeeds hosts.nntp nnrp.access inn.conf passwd.nntp moderators The last group of files are utility scripts you might find useful: inncheck makegroup innwatch scanspool _I_n_n_c_h_e_c_k is a Perl script to check the syntax and permis- sions of an installed INN system. _I_n_n_r_e_p_o_r_t is an alternate way of summarizing the server's log file. It is a Perl script. _I_n_n_w_a_t_c_h is a shell script to monitor the system and stop the server when you are running low on disk space or inodes; it could be run out of your ___P_A_T_H___N_E_W_S_B_O_O_T script. You might have to edit it to understand your _d_f output format. _M_a_k_e_g_r_o_u_p is a front-end to _r_n_e_w_s that helps you write a control message to create a newsgroup. You should review this script because you might have to change the way the output of the _d_a_t_e command is parsed, and because you might might want to change the default February 14, 1992 -33- distribution. _S_c_a_n_s_p_o_o_l is a Perl script to make sure that the active file and the contents of your spool tree agree. Once you have made the necessary modifications (and I admit that some of this -- especially the _n_e_w_s_f_e_e_d_s file -- will be difficult), you should type the following: make install Make sure you have _r_c_._n_e_w_s installed in the right place, as explained in the ``Paths to common programs'' section, above. You might find it useful to read the ``First-Time Usenet or NNTP Installation'' appendix for help on navigat- ing through the INN configuration files. There are now only a couple more things to check. First, make sure you have an _a_c_t_i_v_e file and a _h_i_s_t_o_r_y database! The appendices explain how to convert your exist- ing files; the _B_U_I_L_D script will create new ones for you. If you have Perl, run _i_n_n_c_h_e_c_k to make sure that you have the datafiles configured correctly. The second is make sure that you have correctly updated your _s_y_s_l_o_g_._c_o_n_f file to match the filenames and logging levels required by INN. See _s_y_s_l_o_g_/_s_y_s_l_o_g_._c_o_n_f for an example of what to do. Once you have done all of this, InterNetNews is now installed, and ready to run -- have fun! 88.. HHeetteerrooggeenneeoouuss CClliieenntt IInnssttaallllaattiioonnss The _i_n_e_w_s program is used by user newsreaders. Pro- grams such as _r_n (which call _P_n_e_w_s) prepare a news article and feed it into _i_n_e_w_s. _I_n_e_w_s validates the news headers, adds its own, and feeds the article to the campus _i_n_n_d server. The _i_n_e_w_s that comes with INN is more useful then the ``mini-inews'' that comes with the reference implementa- tion. You cannot run the standard B2.11 _i_n_e_w_s. You can run the C News _i_n_e_w_s, but only on client machines (i.e., those with a _$_N_E_W_S_C_T_L_/_s_e_r_v_e_r file). I recommend that you install INN's _i_n_e_w_s on all the clients in your campus. INN comes with a _M_a_k_e_I_n_e_w_s script to make it easier to build and install _i_n_e_w_s on a wide variety of hosts. This script creates a directory and copies all the necessary files (headers, sources, configuration files) into it. The script takes an optional argument, which should name the client machine's architecture. For example: cd $inn ./MakeInews sun3 will create an _i_n_e_w_s_._s_u_n_3 directory. You can then examine the Makefile in that directory, and build and install _i_n_e_w_s on your Sun-3 clients. This is easiest if the client NFS- February 14, 1992 -34- mounts the source directory -- that way you can keep all your _i_n_e_w_s sources in one place. _R_n_e_w_s only has to be available on the machine where you run UUCP (and perhaps a mail-news gateway). If this is not the same machine as where _i_n_n_d is running, then the _M_a_k_- _e_R_n_e_w_s script can be used in the same manner as the _M_a_k_e_I_n_e_w_s script. 99.. KKnnoowwnn PPrroobblleemmss If you use NIS (formerly Yellow Pages) on SunOS, you will need to add a ``domainname'' entry to your _i_n_n_._c_o_n_f file if your hosts do not contain fully-qualified domain names. The most common symptom of this is that _i_n_e_w_s will fail because it cannot generate a Message-ID. Another prob- lem with NIS is that reverse name lookups do not return the fully-qualified domain name. If you know that none of your local clients have a period in their name, you can use a pattern like ``*[^.]*'' in your _n_n_r_p_._a_c_c_e_s_s file. SunOS4.1.1 has a bug where _w_r_i_t_e(2) can return EINTR. The most common symptom is the following fatal error message from _i_n_n_d: Can't sync history, interrupted system call This is Sun bug 1052649. It is fixed in patch 100293-01. According to the release manual, it is also fixed in all releases of SunOS since 4.1.2. If you have _N_O_F_I_L_E___L_I_M_I_T set you should know that the standard I/O library in SunOS4.x has trouble with more than 127 descriptors. The most common symptom is the following fatal error message from _i_n_n_d: can't fopen /usr/local/news/history, invalid argument This occurs after doing a _c_t_l_i_n_n_d ``reload'' command. For a work-around, reboot your server instead of trying to ``reload.'' Another symptom is that _i_n_n_d will exit if you do a _c_t_l_i_n_n_d ``flush'' command while the server is paused or throttled. This is Sun bug 1045141. Sun does not plan to fix it for any 4.x release. One site has reported the same error message happens after doing a sequence of ``throttle'' and ``go'' commands. It does not appear to be related to the bug mentioned above, although the symptom is the same. If you replace the body of INN's _x_f_o_p_e_n_a routine with the following, it will work: return fopen(p, "a+"); This is in the file _l_i_b_/_x_f_o_p_e_n_a_._c. February 14, 1992 -35- If you use Sun's unbundled compiler, _a_c_c, you must make sure to use the unbundled assembler, too. You might also get lots of ``left operand must be modifiable lvalue'' errors. Setting _U_S_E___C_H_A_R___C_O_N_S_T to ``DONT'' will help. There have been reports that the VAX Ultrix 4.2 _m_a_l_l_o_c doesn't work well with _i_n_n_d, causing it to slowly fill up all swap space. I believe that all of the memory leaks in _i_n_n_d have been fixed, but you might want to look at using a different _m_a_l_l_o_c package. The Kingsley/Perl _m_a_l_l_o_c package is provided in the _l_i_b directory. Add ``malloc.c'' and ``malloc.o'' to the MISSING_SRC and MISSING_OBJ lines in _c_o_n_f_i_g_._d_a_t_a and rebuild. I have been told that on SunSoft Interactive UNIX Sys- tem V Release 3.2 Version 3.0 systems has been broken up into separate files. The easiest way to work around this problem is to add ``#include '' to _i_n_c_l_u_d_e_/_c_l_i_b_r_a_r_y_._h. If you use 386BSD (the Jolitz port, not the BSDI prod- uct) you will have to set _A_C_T___S_T_Y_L_E to ``READ''. If you do not do this then the active file will not get updated. Another work-around is to insert an ``msync'' call in the ICDwriteactive routine in _i_n_n_d_/_i_c_d_._c. This is not supported because I consider the 386BSD behavior to be buggy. The default configuration of some Sequent kernels does not provide enough descriptors for _i_n_n_d to run. You might have to rebuild your kernel with the ``MAXNOFILE=128'' and ``NOFILEEXT=64'' options. You will also have to had a ``setdtablesize(nnn)'' call in the main routine of _i_n_n_d, and a ``setdtablesize(0)'' call in the Spawn routine. I have been told that some older versions of the SCO _o_p_e_n_d_i_r routine have file descriptor leaks. The most noticeable symptom is probably that _i_n_n_d will die while try- ing to renumber the _a_c_t_i_v_e file. You might want to use a freely-redistributable ``dirent'' package such as one dis- tributed by the Free Software Foundation. You should also make sure to set _C_L_X___S_T_Y_L_E to ``FCNTL'' Finally, you might also want to edit _i_n_n_d_/_i_n_n_d_._c where it calculates the ``Max- Outgoing'' parmater and increase the fudge number to four. On some SVR4 systems, attempting to set the socket buffer size is either not supported or, even worse, might result in _i_n_n_d's data size growing. The most noticeable symptom is ``cant setsockopt(SNDBUF)'' messages in your _s_y_s_- _l_o_g output. To fix this, either comment out the calls to _s_e_t_s_o_c_k_o_p_t in _i_n_n_d_/_n_c_._c or add ``-USO_SNDBUF'' to your _D_E_F_S config parameter. I have heard that Sony SVR4 systems have lots of prob- lems. You must set _H_A_V_E___U_N_I_X___D_O_M_A_I_N to ``DONT''; sockets in February 14, 1992 -36- general seem to have problems, including kernel crashes and a blocked _i_n_n_d. If you use the GNU _s_e_d in the ___P_A_T_H___S_E_D configuration parameter, make sure you get version 1.13; earlier versions have a bug that breaks the _p_a_r_s_e_c_o_n_t_r_o_l scripts. The most noticeable symptom is that all ``newgroup'' control messages result in mail saying that they are unparseable. Some versions of the shell in HP-UX do not properly parse a quoted ``['' when it is in a pattern for a _c_a_s_e statement. The most noticeable symptom is that _n_e_w_s_._d_a_i_l_y does not properly expire articles if _i_n_n_w_a_t_c_h has throttled the server. Contact HP and get a fix for SR # 5003-009811. On some versions of AIX on the RS/6000, using memory- mapping can eat up all the page space or crash the machine. This will be noticeable if you have _A_C_T___S_T_Y_L_E set to ``MMAP'' and/or have ``-DMMAP'' in _D_B_Z_C_F_L_A_G_S. Ask your IBM representative for the ``U413090'' PTF and prerequisites to apply it; it is believed that this will fix it. On some systems processes spawned by _i_n_n_d never exit. The most likely cause is that _C_L_X___S_T_Y_L_E should be set to ``FCNTL.'' February 14, 1992 -37- AAppppeennddiixx II:: DDiiffffeerreenncceess ffrroomm ootthheerr NNeewwss ssooffttwwaarree Administrators will find that INN is fairly incompati- ble with B and C News. This section tries to mention the most important places where INN differs from the other news systems. If you have not maintained B or C News, you should probably skip this section. Users will generally only notice is that INN is faster; it should be 100% compatible with the other systems at the user level. If you had particular problems that aren't men- tioned here, please let me know. Note, however, that this is _n_o_t a tutorial on how to set up a new INN system, or con- vert older software to it; no such document exists. 11.. CCoonnffiigguurraattiioonn FFiilleess Below is a list of the data files used by B and C News, and the reference NNTP implementation, along with a short summary of how they map into INN configuration files. The syntax is always different: INN files are almost always a set of colon-separated fields where lines beginning with a poundsign are ignored. _e_x_p_l_i_s_t This is replaced by the similar _e_x_p_i_r_e_._c_t_l file. Archiving is done by a separate pro- gram. _m_a_i_l_p_a_t_h_s This is replaced by the _m_o_d_e_r_a_t_o_r_s file. The ``default'' entry in _m_a_i_l_p_a_t_h_s is replaced by either a full wildcard (``*'') entry in the _m_o_d_e_r_a_t_o_r_s file, or by a ``moderatormailer'' entry in the _i_n_n_._c_o_n_f file. _n_n_t_p_._a_c_c_e_s_s This is replaced by the _h_o_s_t_s_._n_n_t_p (for NNTP peers) and _n_n_r_p_._a_c_c_e_s_s (for newsreading) files. _n_n_t_p_._s_y_s This is a password file used if NNTP is com- piled with the ``AUTH'' option. It is replaced by the _p_a_s_s_w_d_._n_n_t_p file. Note that _i_n_e_w_s and _r_n_e_w_s will also try to read _p_a_s_s_w_d_._n_n_t_p. Therefore, you will probably want to have one-line versions of it for your on-campus clients. _o_r_g_a_n_i_z_a_t_i_o_n This is replaced by the ``organization'' entry in the _i_n_n_._c_o_n_f file. _r_n_/_s_e_r_v_e_r This is replaced by the ``server'' entry in the _i_n_n_._c_o_n_f file. _w_h_o_a_m_i This is replaced by the ``pathhost'' and ``fromhost'' entries in the _i_n_n_._c_o_n_f file. February 14, 1992 -38- 22.. NNeewwssggrroouuppss,, AAccttiivvee,, SSyyss,, aanndd NNeewwssffeeeeddss The biggest difference is how the _n_e_w_s_f_e_e_d_s file com- pares with the _s_y_s file. Newsgroup patterns like ``all.all.ctl'' are completely gone. All newsgroup patterns are shell-style wildcards, matched against the _a_c_t_i_v_e file. The _a_c_t_i_v_e file is taken to be the definitive list of newsgroups that you want to receive. With B and C news, an article must match the subscription list of the local site as specified in the _s_y_s file. If it matches, each newsgroup is then looked up in the _a_c_t_i_v_e file. If none of the news- groups are found, then the article is filed into the news- group named ``junk''. INN's behavior is much simpler. If a newsgroup does not appear in the _a_c_t_i_v_e file, it is ignored. If none of the groups are mentioned, then the article is rejected: nothing is written to disk. This is a deliberate design decision: if you do not want a particular newsgroup to take up your disk space, remove it from the _a_c_t_i_v_e file; if your neighbors have not gotten around to updating your newsfeed, then the only thing that will happen is that some network bandwidth will have been wasted when they send you the arti- cle. You can change INN's behavior so that it resembles the other systems. To do this, compile with _W_A_N_T___T_R_A_S_H set to ``DO.'' Note that this will accept _e_v_e_r_y_t_h_i_n_g. Because there is no subscription list, you cannot say ``give me all of the foo hierarchy (filed into junk), but not the alt hierarchy.'' You must list the group in the _a_c_t_i_v_e file. INN strictly believes in distributions. If the site named _M_E has any distributions, then incoming articles must either have no Distribution header, or the header must match the distribution list. If you want to blindly accept all distributions, make sure you do not have a ``/distrib,...'' section in your _M_E entry. Distributions are fixed strings -- there are no patterns or special wildcards like ``all.'' For more details on these items, see _d_o_c_/_n_e_w_s_f_e_e_d_s_._5. 33.. CCoonnttrrooll MMeessssaaggeess Like C News, INN implements all control messages other than cancel as shell scripts. The number and type of param- eters is different from that of C News. All control mes- sages consult the file _c_o_n_t_r_o_l_._c_t_l before acting on the mes- sage. If the sender's address matches with the list of authorized addresses (e.g., ``tale@uunet.uu.net'', ``*'', etc.), the control message is either acted upon, mailed to the news administrator, or logged. For example, messages from ``tale@uunet.uu.net'' (the current moderator of February 14, 1992 -39- news.announce.newgroups) are honored. The ``control'', ``junk'', and ``to'' newsgroups can be explicitly sent or not sent. See _d_o_c_/_n_e_w_s_f_e_e_d_s_._5 and _d_o_c_/_i_n_n_d_._8. The _c_t_l_i_n_n_d program is what really directs the server to create or remove newsgroups. This results in a semi- recursive process: the control message arrives, and a script is invoked to process the message. If approved, the script invokes _c_t_l_i_n_n_d to send a message back to the server telling it to create or remove the group. 44.. LLoocckkiinngg A running news system has many open files. These files can be divided into two groups. The first group includes the history database and _a_c_t_i_v_e file. The second group includes the logfiles and batch files used to send articles to your feeds. B news uses an internal protocol for the first group. For the second group, since _i_n_e_w_s does ``atomic appends,'' no locking is necessary. C news uses the _l_o_c_k_n_e_w_s and _n_e_w_s_l_o_c_k scripts for the first group, and provides no fine- grain mechanism for the second group. With INN, the server is running all the time and all locking is done under the direction of _c_t_l_i_n_n_d. The first group is generally handled by using the ``throttle,'' ``pause,'' and ``go'' commands (sometimes ``reload'' will be necessary). The second group is handled by the ``flushlogs'' and ``flush'' commands. See the _d_o_c_/_c_t_l_i_n_n_d_._8 manpage; examples of their use can be found in various scripts in the _s_a_m_p_l_e_s directory. February 14, 1992 -40- AAppppeennddiixx IIII:: CCoonnvveerrttiinngg ffrroomm ootthheerr NNeewwss ssooffttwwaarree INN is a complete news transport and expiration system. Since few people will be installing INN from scratch, this section should help you determine what you can ``throw out'' from your earlier news setups. It is also compatible with much of the existing news software, so you can create a mixed environment if you want to, and if you are careful. 11.. CC NNeewwss EExxppiirree The _e_x_p_i_r_e program that is distributed with INN does not do any archiving. Since the history databases currently have the same format, it is possible to use the C News _e_x_p_i_r_e if you want to. (The INN history database may change, however, so you should only do this if you really have to -- you really should use INN's _e_x_p_i_r_e.) There are three ways to do this. The first way is to change your _d_o_e_x_p_i_r_e script so that it calls _c_t_l_i_n_n_d to ``throttle'' _i_n_n_d just before _e_x_p_i_r_e runs. It should then issue a _c_t_l_i_n_n_d ``go'' command after _e_x_p_i_r_e is done. The drawback to this method is that no incoming news is accepted until all expiration is finished. The second way is to compile _l_i_b_/_l_o_c_k_._c and add it to your C News library _l_i_b_c_n_e_w_s_._a, replacing the provided lock functions. You should then remove _e_x_p_i_r_e and relink it. This method has not been tested very thoroughly, but it is rather simple. The third way is to teach the C News _e_x_p_i_r_e to talk to _i_n_n_d and tell it to cancel articles that it would remove. To do this, apply the patch file _e_x_p_i_r_e_/_e_x_p_i_r_e_._p_c_h to your C News _e_x_p_i_r_e_._c sources. You will also have to add _l_i_b_/_i_n_n_d_- _c_o_m_m_._o to _l_i_b_c_n_e_w_s_._a and then rebuild _e_x_p_i_r_e. 22.. SSttaannddaarrdd NNNNTTPP ddaaeemmoonn You can use the ``standard'' _n_n_t_p_d server. You should only have to do this if you have hosts that feed you news, and where the users on that machine also want to read news on your machine. Make sure that you configure _n_n_t_p_d so that it is using DBZ, and have it feed each individual article to _i_n_e_w_s; don't use the ``batched input'' option. It should also be set up so that it acts as if it is running under _i_n_e_t_d. You should also make sure that _i_n_e_t_d does nothing with the NNTP port, number 119. February 14, 1992 -41- 33.. NNNNTTPP--bbaasseedd nneewwssrreeaaddeerrss If you already have your NNTP-using newsreaders installed and running, you do not have to do anything. This includes _x_v_n_e_w_s, _x_r_n, _r_r_n and so on. INN implements the standard NNTP protocol, with some extensions. INN does not provide the extensions used by _t_r_n, _t_i_n or other newsread- ers. INN will not implement all the different indexing sys- tems because the right solution is to have a generic inter- face that all readers can use.) For administrative convenience, however, you might wish to have all your newsreaders use the INN library and config- uration files to talk to the server. The next section describes how to do that for _r_n. It is provided as an exam- ple, to help you convert other programs you might have. INN does not provide, nor fully support, any newsreaders. 44.. RReemmoottee rrnn The ``remote'' version of _r_n (also called _r_r_n) uses a set of routines in the NNTP ``clientlib'' file. INN can emulate these routines; see _d_o_c_/_c_l_i_e_n_t_l_i_b_._3. If you need to build _r_n for client machines that don't have the entire INN distribution available, use the _M_a_k_e_L_i_b script to build a distribution directory of the necessary routines. Use this script the same way you use the _M_a_k_e_I_n_e_w_s script. _R_n, _r_r_n, and _t_r_n are moving targets so these instruc- tions may be out of date. The maintainers have agreed to officially support INN, however, which is a good thing. There are two ways to build _r_n so that it uses the INN library. If you don't have the NNTP distribution installed you will have to use the first way. The first way is to apply a patch to the latest _r_n _C_o_n_- _f_i_g_u_r_e script and then execute it and rebuild the program. To do this, type the following: cd _r_n___s_o_u_r_c_e patch <$inn/frontends/rn.pch ./Configure make At some point, _C_o_n_f_i_g_u_r_e will ask you if you want to use the InterNetNews library; answer _y_e_s. You can then use either the full sources, or a special library that contains just the needed header and sources files. Tell _C_o_n_f_i_g_u_r_e the appropriate pathnames, and then proceed with the rest of the _r_n installation. The second way is to edit a couple of files after you have run _C_o_n_f_i_g_u_r_e and set it up to build the remote rn. February 14, 1992 -42- First, replace the _r_n file _s_e_r_v_e_r_._h with the INN file _i_n_c_l_u_d_e_/_m_y_s_e_r_v_e_r_._h. The next step is to edit the _r_n Make- file to remove the ``clientlib'' file from the source and object file lists. This can probably be done by commenting out the definitions of the _c_5 and _o_b_j_5 variables. You must also edit the Makefile to add the INN library to the list of libraries that are linked in. This can probably be done by editing the line that defines the _l_i_b_s variable so that the full pathname to _l_i_b_i_n_n_._a is the first item after the equal sign. 55.. RReemmoovviinngg tthhee OOtthheerr SSttuuffff The names below assume a ``standard'' news setup; things might be different on your machine. Also, many pro- grams have alternate names and links; make sure you chase down and remove aallll of them. You might find it easiest to rename your _/_u_s_r_/_l_i_b_/_n_e_w_s (and _/_u_s_r_/_l_i_b_/_n_e_w_s_b_i_n) directories to something else and start with a clean slate, copying over the files as they are needed. Make ssuurree that your news processing is completely stopped before you begin this process. That includes any _c_r_o_n jobs that may be running. The _/_u_s_r_/_l_i_b_/_n_e_w_s directory can become cluttered -- that's why C News split everything up into separate directo- ries. The following files are compatible with C News and B2.11 News, and should be _k_e_p_t: active active.times If you are running C News keep these files, otherwise delete them and use _m_a_k_e_h_i_s_t_o_r_y to rebuild them: history history.dir history.pag _R_n does not have to be modified so leave this directory alone (or copy it back if you moved your original): /usr/local/lib/rn If you set up _r_n to use the INN library, remove this file: /usr/local/lib/rn/server The input system is completely replaced. Remove the following programs and their manpages: February 14, 1992 -43- /bin/cunbatch /bin/inews, /usr/lib/news/inews, etc... /bin/rnews, /usr/bin/rnews, etc... /usr/lib/news/rnews.stall /etc/nntpd, /usr/etc/nntpd, etc... Also remove the following directories and everything within them: /usr/lib/news/bin/input /usr/lib/news/bin/relay /usr/lib/news/bin/ctl /usr/lib/news/bin/inject /usr/lib/news/nntp (mkgrdates, nntp_access, shlock, etc) The transmission facility is completely replaced. You may keep your current feed subsystem if you want to, but it will require some changes to make sure that batchfiles are properly flushed; see the _s_e_n_d_-_x_x_x scripts for examples. Remove these files and programs: /usr/lib/news/batchparms /usr/man/man8/newsbatch.8 Remove the following directory and everything within it: /usr/lib/news/bin/batch You can continue to use _n_n_t_p_l_i_n_k, _n_e_w_s_x_d, and the like, sub- ject to the caveat just mentioned. Article expiration and maintenance of the history and active files is completely replaced. Remove this file: /usr/lib/news/explist Remove the following directories and everything within them: /usr/lib/news/bin/expire /usr/lib/news/bin/maint If you do not remove the _e_x_p_i_r_e directory, you will probably have problems installing INN's _e_x_p_i_r_e, which is a program that often has the same name as the C News directory. The following programs in _/_u_s_r_/_l_i_b_/_n_e_w_s_b_i_n are not needed and can be deleted. Keeping them around is harmless, and if you find them useful don't delete them: February 14, 1992 -44- canonhdr newshostname ctime newslock dbz queuelen getabsdate sizeof getdate spacefor gngp Note that _c_t_i_m_e, _g_e_t_a_b_s_d_a_t_e, and _g_e_t_d_a_t_e are replaced by _c_o_n_v_d_a_t_e. More importantly, _n_e_w_s_l_o_c_k does not lock _i_n_n_d; it is best to remove it. The following files are replaced by INN configuration files. You should delete them, just to avoid confusion: mailname sys mailpaths whoami organization If you have other software that uses them (except _s_y_s), you can keep them. The following will be rebuilt (or overwrit- ten) by _i_n_n_d and _s_c_a_n_l_o_g_s so you should remove them: errlog log In addition to the manpages for the programs listed above, the following manual pages should be removed: active.times.5 newsmail.8 expire.8 newsmaint.8 mkgrdates.8c nntpd.8c news.5 nntpxmit.1 newsaux.8 Any other files and directories can probably also be discarded. February 14, 1992 -45- AAppppeennddiixx IIIIII:: SSeettttiinngg uupp ddiiffffeerreenntt ffeeeeddss This section gives some notes and advice on how to set up different types of outgoing news feeds. It duplicates and expands upon the information in the manual pages. 11.. IIhhaavvee//sseennddmmee ffeeeedd For a standard UUCP newsfeed, a site batches up all the articles it receives and sends them to the downstream site, which unpacks the batch and processes each article. If the downstream site has multiple feeds, however, it might want to ``filter'' the articles that get sent. This is done by having the feeding site send a list of Message-ID's as an ``ihave'' control message. The receiving site examines the list to see which articles it does not currently have, and sends it back to the upstream site as a ``sendme'' message. The original site receives this message and prepares a batch in the standard way. Note that this has nothing to do with NNTP. It is a specialized type of batched feed that is not used very often. To do ihave/sendme with a site named remote, the local site must either have a ``to.remote'' newsgroup or be compiled with MERGE_TO_GROUPS set to ``DO'' Accepting an ihave/sendme feed is easy. Suppose an ``ihave'' message is received from a site named remote. When _i_n_n_d processes the message it will invoke the appropri- ate control script, _/_u_s_r_/_l_o_c_a_l_/_n_e_w_s_/_b_i_n_/_c_o_n_t_r_o_l_/_i_h_a_v_e. The script will filter the body using _g_r_e_p_h_i_s_t_o_r_y, creating a list of Message-ID's not found in the _h_i_s_t_o_r_y database. It uses this output to create a ``sendme'' control article which is posted to the ``to.remote'' newsgroup using _i_n_e_w_s. This article will then be queued and sent to remote in the normal way. The remote site will then send the desired articles back. Providing an ihave/sendme feed is a bit more compli- cated. First, you must create two entries in your _n_e_w_s_f_e_e_d_s file. The first should be named remote.ihave. Make this a ``Tf,Wm'' feed that contains the remote's subscription list. This entry results in a a file that accumulates the Message- ID's of all articles that remote might want. The other entry should be named remote. This should be a ``Tf,Wn'' feed that only subscribes to the ``to.remote'' newsgroup. (Actually, if you feed some groups as a standard feed, you can put them on the remote entry, rather then the remote.ihave entry.) The next step is to have the ``ihave'' control messages sent out. To do this, review the _s_e_n_d_-_i_h_a_v_e script and make sure it is invoked as needed (usually out of _c_r_o_n). It splits the batchfile from the remote.ihave _n_e_w_s_f_e_e_d_s entry February 14, 1992 -46- and posts ``ihave'' control messages into the ``to.remote'' newsgroup. These messages will get queued for the remote entry. The next step is to send out any articles queued for the remote entry. Treat this as a standard UUCP feed, invoking _s_e_n_d_-_u_u_c_p or _s_e_n_d_b_a_t_c_h as appropriate, typically a few minutes after _s_e_n_d_-_i_h_a_v_e runs. When the remote site receives the ``ihave'' message it will filter it and send back a ``sendme'' message whose body is the list of desired Message-ID's. When _i_n_n_d processes this message it will invoke the appropriate control script, _/_u_s_r_/_l_o_c_a_l_/_n_e_w_s_/_b_i_n_/_c_o_n_t_r_o_l_/_s_e_n_d_m_e. This script will call _g_r_e_p_h_i_s_t_o_r_y to turn the list into a list of files appended to the batchfile for remote. Examine this script (the file- name should probably match the filename in the remote _n_e_w_s_- _f_e_e_d_s entry) and send the batch to the remote site (using _b_a_t_c_h_e_r, often called by _s_e_n_d_-_u_u_c_p or _s_e_n_d_b_a_t_c_h). 22.. FFeeeeddiinngg aa llaarrggee nnuummbbeerr ooff ssiitteess _I_n_n_d tries to keep as many batchfiles open for as long as possible. It will normally open as many as it can, using all the available descriptors minus a fixed number for internal use (log files, etc.). You can explicitly set the number of files to open by using the ``-o'' flag. If you have more outgoing feeds than available descrip- tors, _i_n_n_d will recycle the files on a a ``least recently used'' basis. If most of your feeds get most articles (or you have vastly more feeds then available descriptors), this will lead to ``file thrashing,'' closing and opening all the excess feeds on each article. To reduce this, you can have _i_n_n_d use an internal buffer for a site by using the ``I'' parameter in the _n_e_w_s_f_e_e_d_s file. If a site does not have its batchfile open, the server will not try to open it until there is more data to be written then will fit in the buffer. For example, suppose _i_n_n_d was started with ``-o10'' and there are 12 sites, all with ``I512'' in their _n_e_w_s_f_e_e_d_s entry. If each article generates 50 bytes (a pathname and a Message-ID), then _i_n_n_d will close and re-assign two descrip- tors every 10 or so articles. A better alternative is to use funnels and an exploder. Funnels, specified in the _n_e_w_s_f_e_e_d_s file, let multiple sites send their output down a single stream. The advantage of funnels is that this stream can be a channel; the primary disadvantage is that the funnel specifies what data is to be written, not the individual sites. (Since most feeds will want either ``Wn'' or ``Wnm'' entries, this is usually not a problem.) February 14, 1992 -47- In order for the funnel output to be useful, it usually must be split into individual, per-site, files. Programs that do this type of splitting are called ``exploders.'' INN provides two exploders, _f_i_l_e_c_h_a_n and _b_u_f_f_c_h_a_n. _F_i_l_e_c_h_a_n is the simplest, and most inefficient, exploder. It does not keep any files open and is very sys- tem-call intensive. It can be used to provide behavior (and performance!) that is similar to B2.11 _i_n_e_w_s. It can, how- ever, be used as the funnel for an unlimited number of sites. _B_u_f_f_c_h_a_n keeps all its output files open all the time. It should not be used for more sites then a single process can have open. _B_u_f_f_c_h_a_n also has flags to automatically flush its files, as well as close and re-open them, every specified number of articles. (The re-open capability is useful for things like _n_n_t_p_l_i_n_k in its ``watch the batch- file'' mode.) Using _b_u_f_f_c_h_a_n with the ``-l1 -c50'' flags will give behavior that is similar to the C News _r_e_l_a_y_n_e_w_s. _B_u_f_f_c_h_a_n can be run as a full exploder (``Tx'') in the _n_e_w_s_f_e_e_d_s file. This means that you can use _c_t_l_i_n_n_d to send a command line down _b_u_f_f_c_h_a_n's input stream. (_I_n_n_d will also send a command whenever newsgroups are modified.) The only useful message is ``flush'' which will close, and re- open, the specified site files. You should also note that the flow is one-way; full exploders cannot send any acknowl- edgement back. 33.. MMaasstteerr//ssllaavvee rreepplliiccaattiioonn INN supports a simple model of replicated news databases: a single master host pushes out updates to its slaves. The master is the only host that receives articles -- this includes both outside newsfeeds and articles written by local users. The slaves only receive articles from the master. No special work is required to set up a master host. A slave is set up by starting _i_n_n_d with the ``-S'' flag to specify the name or IP address of the master host. This should be done by modifying the ``FLAG'' variable in your ___P_A_T_H___N_E_W_S_B_O_O_T script. If _i_n_n_d is started with the ``-S'' flag it will pass this flag on to _n_n_r_p_d. This means that when anyone connects to the slave and does a ``POST'' com- mand, _n_n_r_p_d will connect to the master and offer the arti- cle. You must make sure that the slave's entry in the mas- ter's _n_e_w_s_f_e_e_d_s file sends all articles (e.g., don't have any exclusions). Since the _n_n_r_p_d on the slave host will usually add its name to the Path header, you should add February 14, 1992 -48- ``Ap'' to the _f_l_a_g_s field of the slave's entry on the mas- ter. Once the slave has been set up it is necessary to have the master feed it. This is done by using an extension to the NNTP protocol. This extension, the ``XREPLIC'' command, is documented in _i_n_n_d_._8. In order to do this you will have to set up a _n_e_w_s_f_e_e_d_s entry for the slave. This should be a standard entry except that you will need to have both the filename and the replication information written int the batchfile. To do this, put ``WnR'' in the _f_l_a_g_s field of the entry. When you want to actually send the articles to the slave site you will have to specify the ``-S'' flag in your _i_n_n_x_m_i_t command. Current versions of _n_n_t_p_l_i_n_k use the ``-x'' flag. When running as a slave, _i_n_n_d is very paranoid about staying synchronized with its master. Most noticeably, you should make sure that all newgroup and rmgroup control mes- sages are handled identically on both systems. Finally, note that postings made on the slave (via _n_n_r_p_d) are actualy sent to the _i_n_n_d on the master, so the slave must be treated as a sending host, not a newsreading host. February 14, 1992 -49- AAppppeennddiixx IIVV:: FFiirrsstt--ttiimmee UUsseenneett oorr NNNNTTPP IInnssttaallllaattiioonn Since the needs and administration of systems varies so much, I can only give some general guidelines and advice in this section. Like UNIX system administration in general, it is unfortunately still true that most of the job will be learned ``in the heat of the moment.'' Once you have INN set up, however, it should not require much attention. For general problems, try posting to ``news.admin.misc''; use ``news.software.nntp'' and ``news.software.b'' for installa- tion problems. Once all the software has been compiled and installed, you must now get a newsfeed. This involves having one (or more) sites pass along to you all the articles that they have received. Getting articles is a passive action, because it is generally more efficient that way. (The _n_n_t_p_g_e_t program is primarily a debugging aide and utility program. It is not the recommended way to get a newsfeed, and most sites will prefer you not to use it for that.) If you already have Usenet access, you could post a note to ``news.admin.misc'' asking for a feed. Make sure to say that you are looking for an NNTP connection! If you are a member of an NSFNet regional network, or subscribe to a commercial IP network, ask your contact there at the network center. If they do not provide feeds directly, they can probably help you find one. You also might try writing to the mailing list. This will reach the news administrators of many NNTP sites on the Internet. (If you want to join the list, remember to send it to nntp-managers-request, nnoott nntp-managers!) Once have a site willing to give you a feed, you need to get the list of groups that they will give you. You also need to create those groups on your machine. The easiest way to do this is usually to ask them for a copy of their active file, and for you to add the entries of the groups that you're interested in. The location of the active file is system-dependent, but the manpage should tell you (or the other administrator) where it is, often within the first paragraph. If your feed can't send you their active file, then you might want to find a more competent feed! The fol- lowing command will zap an active file, setting up the arti- cle numbers for a new site: sed active.new \ -e 's/^\([^ ]*\) [0-9]* [0-9]* \([^ ]*\)$/\1 0000000000 000000001 \2/' Once the groups are set up, your newsfeed will periodi- cally connect to your NNTP server and offer it any new arti- cles that have arrived since the last connection. _I_n_n_d will accept the connection, receive the articles, and queue them February 14, 1992 -50- up for any sites that you feed. The next step is to set it up so that your articles are sent back to your newsfeed. To do this, create a _n_e_w_s_f_e_e_d_s entry, using the same name that shows up in the Path header that you see. (If you use a different name, then use the ``excludes'' sub-field to avoid offering back everything they offer you.) This is usually done by giving them all non-local articles as a file feed. For example, ``Foo, Incorporated'' does not give any foo.* articles to anyone else. When someone at your site writes an article, _i_n_n_d will record the filename in the batch file for your upstream site. Either _s_e_n_d_-_n_n_t_p or _n_n_t_p_s_e_n_d will flush and lock the batchfile, and then call _i_n_n_x_m_i_t to connect to the remote site and send these queued articles out. You should edit the script to list the sites you want, and arrange for _c_r_o_n to run this script on a regular basis. You can run it as often as you like, but 10 minutes is a common interval. If you want to feed any sites via UUCP, then you will have to set up file feed entries for them in the _n_e_w_s_f_e_e_d_s file, and arrange to have _c_r_o_n run the _s_e_n_d_-_u_u_c_p script as desired. (UUCP batches are typically only done every few hours.) Once you have news flowing in and out of the system, you will have to expire it or your disks will fill up. The _n_e_w_s_._d_a_i_l_y script should be run by _c_r_o_n in the middle of the night. It will summarize that day's log files, and then call _e_x_p_i_r_e to purge old news. You might also want to have _c_r_o_n run _r_n_e_w_s hourly to pick up any stalled batches. Finally, if your feeds change IP address, you might want a daily job that does ``ctlinnd reload hosts.nntp "flush cache"''. This is because _i_n_n_d does not currently time-out DNS entries. You will generally want to set up the _c_r_o_n jobs so that they are run as the news administrator, and not as root. A good version of _c_r_o_n that makes it easy to do this can be found on gatekeeper.dec.com in pub/misc/vixie/cron.tar.Z. You will also need to get one or more programs to read news. There are several freely-available programs around. _R_n is popular, and is probably the best place to start. The official distribution is available for anonymous FTP at tmc.edu in the _r_n directory. Welcome to Usenet, and have fun! February 14, 1992 -51- AAppppeennddiixx VV:: NNeewwss oovveerrvviieeww ddaattaabbaassee There are now many newsreaders available that are able to do ``threading.'' This is the ability to track a discus- sion within a newsgroup by using the References header (or other data), regardless of changes in headers like the Sub- ject line. Examples of these readers include _n_n, _t_r_n, and _g_n_u_s, and more are becoming available. Until recently, a major problem with these readers is that they all required a specialized external database that contained the threading data. In late 1992, Geoff Collyer released the _n_o_v, or ``news overview,'' package. This included database tools and, and client access routines, that let the current threaded newsreaders use a common, tex- tual, database. An overview database typically adds adds about 7-9% to your storage requirements. By default, the overview files are stored in the spool directory; you can change this to use an alternate tree that mirrors the spool hierarchy by changing the ___P_A_T_H___O_V_E_R_V_I_E_W_D_I_R parameter. INN includes full support for creating and expiring news overview databases. To do this, add an entry like the following to your _n_e_w_s_f_e_e_d_s file: overview:*:Tc,WO:/path/to/bin/overchan (Make sure to replace _/_p_a_t_h_/_t_o_/_b_i_n with the value of your ___P_A_T_H___N_E_W_S_B_I_N parameter.) Then reload the _n_e_w_s_f_e_e_d_s file or restart your server. To create the initial database, run the following command after you have started _o_v_e_r_c_h_a_n: expireover -a -s You will also need to expire the overview data. The easiest way to do this is to add the ``expireover'' keyword to the _c_r_o_n job that runs _n_e_w_s_._d_a_i_l_y. The _n_n_r_p_d server includes two command extensions to access the database; they are documented in the ``protocol extensions'' part of _d_o_c_/_n_n_r_p_d_._8. INN does not include any client code or modifications to any newsreaders to use the overview data. Most maintainers have agreed to support the overview database, including the INN extensions for remote access. You can find prototype versions of many readers (work done by Geoff) on world.std.com in the directory src/news; look for files named _r_e_a_d_e_r.dist.tar.Z. February 14, 1992 -52- AAppppeennddiixx VVII:: LLiimmiitteedd MMIIMMEE SSuuppppoorrtt This version of INN includes limited support for MIME, the Multipurpose Internet Mail Extensions, described in RFC 1341. The support is the ability to do limited transport of arbitrary MIME messages, and _n_n_r_p_d can add MIME headers to all local postings that do not have them. In addition, there are patches available for _n_n_t_p_l_i_n_k that allow it to do MIME transport. The patches are not (yet) part of the official release; if you need them, con- tact Christophe Wolfhugel ; he did most of the INN work, too. You should be very careful if you have _n_n_r_p_d add MIME headers. To do this, edit _i_n_n_._c_o_n_f as indicated in _d_o_c_/_i_n_n_._c_o_n_f_._5. Once this is done, aallll articles posted will get MIME headers added. Existing MIME headers will not be modified, but missing ones will be added. The default val- ues to add to _i_n_n_._c_o_n_f are these: mime-version: 1.0 mime-contenttype text/plain; charset=us-ascii mime-encoding: 7bit An internationalized site might want to use these values: mime-version: 1.0 mime-contenttype: text/plain; charset=iso-8859-1 mime-encoding: 8bit It is possible to use these values because INN provides a clean eight-bit data path. Unless you make special arrange- ments with your peers, however, you must transmit seven-bit data. Doing this will require special transmit agents. Note that _n_n_r_p_d is not a Mime-compatible reader. You must have software to extract the data and present it appropri- ately. If you configure your site to use seven-bit data, then you must also make sure that none of your software creates eight-bit articles. _N_n_r_p_d does not verify this. If you configure your site to use eight-bit data, then ASCII works fine, but remember that in quoted-printable long lines are cut and that the equal sign (``='') is quoted; this is really bad for source code postings, among others. The character set can also cause problems. If you use ``iso-8859-1'' you must make sure that your posting software uses this character set (e.g., not CP-437 under MS-DOS) because _n_n_r_p_d does not do any conversion. In general, be very cautious. February 14, 1992 -53- MIME articles can only be sent using _i_n_n_x_m_i_t; work on _b_a_t_c_h_e_r is in progress. Unless the ``-M'' flag is used, no MIME conversions are done. If the flag is used, the follow- ing happens: Articles with a Content-Transfer-Encoding header of ``8bit'' or ``binary'' are forwaded in ``quoted- printable'' format (the ``base64'' format will be available soon). All other articles -- in particular, those without MIME headers, those of type ``message'' or ``multipart,'' those with Content-Transfer-Encoding header of ``7bit'' -- are forwarded without any change. February 14, 1992