newsfeeds - determine where Usenet articles get sent


DESCRIPTION

       The  file  <pathetc  in  inn.conf>/newsfeeds specifies how
       incoming articles should be distributed  to  other  sites.
       It  is  parsed  by the InterNetNews server innd(8) when it
       starts up, or when directed to by ctlinnd(8).

       The file is interpreted as a set of lines according to the
       following  rules.   If  a line ends with a backslash, then
       the backslash, the newline,  and  any  whitespace  at  the
       start of the next line is deleted.  This is repeated until
       the entire ``logical'' line is collected.  If the  logical
       line is blank, or starts with a number sign (``#''), it is
       ignored.

       All other lines are interpreted as feed entries.  An entry
       should  consist of four colon-separated fields; two of the
       fields may have  optional  sub-fields,  marked  off  by  a
       slash.  Fields or sub-fields that take multiple parameters
       should be separated by  a  comma.   Extra  whitespace  can
       cause  problems.   Except for the site names, case is sig­
       nificant.  The format of an entry is:
              sitename[/exclude,exclude,...]\
                   :pattern,pattern...[/distrib,distrib...]\
                   :flag,flag...\
                   :param
       Each field is described below.

       The sitename is the name of the site to which a news arti­
       cle  can  be sent.  It is used for writing log entries and
       for determining if an article should  be  forwarded  to  a
       site.   If  sitename already appears in the article's Path
       header, then the article will not be  sent  to  the  site.
       The name is usually whatever the remote site uses to iden­
       tify itself in the Path line, but can be almost  any  word
       that makes sense; special local entries (such as archivers
       or gateways) should probably end with an exclamation point
       to  make  sure  that they do not have the same name as any
       real site.  For example, ``gateway'' is  an  obvious  name
       for  the local entry that forwards articles out to a mail­
       ing list.  If a site with the name  ``gateway''  posts  an
       article,  when the local site receives the article it will
       see the name in the Path and not send the article  to  its
       own  ``gateway''  entry.   See also the description of the
       ``Ap'' flag, below.  If an entry  has  an  exclusion  sub-
       field,  then  the article will not be sent to that site if
       any of the names specified as excludes appear in the  Path
       header.   The  same sitename can be used more than once --
       the appropriate action will be taken for each  entry  that
       should  receive  the  article,  regardless  of the name --
       although this is recommended only  for  program  feeds  to
       avoid confusion.  Case is not significant in site names.

       are interpreted to build a ``subscription list''  for  the
       site.  The default subscription is to get all groups.  The
       patterns in the field are wildmat(3)-style  patterns,  and
       are  matched  in order against the list of newsgroups that
       the local site receives.  If the first character of a pat­
       tern  is an exclamation mark, then any groups matching the
       pattern are removed from the subscription,  otherwise  any
       matching  groups  are  added.  For example, to receive all
       ``comp'' groups, but  only  comp.sources.unix  within  the
       sources  newsgroups,  the following set of patterns can be
       used:
              comp.*,!comp.sources.*,comp.sources.unix
       There are three things to note about  this  example.   The
       first is that the trailing ``.*'' is required.  The second
       is that, again, the result of the last match is  the  most
       important.   The third is that ``comp.sources.*'' could be
       written as ``comp.sources*'' but this would not  have  the
       same effect if there were a ``comp.sources-only'' group.

       There  is  also  a  way  to subscribe to a newsgroup nega­
       tively.  That is to say, do not send this  group  even  if
       the article is cross-posted to a subscribed newsgroup.  If
       the first character of a pattern is an  atsign  ``@'',  it
       means that any article posted to a group matching the pat­
       tern will not be sent  even  though  the  article  may  be
       cross-posted  to  a  group  which is subscribed.  The same
       rules of precedence apply in that the last  match  is  the
       one which counts.  For example, if you want to prevent all
       articles posted to  any  "alt.binaries.warez"  group  from
       being  propagated  even  if  it is cross-posted to another
       "alt" group or any other group for that matter,  then  the
       following set of patterns can be used:
              alt.*,@alt.binaries.warez.*,misc.*
       If  you  reverse  the  alt.* and alt.binaries.warez.* pat­
       terns, it would nullify the atsign because the  result  of
       the  last  match  is the one that counts.  Using the above
       example, if an article is posted to one  or  more  of  the
       alt.binaries.warez.*   groups   and   is  cross-posted  to
       misc.test, then the article is not sent.

       See innd(8) for details on the propagation of control mes­
       sages.

       A  subscription  can  be  further  modified  by specifying
       ``distributions'' that  the  site  should  or  should  not
       receive.  The default is to send all articles to all sites
       that subscribe to any of the  groups  where  it  has  been
       posted  ,  but if an article has a Distribution header and
       any distribs are specified, then they are checked  accord­
       ing to the following rules:

       1.     If  the Distribution header matches any of the val­
              ues in the sub-field, then the article is sent.

              it  matches the Distribution header, then the arti­
              cle is not sent.

       3.     If Distribution header does not match  any  distrib
              in  the  site's  entry, and no negations were used,
              then the article is not sent.

       4.     If Distribution header does not match  any  distrib
              in  the  site's entry, and any distrib started with
              an exclamation point, then the article is sent.

       If an article has more than  one  distribution  specified,
       then  each one is according to the above rules.  If any of
       the specified  distributions  indicate  that  the  article
       should  be  sent, it is; if none do, it is not sent -- the
       rules are used as a ``logical or.''  It  is  almost  defi­
       nitely a mistake to have a single feed that specifies dis­
       tributions that start with an exclamation point along with
       some that don't.

       Distributions  are  text words, not patterns; entries like
       ``*'' or ``all'' have no special meaning.

       The flags parameter  specifies  miscellaneous  parameters.
       They may be specified in any order; flags that take values
       should have the value immediately after  the  flag  letter
       with no whitespace.  The valid flags are:

       <size  An  article  will only be sent to the site if it is
              less than size  bytes  long.   The  default  is  no
              limit.

       >size  An  article  will only be sent to the site if it is
              greater than size bytes long.  The  default  is  no
              limit.

       Achecks
              An  article  will  only  be  sent to the site if it
              meets the requirements  specified  in  the  checks,
              which should be chosen from the following set:
                   c    Exclude all kinds of control messages
                   C    Only include all kinds of control messages
                   d    Distribution header required
                   e    Only include message whose newsgroups in
                        Newsgroups header all exist in active
                   o    Overview data is created
                   O    Propagate articles without X-Trace header
                        even if ``O'' flag is specified
                   p    Do not check Path header for the sitename
                        before propagating (the exclusions are
                        still checked).
              If  both  ``c''  and ``C'' are specified simultane­
              ously, the last specified one is adopted.

              If a site is being  fed  by  a  file,  channel,  or
              exploder  (see  below),  the  server  will normally
              start trying to write the information  as  soon  as
              possible.   Providing a buffer may give better sys­
              tem performance and help smooth out overall load if
              a  large  batch of news comes in.  The value of the
              this flag should be  two  numbers  separated  by  a
              slash.   The first specifies the point at which the
              server can start draining the  feed's  I/O  buffer,
              and  the  second specifies when to stop writing and
              begin buffering again; the units  are  bytes.   The
              default  is  to  do no buffering, sending output as
              soon as it is possible to do so.

       Ccount If this flag is specified, an article will only  be
              sent to the site if the following is true: The num­
              ber of groups it is posted to, plus the  square  of
              the  number of groups followups would appear in, is
              no more than count newsgroups.  30 is a good  value
              for this flag.

       Fname  This  flag  specifies  the  name  of  the file that
              should be used if it is necessary to begin spooling
              for  the site (see below).  If name is not an abso­
              lute pathname,  it  is  taken  to  be  relative  to
              <pathoutgoing in inn.conf>.   Then, if the destina­
              tion is a directory, the file togo in  that  direc­
              tory will be used as filename.

       Gcount If  this flag is specified, an article will only be
              sent to the site if it is posted to  no  more  than
              count  newsgroups.  This has the problem of filter­
              ing out many FAQs, and  also  RFDs/CFVs  for  group
              creation. The C or U flags are a better solution.

       Hcount If  this flag is specified, an article will only be
              sent to the site if it has count or fewer sites  in
              its  Path line.  This flag should only be used as a
              rough guide because of the loose interpretation  of
              the  Path  header; some sites put the poster's name
              in the header, and some sites that might  logically
              be considered to be one hop become two because they
              put the posting workstation's name in  the  header.
              The default value for count is one.

       Isize  The  flag specifies the size of the internal buffer
              for a file feed.  If there are more file feeds than
              allowed by the system, they will be buffered inter­
              nally in least-recently-used order.  If the  inter­
              nal  buffer  grows bigger then size bytes, however,
              the data will be written  out  to  the  appropriate
              file.   The  default  value is <SITE_BUFFER_SIZE in
              include/config.h> (typically (16 * 1024 )) bytes.

              The newsgroups that a site  receives  are  modified
              according  to the modifiers, which should be chosen
              from the following set:
                   m    Only moderated groups
                   u    Only unmoderated groups

       OOriginator
              If this flag is used then  articles  sent  to  this
              feed  must  contain  a X-Trace header and the first
              field in the header must match originator.   Origi­
              nator  can  be  a list of wildmat(3)-style pattern.
              The list is separated by ``/''.  Article  is  never
              sent,  if the first character of the pattern begins
              with ``@'' and rest of the  pattern  matches.   One
              use of this flag is to restrict the feed to locally
              generated posts.

       Ppriority
              The nice priority that this channel or program feed
              should  receive.   This should be a positive number
              between 0 and 20, and is the priority that the  new
              process  will  run  with.  This flag can be used to
              raise the priority to normal if you are  using  the
              ``nicekids'' in inn.conf(5).

       Ssize  If  the  amount of data queued for the site gets to
              be larger than size bytes,  then  the  server  will
              switch  to  spooling, appending to a file specified
              by   the   ``F''   flag,   or   <pathoutgoing    in
              inn.conf>/sitename  if the ``F'' flag is not speci­
              fied.  Spooling usually happens only for channel or
              exploder feeds.

       Ttype  This  flag specifies the type of feed for the site.
              Type should be a letter chosen from  the  following
              set:
                   c    Channel
                   f    File
                   l    Log entry only
                   m    Funnel (multiple entries feed into one)
                   p    Program
                   x    Exploder
              Each feed is described below in the section on feed
              types.  The default is Tf.

       Ucount If this flag is specified, an article will only  be
              sent to the site if followups to this article would
              be posted to no more than count newsgroups.

       Witems If a site is fed by  file,  channel,  or  exploder,
              this flag controls what information is written.  If
              a site is fed  by  a  program,  only  the  asterisk
              (``*'') has any effect.  The items should be chosen

                   b    Size of wire formatted article in bytes
                   e    The time article will be expired as
                        seconds since epoch
                        ``0'' means there is no ``Expires'' header
                   f    Token of the article (equivalent to ``n'')
                   g    The newsgroup the article is in;
                        if cross-posted, then the first of the
                        groups this site gets (the differnece from
                        ``G'' is that article belongs to its
                        newsgroup even if it is congrol message)
                   h    Article's Message-ID hash key
                   m    Article's Message-ID
                   n    Token of the article
                   p    The time the article was posted as seconds
                        since epoch
                   s    The site that fed the article to the
                        server; from the Path: header or the IP
                        address of the site that sent the article
                        depending on the ``logipaddr'' field in
                        inn.conf(8);
                        if ``logipaddr'' is ``true'' and IP
                        address is ``0.0.0.0'' which means the
                        article is feeded from localhost (e.g.
                        from rnews), the result will be retrieved
                        from Path: header
                   t    The time article was received as seconds
                        since epoch
                   *    Names of the appropriate funnel entries;
                        or all sites that get the article
                   D    Value of the Distribution header; ? if
                        none present
                   G    Where the article is stored; if cross-
                        posted, then the first of the groups this
                        site gets (the differnece from ``g'' is
                        that cmsg belongs to ``control.cmsg'')
                   H    All headers
                   N    Value of the Newsgroups header
                   P    Article's Path header
                   O    Overview data
                   R    Information needed for replication
              More than one letter can be used; the entries  will
              be  separated  by a space, and written in the order
              in which they are specified.  The default is Wn.

              The ``H'' and ``O'' items are intended for  use  by
              programs  that  create news overview databases.  If
              ``H'' is present, then the all the article's  head­
              ers  are written followed by a blank line.  An Xref
              header (even if one does not appear  in  the  filed
              article)  and  a Bytes header, specifying the arti­
              cle's size (see ``b'' description below for  clari­
              fying size meaning), will also be part of the head­
              ers.  If used, this should be the only item in  the

              line will  be  written  before  the  headers.   The
              ``nteO''  generates  input  to the overchan(8) pro­
              gram.

              The asterisk has special meaning.  It expands to  a
              space-separated list of all sites that received the
              current article.  If the site is the  target  of  a
              funnel  however  (i.e.,  it is named by other sites
              which  have  a  ``Tm''  flag),  then  the  asterisk
              expands  to  the  names  of  the  funnel feeds that
              received the article.  If the site is fed by a pro­
              gram,  then  an asterisk in the param field will be
              expanded  into  the  list  of  funnel  feeds   that
              received the article.  A site fed by a program can­
              not get the site list unless it is  the  target  of
              other ``Tm'' feeds.

       The  interpretation of the param field depends on the type
       of feed, and is explained in more detail below in the sec­
       tion on feed types.  It can be omitted.

       The  site  named ME is special.  There must be exactly one
       such entry, and it should be the first entry in the  file.
       If the ME entry has an exclusion sub-field, then the arti­
       cles are  rejected  if  any  of  the  names  specified  as
       excludes appear in the Path header.  If the ME entry has a
       subscription  list,  then  that  list   is   automatically
       prepended  to  the subscription list of all other entries.
       For example, ``*,!control,!junk,!foo.*'' can  be  used  to
       set up the initial subscription list for all feeds so that
       local postings are not propagated unless ``foo.*'' explic­
       itly  appears  in the site's subscription list.  Note that
       most subscriptions should have ``!junk,!control'' in their
       pattern  list;  see the discussion of ``control messages''
       in innd(8).  (Unlike other  news  software,  it  does  not
       affect  what  groups  are  received;  that  is done by the
       active(5) file.)

       If the ME entry has a  distribution  subfield,  then  only
       articles  that  match  the distribution list are accepted;
       all  other  articles  are  rejected.   A  commercial  news
       server,  for  example,  might  have  ``/!local'' to reject
       local postings from other, misconfigured, sites.


FEED TYPES

       Innd provides four basic types of feeds: log,  file,  pro­
       gram, and channel.  An exploder is a special type of chan­
       nel.  In addition, several entries can feed into the  same
       feed;  these are funnel feeds, that refer to an entry that
       is one of the other types.  Note that the term ``feed'' is
       technically a misnomer, since the server does not transfer
       articles, but reports that an article should  be  sent  to
       the site.

       Other   than   a   mention   in    the    news    logfile,
       <pathlog in inn.conf>/news,  no  data is ever written out.
       This is equivalent to a ``Tf'' entry writing to  /dev/null
       except that no file is opened.

       A  site  fed  by a file is the next simplest type of feed.
       When the site should receive an article, one line is writ­
       ten to the file named by the param field.  If param is not
       an absolute pathname,  it  is  taken  to  be  relative  to
       <pathoutgoing in inn.conf>.    If   empty,   the  filename
       defaults  to  <pathoutgoing in inn.conf>/sitename.    This
       name should be unique.

       When  a  site  fed by a file is flushed (see ctlinnd), the
       following steps are performed.  The script doing the flush
       should  have  first renamed the file.  The server tries to
       write out any buffered data, and  then  closes  the  file.
       The  renamed  file  is  now available for use.  The server
       will then re-open the original file, which  will  now  get
       created.

       A  site  fed  by a program has a process spawned for every
       article that the site receives.  The param field must be a
       sprintf(3) format string that may have a single %s parame­
       ter, which will be a token.  Standard input  will  not  be
       set to the article.  Standard output and error will be set
       to the error log ( <pathlogininn.conf>/errlog.)  The  pro­
       cess  will  run with the user and group ID of the <pathrun
       in inn.conf> directory.  Innd will try to avoid spawning a
       shell  if  the  command has no shell meta-characters; this
       feature can be defeated by appending a semi-colon  to  the
       end  of  the command.  The full pathname of the program to
       be run must be specified; for security,  PATH  environment
       is not searched.

       If  the entry is the target of a funnel, and if the ``W*''
       flag is used, then a single asterisk may be  used  in  the
       param  field where it will be replaced by the names of the
       sites that fed into the funnel.  If the  entry  is  not  a
       funnel, or if the ``W*'' flag is not used, then the aster­
       isk has no special meaning.

       Flushing a site fed by a program does no action.

       When a site is fed by a channel  or  exploder,  the  param
       field  names  the process to start.  Again, the full path­
       name of the process must be given.  When the  site  is  to
       receive  an  article,  the  process receives a line on its
       standard input telling it  about  the  article.   Standard
       output  and  error,  and  the user and group ID of the all
       sub-process are set as for a program feed, above.  If  the
       process  exits, it will be restarted.  If the process can­
       not be started, the server will  spool  input  to  a  file

       try to start the process some time later.

       When a site fed by a channel or exploder is  flushed,  the
       server  closes down its end of the pipe.  Any pending data
       that has  not  been  written  will  be  spooled;  see  the
       description  of the ``S'' flag, above.  No signal is sent;
       it is up to the program to  notice  EOF  on  its  standard
       input and exit.  The server then starts a new process.

       Exploders are a superset of channel feeds.  In addition to
       channel behavior, exploders can  be  sent  command  lines.
       These  lines  start  with  an exclamation point, and their
       interpretation is up to the exploder.  The following  mes­
       sages are generated automatically by the server:
              newgroup group
              rmgroup group
              flush
              flush site
       These  messages  are  sent when the ctlinnd command of the
       same name is received by the  server.   In  addition,  the
       ``send''  command can be used to send an arbitrary command
       line to the exploder child-process.  The primary  exploder
       is buffchan(8).

       Funnel feeds provide a way of merging several site entries
       into a single output stream.  For a site  feeding  into  a
       funnel,  the  param field names the actual entry that does
       the feeding.


EXAMPLES

              ##  Initial subscription list and our distributions.
              ME:*,!junk,!foo.*/world,usa,na,ne,foo,ddn,gnu,inet\
                   ::
              ##  Feed all moderated source postings to an archiver
              source-archive!:!*,*sources*,!*wanted*,!*.d\
                   :Tc,Wn:<PREFIX specified with --prefix at configure>/bin/archive -f -i \
                       /usr/spool/news.archive/INDEX
              ##  Watch for big postings
              watcher!:*\
                   :Tc,Wbnm\
                   :exec awk '$1 > 1000000 { print "BIG", $2, $3 }' >/dev/console
              ##  A UUCP feed, where we try to keep the "batching" between 4 and 1K.
              ihnp4:/world,usa,na,ddn,gnu\
                   :Tf,Wnb,B4096/1024:
              ##  Usenet as mail; note ! in funnel name to avoid Path conflicts.
              ##  Can't use ! in "fred" since it would like look a UUCP address.
              fred:!*,comp.sources.unix,comp.sources.bugs\
                   :Tm:mailer!
              barney@bar.com:!*,news.software.nntp,comp.sources.bugs\
                   :Tm:mailer!
              mailer!:!*\
                   :W*,Tp:/usr/ucb/Mail -s "News article" *
              ##  NNTP feeds fed off-line via nntpsend or equivalent.

              peer.foo.com:foo.*:Tf,Wnm:peer.foo.com
              ##  Real-time transmission.
              mit.edu:/world,usa,na,ne,ddn,gnu,inet\
                   :Tc,Wnm:<PREFIX specified with --prefix at configure>/bin/nntplink -i stdin mit.edu
              ##  Two sites feeding into a hypothetical NNTP fan-out program:
              nic.near.net:\
                   :Tm:nntpfunnel1
              uunet.uu.net/uunet:!ne.*/world,usa,na,foo,ddn,gnu,inet\
                   :Tm:nntpfunnel1
              nntpfunnel1:!*\
                   :Tc,Wmn*:<PREFIX specified with --prefix at configure>/bin/nntpfanout
              ##  A UUCP site that wants comp.* and moderated soc groups
              uucpsite!comp:!*,comp.*/world,usa,na,gnu\
                   :Tm:uucpsite
              uucpsite!soc:!*,soc.*/world,usa,na,gnu\
                   :Tm,Nm:uucpsite
              uucpsite:!*\
                   :Tf,Wnb:/usr/spool/batch/uucpsite

       The last two sets of entries show how funnel feeds can  be
       used.   For  example, the nntpfanout program would receive
       lines like the following on its standard input:
              <123@litchi.foo.com> comp/sources/unix/888 nic.near.net uunet.uu.net
              <124@litchi.foo.com> ne/general/1003 nic.near.net
       Since the UUCP funnel is only destined for one  site,  the
       asterisk is not needed and entries like the following will
       be written into the file:
              <qwe#37x@snark.uu.net> comp/society/folklore/3
              <123@litchi.foo.com> comp/sources/unix/888


HISTORY

       Written by Rich $alz  <rsalz@uunet.uu.net>  for  InterNet­
       News.  This is revision 1.29.2.1, dated 2000/08/17.


SEE ALSO

       active(5),  buffchan(8), ctlinnd(8), inn.conf(5), innd(8),
       wildmat(3).


Man(1) output converted with man2html