actsync, actsyncd - synchronize newsgroups


SYNOPSIS

       actsync [-b hostid] [-d hostid] [-g max] [-i ignore_file]
               [-I hostid] [-k] [-l hostid] [-m] [-n name]
               [-o fmt] [-p min_%_unchg] [-q hostid] [-s size]
               [-s spool_dir] [-t hostid] [-T] [-v verbose_lvl]
               [-z sec] [host1] host2

       actsyncd [-x] actsync.cfg [debug_level [debug_outfmt] ]


DESCRIPTION

       Actsync(8)  permits  one  to synchronize, compare or merge
       two active(5) files.   With  this  utility  one  may  add,
       change  or  remove  newsgroups on the local news server to
       make it similar  to  the  list  the  newsgroups  found  on
       another  system  or file.  The synchronization need not be
       exact.  Local differences in newsgroup lists may be  main­
       tained  and  preserved.   Certain  newsgroup errors may be
       detected and optionally corrected.

       There are several  reasons  to  run  actsync(8)  (or  act­
       syncd(8)), on a periodic basis.  Among the reasons are:

            A  control  message  to add, change or remove a news­
            group may fail to reach your site.

            Your control.ctl(5) is out of date or incomplete.

            News articles for a new newsgroup arrive ahead (some­
            times days ahead) of the control message.

            Control  messages  may  be forged, thus bypassing the
            restrictions found in control.ctl(5).

            Your active(5) file may have been trashed.

       If either host1 or host2 begin with  a  ``.''   or  ``/'',
       then they assumed to be a name of a file containing infor­
       mation in the active(5) format.   The  getlist(1)  utility
       may  be  used to obtain copy a remote system's active file
       via its NNTP server, or an FTP  client  program   can  get
       retrieve  such  a  file  from  an  FTP  archive  (such  as
       ftp://ftp.isc.org/pub/usenet/CONFIG/active; see more about
       this  below).   Newsgroup  information  from a file may be
       treated as if it was obtained from a host.   In  this  man
       page  host1  and  host2 are called hosts, even though they
       may be file names.

       If a host argument does not begin with ``.''  or ``/'', is
       assumed  to  be  a  hostname or Internet address.  In this
       case, actsync(8) will attempt to use the NNTP protocol  to

       If the host argument contains a ``:''  ,  the  right  side
       will  be  considerd  the  port to connect to on the remote
       system.  If no port number ist specified, actsync(8)  will
       connect to port 119.

       Regardless  how  the  active file information is obtained,
       the actions of actsync(8) remain the same.

       If only one host is specified, it is assumed to be  host2.
       If  host1,  is not specified, it assumed to be the default
       local NNTP server as specified by the NNTPSERVER  environ­
       ment   variable,   or   by   the  server  value  found  in
       inn.conf(5).

       The newsgroup synchronization  by  default,  involves  all
       newsgroups  found on both hosts.  One may also synchronize
       on a subset  of  newsgroups  by  directing  actsync(8)  to
       ignore certain newsgroups from both systems.

       The  actsyncd(8) daemon provides a convenient interface to
       configure and run actsync(8).  If a host is not  initially
       reachable, the daemon will thrice retry 9 times, waiting 6
       minutes before each retry.  This daemon runs in the  fore­
       ground,  sending  output  to  standard output and standard
       error.

       If the -x flag is given to actsyncd(8), then  a  ctlinndx­
       exec  will  be used instead of a ctlinndreload to load the
       newly modified active file.

       The configuration filename for the daemon is given in  the
       actsync.cfg  argument.   The  actsync.cfg file understands
       the following options:

            host=host2
            ftppath=/remote/path/to/active/file
            spool=<normally patharticles in inn.conf>
            ignore_file=ignore_file
            flags=actsyncd (8) options

       The host, ignore_file and flags lines are mandatory.

       The keyword must start at the beginning of the  line,  and
       there  may  be  no  whitespace before the ``='' character.
       Blank lines are ignored.  Comments start  with  ``#''  and
       are  ignored.   All  other  lines  may  produce  undefined
       results.

       The host config file line refers to  the  host2  value  to
       sync  off  of.   The  ftppath directive causes the machine
       named in the host line  to  accessed  as  an  ftp  server,
       retrieving the file named.  If the filename ends in .gz or
       .Z, then  it  will  automatically  be  uncompressed  after

       top the news spool tree is to be found.   The  ignore_file
       config  file line names the ignore file to be used by act­
       sync(8).  The flags config file line refers to  all  flags
       that you wish to pass to actsync(8).

       Note  that  the  -i  ignore_file  option and the -o format
       option should not be given in the flags= line because they
       are automatically taken care of by actsyncd(8).

       INN is shipped with default values of ftp.isc.org for host
       and /pub/usenet/CONFIG/active for ftppath.  You  can  read
       about  the  policies used for maintaining that active file
       at  ftp://ftp.isc.org/pub/usenet/CONFIG/README.   Consider
       sychronizing  from  this  file  on  a daily basis by using
       cron.


OPTIONS

       The options to actsync(8) are as follows:

       -b hostid
              This flag causes actsync(8)  to  ignore  newsgroups
              with  ``bork.bork.bork''  style  names.   That  is,
              newsgroups whose last 3 components  are  identical.
              For  example,  the  following  newsgroups have bork
              style names:

                   alt.helms.dork.dork.dork
                   alt.auto.accident.sue.sue.sue
                   alt.election.vote.vote.vote

              The value hostid determines  on  which  hosts  this
              action is performed:

                   0    neither host
                   1    local default server
                   2    remove server
                   12   both servers
                   21   both servers

              The  default  is  -b  0,  no  bork  newsgroups  are
              ignored.

       -d hostid
              This flag causes actsync(8)  to  ignore  newsgroups
              that  have all numeric path components.  The hostid
              value is interpreted the same as in -b.  For  exam­
              ple,  the  following  newsgroups  have numeric path
              components:

                   alt.prime.chongo.23209
                   391581.times.2.to_the.216193.power.-1
                   99.bottles.of.treacle.on.the.wall


              The newsgroups directory of a newsgroups with a all
              numeric  component  could  conflict with an article
              from another group.  For example, the directory for
              the  first  newsgroup listed above is the same path
              as article number 23209 from the newsgroup:

                   alt.prime.chongo

              The default is -d 0, all  numeric  newsgroups  from
              both hosts will be processed.

       -g max Ignore  any  newsgroup  with  more than max levels.
              For example, -g 6 would ignore:

                   alt.feinstien.votes.to.trash.freedom.of.speech
                   alt.senator.exon.enemy.of.the.internet
                   alt.crypto.export.laws.dumb.dumb.dumb

              but would not ignore:

                   alt.feinstien.acts.like.a.republican
                   alt.exon.admendment
                   alt.crypto.export.laws

              If max is 0, then the max  level  feature  is  dis­
              abled.

              By default, the max level feature is disabled.

       -i ignore_file
              The ignore_file allows one to have a fine degree of
              control over which newsgroups are ignored.  It con­
              tains  a  set  of  rules that specifies which news­
              groups will be checked and which will be ignored.

              By default, these rules apply to both hosts.   This
              can be modified by using the -I hostid flag.

              By  default,  all  newsgroups  are  checked.  If no
              ignore_file if specified, or  if  the  ignore  file
              contains  no  rule  lines,  all  newsgroups will be
              checked.

              Blank lines, and text after a ``#'' are  considered
              comments and are ignored.

              Rule  lines  consist of tokens separated by whites­
              pace.  Rule lines may be one of two forms:

                   c    newsgroup [type ...]
                   i    newsgroup [type ...]

              certain  newsgroups  to  be  checked.   If the rule
              begins with an i then  the  rule  requests  certain
              newsgroups  to be ignored.  The newsgroup field may
              be a specific newsgroup, or a wildmat(3) pattern.

              If one or more types are specified, then  the  rule
              applies  to  the newsgroup only if is of the speci­
              fied type.  Types refer to the  4th  field  of  the
              active(5) file.  A type may be one of:

                   y
                   n
                   m
                   j
                   x
                   =group.name

              Unlike  active files, the group.name may be a news­
              group name or a wildmat(3) pattern.  Also, ``='' is
              equivalent to ``=*''.

              For given rule line may, one may not repeat a given
              pattern type.  For example, one may not  have  more
              than  one  type  that  begins with ``='', per line.
              However, one may achieve  the  effect  of  multiple
              ``=''  types  by  using multiple rule lines for the
              same group.

              By default, all newsgroups  are  candidates  to  be
              checked.  If an ignore file is used, each newsgroup
              in turn is checked against  the  ignore  file.   If
              multiple  lines  match  a given newsgroup, the last
              line in the ignore file is used.

              For example, consider  the  following  ignore  file
              lines:

                   i *.general
                   c *.general m
                   i nsa.general

              The  newsgroup:  ba.general  would be ignored if it
              was  not  moderated.   The  newsgroup:  mod.general
              would  be  checked  if it was moderated.  The news­
              group: nsa.general would be ignored even if it  was
              moderated.

       -I hostid
              This  flag  restricts  which hosts, the ignore file
              applies.  The hostid value is interpreted the  same
              as in -b.

              This  flag may be useful in conjunction with the -m


                   actsync -i actsync.ign -I 2 -m host1 host2

              will keep all newsgroups currently  on  host1.   It
              will  also will only compare host1 groups with non-
              ignored newsgroups from host2.

              The default is -I 12, newsgroups from both hosts to
              be ignored per the -I  hostid flag.

       -k     By default, any newsgroup on host1 that is in error
              will be considered for removal.  This  causes  act­
              sync(8)  simply ignore such newsgroups.  This flag,
              in combination with -m will prevent  any  newsgroup
              from being scheduled for removal.

       -l hostid
              Flag problem newsgroups of type ``='' from host1 or
              host2 as errors.  The hostid value  is  interpreted
              the  same  as  in -b.  Newsgroups of type ``='' are
              newsgroups active entries that have 4th field  that
              begins  with  ``=''.   I.e.,  a  newsgroup  that is
              equivalent to another newsgroup.

              A newsgroup that is equivalent to itself,  or  that
              is  in  a  equivalence  chain  that loops around to
              itself is a problem.  A  newsgroup  that  is  in  a
              chain that is longer than 16 is a problem group.  A
              newsgroup that  is  equivalent  to  a  non-existent
              newsgroup is a problem.  A newsgroup that is equiv­
              alent to a newsgroup that is has a  error  of  some
              kind  a  problem.   However,  a  newsgroup  that is
              equivalent to an ignored newsgroup is not  a  prob­
              lem.

              By  default, problem newsgroups from both hosts are
              marked as errors.

       -m     Merge newsgroups instead of sync.  By default, if a
              newsgroup exists on host1 but not host2, it will be
              scheduled to be removed.  This flag  disables  this
              process,  permitting  newsgroups unique to host1 to
              be kept.

       -n  name
              Newsgroups that are created, are  created  via  the
              ctlinnd(8)  command.   By default, the creator name
              used is actsync.  This  flag  changes  the  creator
              name to name.

       -o  fmt
              Determine  the output / action format of this util­
              ity.  The fmt may one of:

                   a1   output in active(5) format,
                            and output host1 non-error ignored groups
                   ak   output in active(5) format, but use host2
                            hi & low (2nd & 3rd active fields) values
                            for any newsgroup being created
                   aK   output in active(5) format, but use host2
                            hi & low (2nd & 3rd active fields) values
                            for all newsgroups found in host2
                   a1k  output in active(5) format, but use host2
                            hi & low (2nd & 3rd active fields) values
                            for any newsgroup being created,
                        and output host1 non-error ignored groups
                   a1K  output in active(5) format, but use host2
                            hi & low (2nd & 3rd active fields) values
                            for all newsgroups found in host2,
                        and output host1 non-error ignored groups
                   ak1  same as a1k
                   aK1  same as a1K
                   c    output in ctlinnd(8) format
                   x    no output, directly exec ctlinnd(8) commands
                   xi   no output, directly exec ctlinnd(8) commands,
                            in an interactive mode

              The a, a1, ak, aK, a1k, a1K, ak1 and aK1 style for­
              mats allow one to form a new active file instead of
              producing ctlinnd(8) commands.  They use hi  &  low
              values  of  0000000000  and 0000000001 respectively
              for newsgroups that are created.   The  ak  and  aK
              variants  change the the hi & low (2nd & 3rd active
              fields).  In the case  of  ak,  newsgroups  created
              take their hi & low values from host2.  In the case
              of aK, all newsgroups found on host2 take their  hi
              & low values from host2.

              The  c  format  produces  ctlinnd(8)  commands.  No
              actions are taken because actsync(8) simply  prints
              ctlinnd(8)  commands  on standard output.  The sync
              (or merge if -m) with host2 may be accomplished  by
              piping  this  output into sh(1).  A paranoid person
              might prefer to use x or xi  in  case  a  newsgroup
              name  or  type contains bogus characters that might
              be interpreted by sh(1).  Even so, this output for­
              mat  is  useful  to  let  you  see how host1 may be
              synced (or merge) with host2.

              The sync (or  merge  if  -m)  may  be  accomplished
              directly  by  use of the x.  With this format, act­
              sync(8) uses the execl(2) system call  to  directly
              executes ctlinnd(8) commands.  Because of the exec,
              there is no risk  of  bogus  newsgroups  containing
              bogus  characters  causing  a shell to do bogus (or
              dangerous) things.  The output of such execs may be
              seen of the verbosity level is at least 2.

              before  each  command  is  executed  if  -o  x   is
              selected.  See the -z sec flag below.

              The  xi  format  interactively  prompts on standard
              output and reads directives on standard input.  One
              may pick and choose changes using this format.

              Care  should be taken when producing active(5) for­
              matted output.  One should check to  be  sure  that
              actsync(8) exited with a zero status prior to using
              such output.  Also one  should  realize  that  such
              output  will  not  contain  lines ignored by the -i
              ignore_file process even if -p 100 is used.

              By default, -o c is assumed.

       -p min_%_unchg
              By default, the actsync(8) utility  has  safeguards
              against  performing massive changes.  If fewer than
              min_%_unchg percent of the non-ignored  lines  from
              host1  remain unchanged, no actions (output, execu­
              tion, etc.)  are  performed  and  actsync(8)  exits
              with  a  non-zero exit status.  The min_%_unchg may
              be a floating point value such as 66.666.

              A change is considered a host1 line that was  found
              to  be  in  error,  was  removed,  was added or was
              changed.  Changing the 2nd or 3rd active fields via
              -oak or -o aK are not considered changes by -p.

              To force actsync(8) to accept any amount of change,
              use the -p 0 option.  To force actsync(8) to reject
              any changes, use the -p 100 option.

              Care  should be taken when producing active(5) for­
              matted output.  One should check to  be  sure  that
              actsync(8) exited with a zero status prior to using
              such output.  Also one  should  realize  that  such
              output  will  not  contain  lines ignored by the -i
              ignore_file process even if -p 100 is used.

              By default, 96% of the lines not ignored  in  host1
              must  be  unchanged.  That is, by default, -p 90 is
              assumed.

       -q hostid
              By default, all newsgroup errors  are  reported  on
              standard  errors.   This  flag  quiets  errors from
              host1 or host2.  The hostid  value  is  interpreted
              the same as in -b.

       -s size
              If size>0, then ignore newsgroups with names longer

              names longer than size.  Length checking is perform
              on both the local and remote hosts.

              By default, size is 0 and thus no  length  checking
              is performed.

       -t hostid
              Ignore  improper  newsgroups with only a top compo­
              nent from host1 or  host2.   The  hostid  value  is
              interpreted the same as in -b.  The following news­
              groups are considered  proper  newsgroups  for  top
              only names:

                   control
                   general
                   junk
                   test
                   to

              For  example,  the  following  newsgroup  names are
              improper because they only contain a top level com­
              ponent:

                   dole_for_pres
                   dos
                   microsoft
                   windoes95

              By  default, all improper top level only newsgroups
              from the remote ( -t 2 ) are ignored.

       -T     This flag causes host2 newsgroups from new  hierar­
              chies  to  be  ignored.  Normally if only host2 has
              the newsgroup chongo.was.here then it will be  cre­
              ated for host1.  However if host1 does not have any
              'chongo.*' newsgroups and this flag is given,  then
              chongo.was.here  will  be  ignored  and will not be
              created on host1.

       -v verbose_lvl
              No default, actsync(8) is not verbose.   This  flag
              controls the verbosity level as follows:

                   0    no debug or status reports (default)
                   1    print summary,
                            if work was needed or done
                   2    print actions, exec output & summary,
                            if work was needed or done
                   3    print actions, exec output & summary
                   4    full debug output

       -z sec If  -o x is selected, actsync(8) will pause for sec
              seconds before  each  command  is  executed.   This

              large number of  ctlinnd(8)  commands  are  needed.
              One can disable this sleeping by using -z 0.

              By  default,  actsync(8)  will  pause for 4 seconds
              before  each  command  is  executed  if  -o  x   is
              selected.


EXAMPLES

       Determine  the  difference  (but  don't  change  anything)
       between your newsgroup set and uunet's set:

            actsync news.uu.net

       Same as above, with full debug and progress reports:

            actsync -v 4 news.uu.net

       Force a site to have the same newsgroups some other site:

            actsync -o x master

       This may be useful to sync a slave site to its master,  or
       to sync internal site to a gateway.

       Compare  your  site  with uunet, disregarding local groups
       and certain  local  differences  with  uunet.   Produce  a
       report if any differences were encountered:

            actsync -v 2 -i actsync.ign news.uu.net

       where actsync.ign contains:

            # Don't compare to.* groups as they will differ.
            #
            i    to.*

            # These are our local groups that nobody else
            # (should) carry.  So ignore them for the sake
            # of the compare.
            #
            i    nsa.*

            # These groups are local favorites, so keep them
            # even if uunet does not carry them.
            #
            i    ca.dump.bob.dorman
            i    ca.keep.bob.dorman
            i    alt.tv.dinosaurs.barney.die.die.die
            i    alt.tv.dinosaurs.barney.love.love.love
            i    alt.sounds.*   =alt.binaries.sounds.*

       To  interactively sync against news.uu.net, using the same


            actsync -o xi -v 2 -i actsync.ign news.uu.net

       Based on newsgroups that you decided to  keep,  one  could
       make changes to the actsync.ign file:

            # Don't compare to.* groups as they will differ.
            #
            i    to.*

            # These are our local groups that nobody else
            # (should) carry.  So ignore them for the sake
            # of the compare.
            #
            i    nsa.*

            # These groups are local favorites, so keep them
            # even if uunet does not carry them.
            #
            i    ca.dump.bob.dorman
            i    alt.tv.dinosaurs.barney.die.die.die
            i    alt.sounds.*   =alt.binaries.sounds.*

            # Don't sync test groups, except for ones that are
            # moderated or that are under the gnu hierarchy.
            i    *.test
            c    *.test    m    # check moderated test groups
            c    gnu.*.test
            c    gnu.test  # just in case it ever exists

       Automatic  processing  may be setup by using the following
       actsync.cfg file:

            # host to sync off of (host2)
            host=news.uu.net

            # location of the ignore file
            ignore_file=<PREFIX specified with --prefix at configure>/etc/actsync.ign

            # where news articles are kept
            spool=<patharticles in inn.conf>

            # actsync(8) flags
            #
            # Automatic execs, report if something was done,
            #    otherwise don't say anything, don't report
            #    uunet active file problems, just ignore
            #    the effect entries.
            flags=-o x -v 2 -q 2

       and then by running actsyncd(8) with the path to the  con­
       fig file.

            ure>/etc/actsync.cfg

       One may produce a trial actsyncd(8) run  without  changing
       anything on the server by supplying the debug_level arg:

            actsyncd  <PREFIX  specified with --prefix at config­
            ure>/etc/actsync.cfg 2

       The debug_level causes actsyncd(8) to run actsync(8)  with
       an  -v  debug_level  (overriding  any -v flag on the flags
       line),  prevents  any  changes  from  being  made  to  the
       active(5)  file, writes a new active file to standard out­
       put and writes debug messages to standard error.

       If the debug_outfmt arg is also given to actsyncd(8)  then
       the  data  written  to  standard  output  will  be  in  -o
       debug_outfmt instead of in -o a1  format.   The  following
       /bin/sh command:

            actsyncd  <PREFIX  specified with --prefix at config­
            ure>/etc/actsync.cfg 4 >cmd 2>dbg

       Will operate in debug mode, not change the active(5) file,
       write  ctlinnd(8)  style  commands  to cmd and write debug
       statements to dbg.

       To check only the major hierarchies  against  news.uu,net,
       use the following actsync.ign file:

            # by default, ignore everything
            i *

            # check the major groups
            c    comp.*
            c    gnu.*
            c    sci.*
            c    alt.*
            c    misc.*
            c    news.*
            c    rec.*
            c    soc.*
            c    talk.*

       and running:

            actsync -i actsync.ign news.uu.net

       To  determine  the differences between your old active and
       your current default server:

            actsync <pathetc in inn.conf>/active.old -

       To report but not fix  any  newsgroup  problems  with  the


            actsync - -

       To  detect  any newsgroup errors on your local server, and
       to  remove  any  *.bork.bork.bork  style  silly  newsgroup
       names:

            actsync -b 2 - -

       The active file produced by:

            actsync ... flags ... -o x erehwon.honey.edu

       or by:

            actsync ... flags ... -o c erehwon.honey.edu | sh

       is effectively the same as the active file produced by:

            ctlinnd pause 'running actsync'
            rm -f active.new
            actsync ... flags ... -o a1 erehwon.honey.edu > active.new
            rm -f active.old
            ln active active.old
            mv active.new active
            ctlinnd reload active 'running actsync'
            ctlinnd go 'running actsync'

       It  should  be  noted  that  the above 'pause', 'actsync',
       'reload' and 'go' method is faster.


CAUTION

       Careless use of this  tool  may  result  in  the  addition
       change  or removal of newsgroups that you don't want.  You
       should avoid using the x output format until you are  sure
       it will do what you want.


BUGS

       If  a  newsgroup  appears  multiple times, actsync(8) will
       treat all copies as errors.   However,  if  the  group  is
       marked for removal, only one rmgroup will be issued.

       The timeout for ctlinnd(8) commands is fixed at 30 seconds
       when running in ``x'' or ``xi''  output  format.   Perhaps
       the  timeout value should be controlled via a command line
       option?


SEE ALSO

       active(5),
       simpleftp(1),
       mod-active(8),
       ctlinnd(8),

       inn.conf(5).


HISTORY

       Written by Landon Curt Noll <chongo@toad.com>  for  Inter­
       NetNews.    Updated  to  support  ftp  fetching  by  David
       Lawrence <tale@isc.org>.


Man(1) output converted with man2html