Subject: (Copy) NSF STIS
From: Dot Baker <[email-masked]>
To: [email-masked]
Date: Mon, 23 Mar 92 08:37 EST

Paul,
   Please post this to the NIH Listserver as:
    NSFSTIS.920330
 ---------------------------- Text of forwarded message -----------------------
Title       : STIS Internals
Type        : STIS
NSF Contact : DIS
Date        : March 11, 1992
 
      This document contains information on how the STIS system is
constructed.
 
      From the user's point of view, there are four distinct parts
to STIS:
 
 1.   The On-line system.  This is a computer at NSF that anyone
      can log onto and use the text search and retrieval software.
 
 2.   Anonymous ftp.
 
 3.   Documents via E-mail
 
 4.   Direct E-mail.  With this feature users can have documents
      sent to them automatically when they appear on STIS.
 
      From the system administrator's point of view, there are
some additional areas of interest:
 
 5.   System administration.
 
 6.   Document administration.  This is the procedures used by NSF
      staff to get documents onto STIS.
 
      Each of these areas is discussed below:
 
The On-line System
------------------
      The on-line system runs on a Sun workstation, a Sparc 2.  It
has about 1.5 Gigabytes of disk space.  It is directly connected
to the Internet, and that's how most people log on.  There are
four 9600 baud modems that can also be used for logging on to
STIS; these are funneled through a terminal server.  There is a
single Unix username, "public", for all users.  The STIS software
further identifies users.
 
      The on-line system is a blend of custom software, written at
NSF, commercial software (TOPIC) from Verity Corp., and public
domain software.  TOPIC was chosen largely because it has a
character-based full-screen that can run over the Internet.  In
effect we have written a shell around TOPIC.  The shell does
simple bulletin-board software functions such as asking users who
they are, setting the terminal type, and providing help screens.
The shell then invokes TOPIC for text search and retrieval.
 
      TOPIC provides several hooks that we use.  For example,
TOPIC has no internal download features, which we needed for
STIS.  Instead, we added a menu item for "download".  When users
select this item, TOPIC invokes a program written at NSF which in
turn invokes one of a variety of download programs (e.g. kermit).
We've also added a "read me first" menu item that describes how
to use TOPIC on the NSF databases.
 
      Of the download options provided on the on-line system, the
most popular is "mail me a copy."  In this method, the user
provides an E-mail address and STIS simply mails out the selected
document.
 
      We do not use the "topics" feature of TOPIC, because we
found it confusing to users and required a lot of human resources
to maintain.
 
Anonymous FTP
-------------
      All STIS documents are available via anonymous FTP.  We use
a slightly hacked version of ftpd from the network.  The only
reason we don't use Sun's ftpd is that we wanted better logging
of the file transfers.
 
      We provide a file, called "ftpindex", that provides a list
of all available documents.  Originally, documents were located
in subdirectories, but we found that users had a lot of trouble
retrieving these documents.  Now we create a symbolic link in the
ftp root for every document, and our user support requirements
have dropped to essentially zero.
 
      We do log requests, and find that anonymous ftp accounts for
over 2/3's of our downloads.
 
Documents Via E-Mail
--------------------
      This service is currently supported by the Info-Server
software from CSNET (now part of CREN).  This software requires
requests of the form:
 
      Request: stis
      Topic: nsf91101
 
      Commands are available to split documents into multiple mail
messages, for mail systems that refuse messages over an arbitrary
size.
 
      Info-Server has been effective and reliable, but the command
syntax is a bit cumbersome.  In the future we plan to move to a
Perl-based system that will be easier to modify, and provide a
friendlier command syntax.  Info-Server accounts for about 20% of
STIS downloads.
 
Direct E-mail
-------------
      We have two lists for direct E-mail.  The first list is for
(mostly) institutions that want to get the full text of every
document added to STIS.  The second list gets just a weekly
summary of the new documents for that week.
 
      Our original intent was to maintain a small list of
addresses.  We would only allow institutions (not individuals) to
get on the list, and the institution would be responsible for
further distribution.  Although this is still our current system,
we have all but abandoned the concept for the following reasons:
1) we found that few institutions have the mail expertise to re-
distribute, 2) even with the restricted list we still have
significant maintenance headaches (bounced mail and user
requests), and 3) we think we can develop a system that is
totally automated.
 
      So in the near future we will be implementing Anastasios
Kotsikonas' "Listserv for Unix" package to automate the mailing
list management functions.
 
System Administration
---------------------
      System administration on STIS is automated almost entirely
with scores of Perl scripts, mostly running from crontab.  These
tasks include:
 
 1.   Usage reports.
 2.   Disk backups to tape.
 3.   Verifying, reformatting, and adding documents and rebuilding
      the text database.
 4.   Deleting expired documents.
 5.   Sending documents to the mailing lists.
 6.   Creating the (human-readable) indexes for anonymous ftp and
      Info-Server users, and the machine-readable index for Info-
      Server.
 
Document Administration
-----------------------
      Documents at NSF are prepared on a wide variety of hardware
and software platforms.  STIS requires ASCII versions of
documents because of the diverse resources of our users.  Getting
the ASCII version is the most difficult task in getting documents
on STIS.  Some software packages, such as Ventura Publishing,
cannot produce reasonable ASCII versions of documents at all, and
others, such as WordPerfect, that do a good job, require
considerable knowledge of the software.
 
      We do try to provide PostScript versions of documents when
possible, but we don't do this often, and it's not without
pitfalls.  For example, PostScript generated by most MacIntosh
computers will not print on non-Mac systems.
 
      We provide access to all NSF award abstracts.  These are
extracted once a month from the NSF mainframe.  They are copied
to the STIS system, and added to the text database with a Perl
script.

More Information
----------------
      If you have any questions, write to "[email-masked]."
 
..expires with new edition
 
-----------------------------------------------------------------------
** FOR YOUR REFERENCE **
 
This document is mailed as a feature of NSF's STIS system (Science and
Technology Information System).  If you would like to know more about
STIS, send a message to [email-masked] (Internet) or stisserv@NSF
(BITNET).  The subject will be ignored.  Put the following request in
the *text* of the message:
 
   Request: stis
   Topic:   nsf9110
 
For information about joining this list, send the following request:
 
   Request: stis
   Topic:   stisdirm
 
If you want to get off this list, or change your address, first
determine if you received this message from LISTSERV, by checking the
mail header.  If you got this message from LISTSERV, you must get off
by sending a message to LISTSERV.  The document "stisdirm" shown above
has information on how to do this.  If you receive this document
directly from NSF, change requests should be sent to [email-masked]
(Internet) or stis@NSF (BITNET).