


      Regular Broadcasting of PACSAT File Headers 
      DRAFT
      04 November 1991
      
      [All changes and additions in this DRAFT are PROPOSED, and should not be 
      considered final until a subsequent RELEASE version of this document is 
      published.]
      
      JW Ward, G0/K8KA
      HE Price, NK6K
      
      DISCUSSION 
      
      In the past 18 months the PACSAT Broadcast Protocol has proven to be a very 
      efficient means of downloading information from the LEO store-and-forward 
      satellites. It makes good use of the available bandwidth, and especially 
      efficient use of available pass time. The authors always intended to extend 
      this datagram facility to the provision of a "file directory" for the PACSAT 
      server. Since all stations need an up to date directory, this is a prime 
      candidate for point-to-multipoint distribution.
      
      A PACSAT file server's "directory entries" are, in fact, PACSAT File Headers 
      (PFHs). The PFH contains all of the important information about a file (name, 
      number, upload time, title, addresses, etc.). This information is present in 
      the PFH form on the server, and routines for interpretation of the PFH are 
      present in the client software. The PFH is the natural data structure in which 
      to present "broadcast directories."
      
      The broadcast directory service will, therefore, consist of periodic 
      transmission of PFHs in special Directory Broadcast datagrams. The 
      transmissions will be either automatic or client initiated. This protocol 
      (see Appendix A) specifies the format for Directory Broadcasts and for 
      Directory Broadcast Requests. The protocol does not specify when a server 
      might periodically transmit PFHs without client requests, but such automatic 
      directory broadcasts will be implemented on the existing PACSAT servers.
      
      CHOOSING A HANDLE
      
      Having decided that a "directory entry" is a PFH and that "directory 
      broadcasting" is PFH broadcasting, we must address the problem of how the 
      client can assure that it has received a complete and up-to-date directory 
      from the server. 
      
      Although the server uniquely identifies each file with an identification 
      number, this is not necessarily the best way to for the client to request 
      directory entries from the server. A system based on file numbers will not 
      always assure that the client receives directories for all "new" files. The 
      following cases illustrate this:
      
      (1) Partial Uploads
      
      When a client begins uploading a file, the server assigns a file number. The 
      client may not finish uploading the file for some time. Only when the file 
      upload is finished is the file a "new" file.
      
      (2) Non-existent Files
      






      Some file numbers are used only briefly (e.g. for temporary files) and may 
      never be associated with a "new" file (as seen by the clients).
      
      (3) Non-PACSAT Files
      
      Files created by on-board processes, but lacking the PFH. They cannot be 
      downloaded or uploaded. As far as the PACSAT protocol suite is concerned, 
      these are like (2).
      
      (4) Daily Log Files
      
      The server and its processes also maintain daily logs which, although they may 
      become "new" several times during a day, will cease to grow after midnight.
      
      (5) Continuous Log Files
      
      The server and processes on the server maintain log files. These files become 
      "new" every time something is written into them.
      
      During the design of the directory broadcast protocol, we attempted to solve 
      these problems using simple measures which would still allow file number to be 
      the primary directory request handle. Problems persisted both in the complete 
      specification of the protocol and in a prototype implementation of the 
      groundstation receiving software.
      
      Eventually the file-number-based scheme was abandoned in favour of a slightly 
      more complex protocol providing two "time limits" with each broadcast 
      directory entry: an "older limit" , t_old,  and a "newer limit", t_new. Thus, 
      as well as containing a PFH, each broadcast directory entry gives the client a 
      pair numbers (t_old, t_new). The proper interpretation of this pair is 
      
           "There   are  no  files  other  than  this  file   with 
           UPLOAD_TIME >= t_old and <= t_new."
      
      The client maintains a list of holes in its directory time line using the 
      t_old and t_new information. Requests to the server for hole fills use these 
      times, rather than file numbers.
      
      EXAMPLES
      
      [N.B.: Small times have been used deliberately in the examples; real satellite 
      times are the number of seconds elapsed since 00:00:00 1/1/80. Also, the 
      satellite file system assures that no two files will ever have the same 
      UPLOAD_TIME.]
      
      For example, a client with no previous directory information assumes that it 
      has a directory hole from
      
      0 - FOREVER
      
      Upon receiving a directory broadcast with (t_old, t_new)
      
      (0, 50)
      
      the client's directory hole list would look like
      
      51 - FOREVER






      
      and the client software is confident that there are no older files.
      
      The list can get fragmented, e.g. if the client now receives
      
      (120, 150)
      
      and the time hole list looks like:
      
      51 - 119
      151 - FOREVER
      
      Requests from the client to the sever are time hole lists. In effect, they 
      request that the server transmit any directory entries in the hole OR an entry 
      which shows that there are no directories in the hole. 
      
      FLEETING HOLES
      
      The server sets time limits for a particular directory broadcast by examining 
      the next newer and previous older files on the system. These files, however, 
      may move or be deleted, hence the need to show that a hole no longer holds any 
      directories. Starting from the previous example, say that the server creates 
      files at t=151, t=152 and t=153, thus it would transmit
      
      [a] (120, 151) (PFH for t=151)
      [b] (152, 152) (PFH for t=152)
      [c] (153, 153) (PFH for t=153)
      
      A client receiving [a] and [c], but missing [b] would have a hole
      
      152 - 152 
      
      in its directory time line, and might assume that there was a file to fill 
      that hole. But the server might delete the file t=152. Thus, when the client 
      requests a fill for
      
      (152,152)
      
      the server would respond with
      
      [c'] (152, 153) (PFH for t=153)
      
      which allows the client to fill its hole, but does not contain a directory 
      entry for t=152. 
      
      MARKING THE LIMITING FILES
      
      It is convenient for the client to know when it has received the directory for 
      the current newest file. The server will mark this file with a special flag. 
      The oldest file will always have t_old==0.
      
      BACKWARD COMPATIBLE IMPLEMENTATION 
      
      Unfortunately, adding new data and request frame formats to the existing 
      broadcast protocol in a backward compatible manner required some compromises. 
      On inspection of the existing FRAME_HEADER for broadcast data packets, there 
      is no clear indication of frame type! Strictly speaking, the reserved bits of 






      the FRAME_HEADER.flags byte could be used to indicate that the frame was a 
      directory broadcast frame. Unfortunately, PB at least does not check these 
      bits, and any new directory broadcasts would terminally confuse PB. We expect 
      that clients will try to get updated broadcast receiver software, but we don't 
      want to force the pace of this change by causing the current implementation to 
      crash.
      
      We completely avoid this problem, by specifying a separate PID for directory 
      broadcast and request frames. This should allow existing groundstation 
      software to co-exist with the new broadcast directories.
      
      IMPLEMENTATION NOTES
      
      A single implementation of PACSAT Protocol Suite software runs on UO-14, AO-
      16, LO-19 and UO-22. Although there are minor variations from satellite to 
      satellite, these are generally related to ongoing upgrades rippling through 
      the system. There are some implementation peculiarities, and some ad-hoc 
      additions have been made to the standard broadcast protocol. These matters are 
      documented here, but are not added to the formal specification.
      
      Queue Management
      ----------------
      
      The server maintains a queue with 10 entries; each entry is a hole list 
      request or broadcast start request. A particular station (as identified by 
      callsign, not including SSID) can have only one entry on the queue.
      
      Entries are removed from the queue: 
           after 10 minutes;
           after a hole list has been completely transmitted (for hole list);
           after a file has been completely transmitted (for start request);
           when a new request is received from a station already in the queue;
           if the file associated with the entry cannot be opened and read;
      
      Broadcast Request Response Messages
      -----------------------------------
      
      When the server receives a request frame from a client, it will respond 
      immediately indicating the success or failure of the request. This response is 
      intended for use in manually operated stations; because of timing and 
      reliability problems, its use in automated broadcast request systems is 
      discouraged.
      
      The request response message is an AX.25 UI frame from the server callsign to 
      the client callsign with the PID 0xf0. [NB - this is a change of the PID.] The 
      data portion of the packet has no header or CRC. The data is in ASCII as 
      follows:
      
      <"OK "|"NO ">[<errorcode>]<callsign><CR>
      
      "OK " indicates that the request was successful. There is no <errorcode>.
      
      "NO " indicates that the request failed for some reason. In this case, the 
      <errorcode> is an ascii representation of an error number followed by one 
      blank space, e.g. "NO -1 ". The error codes for broadcast start requests and 
      hole list requests should be interpreted as follows:
      






                     -1       Temporary problem, e.g. queue full or server closed
                     -2       Permanent problem, e.g. file does not exist.
      
      
      Broadcast Server Status Message
      -------------------------------
      
      The format of the PBLIST message will be altered to include (at the option of 
      spacecraft controllers) the number of the file being broadcast. The format of 
      the PBLIST message will now be one of the following - 
      
      If file numbers are disabled:
      
      PB: <call> ...
      PB: <call>\P ...
      PB: <call>\D ...
      
      If file numbers are enabled:
      
      PB: <call>\<fnumber> ...
      PB: <call>\P\<fnumber> ...
      PB: <call>\D ...
      
      \P        indicates that the request is permanent 
      \D        indicates that the request is a directory request
      <call>     is the callsign of the requesting station, without SSID
      <fnumber>  is the hexadecimal file number of the requested file
      
      If the queue is empty, the message will be :
      
      PB: Empty.
      
      These messages will be transmitted to PBLIST-0 with PID 0xF0.
      
      File Types
      ----------
      
      There is still some confusion regarding compressed files. The intent was to 
      place all compression information in the compression_type field and all file 
      format information in the file_type field. Thus, as per the FTL0 update 
      released 22 March 1991:
      
           Delete  file  type 0x0a, "Simple ASCII text  file,  but 
           compressed"  from Appendix A, PACSAT FILE  TYPES.  This 
           type of file should be indicated by file_type 0x00  and 
           a non-zero compression_type item.
      
      We propose to fix the conflict between "miscelaneous binary" and "activity 
      log" by changing the activity log file type. Since only Harold and I have to 
      take action to implement this solution, it is more likely to succeed than 
      trying to recall every copy of PFHADD.
      
      Suggestions regarding additional, more-informative, file types are solicited. 
      
      Compression Types
      -----------------
      






      The PFH protocol was designed to be flexible. We should now add a compression 
      type for the popular LHARC program. Add to APPENDIX B of the document "PACSAT 
      File Header Definition"
      
           0x03      body compressed using LHARC.
      
      --------------------------------------------------------------
      --------------------------------------------------------------
      BROADCAST DIRECTORY PROTOCOL PROPOSED ADDITIONS TO APPENDIX A
      OF THE PACSAT BROADCAST PROTOCOL SPECIFICATION
      --------------------------------------------------------------
      
      PACSAT BROADCAST PROTOCOL ADDENDUM - PFH BROADCASTS
      
      A.5 Directory Broadcasts
      
      The server will transmit directory information automatically and/or as the 
      result of Directory Requests from clients. The directory information provided 
      is the PACSAT File Header (PFH). For a particular file, the directory entry 
      will be transmitted in one or more Directory Broadcast frames, each of which 
      is carried in the data portion of an AX.25 UI frame. Each frame will also 
      contain the DIR_HEADER described below.
      
      To aid the client in maintaining its directory database and requesting up-to-
      date directory entries, Directory Broadcast frames include time limits. These 
      limits allow the client to maintain a "time line" of directory entries and to 
      determine when the time line is complete and up-to-date.
      
      Servers will periodically transmit the directories for the most recently 
      uploaded files. Other directory entries can be requested by the client using a  
      Directory Request frame. The Directory Request format allows clients to 
      request that the server transmit directories for all files with UPLOAD_TIME in 
      a particular range. This is equivalent to filling holes in the directory time 
      line.
      
      A.5.1 Directory Broadcast frame format
      
      A directory entry is transmitted as one or more directory broadcast frames.
      
      Directory broadcast frames consist of header, data and CRC. The header is 
      fixed length, and the data is variable length.
      
      A directory broadcast frame is wholly contained in the data portion of an 
      AX.25 UI frame.
      
      A <UI> frame containing a directory broadcast frame uses a PID of 0xbd.
      
      A <UI> frame containing a directory broadcast frame uses a source address of 
      the transmitting station, and a destination address of QST-1.
      
      A.5.1.1 Frame Header
      
      Each directory broadcast frame consists of a header occupying the first 17 
      bytes of the frame. The structure of the frame header is as follows:
      
      <flags><file_id><offset><t_old><t_new>
      






      This structure follows the PACSAT Data Specification Standards as regards 
      field size and byte order; these are defined in a separate document.
      
      struct DIR_HEADER {
           unsigned char flags;
           unsigned long file_id;
           unsigned long offset;
           time_t t_old;
           time_t t_new;
      }
      
      
      flags          A bit field as follows:
      
           7  6  5  4  3  2  1  0
          /----------------------\
          |*  N  E  0  V  V  T  T|
          \----------------------/
      TT                  Two bit frame type identifier
                          00   PFH broadcast 
                          01   reserved
                          10   reserved
                          11   reserved
      
      VV                  Two bit version identifier.  This version is 00.
      
      0                   Always 0 indicates a server generated frame.
      
      E              1    Last byte of frame is the last byte of the directory PFH.
                     0    Not the last frame.
      
      N              1    This is the newest file on the server.
                     0    This is not the newest file on the server.
      
      *                   Reserved, always 0.
      
      
      file_id    A number which identifies the file.  All directory broadcast 
      frames which are part of the same file's PFH are tagged with this number.
      
      offset     This is  the offset from the start of the PFH for the first data 
      byte in this frame.  
      
      t_old     Number of seconds since 00:00:00 1/1/80. See below.
      
      t_new     Number of seconds since 00:00:00 1/1/80. See below.
      
           There  are no files other than the file  identified  by 
           file_id with t_old <= UPLOAD_TIME <= t_new.
      
      A.5.1.2 Data
      
      The data portion of a directory broadcast frame will contain all or part of 
      the PACSAT File header from the file identified by <file_id>. The <offset> 
      field indicates where the data from the current frame belongs in the PFH. 
      
      An <offset> of 0 and a <flags> field with the E bit set to 1 indicates that 






      this directory broadcast frame contains the entire PFH for the identified 
      file.
      
      A.5.1.3 CRC
      
      The CRC covers the header and data portions of the directory broadcast frame. 
      It is the XMODEM CRC as defined in section A.2.1.3.
      
      A.5.2 Directory Broadcast Requests
      
      Clients may request that the server transmit directory broadcasts for files 
      with UPLOAD_TIME within a certain range. If entries from several ranges are 
      required, multiple ranges can be included in a single request.
      
      A.5.2.1 Request Frame
      
      A directory broadcast request frame consists of header and data. The header is 
      fixed length, and the data portion is variable length.
      
      The directory broadcast request frame is wholly contained within a standard 
      AX.25 <UI> frame.  The total length of the request frame may not exceed the 
      length of a legal AX.25 <UI> frame.
      
      A <UI> frame containing a request frame uses the same PID as a directory 
      broadcast frame: 0xbd.  The source address is the address of the transmitting 
      station, the destination address is the address of the server to which the 
      request is directed.  
      
      A.5.2.2 Request Header
      
      The format of the directory broadcast request header is:
      
      struct DIR_REQ_HEADER{
           unsigned char flags;
           int block_size;
      }
      
      flags          - A bit field as follows:
      
           7  6  5  4  3  2  1
          /-------------------\
          |*  *  1  V  V  C  C|
          \-------------------/
      
      CC                  Two bit field as follows:
      
                          00   directory fill request.
                          01   reserved
                          10   reserved
                          11   reserved
      
      VV                  Two bit version identifier.  This version is 00.
      
      1                   Always 1 indicates a client-generated frame.
      
      *                   Reserved, must be 0.
      






      
      <block_size> is the number of data bytes which the server should place in each 
           directory broadcast frame.
      
      5.2.3 Data
      
      If the CC field is 00, a list of UPLOAD_TIME ranges is present. 
      
      The format of the list is pairs times (number of seconds since 00:00:00 
      1/1/80.)
      
      struct PAIR {
           time_t start
           time_t end
      };
      
      For each PAIR, the server will transmit directories for those files with 
      UPLOAD_TIME greater than or equal to <start> and less than or equal to <end>. 
      
      If there are no files within the range, the directory for the first file with 
      upload time greater than <end> will be transmitted. If there is no such file, 
      the directory for this first file with upload time less than <start> will be 
      transmitted. In either case, the <t_old> and <t_new> fields in this directory 
      will indicate to the client that there are no entries between <start> and 
      <end>. 
      
      <END>



