pn-172 Users Guide to RSX-11M Communications Software Ben Burch Vicky White Fermilab P.O.Box 500, Batavia, Il 60510 December 2 1982 This document describes the Fermilab RSX software for task-to-task communication between tasks in machines connected together by a DR11-W link. Tasks within a single machine may also use this communications software for inter-task communication This Guide explains how to install and use the communications software Users Guide to RSX-11M Communications Software Page 2 1.0 INTRODUCTION . . . . . . . . . . . . . . . . . . . . 3 1.1 References For Further Information . . . . . . . . 3 2.0 SOFTWARE NEEDED . . . . . . . . . . . . . . . . . . 4 3.0 CONTENTS OF THE ARCHIVE TAPE IN DETAIL . . . . . . . 5 3.1 Files Normally Needed On User's System . . . . . . 5 3.2 Other Files Included On The Tape But Not Normally Needed . . . . . . . . . . . . . . . . . . . . . . 6 4.0 INSTALLING THE SOFTWARE . . . . . . . . . . . . . . 8 4.1 From Tape To Disc . . . . . . . . . . . . . . . . 8 4.2 Building The Driver . . . . . . . . . . . . . . . 8 5.0 USING THE COMMUNICATIONS SOFTWARE . . . . . . . . . 9 6.0 TEST PROGRAMS . . . . . . . . . . . . . . . . . . 10 6.1 RCVLUP,XMTLUP,RCVTST,XMTTST . . . . . . . . . . 10 6.2 CDTEST . . . . . . . . . . . . . . . . . . . . . 10 6.3 RCVXMT . . . . . . . . . . . . . . . . . . . . . 11 7.0 THE CD DRIVER TRACE IN DETAIL . . . . . . . . . . 11 7.1 The Trace In The Driver . . . . . . . . . . . . 11 7.2 Trace Dump And Control Program CDDUMP . . . . . 12 7.3 Interpretation Of Trace . . . . . . . . . . . . 12 Users Guide to RSX-11M Communications Software Page 3 1.0 INTRODUCTION Unibus machines, not more than 50 ft. apart may be directly connected by a DR11-W to DR11-W high speed parallel link. Software exists for both RSX-11M and RT-11 (and soon VMS) to allow a program in one machine to send a message to a program in the other machine. A message is either a block of data (between 1 and 32767 16-bit _ _______ words) or a single byte of control information, termed a Signal. A block of data is transferred directly from the sending task's buffer to the receiving task's buffer. The destination of a message is denoted by a single word identifier, called a packet type code (PTC). Packet type codes may take any value between 0 and 255. Packet type codes in the range 0 to 100 should be reserved for computer department use, for system-wide identifiers. Packet type codes over 100 are available for any task to use. The software takes the form of services performed by a device driver. This driver handles the hardware and the job of routing messages to the appropriate task. The RSX-11M driver handles both physical DR11-W links and and internal pseudo link. An RT-11 driver with the same link transmission protocol exists and so a task under RSX may communicate with an RT-11 program. Fortran callable subroutines have been provided to make calls to the communications drivers in an operating system independent way. The Fermilab RSX data acquisition system is implemented using this communications software. Even if the data acquisition system is to be used only on a single machine, a communications driver (internal link only) will still be needed for inter-task communications. 1.1 References For Further Information For a more detailed discussion of the DR11-W hardware, the link level protocol, the general concepts of routing of blocks of data to the appropriate task and the overall framework of interprocessor communication the reader is referred to the following documents. 1. Digital Equipment Corporation: DR11-W Direct Memory Interace Module User's Guide. Users Guide to RSX-11M Communications Software Page 4 2. DS-77 - DR11-W as a link device - The Transmission Protocol. 3. DS-80 - Application Level Protocol. 4. PN-160 - High Speed Interprocessor Data links using the DR11-W. 5. PN-161 - Using the DR11-W DMA device for Interprocessor Communications in RT-11. 6. PN-162 - An RSX-11M Device Driver Implementing Network Protocols on the DR-11W. 7. PN-166 - The RSX-11M Communications Driver PN162 contains details on performance and size of the driver. To give an idea of size and speed: each data block transfer over the link has an overhead of under 6 m.s., each data word passed across the link takes roughly 3 microseconds. The device driver (no trace, but with DR11-W and internal link support) is about 2000 words in size. 2.0 SOFTWARE NEEDED 1. The CD device driver, which forms a part of the RSX executive, must be built and loaded in the system. The driver may optionally include a trace feature as a debug and troubleshooting aid. 2. The tasks to install and interpret the CD driver trace should also be built, to be available in case of problems. 3. The object library of Fortran callable subroutines (CDPACK.OLB) needs to be put on some convenient UIC. Then all tasks using these routines can be task built including a CDPACK library search. 4. Some test programs, which exercise the link and link software are available. Either the task files, or if desired the sources and command files may be copied to your Users Guide to RSX-11M Communications Software Page 5 system. It is advisable to have at least some of these test programs on your system(s). Then, in case of problems in using the link software in your own program, you can verify correct operation of the link with the test programs. 3.0 CONTENTS OF THE ARCHIVE TAPE IN DETAIL For completeness the archive tape contains all software and test software relating to the RSX communications driver and the CDPACK package of routines. Some of this software is not normally needed on a users system. The contents of the tape by UIC is given below. 3.1 Files Normally Needed On User's System [77,1] Command files UFD.CMD Creates the UFDs used by this tape OFFBRU.CMD Copies a complete UIC from this tape to disc SAVECD.CMD Copies each of the UIC's on this tape from some other UIC (only the command file OFFBRU.CMD is needed) [77,2] CD driver and trace sources and command files CDTAB.MAC CD Driver tables CDDRV.MAC CD Driver source CDPREF.MAC CD Driver Prefix file CDDRV.CMD Command file to assemble and build CD Driver and also the trace programs CDDUMP.MAC Trace Dump program CDDUMP.CMD Command file to assemble and build CDDUMP CDTRIN.MAC Program to create a dynamic common region CDTRAC CDTRIN.CMD Command file to assemble and build CDTRIN [77,4] Library of Fortran callable routines CDPACK.OLB CDPACK object library Users Guide to RSX-11M Communications Software Page 6 [77,11] Test programs for Link Software RCVLUP.TSK Loop receiving a single data block XMTLUP.TSK Loop transmitting a single data block RCVTST.TSK Receive a single data block XMTTST.TSK Transmit a single data block CDTEST.TSK Test all driver functions and Echo test RCVXMT.TSK Multiple receives and transmits of data blocks QIORCV.TSK Receive a single data block (QIO - not CDPACK) QIOXMT.TSK Transmit a single data block (QIO - not CDPACK) [77,100] Libraries for building test programs CERNLIB.MLB Macro library RSXLIB.OLB Object library - shortened version of CERN RSX object library (these are needed only to build CDDUMP or if you intend to rebuild any of the other test programs) 3.2 Other Files Included On The Tape But Not Normally Needed [77,3] Sources of Fortran callable subroutine package CDPACK CDASGN.MAC CDASGN subroutine CDDRV.MAC Macros and definitions CDERRM.MAC CDERRM subroutine CDERRN.MAC CDERRN subroutine CDERRT.MAC CDERRT subroutine CDMODE.MAC CDMODE,CDCHAN,CDRCVP,CDXMTP,CDRBUF,CDXBUF, CDTMO,CDUSER subroutines CDPACK.MAC CDOPEN,CDCLOS,CDXMT,CDXMTW,CDRCV,CDRCVW CDSIG,CDSIGW subroutines CDPREF.MAC prefix file for assembling routines CDRSX.MAC defines conditional for RSX operating system CDRT.MAC defines conditional for RT operating system CDSTAT.MAC CDSTAT subroutine CDUTIL.MAC internal routines called by other modules CDWAIT.MAC CDWAIT subroutine CDCMPL.MAC AST (RSX) and completion routine (RT) [77,5] Sources and command files for test programs RCVLUP.FTN RCVLUP.CMD XMTLUP.FTN XMTLUP.CMD Users Guide to RSX-11M Communications Software Page 7 RCVTST.FTN RCVTST.CMD XMTTST.FTN XMTTST.CMD (test programs using CDPACK and described under [77,11]) [77,6] Source and command files for test programs QIORCV.MAC QIORCV.CMD QIOXMT.MAC QIOXMT.CMD (test programs using QIO's and described under [77,11]) [77,7] Sources and command files for CDTEST program CDTEST.FTN Single function and Echo test program CDTEST.INC CDTEST.CMD CDWAKE.FTN Fortran version of a CDPACK routine [77,10] Sources and command files for RCVXMT program RCVXMT.FTN Main program RXINIT.FTN Initialisation part RXSUMM.FTN Summary part RCVXMT.BLD Task build command file RCVXMT.TXT Combined form of all source for RT-11 RCVXMT.CMD Command file to compile and build RCVXMT [77,20] Test programs used in developing CDPACK CDUTST.FTN To test CDUTIL package CDUTST.COM to build CDUTST on VAX BADARG.FTN to give bad arguments to CDPACK BADARG.CMD to build BADARG ERRTST.FTN to test CDPACK error routines ERRTST.BLD task build command file CDUINT.MAC to use CDUTIL routines from fortran [77,200] Documentation DS77.RNO DS-77 -link transmission protocol DS77.MEM DS80.RNO DS-80 -logical link protocol DS80.MEM PN159.RNO PN-159-CDPACK documentation PN159.MEM Users Guide to RSX-11M Communications Software Page 8 PN159.BTC PN160.RNO PN-160-DECUS paper on overall connected PN160.MEM machines project PN161.RNO PN-161-DECUS paper on RT-11 Driver PN161.MEM PN162.RNO PN-162-DECUS paper on RSX-11M Driver PN162.MEM PN166.RNO PN-166-The RSX-11M Communications Driver PN166.MEM PN172.RNO PN-172-User's Guide to RSX-11M Communications PN172.MEM Software 4.0 INSTALLING THE SOFTWARE 4.1 From Tape To Disc The tape has been created with the BRU utility, at 1600 bpi. At the time of writing this User's Guide it contains only one backup dataset name - DEC82, which is the date of the release of the software. There may in future be several different backup datasets on the tape. The backup dataset names on a tape can be displayed by the command BRU /DIRECTORY Mxn: where Mx may be MJ for STC 1900 or MM for TJE-16 tape drives You should have a disc available where the UIC's [77,*] can be used without conflict with existing software. Having chosen the appropriate dataset name on the tape the command files can be transferred to the disc with the command BRU /NOINI/BACKUP:backupname from: Mxn:[77,1] to: Dxn: (your chosen disc) By repeatedly executing the command file @OFFBRU each of the UIC's which are required can be copied to the disc Alternatively the whole tape may be copied to disc with the command BRU /NOINI Users Guide to RSX-11M Communications Software Page 9 from: Mxn: to: Dxn: 4.2 Building The Driver The software, once on a disc may be copied to any convenient UIC's. The clustering of files within a particular UIC should be maintained. To build the communications driver and the related trace and dump programs invoke the command file by @CDDRV.CMD You will need to know the CSR address, interrupt vector of each of your DR11-W's as well as the priority of the device (normally 5). For CD0 the standard CSR and vector at Fermilab will be suggested to you. You will be required to answer some questions in order to generate the driver conditionals. Initially anyway, it is a good idea to include the trace feature in the driver. This takes up about 800 words of extra space. Use of the trace feature adds up to 3 ms of overhead to each transfer. but the feature does not need to be used except in case of problems. A copy of the trace dump program should be build for each CD unit number in the system The trace partition size can be specified. A 1K buffer should be sufficient (4K is the maximum). Give the block size as an octal number e.g. 100 for a 2K buffer. Except in case of problems the trace partition CDTRAC need not actually be created. A command file for loading the CD driver will be created for you CDINS.CMD This can be put into the system startup command file or just invoked when needed. It can be edited to include/exclude creation of a trace common region. Users Guide to RSX-11M Communications Software Page 10 5.0 USING THE COMMUNICATIONS SOFTWARE If you wish to write your own programs using this software, you will need to do one of the following: 1. Make FORTRAN calls to transmit and receive messages over a link. See PN-159 for a complete description of CDPACK and how to use it. CDPACK subroutine calls and error codes are operating system independent 2. Make QIO calls to the CD driver directly. PN-166 describes the QIO functions available, how to use them and the status returns from the communications driver. 6.0 TEST PROGRAMS 6.1 RCVLUP,XMTLUP,RCVTST,XMTTST These are self-documenting simple test programs which receive or transmit a block of data with/to a particular packet type code either one or many times. The xxxLUP programs give transfer timings. When the programs are started up, default values for parameters can be used by pressing carriage return in answer to questions. 6.2 CDTEST This is a more complex test program with a menu of options. Press the key with the letter alongside an option to select that option. An option either allows you to change a parameter, start an action or go into further menus or dialogue and prompts. The program operates in two modes 1. Test program for each driver function singly. 2. Echo type test program with a partner program echoing For Echo mode operation both the options Users Guide to RSX-11M Communications Software Page 11 E AUTOMATIC WAIT FOR MESSAGE AFTER SEND [ON] F AUTOMATIC ECHO OF MESSAGE AFTER RECEIVE [ON] must be set to ON One end of the link must start with option A A WAIT FOR MESSAGE TO ARRIVE and the other end must then start the echo process with option C TRANSMIT MESSAGE The size of the data blocks and packet type codes to be used, timeouts etc. should be set up initially using option K K SET PARAMETERS (PTCS,BUFFERS,WC,ETC) The side which started by transmitting may check on the validity of the data each time it receives it back. The set of parameters specified may be saved on disc in a named file -see option K. By default an attempt will be made to save them on SY:[250,1]CDECHO.LAS. If this fails some error messages relating to Parameter files may be seen, but will cause no problems. 6.3 RCVXMT This is a test program which allows multiple simultaneous transmits and receives on one or more links, to be issued. It is a self documenting test task. A complex test involving each task receiving messages with up to 20 different packet type codes, and transmitting messages with up to 20 different packet type codes, each with a different length and possibly different CD unit number, may be set up. Such a set-up is quite laborious and so once done the parameters for this particular test may be "saved" on disc and re-used later to repeat the same test Default values for parameters may be obtained by pressing carriage return in answer to questions. Users Guide to RSX-11M Communications Software Page 12 This same program will run under either RSX-11M or RT-11. 7.0 THE CD DRIVER TRACE IN DETAIL 7.1 The Trace In The Driver The trace buffer may be a static common partition name CDTRAC. The task CDTRIN creates a dynamic common region CDTRAC in GEN for use as a trace buffer. Either will do. If some partition named CDTRAC does not exist when the driver is loaded then tracing will be disabled. If the partition is later created you can force the driver to start tracing operations by forcing a protocol re-start. To do this you must create a condition where there is link activity but the partner machine does not respond - e.g. Halt it momentarily. 7.2 Trace Dump And Control Program CDDUMP CDDUMP should be installed in the system with a name of the form ..XXX e.g. by default it will be installed as ...CDD. The dump for CD unit 1 might for example be installed as ...CD1. Command lines can then be passed to the dump program. e.g. CDD T,L to list the trace on the line printer The commands available are: A Display all available data and trace D Display UCB and SCB data H Help - display list of commands L Select line printer T Display trace F Freeze trace - no further entries will be made until unfrozen U Unfreeze trace - continue tracing P List Packet type code owners and their Read QIOs more than one command may be used on a line separated by commas Users Guide to RSX-11M Communications Software Page 13 7.3 Interpretation Of Trace No user is expected to make serious use of this trace. It is for the expert system programmer. However for posterity and interested persons it is described in this document The trace shows all the link transactions, all the entries and exits from the driver and many other significant events. At each trace one or more relevant parameters are displayed The first item in the trace is the last event which occurred. Reading down the page leads backwards in time through the events Some typical trace elements are: 58 EXIT FROM DRIVER, FFL 000001 STATE=IDLE 58 CDINI,STATE=IDLE FLG 000000 FFL 000001 S.BYS 000000 58 IODON,STATE=FORK UCB 056042 IO STATUS 000001 07640 PKT 054730 58 IN FORK CODE,FFL 000001 STATE=FORK FINAL STATE=IDLE 58 INTERRUPT EXIT,FFL 000001 STATE=FORK 58 FORK,FFL 000001 STATE=TRAN FINAL STATE=IDLE 58 * INTERRUPT,STATE=TRAN CSR 121330 ERR 000001 IDR 110040 (:EOM) UCB 056042 57 INTERRUPT EXIT,FFL 000001 STATE=TRAN 57 * INTERRUPT,STATE=WRWC CSR 125302 ERR 000001 IDR 003720 (WC=2000) UCB 56042 57 INTERRUPT EXIT,FFL 000001 STATE=WRWC 57 SEND LINK CODE 003720 (WC= 2000) 57 * INTERRUPT,STATE=WPAK CSR 121302 ERR 000001 IDR 120001 (:ACK) UCB 56042 57 INTERRUPT EXIT,FFL 000001 STATE=WPAK 57 SEND LINK CODE 140001 (:PTC) 57 * INTERRUPT,STATE=WLAK CSR 121302 ERR 000001 IDR 120005 (:ACK) UCB 56042 57 EXIT FROM DRIVER,FFL 000001 STATE=WLAK 57 SEND LINK CODE 104000 (:RLK) 57 CDCHK,STATE=IDLE IO FN 400 WRITE TASK ...XMT UCB=056042 PKT 054730 The above trace entries are those for one normal transmit (write) of a block of data over the link. Column one gives the ticks counter at the time of the trace entry. The asterisk is put beside an interrupt. Most of the rest is self explanatory for anyone familiar with the link protocol. The UCB address is used to distinguish between the different CD unit numbers. Various interesting words within the UCB are displayed.