.; .; The text of a paper given at the DAARC sessions .; of the Fall 83 DECUS symposium in Anneheim, Cal. .; For correct printing use Bonner Lab Runoff .; .ps 72,90,0,0,1 .no header .req 'sys$library:greekmath' .no flags underline .no flags special <> .define escape '^&' LCK 27,'E' .define escape '\&' LCK 27,'R' .;define substitute /mu$/u .flags substitute .flags hyphen ~ .define item 'TABL$' .define item 'FIG$' .C TEXT ^*REALTIME FAST DATA ACQUISITION USING AN MBD AND CAMAC\* .S2 John Clement Bonner Nuclear Lab Rice University Houston, Tx 77251 .s4 ^&Abstract\& .end center .lm+20 .rm -20 To satisfy our need for a fast flexible data acquisition system we developed a data acquisition system built around an intelligent CAMAC interface known as the MBD. In conjunction with the hardware we designed software to control the interface. The software includes MBD microcode, a device handler, a compiler and some FORTRAN subroutines to use the MBD. In the general overview of this system no prior experience with this type of hardware is assumed. .ps ,43 .s4.c;^&Introduction\& .p At Bonner Lab the primary activity is taking and analyzing nuclear physics and high energy data. The problems of nuclear data acquisition are very similar to those encountered by industrial research labs, and process control systems. The main difference is that we never do the same experiment for more than a couple of months in a row, and our data rates can vary from about 1 byte per second to over 100 kbytes per second. Furthermore our data rate is not uniform even within a single experiment. These varying rates can severely tax any computer system. .p Another problem shared with the industrial world is the presence of deadlines. We are given only limited time at a big accelerator to complete our experiment. Any software or hardware failures reduce the amount of data we can take so an unreliable system is usually a disaster. To make sure that the data is good we must partially analyze and look at it interactively in real~time while acquiring it. .p To meet the flexibility requirements we settled on CAMAC as our interface standard. To relieve the load on the CPU we helped design an attached processor to perform most of the data acquisition chores. The PDP-11/34 is then free to do all of the analysis. Essentially we have a multiprocessor system. Finally to help insure reliability we designed a user friendly structured programming system. The Bonner lab system is shown in Figure 1 .figure deferred 26 .s.tt4.c;^&CAMAC\& .p CAMAC is a standard interface originally designed to meet the need of the nuclear physics community. It consists of a series of crates strung together on a bus called a branch highway. Each crate contains an independent power supply, a crate controller and slots for up to 24 modules. The bus has a controller which attaches it to the computer. Each module is an independent device or peripheral. For example a CAMAC module can contain several analog to digital converters, scalers, voltmeters or any of a large variety of devices. These modules can be readily purchased from a variety of vendors, and more importantly we can borrow them from equipment pools at the accelerator labs. To use a module, the experimenter just plugs it into a crate and specifies it in his program. .p Modules can be input, output, or both. To use a module we specify the operation as a set of numbers which determine what is to be done and which module does it. These numbers are called the FCNA. .text .list 0 .lm -3 .le;F the function (0-31). .le;C the crate number (1-7) .le;N the slot address within the crate (1-31) .le;A the subaddress within the module (0-15) .els 0 .end text The function (F) is defined by the CAMAC standard to have certain meanings. For example 0 means read, while 2 is read and clear, and 16 is write. Some functions control the module without transferring any data. Each module can contain up to 16 sections specified by the subaddress. For example you might have 16 separate voltmeters in the module. .p In addition each module can independently signal the computer it is ready to transfer data by generating a LAM (look at me) signal on one of 24 lines. This is analogous to an interrupt generated by a normal computer peripheral. .text deferred .fig 60 .c;Figure 1##################### .c;Bonner Lab Acquisition System .s .etx .s.tt4.c;^&MBD\& .p The MBD (Microprogrammed Branch Driver) is an attached processor which was designed by us in collaboration with Los Alamos National Lab. The chief designer at Rice on the project was James Buchanan. We are still using the model we originally built though it is currently commercially available through BiRa systems. It is used by labs both in the U.S. and Europe. Basically it must transfer data from CAMAC to memory or from memory to CAMAC via the UNIBUS. As one of the design goals it must handle several simultaneous data streams at maximum speed. The result was a 16 bit reduced instruction set computer (RISC) containing 128 general purpose registers and 8 special purpose registers. The instructions are optimized to do the operations necessary for data transfer and they have little arithmetic capability. Each instruction can perform several overlapped operations to achieve maximum speed. For example data can be modified, transferred to a register, and written to the CAMAC branch simultaneously. .p The MBD is organized into 16 priority ordered channels each of which is activated by a separate request. The 128 registers are divided into groups of 8 assigned to only 1 channel. Each channel may correspond to a separate data stream. Each channel has a separate vector, but all are controlled through a single CSR (Control and Status register). When the MBD switches from processing one data stream to another, the context switch takes only a $mu$sec because the registers do not need to be saved or restored. This may be contrasted with conventional computers which require tens to hundreds of $mu$sec for a context switch. .p Early in the design of this system we decided to use the MBD as a classic microprocessor, not a microcomputer. The MBD programs are like microcode in control store and the user programs reside in the PDP-11 memory. The MBD must then fetch instructions from the 11 memory and execute them. This could be considered to be a coprocessor or a list processor similar to the VS11. The major difference between this and a conventional list processor is the capability of handling up to 16 independent data streams. This type of design has the advantage of simplicity, and the memory is only limited by the available PDP-11 memory, not the MBD control store. The MBD code need only be written once, and may be designed carefully so that disastrous errors are avoided. For example it is possible in the MBD code to lock the bus, thus bringing the PDP-11 to a halt. Another problem is the possibility of accidentally writing over user code inside the PDP-11 memory. The major drawback is that maximum processing speed may not be achieved. .s.tt4.c;^&MBD Code\& .p The MBD code is written using the PDP-11 MACRO assembler, then it is downloaded into the MBD control store. Then the MBD does not run until started by a command from the 11. The MBD once started, fetches a buffer address, buffer length, and command list address from the 11 memory. The command list (CLIST) contains all of the instructions to the MBD, while the buffer contains data to be output to CAMAC and/or space for data to be acquired. The MBD only reads the CLIST while it may read or write the buffer. In effect this divides the 11 memory into instruction and data space as far as the MBD is concerned. .p Each instruction in the CLIST specifies how data is to be transferred. For example an instruction may specify transfer 25 words of data from CAMAC into the buffer using a given FCNA. This is equivalent to a direct memory access (DMA) transfer by a normal peripheral such as magnetic tape. Another command might involve transferring data from different modules specified by a list of FCNAs. There are some specialized commands which will take data from CAMAC, interpret it as an address, and increment a memory location in the buffer. This would be equivalent to a memory resident scaler or an automatic histogram option. This is often called direct memory increment (DMI). Data can be added to the buffer as well as transferred to it. Both 16 bit and 32 bit transfers are possible. The instruction format is extremely variable then depending on the application. Each instruction is executed by a section of microcode inside the MBD and the time to execute that instruction could vary from 10 to thousands of $mu$sec depending on the amount of data transferred. .p Since the MBD must transfer data on demand and time share acquisition over several data streams, three different modes of execution are used. In the normal mode of execution MBD programs give up control at regular intervals to allow higher priority channels a chance to execute. This is typically done in between each data word transferred to the 11 memory. The second mode or fast mode is where the command is executed with no interruptions. The third or wait mode is where the MBD waits for a LAM to signal the availability of data. In wait mode the MBD program ceases execution and allows any other pending channels to run. The waiting channel does not continue execution until the appropriate LAM signal is generated by a CAMAC module. .p The fast mode is used for time critical applications where the speed of transfer affects the amount of data acquired. It is also useful in resolving conflicts between different channels that must use the same modules. A mixture of modes may be used within a single CLIST. Normally data acquisition requires waiting for data to be ready and then transferring it quickly in the fast mode. At the same time utility programs may be monitoring other devices such as voltmeters, by reading them in normal mode. .s.tt4.c;^&The Device Handler\& .p A very important component of the system is the device handler which controls the execution of the MBD. This is the component that actually activates a given channel and reports the operation done to the user program. It was constructed to look like a standard RSX-11M device handler and in some respects it was patterned after the laboratory peripheral display driver, but it looks like 16 independent devices (MB0: to MB15:). Table $TABL$ shows the various device handler functions available. .text deferred .s .tab stops '-' 80 .tab stops +8,+8,+8,+8,+8 .s .c;Table $TABL$ device handler MB: functions .s .list 0 .lm -3 .le;IO.KIL - Stop all activity on the specified device. All requests by a given task are aborted. .le;IO.RBC - Start execution of the specified CLIST and buffer. .le;IO.ATT - Attach the device and a specific crate .le;IO.DET - Detach the device If a CLIST is still executing then the request is queued for execution later. .els 0 .s .tab stops '-' 80 .tab stops +8,+8,+8,+8,+8 .s .etx .p Since speed is one of the most important needs the device handler was designed in a slightly non standard fashion. The device handler is completely standard as long as a device is not attached. When a particular channel MBnn: is attached, the device handler queues the requests internally instead of relying on the DEC standard mechanism. This shortens the time to switch CLISTs and buffers from about 2 msec to 200 $mu$sec, or a 10 fold decrease in time. .p The device handler also has a mixture of standard and specialized features. .number item 'TABL$' +1 These are presented in table $TABL$. The status return needs to be carefully interpreted in light of the request made. For example the number of bytes of data transferred is probably irrelevant if the DMI routines are used by the CLIST. .text deferred .s .tab stops '-' 80 .tab stops +8,+8,+8,+8,+8 .s .c;Table $TABL$ Device Handler Features .s .list 0 .lm -3 .le;Timeout may be specified as time in seconds, infinite, or default. (standard feature) .le;Multiple CAMAC crates may be attached as specified by 8 bit mask when IO.ATT is performed. .le;Crates to be used are specified with IO.RBC. .le;Command list may be restarted with a new buffer. .le;Status returns standard error code and the number of bytes of data transferred. .le;Status return indicates if CLIST needs continuation. .els 0 .s .tab stops '-' 80 .tab stops +8,+8,+8,+8,+8 .s .etx .s.tt4.c;^&CLIST Compiler\& .p All of the preceding discussion was about the portions of the system which are designed and maintained by a system programmer. The actual construction of the CLIST and acquisition program is often left to the user. To simplify this task a CLIST compiler generates code from a simple "high level" transfer language. The compiler was written in FORTRAN and it produces both listings and object code. The object code is in standard RSX-11 format which is linked to the program by the task builder. The object modules are not designed to be executed, but are passed to an assembly language camac program along with a buffer. The camac program starts a transfer by issuing a QIO and passing the CLIST along with the buffer to the device handler. .p Since the CLIST is performing a very specialized set of operations most available languages are not suitable. Hardly any arithmetic operations are available so FORTRAN would of course be useless. Since most scientific programs are written in FORTRAN some common notation was used. The basis of the language is a verb or command which must appear first on each line. Following the command are modifiers approximating a simple English sentence. .number item "TABL$" +1 A sample of the syntax is presented in table $TABL$. The compiler is clever enough to decode the syntax when the TO preceds the FROM key word. Data is moved from a source to a destination, or moved from a source and added to a destination. The FCNAs are represented as . Several modules may be represented as . This means all As between A1 and A2 and so on. The notation is incremented from right to left, or the As are incremented before the Ns. .text deferred .s .tab stops '-' 80 .tab stops +8,+8,+8,+8,+8 .s .c;Table $TABL$ CLIST syntax. .s .c;[ ] Denote optional elements .c;Uppercase elements are keywords .nf.list 0 .lm -3 .s .le;DEFINE symbol=source/destination .le;MOVE [FROM] source [TO] destination [nnn] [TIMES] .le;ADD [FROM] source [TO] destination [nnn] [TIMES] .le;STOP ! terminate execution .le;WAIT ! wait for lam .le;IF source GT value .i4;Operators: GT LT GE LE EQ NE .i2;ELSEIF source GT value .i2;ELSE .i2;ENDIF .le;DO [WHILE source GT value] .le;ENDDO .le;EXECUTE .le;SET [FAST] [LONGWORD] [WHENLAM] .le;GOTO 10 20 30 40 .#.#.# source .i2;n:###BLOCK .i2;n:###CONTINUE .i2;ENDGO .le;ADDRESS nnn .els 1 Sources/destinations: .lm +4 .i-2;BUFFER .i-2;REGISTER is a single temporary storage location. .i-2;Constants can be decimal octal or hex sources. .i-2;LIST is a indicates string of following the command. .i-2;COUNT is the remaining size of the buffer .s.i-4;Modifiers for any command: .i-2;WHENLAM FAST WORD LONGWORD .lm .f .s .tab stops '-' 80 .tab stops +8,+8,+8,+8,+8 .s .etx .p .text An example of the code would be: .br;MOVE <2,5,2:5,0:15> TO BUFFER LONGWORD .br This would read and clear 24 times and put the data into the buffer. 3 modules in crate 5 are used. The data occupies 96 bytes or 24 32-bit words. .text .s.br;MOVE <0,2,10,0> TO BUFFER 25 TIMES WHENLAM transfers 25 words to the buffer from a single CAMAC module. The MBD waits for a LAM before each transfer. The final data occupies 50 bytes. .text .s.br;MOVE 35 TO <16,2,10,0> writes the number 35 to the module in crate 2, slot 10, subaddress 0. .text .s.br;ADD LIST TO BUFFER .br;<0,2,10,0> <2,3:4,1:4,0:15> .br;reads CAMAC 129 times and the data is added to the data already in the buffer. The data is treated as 16-bit words. .etx .p The listings produced by the compiler are automatically indented to reveal the structure of the CLIST. Comments may be added to each line after an exclamation point as in DEC FORTRAN. Top down structured programming is enforced. Once the user grasps the operations being performed, the syntax is relatively simple. For example MOVE is used to fill a buffer from CAMAC devices or to output data to CAMAC. ADD is used to add the data from CAMAC to the existing contents of a buffer. When combined with an unlimited DO and the ADDRESS command, ADD creates a bank of memory resident scalers. Normally only 16 bit data is used, but the modifier LONGWORD treats data as 32 bit. The modifiers may be specified for all code with the SET command. .p An important function of the compiler is protecting the integrity of the user program. It keeps track of which crates are used and puts that information into a header of the CLIST. The device handler is passed this information and checks it against an access list. If a task attempts to use a crate attached to another task, the device handler rejects the request with an error code. Real~time access checking of each FCNA is impossible because of time and memory limitations. Some form of checking is vital. With protection we may use CAMAC for a variety of purposes rather than restricting it to primary data acquisition. .s.tt4.c;^&Conclusion\& .p The design goals of this system were met by the combination of hardware and software we have described. Many of the goals conflict. For example a fast system is usually neither user friendly nor secure. A flexible system is often complicated and not fast. We believe that we have made reasonable compromises. .p The sources for all the software mentioned here will be included in the FALL 84 RSX DECUS tape. One caution is in order. Our MBD differs from the commercially available version in several ways, so any user who wishes to use our code must modify it for the available hardware. This is made easy because our model was severly limited in memory compared to the commercial version manufactured by BiRa systems. .text .s.c;^&Bibliography\& .s .list 0 .lm -3 .no period .le;"An MBD Primer," LA-5511-MS Informal report, Los Alamos Nationa Lab, Los Alamos, New Mexico 87544 .le;"Programming the MBD Model II," LA-5404-MS Informal report, Los Alamos National Lab .le;"A User-Oriented Structured CAMAC I/O Software System," I.E.E.E. Transactions on Nuclear Science, Vol. NS-28 .le;"CAMAC Multi-microprogrammed IO Processor," I.E.E.E. Transactions on Nuclear Science, Vol. NS-19 .etx