DS077 DR11W AS A LINK DEVICE - The Transmission Protocol Ben Burch Joe Biel Vicky White January 8th 1982 DR11W as a link device - The Transmission Protocol Page 2 1.0 INTRODUCTION The decision to provide data acquisition systems consisting of two or more connected CPU's was made some time ago. The motivation for this can be summarised in the following list of goals: 1. Increase event data rates by allowing data transfers on logically separate parts of an apparatus, to be done in parallel. 2. Decrease event dead-time by allowing data transfers on logically separate parts of an apparatus, to be done in parallel. 3. Provide sub-systems which can be dedicated to a particular piece of apparatus or sub-group of experimenters. 4. Provide extra memory for analysis and other functions which much be performed on-line. 5. Make best use of the existing resources, with limited extra expenditure. The existing PDP 11's, even with bulk memory added are limited for program space. 6. Increase the number of instructions/sec which can be executed in an experimental system, for whatever purposes. The various ways in which these goals might be achieved and the relative merits of the possible approaches has been discussed extensively in a series of Bison Group notes: IN-32,IN-35,IN-40,IN-48,IN-50 and Bison Meeting of September 24 1980. The chosen method for Interconnecting CPU's is by direct parallel link, using the DR11W DMA device. This assumes that all CPU's involved are Digital Equipment Corporation UNIBUS machines. The use of sufficient quantities of bulk-memory on some or all of the data acquisition machines is assumed, in cases where high speed or low-dead-time data acquisition is required, or where the volume of data is large. The methods by which the various goals might be achieved using this interconnection hardware are numerous and each goal, in isolation, has perhaps a different optimal solution. DR11W as a link device - The Transmission Protocol Page 3 The types of interconnected systems which could be used fall into two broad categories: a) CONCAT ---- Mag tape b) DA --- Mag tape | | | | _________ ... ____ _________ ... __ | | | | | DA DA DA EXTRA EXTRA a) the acquisition of data is performed on one or more DA machines. The assembly of the data into a single event and its logging to tape is performed on a separate Concatenation machine. b) the acquisition and logging of data are performed on a single machine but separate machines are used to perform extra analysis or equipment monitoring. Hybrid systems combining these two basic types are of course possible. Apart from the functions of Data acquisition Logging of Data to tape Reading back data from tape whose location is defined by this categorisation, the other functions needed by an experiment Data analysis Calibration/Testing Run control Error logging (+ program development) might be distributed in any possible way between the available machines, depending on the specific needs of the experiment. The distribution of these functions arbitrarily and flexibly requires the use of the link (a half-duplex communications device) in a full duplex fashion. It may also require that the link be 'shared' between several different 'tasks' or applications in one machine. The very act of interconnecting machines itself generates further uses of the link - file transfer, message broadcasts, central error reporting, etc. immediately become desirable features. For this reason, for an RSX system a Communications driver has been ___ __ ___ ______ envisaged. This will allow several applications to use and share the link in a controlled way. See IN-57 for design details. DR11W as a link device - The Transmission Protocol Page 4 However the overhead involved in performing a system I/O request for each message to be transferred across the link is high (1.6-2.0 ms) , and ____ _______ in case a) conflicts with the goals of high rate, low dead-time data acquisition. So, in cases where such 'fast' data acquisition is the primary goal, a mechanism needs to be provided for temporarily giving dedicated use of the link to the software responsible for: - transmitting the data from a DA machine to a concatenation one - receiving and assembling the data from one or more DA machines ready to be logged on tape Such a mechanism has been discussed in DS-68, and IN-57, for an RSX system. We cannot definitely say what types of machines will be used in a particular connected configuration, or which operating systems each of the machines will use. Therefore it is desirable to define the basic protocol __ __ _________ __ ______ ___ _____ ________ by which messages are transferred over the link, regardless of which piece __ _____ ________ ___ ___________ ____ ___ ____ of software may be responsible for the transfer: a driver, a dedicated process, or whatever. The needs are different however. The dedicated use of the link for a single function, with a well defined set of rules (e.g. between the Event receiver and the Event transmitter), is a considerably simpler environment in which to define an adequate protocol than that of using the link for a multitude of different functions. In the latter case the various higher level protocols between the communicating entities are unknown, and so cannot be utilised, for example in assumptions about error recovery procedures. In what follows we attempt to define a protocol suitable for both environments. DR11W as a link device - The Transmission Protocol Page 5 2.0 USE OF THE DR11W AS A DIRECT DATA LINK The DR11W is a general purpose UNIBUS DMA device. A DR11W on one UNIBUS can be connected to another DR11W by means of a suitable interface cable, not exceeding 50ft. Data transfers are 16 bit word (parallel transfer) in either a programmed I/O (single word) mode or in a DMA mode. The DMA may be on a 2 cycle (standard) per bus grant, termed block mode; or on N cycles per bus grant termed burst mode. Burst mode is not supported for a VAX/DR11W link. The DR11W uses six programmable registers WCR - Word count register (read/write) BAR - Bus address register (read/write) IDR/ODR- Input/Output data register (read/write) CSR/EIR- Control and status register (write only) Error and Information register (read only) The DR11W may interrupt the processor, provided this interrupt capability is enabled The DR11W hardware is described in Digital's " DR11W Direct Memory Interface Module User's guide". When the DR11W is used as a link device to interconnect machines the actual communication between the linked systems is established through manipulation of some of the CSR bits of the DR11W. Setting and clearing of bits 1-3 in one DR11W causes setting and clearing of some corresponding bits of the CSR of the partner DR11W. By this means one machine may indicate which mode of data transfer it wishes to use and may interrupt the partner machine. The ODR functions as a write-only register for data transmitted to the other computer. The link operates in a half duplex communications mode; i.e. it has the capability of transmitting data bi-directionally, but in only one direction at a time. Each computer has independent control of its own DR11W, so that the pieces of software in each computer, which are responsible for data transfer over the link, must be compatible, i.e. They must have suitable agreements about which of them is manipulating the relevant 'connected' bits of the CSR at any time; which is interrupting the other next. They must ensure that suitable word counts are set up for DMA operation and that the DMA is properly synchronised. It is this protocol, at the level ________ of transmission of a single message over the link, which is described in the following sections. What is described is the protocol between the two pieces of software (link managers) controlling the DR11W , in order to transmit a single packet of data, or single message. It is not necessarily the end-to-end protocol between any 2 processes in the two machines which wish to communicate over the link. Two processes which communicate using the DR11W-link driver, or Communications driver, will use a higher level protocol described in the Communications Driver DR11W as a link device - The Transmission Protocol Page 6 documentation. We assume that we will be using a 'layered' approach in using the link, in which higher level protocol(s) between processes must be clearly defined. The link transmission protocol provides a limited set of functions, which are summarised below. Other levels of protocol will have to provide the required retries, checking that they do not have the same request twice, data integrity checks, insurances against lockout and deadly embrace situations etc. 2.1 PROTOCOL DESIGN CRITERIA 1. Data blocks of any type of data, and arbitrary number of 16 bit words (up to 32K maximum) may be transferred on the link in either direction. 2. There will be no check on the integrity of the data block, i.e. no checksum or CRC. Although not ideal, this is in keeping with the general level of data integrity in a data acquisition environment, where CAMAC, and often memory, have no parity. Applications needing to ensure data integrity must put suitable checks into their higher level protocol. For example, file transfer utilities would probably implement file checksums. 3. It should be possible to transfer data by a direct DMA from end-user buffer to end-user buffer, for high speed and low CPU usage. 4. The link transmission protocol will be considered to be applicable only to DR11W transmission on UNIBUS machines. 5. Although a final goal of flexible and transparent use of the link for any purpose should be kept in mind, it should not compromise the need for a fairly 'light' low level protocol, for use in the event transmission. Here the choice is between extra DMA(s) between partners for each block of data to be sent, or single word data transfers and handshakes. 6. The protocol should allow for adequate distinction between real hardware errors, and software/timing problems and should permit knowledge of the type of error encountered to be given to the partner. It should not however attempt error recovery. Each end of the link knows only that the message was either sent successfully or failed, with a reason. The handling of retries, and problems that retries bring such as detecting duplicate requests is the responsibility of the higher level protocol. 7. The link transmission protocol will only be used in cases of directly connected transmitters and receivers (no route-through). DR11W as a link device - The Transmission Protocol Page 7 8. The hardware will be assumed to be relatively error-free (unlike a longer-range serial link). Although all errors due to intermittent hardware errors, persistent hardware errors or software bugs must be handled, they will be assumed to be fairly rare events. DR11W as a link device - The Transmission Protocol Page 8 3.0 SINGLE WORD DATA TRANSFERS ON THE LINK Single 16 bit data words are transferred between machines for all control and status information. The sender of such information writes it to his output data register and informs the partner by interrupting him. 16 bit data words are used for - requests to the partner - ACKS and NACKS - End-of-Message status returns - Word Counts - Single data word messages - SIGNALS In general a single data word exchanged between machines has the following format: High Byte ____ ____ - control information Bit 7 - NOTWC - If clear data word is a word count 6 - PTC - If set data byte is a packet type code 5 - ACK - If set data byte is an ACK or NACK of PTC or WC 6 - EOM - If set data byte is the end-of-message status 3 - REQLK - If set sender is requesting the link 2 - START - If set sender is requesting to start or re-start the protocol 1 - SIGNAL- If set data byte is a Signal 0 - DELAY- If set sender request partner to delay before starting to transmit next message Low byte ___ ____ If NOTWC bit of high byte is set the low byte is a data byte with a value between 0 and 255. If the SIGNAL bit is set the data byte may take any value between 0 and 255. Otherwise the value of the data byte is restricted to certain legal limits depending on the control byte. 1-7 ACKS and NACKS 1 - PTC ACK 2 - PTC NACK 3 - START ACK 4 - WC NACK 5 - LINK ACK 8-31 End-of-message error statuses 32 End-of-message - success code 0-255 Packet type codes. The use of the data word in this way serves two purposes 1. Provides some degree of security against corrupt codes and statuses. Relationships between the allowable bits of the control byte exist and correct correlation between the data byte value and the control byte information can be checked. An DR11W as a link device - The Transmission Protocol Page 9 end-of-message status can not be transformed from a success to a failure or vice-versa without the addition of one bit and the removal of another bit. 2. Allows status codes to be combined with request. In particular request for link mastership or request for restart of protocol or request to delay next message transmission can be combined with an end-of-message status. 4.0 STARTING THE LINK PROTOCOL Initiator Start Protocol ==============> Start protocol ACK <============== The link protocol must be started whenever a link controller wishes to initialise, or to resynchronise with its partner after an error in the protocol. This starting of the protocol also establishes a primary/secondary relationship between the partners. The initiator sends a single word Start-protocol code and interrupts its partner. It then waits to receive an acknowledge of protocol startup The receiver of a start-protocol code, who is not itself waiting for a start-protocol acknowledge will send a single word start protocol acknowledge code and interrupt its partner. It then considers the protocol to be active and itself as the secondary machine. When the initiator receives the Start-protocol-acknowledge it also considers the protocol to be active, and itself to be the primary machine. In the case where both partners attempt to start the protocol simultaneously, one or both will receive a start request when waiting for a start acknowledge. Each ignores the request and retries after a random ______ delay _____ Until the next start-protocol code is sent by one of the partners the primary/secondary relationship remains in effect and determines which machine gains mastership of the link in a conflict case. DR11W as a link device - The Transmission Protocol Page 10 5.0 TRANSMITTING A SINGLE MESSAGE 5.1 MESSAGE FORMAT Each message consists of a single data packet and a packet type. A data packet may contain up to 32K 16 bit words. The sender of a message must initiate the transfer. The protocol for the transfer of messages over the link comprises 4 phases: 1. Initiation - to gain mastership of the link for the sender 2. Sending of a packet type code (PTC) 3. Sending of a single data word which is either a Word count (N) for a DMA to follow, or a single word message, termed a Signal. 4. Optionally a block transfer (DMA) of N data words. There are acknowledgements in all phases. 5.2 INITIATION Transmitter Receiver Request Link ==============> Link ACK <============== The transmitter bids for mastership of the link by transmitting a single word, in word mode, and interrupting the receiver. The receiver grants mastership of the link to the transmitter by sending a single word acknowledge code, in word mode, and interrupting the transmitter. If the receiver is in any state other than IDLE or attempting to itself gain mastership of the link, when it receives the request for link mastership, then it will return an error status to the transmitter together with a request to re-start the protocol. If the transmitter does not receive the link acknowledge within a specified time limit, or receives an unexpected code it terminates its bid and the request for the link will be eventually retried. DR11W as a link device - The Transmission Protocol Page 11 If both computers bid for mastership of the link at the same, or nearly the same time, then one or both of them will receive a request for link mastership whilst themselves requesting mastership. Then the primary machine will ignore the request and the secondary machine will back-down from its bid and send a link ACK code to its partner. Once the initiation phase has been successfully completed then phase 2 of the transfer may proceed. 5.3 SENDING OF A PACKET TYPE CODE Transmitter Receiver Packet type code (PTC) ==============> PTC ACK or PTC NACK <============== The packet type code is a simple form of packet header. It contains routing information. The transmitter sends a single data word to the receiver,which describes the packet type.(PTC) PTC's are symbolically defined in the macro prefix file DR11WDEF.MAC and in a Fortran Include file. The PTC contains the information by which the receiver may decide whether or not to allow the transfer of data to take place. How this information is used is really a function of the higher level protocol between communicating processes. On the level of transmission protocol it need not be used at all, other than checking for validity If the receiver is prepared to accept the data, it acknowledges the PTC by sending a single word PTC acknowledge code and interrupting the transmitter. If the receiver is not prepared to accept data from the transmitter then it sends a single word NACK of the PTC and interrupts the transmitter. This terminates the message transfer with an error condition, for both transmitter and receiver. If either the receiver or the transmitter does not recieve the expected PTC, or PTC acknowledge within a certain time, it times out and considers the message transmission terminated with an error condition. DR11W as a link device - The Transmission Protocol Page 12 If either receiver or transmitter receives a request or status code other than that expected, it also terminates the message transmission with an error condition. In both cases of error the terminator sends a single word end-of-message error code together with a request for restart of the protocol. 5.4 SEND WORD COUNT OR SIGNAL Transmitter Receiver Word count or Signal ==============> Receiver's Word count or Signal ACK or WC NACK <============== Once the willingness of the receiver to accept data from the transmitter has been established, an agreement must be made about the amount of data to be sent. The transmitter sends a single data word and interrupts the receiver. This data word may be one of two possible indications: 1. A word count for the DMA data transfer which will follow 2. A single word message or 'Signal' All positive values of the data word are word counts. All negative values of the data word are interpreted as Signals and must therefore conform to the format for Signals They are thus distinguishable from link request codes, packet type codes, and various acknowledge codes. When a Word count has been passed to the receiver it must be followed by either an acknowledgement, or a refusal (WC NACK). In the normal transfer case, the receiver sets up the DR11W DMA ready to receive data and then sends, as an acknowledgement, its receiver word count to the transmitter, and interrupts the transmitter. The receiver's word count is sent as a single word data transfer and is a positive value not exceeding the transmitter's word count. DR11W as a link device - The Transmission Protocol Page 13 If a signal had been passed to the receiver then it must acknowledge receipt of the signal, thus terminating the message transfer. Both a Signal Acknowledge and a Word count NACK are given by sending a single word end-of-message status code, in word mode, and interrupting the transmitter. If either receiver or transmitter times-out on waiting for a word count or word count acknowledge, or receives any invalid data word then the message transfer terminates with an error condition. A request to restart the protocol is sent to the partner together with an end-of-message status error code. 5.5 DMA TRANSFER OF DATA Transmitter Receiver N data words --------------> End-of-message status <=============== When the transmitter has received a WC acknowledge, it may then actually start the DMA of data from its output buffer. The receiver has already set up its DMA from the DR11W into its input data buffer and so the DMA will proceed. When the receiver DMA ends (or times out) the receiver sends a single word End-of-message status and interrupts the transmitter. Because of the nature of the DR11W device, an interrupt to the transmitter which arrived before the end of DMA interrupt had been serviced by the transmitter would be lost. The transmitter must be aware of this possibility and check the input data register for an end-of-message code immediately before clearing the interrupt. The receiver is free to immediately try to gain mastership of the link to transmit, after sending the end-of-message status code. Since the interrupt request for link mastership could similarly arrive before the end-of-message status interrupt has been handled, the receiver must always retransmit the end-of-message status together with the link request. This combined request can in fact be sent straightaway at end-of-message by the receiver, if it already knows that it will need to transmit. Alternatively the end-of-message status can be combined with a delay request, if the receiver wishes to hold off further messages from the transmitter whilst it performs other functions. DR11W as a link device - The Transmission Protocol Page 14 6.0 SYNCHRONISATION OF TRANSMITTER AND RECEIVER Each end of the link transmission protocol is a state machine with 11 possible states. -2 Link Down -1 Waiting for protocol start 0 IDLE 1 Requesting link 2 Waiting for PTC 3 Waiting for PTC ACK 4 Waiting for WC or Signal 5 Waiting for Receiver WC or WC NACK 6 Waiting for end-of-message (or Signal) status 7 Transmitting 8 Receiving The state transitions are initiated by 'events' occurring. These are either a DR11W interrupt, or a time-out on waiting for an interrupt. In the protocol, the occurrence of any event other than the one expected in a particular state causes an abandonment of the current message transfer by that partner, with an error condition. If the unexpected event is a timeout or the receipt of any single word code other than a re-start protocol request then an error status code combined with a restart request is sent to the partner. If the unexpected event is the receipt of a re-start-protocol request (with some end-of-message status code) then the situation procedes as for the Link Start Protocol procedure. It can be seen that the resynchronisation mechanism is to restart the protocol whenever there is an unexpected error in the protocol, informing the partner in doing so of what error was detected by you. After sending a re-start-protocol request, the link is in the state of Waiting-to-start-protocol. Persistent failures to receive a start-protocol acknowledge will result in the link being put in the state 'link down'. In every state one possible "event" MUST be a time-out occurring. ____ All transitions back to the IDLE state imply termination of the message transfer. The transmiter and receiver may independently return to the IDLE state in one of the following conditions: 1. all data transferred successfully 2. some or all data transferred but with error. 3. no data transferred - error condition. DR11W as a link device - The Transmission Protocol Page 15 All transitions back to the link DOWN state imply termination of the message in a condition where either hardware or software failure has caused complete failure of the protocol, necessitating a resynchronisation by restarting the protocol. The overall timeout for a transmitter should normally be longer than that of a receiver. This is to ensure that a receiver who is unable to receive a message informs the transmitter of this with a suitable WC NACK, before the transmitter times out. This case is then not a failure in the protocol, only a normal failure to deliver a message. 7.0 STATE DIAGRAMS The transitions between states of a process controlling link data transfer are shown diagramatically in Figs 1,2 and 3. Receipt of interrupt or timeout is indicated by a single arrow, Sending of data to the partner is indicated by double arrows. OTHER indicates that an event other than the ones on the alternative event lines has occurred. This may be a time-out or any other not expected interrupt. Page 16 8.0 START OF TRANSMISSION AND INTERRUPT HANDLER FLOW CHARTS These show in detail how the state transitions are handled as well as how the actual transmission of data and interrupting of the partner is carried out. Initiating a DR11W Transmit Operation __________ _ _____ ________ _________ Start | | Raise processor priority high enough to prevent DR11W interrupts | | -----<----------------DR11W state = idle ? | | yes Start transmit timer | | Load link-request-code into DR11W output data register | | DR11W state <- waiting-for-link-acknowledge | | CSR-template <- FNCT1 ! ! | | Lower processor priority | | Exit ----->-------------------------- ! Lower processor priority | | Try again later Page 17 DR11W Interrupt Service Routine _____ _________ _______ _______ (Executed at a processor priority high enough to prevent additional DR11W interrupts) | | ! ! Start | | -----<----------------DR11W state = idle ? | | yes DR11W input data reg. = request-link-code ?---> ERROR | | yes DR11W output data reg. <- link-acknowledge-code | | DR11W state <- waiting-for-packet-type-code | | CSR-template <- FNCT1 ! ! Start receive timer | | | | Exit ----->-------------------------- | -----<----DR11W state = waiting-for-link-acknowledge ? | | yes DR11W input data register = link-acknowledge-code ? --> ERROR | | yes DR11W output data reg. <- packet-type-code | | Page 18 DR11W state <- waiting-for-packet-type-code-ack. | | CSR-template <- FNCT1 ! ! | | Exit ----->-------------------------- | -----<--DR11W state = waiting-for-packet-type-code-ack. ? | | yes DR11W input data reg. = packet-type-code-ack-code ?--> ERROR | | yes DR11W output data reg. <- transmit word count | | -----<----------Transmit word count < 0 ? | | no DR11W state <- waiting-for-end-of-message status code | | CSR-template <- FNCT1 ! ! | | Exit ----->----------------------- ! DR11W state <- waiting-for-word-count-acknowledge | | CSR-template <- FNCT1 ! ! Page 19 | | Exit ----->-------------------------- | ---<--DR11W state = waiting-for-end-of-message status code | | yes DR11W input data reg. = end-of-message status code ?-> ERROR | | yes Cancel receive timer | | DR11W state <- idle | | Indicate to program that transmission has been successfully completed | | Exit ----->-------------------------- | --<----DR11W state = waiting-for-word-count-acknowledge ? | | yes DR11W input data reg. = valid receivers word count ?-> ERROR | | yes Transmit word count <- DR11W input data reg. ! ! DR11W address reg. <- lower 16 bits of buffer address | | CSR-template <- upper two bits of buffer address | | DR11W word count reg. <- transmit word count | | Page 20 DR11W state <- waiting-for-transmit-completion | | CSR-template <- FNCT1 + FNCT3 + GO + CYCLE ! ! | | Exit ----->-------------------------- ! ---<----DR11W state = waiting-for-transmit-completion ? | | yes Are any DR11W error bits set or does the DR11W word count register contain a nonzero value ? -----<----------------------- | | yes Indicate to the program that a transmission has been completed, but that there was an error | | DR11W state <- waiting-for-end-of-message status code | | Exit ! ----->----------------------- Indicate to program that a transmission has been successfully completed. | | DR11W state <- waiting-for-end-of-message status code | Page 21 | Exit ----->-------------------------- | -----<----DR11W state = waiting-for-packet-type-code ? | | yes Packet-type-code <- DR11W input data reg. | | Is packet-type-code a legal value ? ---> ERROR ! ! yes -----<------Is packet-type-code acceptable ? | ! no Cancel receive timer | | DR11W output data reg. <- packet-type-code-nak | | DR11W state <- idle ! ! -------------->------------ ----->---------------------- ! DR11W output data reg. <- packet-type-code-ack | | DR11W state <- waiting-for-word-count | --------------<------------ | CSR-template <- FNCT1 ! ! | | Exit Page 22 ----->-------------------------- ! ---<---------DR11W state = waiting-for-word-count ? | | yes word-count <- DR11W input data reg. | | ---------------------word-count < 0 ? | | yes Cancel receive timer | | Signal <- word-count | | DR11W state <- idle | | Inform program that a signal has been received | | DR11W output data reg. <- end-of-message code | | CSR-template <- FNCT1 ! ! | | Exit --->----------------------- ! Does the receiving program (for whatever reason) wish to cancel the message transmission ? ---<----------------------- ! | | yes Cancel receive timer | | DR11W state <- idle Page 23 | | DR11W output data reg. <- word-count-nak-code | | CSR-template <- FNCT1 ! ! | | Exit --->----------------------- ! DR11W output data reg. <- receivers-word-count | | DR11W address reg. <- lower 16 bits of buffer address | | DR11W word count reg. <- word-count | | Set upper two address bits into CSR-template. | | CSR-template <- FNCT1 + FNCT3 + GO | | DR11W state <- waiting-for-receive-completion | | ! ! Exit -------------------------------- ! --<----DR11W state = waiting-for-receive-completion ? | | yes DR11W state <- idle Page 24 | | Cancel receive timer | | Are any DR11W error bits set or does the DR11W word count reg. contain a nonzero value ? ---<----------------------- | | Notify program that a receive has been completed, but that there was an error -------------->--------- --->----------------------- ! Notify program that a receive has been successfully completed --------------<--------- | | DR11W output data register <- end-of-message status code ! ! CSR-template <- FNCT1 ! ! ! ! Exit ----->-------------------------- ! -----<--------------DR11W state = link-down ? | | yes DR11W input data reg. = start-protocol-code---> ERROR | | yes DR11W output data reg. <- start-acknowledge-code Page 25 | | DR11W state <- waiting-for-start-protocol | | CSR-template <- FNCT1 ! ! Start initialisation timer | | | | Exit ----->-------------------------- ! DR11W state = waiting-start-protocol ? --> ERROR | | yes ---<-----DR11W input data reg. = start-acknowledge-code ? | | yes DR11W state <- idle ! ---->--------------------------- ! ! Exit Page 26 Error condition _____ _________ or Transmit or Receive timer time out ________ __ _______ _____ ____ ___ | | Print an error message or notify some program that an error has occured | | If the transmit timer had been started, cancel it | | If the receive timer had been started, cancel it | | DR11W state <- link-down ! ! ---<--DR11W input data register = restart-link-protocol code ? ! | yes DR11W output data reg. <- start-acknowledge-code ! ! CSR-template <- FNCT1 ! ! DR11W state <- idle ! ! ! ! cancel initialisation timer ! ! Exit --->---------------------------- ! ! DR11W output data reg. <- end-of-message status + start protocol ! ! DR11W state <- waiting-to-start-protocol Page 27 ! ! CSR-template <- FNCT1 ! ! ! ! ! ! Exit DR11W as a link device - The Transmission Protocol Page 28 9.0 TRANSMISSION OF DATA AND INTERRUPTS This section describes how the DR11W registers are manipulated to transmit data, interrupt the partner and receive an interrupt. The CSR bits with which we are especially concerned are: Bit Name Function 0 GO Start transfer 1 FNCT1 Set for output data, clear for read data 2 FNCT2 Setting this bit causes ATTN bit in partner DR11W to be set, which causes interrupt if IE is set 3 FNCT3 Indicates mode of DMA. Clear for Burst mode, Set for normal block mode. 6 IE If set DR11W interrupt ability enabled. Many possible conditions cause interrupt. 8 CYCLE Initiates one NPR operation upon setting GO bit 9 STATUSA Set if FNCT3 bit of partner DR11W CSR is set 10 STATUSB Set if FNCT2 bit of partner DR11W CSR is set 11 STATUSC Set it FNCT1 bit of partner DR11W CSR is set 13 ATTN If set generates ERROR flag, which will cause DR11W to interrupt if IE bit set. 14 NEX If set indicates transfer to or from non- existent bus address; generates ERROR flag. 15 ERROR Inclusive OR of several other bits including ATTN,NEX. Causes DR11W interrupt if IE set For further details see Digital's Manual " DR11W Direct Memory Interface Module User's Guide". Most of the following information is available in some form in the Digital DR11W manual. It has however taken considerable time and effort to extract, combined with study of some existing DR11W code. Therefore the results of this study are summarised below. 9.1 Some Folklore 1. The CSR and EIR (Error and Information) Register share the same address. If the ERROR bit of the CSR is written then reading from that address will read out _______ the EIR, until the CSR is re-written with the ERROR bit clear. 2. If an ERROR condition has occurred and the ERROR bit is set, it can only be cleared by removing the condition which caused it to be set. Two of these possible conditions, ATTN or NEX can be cleared by clearing their respective bits in the CSR. Other conditions are cleared automatically at the start of the next DMA transfer. However the ATTN bit cannot be cleared if the partner has ______ DR11W as a link device - The Transmission Protocol Page 29 its FNCT2 bit set. It is for this reason that an interrupt of a partner DR11W is performed by flipping the FNCT2 bit on and off again. Failure to clear the ATTN bit indicates a hard-error. 3. Writing to the CSR/EIR address always writes to the CSR. 4. After writing to the output data register(ODR), the partner will receive the data simply by reading from his input data register (IDR) 5. A DMA transfer will not start until one DR11W (the transmitter in our case) sets both the GO and CYCLE bit. Setting the GO bit twice apparently causes an error, as does setting the CYCLE bit when the DR11W has not finished the previous cycle. 6. Clearing an interrupt, ready for the next one, is done by writing out the CSR with all but the function and IE bits clear. If a partner interrupts again, before the first interrupt has been handled and cleared, then the second interrupt will be lost. The Input data register should be read out, immediately before clearing the interrupt, and used to determine whether a second interrupt had in fact occurred. (e.g. a request link on-top-of an end-of-message status). This then ensures that the window where an interrupt may be lost without trace is only that between the instruction of reading out the IDR and resetting the CSR. 7. BIS and BIC Instructions must be used with the greatest care on the CSR. For if a partner DR11W has caused the ATTN bit to be set (and therefor the ERROR bit), a BIS or BIC will in doing a read/modify/write on the CSR, write the CSR _____ ERROR bit, thus causing subsequent BIS,BIC or reads to read the EIR instead of the CSR. 9.2 Example Routines Which Manipulate The CSR Bits The following two routines are examples of the routines used in the Flow charts of the previous section. 9.2.1 Clear Interrupt and Check Errors ; ; routine to reset the CSR after an interrupt, saving all ; relevant registers. ; MOV CSR,CSRSAV ;save CSR contents MOV CSRSAV,CSRTMP ; DR11W as a link device - The Transmission Protocol Page 30 BIS #ERROR,CSR ;write ERROR bit to read out EIR MOV EIR,EIRSAV ;save EIR contents ; ; error condition can be analysed, either hardware error, non ; existent memory,parity, power fail or just ATTN bit set by partner ; can see which from the bits of the CSR and EIR ; can look at this now or later ; - now clear error condition ; BIC #^C,CSRTMP ;clear all but ;function and IE bit MOV IDR,IDRSAV ;save input data register MOV CSRTMP,CSR ;reset CSR MOV CSR,CSRTMP ;read back CSR BIT #ERROR,CSRTMP ;check that error condition cleared BEQ 100$ ; ; else hard error, either try again to clear or signal error ; and disable interrupt on DR11W - error exit ; 100$: 9.2.2 Interrupt Partner DR11W ; ; on entry to this routine suitable FNCT1,FNCT3,,CYCLE and GO bits have ; been set up in CSRTMP for whatever operation is to be performed ; - a single word transfer ; - a DMA receive data set-up ; - a DMA transmit data ; BIS #IE,CSRTMP ;interrupts enabled throughout MOV CSRTMP,CSR2 ;save template CSR BIS #FNCT2,CSRTMP ;will set partners ATTN bit BIC #,CSRTMP ;dont want to start yet, just ;propagate function bits to partner MOV CSRTMP,CSR ;sets up FNCT bits and causes ;ATTN bit of partner to be set MOV CSR2,CSR ;now set GO and/or CYCLE bit if ;needed and remove FNCT2 bit RETURN