Subject: Interface: Try 2

From: "J. Larry Stephan" <[email-masked]>
Date: Thu, 17 Mar 94 20:03:55 PST

Hi, Jes: This is for your mailing list: a secont attempt. :-)

Here's my second cut at this.  It's still (pre-)alpha, or worse, but I think
it provides a better jumping-off place.  Thanks to Ronnie and Alan for their
comments and assistance.  I haven't followed all of the suggestions given
me, but I think I've got most of them, and good reasons when I didn't.  Can
anyone confirm that there are no mouse protocols that can't be mapped into
MSC? (Three buttons.)  My mouse data, on second reading, doesn't actually
come out and say that all mice use either MS or MSC protocol, and I'd like
to know, although it probably wouldn't change anything. Thanks to Alan for
raising the issue.

Ronnie has suggested a new type of client, which I'll call the direct/X
client.  This is a direct video client with the ability to read video
memory and translate it to X.  This would allow full emulation of all
video modes, but, of course, tie up the console.  If a hercules graphics
card is installed as a second video card, it could be used and the console
wouldn't even be tied up!  The idea is that an X server on another machine 
could take advantage of display hardware to emulate video modes which
would 
otherwise be very difficult to emulate.  A direct/Ncurses client is
feasible, too.

I've re-organized, presenting the flags and the names of other areas at the
front, and details of the mouse, video, and queue areas afterward, in order.
Notice that we are down to 2 areas, with fonts back in the video memory as
some suggested.  The second section, video memory, is now actually a series
of up to 32 areas of 32k each.  This breakdown allows video mapping to be
emulated for all video cards I have data on.

I've also added a short statement of purpose, to clarify why I'm doing all
this, anyway.  That, too, is wide open to comment.  Right now, its mostly
my ideas, heavily influenced by your comments.  (I've lost track of whose
ideas were whose.)
I have a new e-mail address, although they're having some telephone line
problems and the internet connection sometimes disappears (as in bounced
mail).  Its <[email-masked]>.  I'll also check at mail at
<[email-masked]> just in case you get tired of re-mailing.  The holonet
address is probably going to disappear when ssnet gets straightened out.

Have at it! ...And thanks!

------------------------------------------------------------------------------ 

                     DOSEMU Client/Server Interface

Purpose:  This interface definition is written in the hope that it will
provide a framework for further development of the Linux dos emulator.
It is intended to isolate client (user interface) and server (PC emulation)
functions in such a way as will allow both pieces to be developed and
enhanced independently.  It is also intended to be as flexible as possible,
and to allow room for future growth.  Finally, it attempts to keep hardware
and bios emulation issues confined as much as possible to the server, and
user interface issues confined as much as possible to the client.

This definition is permissive. At some future date, some parts will have to
become mandatory, however they will probably have changed by then.  Right
now, its just a direction for people to follow, in the hope it keeps us all
going in the same direction.  Note - if you do follow it, the documentation
is already (partly) written!

Interface:

  The dos emulator client and server will communicate initially via a unix
socket, and will maintain the socket for control purposes.  However, the
bulk of all communication will be via IPC shared memory, using flags to
control access as needed.  The shared memory definitions follow.

Shared Memory Definitions:

Overview:
    Two shared memory areas:
       Functions:

         1.   Client Server Communication area

              A)  General capability and queue data
                    Client/server capability negotiation
                    Device i/o data

              B)  Video capability area
                    Video capability negotiation
                    Video service requests
                    Video buffer data 
                    Palettes

              C)  Device queues
                    (size of area depends on queue size)

        2.  Video buffer
                    Up to 32 shared memory pages of 32k bytes
                    (Size depends on mode)

Client/server communication ares structure:

     Format of listing is:   
         length in bytes / function [instances if > 1] (comments)
        Definition starts with byte 0 of area

                4 / client capability flags    (see definitions below)   
                4 / server capability flags              "           
                4 / client request flags                 "
                4 / server request flags                 "
             
              108 / Mouse support data (see below)

             1208 / video data (see below)
                
              484 / queue descriptor data

               ?? / queues (one queue after the other)

-----------------------------------------------------------------------------

              Flag definitions:

                    In a capability word: 1 = "I can do this"
                    In a request word:    1 = "You do this" 
                                           or "I have done this"
             (Note:  a request is acknowleged by clearing the request bit.)

                    Bits: 0 - 9 = open/close queue 0 - 9
                             10 = switch video emulated/direct
                             11 = switch mouse video client/server
                        12 - 14 = unused
                             15 = reboot
                        16 - 25 = process data in queue 0 - 9
                             26 = process video info ( stuff in dosvidxxx)
                             27 = switch initialize/run
                        28 - 30 = unused
                             31 = exit 

  Note that is only necessary for one process to check the other's request
  word.  If that word is zero, nothing is requested.  If it is non-zero, 
  there is a one instruction inline assembly routine that will return
  the lowest bit number set, so it is easy to figure out what to do.
  There are also one instruction inline assembly routines to atomic-ly
  text and [set,clear] specified bits.

------------------------------------------------------------------------------
 Mouse support data

                4 / mouse int0x33 function enable (see below)
                2 / mouse data support flags      (see below)
                2 / mouse data change flags
                2 / mouse X sensitivity
                2 / mouse Y sensitivity
                2 / mouse X pixel position
                2 / mouse Y pixel position
                2 / mouse exclusion max X position
                2 / mouse exclusion max Y position
                2 / mouse exclusion min X position
                2 / mouse exclusion min Y position
                2 / mouse limit max X position
                2 / mouse limit max Y position
                2 / mouse limit min X position
                2 / mouse limit min Y position
                2 / character X size
                2 / character Y size
                2 / double speed threshold
                2 / mouse hide/show counter
                2 / mouse text AND mask
                2 / mouse text XOR mask
               32 / mouse graphics AND mask
               32 / mouse graphics XOR mask

  This interface supports three basic mouse modes. 
 
      1.  Server reads mouse port and server does mouse video.
           This mode simply lets the dos mouse driver do all the work.
 
      2.  Client reads mouse data and client does mouse video.
           This is the mode one would likely use in an X client,
           with the X server actually doing the display.  From
           dosemu's point of view, both the X client and the
           X server are part of the dosemu client.
 
      3.  Client reads mouse data and server displays mouse video.
           This mode is primarily for running dos programs that do
           strange things with the sprite.  The mouse may be difficult
           to use if the video update rate is slow.  However, in
           some dos programs, the mouse position reported by the mouse
           driver is ignored, and there is no other way to correctly
           show the sprite location.  Actually, this mode only requires
           support of the mouse show/hide function to work.  The dos 
           application does the rest.
 
   See the section "Mouse emulation" below for some hints on how to do
   the second mode.  Ronnie has suggested that the X client should ignore
   exclusion and limits, and force the mouse sensitivity so that the dos
   mouse sprite follows the X server sprite.  This is basically mode 2
   with some extra constraints, and I think he is correct.  However, we
   must still cope with programs that don't use the mouse driver's sprite.
   I think, in that case, the X cursor should be hidden and Mode 3 should be
   used, since we're going to see the dos application's idea of a mouse
   cursor anyway (we can't stop it if it doesn't use the mouse driver's 
   cursor), and a display of two cursors that may not track exactly would be
   distracting.
           
  Mouse int0x33 function enable:
 
  This word contains flags indicating which int0x33 requests should
  be passed to the mouse driver.  A zero indicates the function should
  be ignored without error.
 
              Bits  0 - 24      = functions 0x00 - 0x18
              Bits 25 - 28        (undefined)
              Bits 29 - 31      = functions 0x22 - 0x24
 
  Mouse data support flags
  
  These flags indicate which mouse data is supported by the server.
  A 1 indicates that the server supports the data. 
 
                    Bit    Data
                      0    Mouse sensitivity (x,y)
                      1    Mouse position (x,y)
                      2    Mouse exclusion (x,y)(min/max)
                      3    Mouse limit (x,y)(min,max)
                      4    Character size (x,y)
                      5    Double speed threshold
                      6    Show/hide counter
                      7    Text masks (and,xor)
                      8    Graphics masks (and,xor)
 
  Mouse data change flags
 
  Used to flag a change in mouse data. A 1 indicates a change.  The bit is
  set by the server and reset by the client.  The bits have the same
  significance as the mouse data support flags, above.
 
  Mouse emulation:
      Except, of course, for the case of direct console video, client 
    mouse video emulation may be required to give adequate visual
    response.  My (rather hastily assembled) suggestions are:
          
          1. For server mouse video, just pass stuff on through and perhaps
             speed up video refresh a little.  This is not going to be
             great, but it's the only way I know to deal with applications
             that don't let the mouse driver handle everything for them.
         
          2. For client video, server would do several things:
              
             A.  "Capture" the mouse software and hardware "interrupts" 
                 and preprocess requests, both to maintain the mouse data,
                 and to do the remaining steps.

             B.  Maintain the mouse driver's show/hide counter at -1, to
                 disable server mouse video.

             C.  Maintain the mouse driver's double speed threshold at
                 10,000 to disable double speed.  (Client would implement
                 double speed by replicating mouse motion data sent to
                 server.)

             D.  Switch to server emulation whenever an application "takes
                 over" mouse hardware interrupt, since mouse behavion then
                 becomes unpredictable.

             E.  Maintain mouse sensitivities by comparing motion sent to
                 server with change in pixel position.  If we do our own
                 custom mouse driver, or use one that has no built-in
                 sensitivity adjustment, we can maintain sensitivities
                 directly.
 
          3. For client video, client would do several things:

             A.  Move mouse pointer based on calculated sensitivity, adjusting
                 position to that sent from server whenever mouse queue is
                 empty.

             B.  Show and hide mouse pointer based on exclusion and show/hide
                 counter sent from server.

             C.  Constrain mouse pointer as requested by server.

             D.  Implement double speed by replicating mouse communication
                 blocks sent to server.

             E.  Allow mouse queue to drain and verify mouse position before
                 forwarding any mouse packets specifying button action.

             F.  Disable client mouse video on request of the server.

  Note that the mouse limits are implemented on both client and server. 
  This provides resynchronization of mouse position whenever a limit is
  reached. It's probably not needed, however.

-----------------------------------------------------------------------------

Video Data Area

   This area contains video mode information, registers, palettes, and
   memory layout.  The location of fonts is in the video buffer, where
   they really live.  There are really only three font storage formats.
   Note that video memory is divided into 32 pages of 32 kbytes each.  
   This is the  smallest page structure used by any of the common SVGA 
   cards.

   I have tried to accomodate nearly every mode including SVGA modes,
   however, 16 color graphics modes may be missing something.  I can't
   come up with a reasonable emulation model for 16 color graphics, and
   have opted to simply represent each of the four bit planes in memory,
   hoping that when someone figures out how to do the emulation, it will
   be enough. 

   In the listings below I have attempted to indicate the register(s) where
   the information needed to construct a data value is stored.  The data is
   NOT COMPLETE - I ran out of time, and will fill in more soon.  I have
   ignored the EGA/VGA overflow register in this list, since it is simply
   a set of extensions to other registers.  Numbers in braces {} indicate bit
   numbers, if only a part of a register is used.  Registers are indicated
   by a letter and number.  The letter is the device code, which maps to one
   or more I/O ports.  The number is a hexadecimal register number. 
Column 
   headings are video card types.  If no register is given, the value is
   not related to any ports on that video card.  The register  codes, 
   and corresponding I/O ports (in hex), are:

  Code  Name            -------------I/O PORT----------------------------
                          MDA     HGC     CGA     InColor  EGA     VGA

   c CRT controller       3b4/5   3b4/5   3d4/5   3b4/5    3d4/5   3d4/5
   a Attribute controller                                  3c0     3c0/1
   q sequencer                                             3c4/5   3c4/5
   g graphics controller                                   3ce/f   3ce/f
   s status register      3ba     3ba     3da     3ba      3da     3da
   h configuration switch         3bf             3bf
   m mode control         3b8     3b8     3d8     3b8      ( part of c )

  NOTE: EGA and VGA ports at 3d_ are at 3b_ in a monochrome setup. 

  For example: q3{0-5} means bits 0-5 of sequencer register 3, the font
     selection register.

  Shared memory data will be:

 Bytes  Name                  -----------REGISTER (hex)------------------
                                MDA    HGC    CGA    InColor EGA    VGA

   4 / client capability flags
   4 / server capability flags
   4 / client request flags
   4 / server request flags

   4 / register update flags
   4 / page update flags
   4 / page locks

   4 / physical page map
     1 / page at 0xa0000
     1 / page at 0xa8000
     1 / page at 0xb0000
     1 / page at 0xb8000      
  64 / pages shm ids
     2 / shmid [32]

   4 / screen size (pixels)     c 6    c 6    c 6    c 6     c12     c12
     2 / height                 c 6    c 6    c 6    c 6     c12     c12
     2 / width                  c 1    c 1    c 1    c 1     c 1     c 1
   4 / screen start
     2 / screen start           c c/d  c c/d  c c/d  c c/d   c c/d   c c/d
     1 / row start                                           c 8     c 8
     1 / pixel start                                         a13     a13
   2 / memory bytes/row
   2 / line compare
   4 / character data
     1 / character height       c 9    c 9    c 9    c 9     c 9     c 9
     1 / character width (bits)                 
     1 / char. font height   
     1 / char. width (pixels)   ?      ?      ?      ?       q1{0}   q1{0}
   4 / graphics data
     1 / interleave factor             m{1}          m{1}
     1 / pixel width (bits)     
     1 / number of bit planes   ?      ?      ?      ?       q1{0}   q1{0}
     1 / ? 
   4 / attribute masks
     1 / blink mask             m{5}   m{5}   m{5}   m{5}   a10{3}  a10{3}
     1 / color mask             m{5}   m{5}   m{5}   m{5}   a10{3}  a10{3}
              "                                             q3{0-3} q3{0-5}
     2 / character mask              c14{1-2}      c14{1-2} q3{0-3} q3{0-5}

    16 / font maps
       1 / font subpage [16]
     4 / bit plane maps
       1 / zeroth plane subpage (1 subpage = 4 kbytes = 0x1000 bytes)
       1 / first plane subpage
       1 / second plane subpage
       1 / third plane subpage
   4 / interleave maps 
       1 / zeroth row subpage
       1 / first row subpage
       1 / second row subpage  
       1 / third row subpage

  68 / 16-color palette
       4 / color [17]
         1 / unused
         3 / rgb levels
1024 / 256-color palette
       4 / color [256]
         1 / unused
         3 / rgb levels

Flag definitions:

   Request and capablilty flags:
    (I don't have these mapped out yet.  The mechanism is similar to the
     client/server communication flags at the start of the area, but
     bit assignments are undefined as yet. The character/graphics flag
     is in here somewhere.)

   Register Update flags:

     A change bit for pieces of data.  Not yet mapped out, but somehow
     matches up with the data defined above, so that change notification
     can be done quickly.

   Page update flags:

     These are set to 1 frequently by the server while a video page is 
     mapped into the server's 0xa0000 - 0xbffff address space, whether
     or not that is mapped to the real physical video memory.  Only pages
     mapped to that region are ever changed by a dos program.  These flags
     serve to notify the client that a page may have changed.  When the
     client updates its record of that page, it clears the flag.  This way,
     the server can determine if the client is finished with the page.

   Page locks:

     I don't know if it's needed, but these are flags which can be set
     atomic-ly by either a client or server to claim exclusive control of
     a video page.  May never get used, but its only 32 bits, so why not?

Physical page map:

     For security reasons, it is best for the client to do the initial
     mapping to physical video memory: only the client can be sure that
     proper housekeeping has been done on the vc.  However, once that is
     done, the server should take over control of mapping, since there are
     video i/o requests that do mapping.  These selectors are needed so
that
     the server can find out what the client has set up.

Shared memory ids:

     Since video memory is up to 32 shared 32k byte memory areas, the client
     and server need to know the same list of ids.  Since the shared memory
     is mapped into the server space at most 4 pages at a time, the client
     should probably acquire the pages, and the server just use them as the
     dos program requests.

Other data:
     I'll do more about this, or someone else can. I'm only including
     info now on a few that I think need explanation:

     Attribute masks:
       Various video cards use various attribute bits for color, underline,
       and blink.  Rather than try to model each one, I've settled for
       masks.  I'm undecided about underline, since there isn't really an
       "underline" bit on an MDA.  I'm thinking about putting an underline
       bit into the color palette instead.  Note the character mask.
       Various attribute bits may be used for selecting alternate fonts,
       depending on the video card emulated.  Again, I have settled for a
       mask.

     Font maps:
      
       Locate font in video memory.  EGA and VGA will use fonts 0 and 8,
       mapped according to the font select register.  HGC and InColor will
       use fonts 0 - 15, although only 0 - 11 are legal on the card.  I've
       put in the other four bit combinations (12-15) so they can be mapped
       to something legal, saving one validity check in processing.

     Bit Plane maps:
  
       If number of bit planes is more than 1, these pointers locate those
       planes in video ram.  Nothing I know of uses more than 4 planes.

     Interleave maps:

       Some video cards do a 2- or 4-way interleave of rows in some graphics 
       modes.  These point to the memory areas used for those rows.

     16-color palette:

       The real palette on a VGA card points into the 256 color palette.
       This can be emulated by the video bios emulation code.  Doing
       this as an actual palette simplifies other emulations and the client.
       My decision to use 4 bytes per color is arbitrary.  X uses 6 bytes,
       I think (I need help here) so that could be a better choice.  
       A 2-byte pad would be needed after the 17th (border) color.

    256-color palette:

       My decision to use 4 bytes per color is arbitrary.  X uses 6 bytes,
       (see above) so that could be a better choice.

Video mode changes:

       In the case of a mode change, server will fill in video area with
     data on new mode requested, set video info request flag, and sleep.
     Client will either:
     
          1.  Accept the new mode.  This means making whatever adjustments
              are needed in the supported display, then set the client
              video info request and capability flags, and send a signal to 
              wake up the server.  Seeing the client request and capability 
              flags set, the bios emulation in the server will do whatever it 
              needs and return to the requesting dos program.

          2.  Reject the new mode.  Clear the capability bit for the process
              video info flag, set the request flag, and wake up the server.
              seeing the request flag set and the capability flag cleared,
              the bios emulation in the server will (abort the calling
              program with a suitable screen message?).
          
     (Future options, if the client is on the console: )

          3.  Switch to direct console video.  This means the client will
              do the necessary housekeeping (save screen status, capture
              control of console switching, etc.), remap the server's
              video memory (can client do this?), clear the emulate video
              flag in the client capability word, then set the
              emulate/direct video request flag and wake up the server.
              The server will arrange ioperms to the video system,
              then either pass the mode change on to the dos video
              bios, or do the mode change to the video card via the bios
              emulation.  The bios emulation will continue to monitor
              for mode  changes, so the client can opt to restart emulation 
              if an acceptable mode is requested.  
              
              I'm reasonable certain the client can remap the server's video
              memory, although I don't know IPC very well.  If it can't
              be done, it may best to do both housekeeping and mmap-ing 
              in the server, for security reasons.  I do believe it's
              possible for the server to verify the client is really
              displaying on the console, via the /proc filesystem. I've even
              proposed a way for the server to verify the client
              has access to the physical video memory. I don't think there 
              is any way for server to verify that the client has done  
              housekeeping, however see the section on memory mapping 
              (below) for some ideas.  How much shall we trust the client?

          4.  If the client is already in direct console video mode, it may
              elect to set the emulate video capability bit, then set the
              emulate/direct request bit, thereby requesting the server to
              return to bios emulation and remap its video back to the
              shared memory area.

 Mapping video memory to physical video memory::

     This has been the subject of much discussion.  There are at least a few
     things that are clear:

        1.  Except for assigning physical (direct) video memory, all
            mapping must be under the control of the server, since that
            is where video bios calls that do mapping are processed.

        2.  Access to physical (direct) video memory must be authorized by
            the client, since the server has no way to determine that
            the video state has beed saved on the current vc.

        3.  A mechanism is required so the server can verify that the 
            client has arranged for direct video access, and which video 
            pages it has mapped directly, if any.

        4.  Server access to direct video i/o ports should only be arranged
            after verification that the client has arranged direct addess to
            video memory.

        5.  Either the client or the server can remap the shared video
            memory to physical video memory.

    I propose the following: (I assume mmap() works on shared memory pages)
        
        1. Upon startup, the direct video client will map shared video
           page 0 to the appropriate physical video memory (after required
           housekeeping, of course).

        or
           When an unsupported video mode is requested, the client may elect
           to switch to direct video (if it can gain control of a vc).

        2. The client will notify the server that direct video is mapped,
           and which page is mapped.

        3. The server will compare the first n bytes of the shared memory
           page with the address in /dev/kmem where video memory is.

        4. A bad compare means the client lied.  Do something appropriate.
           A good compare means the client may be honest.  Write a few random
           bytes to the start of video memory, verify that they now show up
           in /dev/kmem, then restore the memory as it was.  At this point,
           we have a trusted client.  Of course, if the bytes we write don't
           show up in /dev/kmem, the client lied again, and we should
           again do something appropriate, maybe like kill client and
           server processes?

        5. Once we have a trusted client, the server can call ioperm() and 
           arrange for direct access to video ports.  It can also take over
           video mapping (since access to the ports controls video mapping,
           this is automatic).

-----------------------------------------------------------------------------
Queue descriptor data:

                4 / number of queues  (10 is probably more than enough )
               12 / first queue descriptor    
                    4 / dos device name (4 char string)
                    1 / bit rate        (0,1,2... = 110,150,300...)
                    1 / IRQ priority    (maps to IRQ number)
                    2 / base port address
                    4 / pointer to queue  (queue 0)
                    
               12 /  additional queue descriptors [9]
               36 / queue header                 [10] 
                     2 / client -> server file descriptor
                     2 / server -> client file descriptor ( may be the same)  
                    16 / client -> server queue
                         4 / queue start
                         4 / queue end
                         4 / queue head
                         4 / queue tail
                    16 / server -> client queue
                         ( as above )
               36 / next queue header          [9]       (as above)

  I envision an initialization protocol wherein the client proposes a 
  queue size (by setting up the header and issuing queue request flags)
  and the server may accept or enlarge queue sizes at any time before
  signaling "initialization complete".  Once initialization is complete,
  queue sizes are fixed.  Of course, we can skip the negotiation at first
  and just use compiled in queue sizes.

  A lack of mouse support would be signaled by clearing the capability bit
  for keyboard/mouse (0).  Keyboard support would be required (big surprise!).

  Note that queue 0 is actually 2 unidirectional queues - one for keyboard
  and one for mouse.  All other queues are bi-directional.  There are two
  possibilities for handling the mouse, COM1 - COMn,  and LPT1- LPTn:  

         1.  Client requests device with queue empty, opens device, 
             and does I/O, passing data via queue.

         2.  Client places a device name in queue before issuing request.
             Server then opens device and processes data directly (still
             via queue), and resets capability flag disallowing use of
             queue by client.  If server can't open a device, it leaves
             that capability flag set, so that client is free to try another
             device for that queue.  
         
      Note:  Client opens and closes device by setting (=open) or clearing
             (=close) client capability bit for open/close and setting
             open/close request bit.
 
  This means that the server can support any of these devices either
  through a link to the client, or by directly accessing /dev/????, at
  the discretion of the client.