Description of Objectives and Activities of the Human Interface Technology Laboratory OVERVIEW: The Human Interface Technology Laboratory (HITL) in Seattle, Washington is pioneering development of virtual interface hardware and virtual environment software, the tools necessary to construct, inhabit, and interact with computer generated, three dimensional, inclusive environments. In conjunction with the Washington Technology Center, the University of Washington and HITL Industry Consortium members, we are building a laboratory infrastructure that will support the definition and creation of a completely new form of computer interface, hardware and software that is primarily responsive to natural human physiology and cognition, systems that emphasize spatial interaction rather than symbolic processing. GOALS: The objective of the Human Interface Technology Laboratory is to develop natural interface techniques, hardware and software designed for experiential rather than symbolic interaction. A virtual reality interface feels as though it were reality, permitting human- machine interaction that calls upon natural human responses, responses that we have been acquiring since birth. Virtual reality (VR) techniques are immediately relevant to computational tasks that model reality, including scientific visualization, computer-aided design and manufacturing, client presentation for architecture and interior design, computer-aided instruction, medical imaging and simulation, interaction with complex display panels and layouts (cockpit design, industrial monitoring, desktop publishing), terrain navigation and landscaping, traffic control, and computer games. In the long term, VR techniques hold the promise of innovative computational applications such as virtual conferencing, prosthetic interfaces, knowledge navigation, virtual sales and merchandising, and inclusive entertainment. The long term goal of HITL is to define a new generation of human- machine interface. This goal is supported by four major objectives: To investigate and understand the fundamentals of human perception and interaction with the world, with computational machines, and with information systems. To pioneer new interface concepts focusing on VR technologies. To create and demonstrate new application areas for VR in aerospace, medicine, education, design and entertainment. To transfer advanced interface technologies to the commercial sector. To achieve these objectives, HITL coordinates and facilitates the efforts of an excellent technical staff, University of Washington professors and graduate students, other VR labs around the world, and affliated professionals from a diversity of application domains. The HITL Industry Consortium provides an active link to corporations wishing to market VR technologies and assures cost effective, state-of-the-art information and technology transfer. The overall scope of our research objectives is embodied in HITL's three tiered functional organization: Infrastructure: Virtual Interface Knowledge Base and Library Virtual Simulation Laboratory Technologies: Low Cost Virtual Display Hardware Virtual Environment Operating System Software Applications: Virtual Prototyping Visualization Televirtuality Virtual Prostheses Infrastructure: The objective of the Virtual Interface Knowledge Base is to provide a world-class repository for experimental data, research findings and other information related to virtual interface technologies. Activities to achieve this objective include establishing a comprehensive literature collection, hosting the USENET newsgroup "sci.virtual-worlds" and preparing hardcopy newsletters on research developments in virtual reality. The Virtual Simulation Laboratory is intended to provide a rapid prototyping environment for the simulation of virtual interface concepts, software and hardware which emphasizes empirical research on human sensory, perceptual and psychomotor behavior in virtual environments. The simulation laboratory will include modules for electro-optical testing, complex control panel modeling, image generation, audio generation and recognition, virtual world design and integration, behavioral instrumentation and measurement, and human physics, neurophysiology and cognition. Technologies HITL will develope two primary technologies to support it's long-term objectives. The virtual display hardware focuses on head-coupled units, which feature visual and audio displays, voice and eye sensors and head movement tracking. The virtual environment operating system is the software substrate which mediates the interface between the symbolic computation that generates the virtual environment and the natural behavior of the patron within the virtual environment. Our objective for the head-mounted display is to produce a commercial prototype that is both high in performance and low in cost. We expect this unit to rival the monitor as the medium of visual (and auditory) display of computational processes in the next decade. Our objective for the virtual environment operating system is to provide a seamless environment which couples input behaviors of the patron to computational processes, manages the activity and modeling of the (parallel) computational processes, and integrates output signals from models and other sources to drive the virtual display devices. The virtual environment operating system incorporates three modules. The signal interpretor receives, validates and integrates sensor information generated by the patron, negotiating ambiguous and erroneous signals. The modeling module maintains and coordinates the representation, processes, and interaction between objects (model elements), managing memory, process allocation, and multiple patrons. We intend to develop a uniform internal architecture for models, combining object-oriented approaches with rulebased logic programming to create objects which act as situated agents. The display integrator integrates standardized output from models, error processes and model- independent hardware, managing viewpoint and perspective and integrating multiple sources of images. The virtual environment operating system provides a wide range of software tools for construction of and interaction with models, including editors of objects, spaces, and abstractions; movement and viewpoint control; object inhabitation; boundary integrity; display, resource and time management; multiple concurrent patrons; programmable internal processes within models; and history and statistics accumulation. Some potential user interface tools include the Wand, for identifying objects, connecting, moving, jacking, grasping, measuring, and drawing; and the Virtual Body for attaching arbitrary hardware sensing devices to arbitrary representations of body components, for collecting physiological measurements of behavior, and for maintaining coherence between a patron's model of physical activity and the virtual representation. Applications: Although VR techniques have an extremely broad range of potential applications, we are choosing to focus on four application areas during the first few years of the Lab's operation. The objectives of the Virtual Prototyping application are to provide tools for the rapid configuration of complex (virtual) machines and control panels; to provide functional connections between display and computational processes; and to enhance the coordination of design activities involving groups of professionals. The Visualization application is designed to provide tools for associating data with 3D visual models; to project, cluster and abstract data patterns; to interact with displayed data while automatically updating the underlying database; and to apply statistical and analytic techniques to displayed data. The Televirtuality application includes the design and development of virtual environments which support multiple interacting patrons; connection of multiple patrons and multiple computational resources over fiber optic networks; and the exploration of techniques for maintenance of inconsistent and incompatable virtual environments over multiple patrons. The objective of the Virtual Prostheses application is to develop aids for physically disabled persons that permit mapping of arbitrary movements onto functional virtual bodies. Two active projects are the integration of VR techniques with advanced wheelchair technology, and the control of textual display for dyslexia and reading disabilities. SUMMARY: VR is an infant field. HITL intends to contribute to communal knowledge to help resolve software, hardware, physiological and cognitive issues in VR. The domains we are addressing include design of multi-sensory display systems, electro-optics and microscanners, multi-sensor integration and synesthesia, VR neurophysiology, psychophysics and psychometrics, occlusive, overlayed and enhancing VRs, cross-validation of varieties of reality, physiological and cognitive design of task-oriented virtual environments, 3D representation techniques and interaction tools, editing techniques for objects, spaces, and abstractions, visual programming languages, virtual bodies and inhabitation techniques, integration of multiple patrons, design of situated and cooperative agents, autonomous objects and inconsistent environments, and form abstraction and the functionality of space. The Human Interface Technology Laboratory Washington Technology Center University of Washington, FU-20 Seattle WA 98195 206-543-5075 Dr. Thomas Furness, Director Dr. Bob Jacobson Dr. William Bricken