Presenting critical information to key personnel who are simultaneously enabled/empowered to respond effectively has long been a problem.

Inadequate capacity for comprehensively addressing the problem of connecting strategic personnel to emerging information and enabling their interaction with that information.

We propose a single, multi-modally integrated interface system for the perceptibility and expressablity of massive quantities of critical information. For the purposes of communication with a larger collective of connected individuals

2.0 Background Problem

The research, prototyping, development and demonstration of technologies to support gathering, interpretation and use of data

Funneling the greatest amount of high quality (prepared/processed) information to key personnel would be the ideal solution.

It is therefore critical that technologies be researched and demonstrated for the representation and interpretation of multiple, diverse data sources, for human interpretation of those data sources, and for powerful, deliberate/calculated and rapid interaction with that information.

Our offering, then, is a integrated interactive perceptualization interface system: a sensorial combinetric integrator for enhanced expressivity.

3.0 Scope

We propose to research and develop an interactive perceptualization system for the purposes of enhancing the dynamic interaction between the human user and a set of complex and diverse data types. This system will be designed to be linked to a large collective of other individuals

The end result of our work will be an interface imbued suit worn by personnel;

the suit itself is the "interactive perceptualization system."

and demonstrate capabilities in the following areas:

Integration and fusion of information from multi-modality human to computer input devices

Innovative mechanisms to support interactive informatic environments.

Development of computer to human multi-sensory data rendering and analysis.

Refinement of experimental protocols for optimal utilization of the system.

We will design and develop a highly advanced, novel and application rich interface system for interfacing with data from sensors information from databases other persons connected to the system

. Ability to transmit any information presented to the sensory technology back to expert personnel is primarily an issue of how intelligent in its reconfigurability, processing and rendering options the interface is.

We propose a system which takes advantage of the entire body's neural capacities for sensation, processing and yielding to consciousness information.

Our system's multisensory interfaces will thus make maximal use of the 'feature extraction' properties of the human senses.

That is, a neurological margin of maximally meaningful input from the outside can be rendered to each sensory system.

The system will accordingly map to this margin for each of the involved senses; system resident artificial intelligence will allow the reconfigurability based on context need. 3-D visual displays,

spatialized audiomorphic

and tactile body surface significations

and other methods will present information from the sensors to the expert.

EMG-like sensors across muscle surfaces, foot activated pressure sensors, voice recognition,

specially signified body movements will all serve as methods for communicating in real time

4.0 Requirements

The contractor shall develop project plans to provide insight into the various phases of the research.

The overall objective will be to develop one single ergonomic 'interactive perceptualization interface system' for simultaneously perceiving and expressing information. With in the context of a large collective of connected individuals and information systems

The following specifications of the system, then, will be heavily structured by the dual requirements for

1.perception: perceptually enhanced experience through generation of a pansensory rendering of relevant information so as to provide an enhanced capability to discriminate between classes of complex dynamic interactions of information; and

2.expression: resultant tactical actions taken based on that information. Ie communication to the system

4.1. Perceptual: having gathered raw data from remote sensors and/or micro-robots, processes which bring the information from "out there" into the mind are computer to human elements of the interface.

1.The contractor shall provide the systems, architecture, and design engineering required to research, prototype, and demonstrate advanced interfacing technology for the rendering of computer information onto multiple human sensory systems to achieve and demonstrate a sustained perceptual effect (i.e., a sensation with a context).

2.The contractor shall provide the systems, architecture, and design engineering required to research, prototype, and demonstrate how the implementation of vision, hearing, and touch technologies can allow for simultaneous sensation of multiple independent and dynamic data sets that can be integrated physiologically into a single perceptual state. Incidentally, from an evolutionary neuro-information processing perspective this technology creates a new potentiality for response to perceptual awareness: it canalizes not a single response to a single stimulus, but rather multiple responses to multiple stimuli born of a single though multi-dimensional sensorial perceptual state.

3.The contractor shall provide the systems, architecture, and design engineering required to research, prototype, and demonstrate the combination of these different rendering modalities with somatotopic placement, in order to achieve and demonstrate spatial coding of the rendered information.

4.The contractor shall provide the systems, architecture, and design engineering required to research, prototype, and demonstrate enhancements to the interface system to match the human nervous system's ability to transduce, transmit, and render to consciousness the necessary information to interact intelligently with information.

Below is some of the thinking which grounds this (i.e., 4.1.4.) our fundamental operational design principle:

Research in human sensory physiology, specifically sensory transduction mechanisms, shows us that there are designs in our nervous systems optimized for feature extraction of spatially rendered data, temporally rendered data, and textures. We will develop these interface techniques and technologies consistent with the basic neuroscience issues of modality, duration, intensity, distribution, frequency, spatial displacement, contrast, inhibition, threshold, adaptation, transduction, conductance and transmission (to name a few)

We can then identify the optimal perceptual state space parameters in which information can best be rendered.

That is, what types of information are best rendered to each specific sense modality, a sense specific optimization of rendered information.

Computational Preparation of Rendered Data: after merely rendering then there is the honing to the nervous system phase (i.e., how to best render for the highest quality information).

4.1.5 The contractor shall provide the systems, architecture, and design engineering required to research, prototype, and demonstrate the implementation of a data analysis subsystem designed to enhance the ways that relevant data may then be rendered optimally to the operators sensory modalities, utilizing such techniques as linear and nonlinear multivariate analysis tools for the processing of multiple data sets in a variety of ways, including graphical analysis (phase portraits, compressed arrays, recurrence maps, etc.) and sound editing (mixing, filtering ).

Possible Sensory Input:

visualization techniques. 3-D sound and graphics. sonoification of data: in these spatialized environments we can shift our attentional focus from source to source for real time comparison of multiple sets of data. Devices now exist which we will exploit so we can stimulate the sensation of pressure, vibration, texture and temperature.

4.2. Expressional: having cognized the information gathered, processes which bring the deliberate response of the human out to the world are human to computer elements of the interface

4.2.1. The contractor shall provide the systems, architecture, and design engineering required to research, prototype, and demonstrate the integration of a set of advanced human to computer input devices into a single interface system. This integration of data input devices into a single system will provide for devices such as, EEG, EMG, EOG, (bioelectric signals from brain, muscle and eye), dynamic bend sensors, pressure sensors, audio and video digitizers, and other devices.

Notes:

Physiologically Oriented Interface Design: the next paradigm of human-computer interface will optimize the technology to the physiology -- a biologically responsive interactive interface. paradigm of interface technology is based on new theories of human-computer interaction which are physiologically and cognitively oriented.

4.2.2. The contractor shall provide the systems, architecture, and design engineering required to research, prototype, and demonstrate data fusion to enable meaningful correlations across various input modalities.

Abstracting function for the sake of machine resident intelligence and user characterization and quantitative/qualitative recording.

4.2.3. The contractor shall provide the Systems, architecture, and design engineering required to research, prototype, and demonstrate the automated detection of trends and correlations, using techniques, such as fuzzy logic in the background or in a post processing mode, and support the demonstration of user alerts indicating detection of areas worthy of further investigation.

5.0. Interactive Environments

In short, the integrated perceptual/expressional suit is the interactive interface environment. It is essentially a "wearable computer." We believe that the optimal interface for the expert personnel in the field implementing the capabilities of remote sensors and actuators is a fully "wired" body suit.

complete with visual, auditory input and motion output as well as controller fitted gloves and boots

This is a primary 'enabling technology' critical to the success of militarily operations in the field.

The suit is itself a perceptual and expressional environment; information gathering and sending respectively. Militarily relevant information capability strives for the greatest volume, most diverse types and the most quickly ascertained information.

Viewing the entire body as a perceptual and expressional technology opens up possibilities for exploiting the heretofore untapped richness and greater volumetric potential of its informatic capacities. Hence, we propose to develop an interactive environment incorporating new ways to render complex information to the user by optimizing the interface system to match the human nervous system’s ability to transduce, transmit, and render to consciousness the necessary information. Such a system will be based on the human user’s neural information processing that directly supports perception. A perceptualization environment such as we are proposing optimizes the human’s ability to discriminate and iteratively refine emergent patterns from any variety of sensor data, proactively.

The perceptual interface systems of the suit will be designed so as to optimize the salience and content of data sets. Some data which conventionally would be displayed visually might be processed so as to be perceived (in the suit) in a tactile or auditory manner

feel the sound, see the pressure and ultimately be able to reconfigure the rendering parameters of the interface based on the specific elements of a situation. Seeing colors may be more appropriate in one context whereas hearing them may be more suitable for another; many factors will determine the tailoring of rendered data: which data will be shunted to which renderer? Novel interface controllers are essential here.

Our success will have been to provide a completely integrated perceptual and expressional system which becomes the sense and agency of the expert