NeatTools Interfacing

Rahul S. Panesar ^{∗}

Proposal for a 1996 NPAC REU Project^{†}

Abstract

The abstract comes here...

1 Sensory Interface

The intrinsic problem of the disabled patient is the limited number of usable channels of communication available to him. In effect, the total possible methods of inputting information and commands independently and willfully into a computer become limited to a few manageable motions. Hence, the need for the NeatTools program becomes apparent – the expansion and increase of input channels from a user of limited biological capabilities. NeatTools is a Java-based visual program that provides a graphic environment for processing data collected from various input devices that are connected to the user. Up to 16 data channels can be designated to various input and output mechanisms on NeatTools and such advantageous displays include the multichannel oscilloscopetype array which can be used to train the user to activate particular channels [1].

The input devices used with NeatTools are external devices that are modified specifically for the the user so as to employ whatever controlable movements he can make. Such input devices vary from commercial joysticks to the ”Thing” devices, which were developed for NeatTools application. The Thing 1 (TNG1) device, for example, is a 4 channel EMG signal detector, developed at the Institute for for Interventional Informatics [2]. TNG1 ulitizes the mechanisms of muscle contraction which are caused by the changes in the potential difference of the muscle tissues. These potential differences can be picked up by the EMG electrodes which are then connected to TNG1 and NeatTools. In turn, upto four different channels can be initiated by NeatTools to give a visual display of the muscle movements in the user. These channels can then be converted by the computer into commands for a set number of rules, a program, and/or communication via text and possible audio generation.

In the case of Eyal Sherman, the task of increasing the number of channels increases in difficulty not so much due to any limitations of the NeatTools program, but rather the limitations of Eyal himself. The workable area of the user is focused on the facial gestures which Eyal has sole control over. The method of exploiting the few biological channels that he has is centered then, to the available input devices that can be made around and on his facial muscles so as to obtain input data. Possible devices will be discussed in the next section.

∗Research Apprentice, 1996 NPAC REU Program, rahulp@npac.syr.edu; Chemistry major, State University of New York at Binghamton, bc71203@bingsuns.cc.binghamton.edu.

†The 1996 Research Experiences for Undergraduates (REU) Program in High-Performance Computing is conducted by the Northeast Parallel Architectures Center (NPAC) at Syracuse University. The National Science Foundation provides primary funding for the NPAC REU program, through NSF Grant CDA-9424332. Additional funding is provided by Syracuse University and NPAC.

Finally, upon determining the input devices that can be used and distinguishing the origins (i.e. muscle regions) of incoming signals, the resulting data can be used to define a computer rendering of the changes that the input data has created. This rendering then can be witnessed by the user in real time and can give an effective, responsive environment for the user to act in.

In the schematic approach, this subset of the project focuses in on the Phx (Life) to Sns (Sensor) to Khi (Computer) modules of the interactive cycle. Specifically, the input devices attached to the user relate the Phx to Sns interfaces. The software/hardware interfaces of the inpute devices and the NeatTools program relate to the Sns and Khi aspects of the cycle. Finally, the rendering of the data coincides with the Khi to Sky portions of the cycle.

2 Proposed Work

The aims of this aspect of the research is to determine the best possible environment for data relay from Eyal Sherman’s limited biological capabilites to coordinate with several input devices. Thereby increasing the number of input channels to the computer and hence, a greater number of different commands to the computer.

The initial investigation will be to construct a stationary frame, or mask, that Eyal’s head can be placed in. Within that, several input devices, either of the same nature or different technologies, can be placed on or near his face. Of the devices under consideration are the EMG electrodes and a pressure- sensitive relay that can detect motion of Eyal’s jaw (which he can motion willfully). This sensor can be placed into a chin strap mounted on the stationary head set. Finally, photo cell sensors can be implemented on or near the skin and can give signals based on the intensity of the light allowed to act on it that is not blocked by the skin. To do so would require a constant light source on each cell which could be accomplished by LED sources directed at each electrode. The selection of input devices will primarily depend on the reproducibility of performance with NeatTools and the feasibility of mounting it on the stationary frame.

The primary goal to setting up a system of input devices is to establish a model of fast, easy communication through the computer. One model under examination is using a pyramid-based model of the alphabet. Within the one large pyramid, there are four smaller triangles with another set of three smaller trianlges in each of the four larger triangles. In each set will be a group of letters and one funcitonal key. To get to a particular letter, the user can chose one of three directions and simplify the pyramid to the triangle pointing in the direction that the user has chosen. The user can continue to simplify the triangles until he has reached the letter he wants. This model is very efficient; three selections in any direction by the user can get him to the letter he wants. In the case of Eyal, the sensor frame will give these three signals so as to allow Eyal to select letters and form complete phrases more quickly than before.

The final stages of the project will be to convert the text to audio output via such hardware as SoundBlaster and to have the stationary frame devices allow Eyal not only to command text, but to move a cursor on the screen, effectively making him able to use the World Wide Web. In sync with the pyramid model, Eyal will be able to naviagate through WWW and command text into prompts when needed.

References