PROPOSAL ABSTRACT submitted by NPAC at Syracuse University, Vanguard Research Inc., John Hopkins Medical School and Maryland University Computer Science Dept. Section I. Administrative (1) BAA number: BAA 97-09 (2) Technical topic areas: "Tools for Sharing Views and Prototyping and Evaluating Collaborative Applications" (3) Proposal title: “Development and Evaluation of Tools for Web Technology Based Collaboration and Visualization in Command and Control, Health Care and Training” (4) Technical point of contact: Prof. Geoffrey C. Fox Director Of the Northeast Parallel Architectures Center Professor of Computer Science and Physics 111 College Place Syracuse, New York 13244-4100, USA Tel.: 315-443-2163, Fax: 315-443-1973 http://www.npac.syr.edu Email: gcf@npac.syr.edu (5) Administrative point of contact: Matthew E. Clark Office of Sponsored Programs 113 Bowne Hall Syracuse NY 13244-4100 Tel: 315-443-9356, Fax: 315-443-9361 Email: clarkme@syr.edu (6) Summary: We will establish the critical features and performance needed in future Web based collaborative environments by assessing prototypes in three significant application areas – Command and Control, Healthcare and Education. These will be implemented in terms of an existing leading edge Web based collaborative system TANGOsim built as part of a Rome Laboratory project CIV. CIV (Collaborative Interaction and Visualization) was co- funded by DARPA. The base capabilities of TANGOsim are one to two years ahead of competitive systems and are being used in a significant C2 application demonstration. TANGOsim already has two key novel features. Firstly it is linked to a Web hosted Oracle database where it can log all transactions of the collaborative sessions and which can be used in asynchronous collaboration as well as our assessment task. Secondly it has generalized the typical session manager of collaborative environments to a full function event driven simulator. This allows TANGOsim to support collaborative linkage of heterogeneous sessions with different views of the same base object. Further it allows one to use both virtual (scripted) and real collaborators which greatly enhances both the assessment process and training for all applications. The project is led by NPAC at Syracuse University which designed and built TANGOsim with PI Geoffrey Fox; the C2 application is the responsibility of the Vanguard Research Inc.; Healthcare applications will be built by a team from John Hopkins Medical School led by Joel Saltz who has a joint appointment with the Computer Science department at Maryland which will be involved in the performance measurements, quality of service guarantees, and in the assessment tasks. The requested budget is $1100K per year for three years. Cost sharing includes use of education applications of TANGOsim being developed by NPAC to support distance education, which will be included in assessment task. Deliverables include enhancements to TANGOsim to support scalable large scale collaboration, multimedia annotation, support for users with varying degrees of Internet connectivity and access to platforms with varying computational capabilities, mobile users and immersive environments. Several different applications in the three areas will be developed in terms of TANGOsim and their use will be analyzed in terms of requirements for performance and functionality of the various multi-use services available in TANGOsim. (7) Contractor's type of business: "Other Educational" Section II. Detailed Proposal Information A. Innovative claims of the proposed research 1. We will exploit EXISTING leading edge Web based collaborative system TANGOsim to evaluate the functionality and performance needed for Intelligent Collaboration and Visualization in three distinct application areas Command and Control, Health Care and Distance Education. 2. We will enhance key features of TANGOsim including use of evolving open Web technology such as Java servers, built in scriptable event driven simulator, database indexed digital audio and video, automatic session logging into attached database, adaptable compression for images and video. 3. We will develop new capabilities to meet application requirements including scalable distributed servers, multimedia annotation, mobile users and immersive environments. 4. We will exploit the built in event driven simulation engine of TANGOsim to support custom filters for application message types which will allow adaptive generation of multiple views of a given object within the flexible master slave model of TANGOsim. 5. Develop several “real” applications in the three chosen areas with C2 applications coordinated with Rome Laboratory (for whom TANGOsim was developed for our first C2 example); Healthcare application prototypes tested at John Hopkins Medical School; and educational and training using TANGOsim used in Syracuse University continuing education and in local K-12 schools within existing ATM based Living SchoolBook project. 6. Integrate program and Internet monitoring tools into TANGOsim. Use integrated performance tools to assure TANGOsim users an acceptable level of interactivity. As resource contention increases, acceptable levels of performance are maintained by altering the services provided in ways that reduce resource requirements 7. Provide ongoing demonstrations of capabilities of base system and applications. B and D. Deliverables and description of the results 1. Use of leading edge Web Technologies to enhance a Java Collaborative framework TANGOsim to support multiple complex heterogeneous applications in both synchronous and asynchronous modes 2. Enhancement of current central server model to a distributed cooperating mode using most appropriate of emerging Java server technologies and JDBC distributed database 3. Allow Web or traditional(C++) applications to be conveniently integrated with TANGOsim and to interact with each other through customizable filters. Support tools to conveniently debug new applications in collaborative mode 4. Allow both real and virtual users of the system with a general Discrete Event Simulation capability integrated into TANGOsim control engine. Support pause and playback of session and real time modification of simulation script 5. Tools to support performance measurement in terms of measurement primitives (based on the Pablo SDDF from Reed’s group at Illinois) built into applications and logged by an attached Web Hosted database 6. TANGO will provide a level of service consistent with the users computational capabilities and network performance. This will allow high end users to make use of compute and communication intensive services and provide lower end users a highly responsive but less resource intensive service 7. Collaborative Services provided by TANGO will include 2D and 3D stereo Geographical Information Systems, digital audio images and video with adaptive codecs and database indexing, real time annotation and review of multi-media sessions, and session management, as well as traditional shared information resources such as white boards, web pages etc. 8. Immersive Environment based emerging web (VRML2) standards and optimized for high performance 9. C2 Applications involving over 10 active participants which can be real or virtual and 100’s of questioners 10. Healthcare applications including distributed conferences designed to allow experts to exchange information and to train residents; team health care with synchronous and asynchronous interaction. Tools to support summary of multimedia records for later expert consultants. 11. Educational applications including University and K-12 distance teaching of computer science and physics with up to 100 active students and low speed links addressed by local caching of course material 12. Tools for evaluation of collaborative sessions logged in database in terms of linked Java visualization applets and statistical analysis libraries. 13. Summarize results of applications in terms of architectures of TANGOsim, choices in its integration technology, performance and functionality of basic services (generic applications) and needed compute and network resource. F. Cost, schedule, and milestones Budget: Year 1 Year 2 Year 3 Task 1)Enhance and support $350K $250K $200K TANGOsim and associated services Task 2)C2 Applications $225K $225K $150K Task 3)Health Care $150K $150K $100K Task 4)Distance Education Application $0K $0K $0K (cost share by Universities) Task 5)Evaluation $100K $200K $325K Task 6) Performance Measurements $150K $150K $150K and Quality of Service Guarantees Task 7)Integration, Administration $125K $125K $175K and Demonstrations Total $1100K $1100K $1100K Task 1: Enhance and support TANGOsim and associated services Year 1: Evaluate TANGOsim in terms of application requirements and other collaborative systems. Initial Enhancements (multimedia annotation, assessment tools) Year 2: Continuing Enhancements (mobile users, scalable service) Year 3: Continuing Enhancements (immersive environments) Tasks 2-4)Applications Year 1: Find requirements and generate initial design of applications in each of 3 areas. Develop first prototype implementations. Year 2: Complete major applications to be used in assessment process Year 3: Continue to improve and extend applications Task 5: Evaluation and Performance Measurement Year 1: Design evaluation and performance measurement process and initial TANGOsim support tools Year 2: Initial evaluation of prototype applications – feed lessons into TANGOsim tools and evaluation process Year 3: Complete analysis of final applications with evaluation of lessons for collaborative systems Task 6: Performance Measurements and Quality of Service Guarantees Year 1: Begin incorporation of performance characterization tools (computer and Internet) into TANGO. Develop and implement TANGOsim QOS API to use input from performance tools to modify resource use of applets. Year 2: Continue incorporation of performance characterization tools into TANGO. Use TANGOsim QOS API with performance tools to demonstrate ability to design applets able to dynamically adjust their use of resources. Year 3: Demonstrate quality of service guarantees in context of targeted applications Task 7: Integration, Administration, and Demonstrations Years 1-3: Semi-annual major demonstrations of selected applications. Integrate applications in terms of common service needs. Reporting and coordination of subcontractors. G. Technical rationale; Technical approach and Implementation Plan This proposal exploits the TANGOsim system which combines collaboration and Discrete Event Simulation capability in an open modern Web Technology framework. TANGOsim has in prototype form many of the key capabilities called for in the first two years of the ICV DARPA program. Thus we see it as attractive to focus on a program of Prototyping and Evaluating applications which are built around TANGOsim which is enhanced in appropriate ways to support the application requirements and DARPA program goals. We recognize that TANGOsim should be viewed as an immature rapid prototype for surely Web technologies will continue to evolve rapidly and will certainly be quite different and far more powerful two years from now! We assume that other participants will pursue these points and aim our proposal at supporting ICV by feeding back early lessons from using the type of system envisioned by ICV in three significant application areas. This will help ICV in terms of performance and functionality tradeoffs and requirements which should be relevant to most Intelligent Collaboration and Visualization approaches. We have chosen three quite different application areas as collaboration takes different forms in different areas and one must understand similarities and differences so that as much as possible one can design the collaborative environment in terms of reusable but customizable services. We will describe our proposed effort roughly in terms of the seven tasks – basic TANGOsim and its enhancement, the three application areas, the evaluation process, network performance and Quality of Service issues and overall integration. TANGO is a basic Web collaborative system built around a custom Java Server and using an elegant Netscape LiveConnect approach to integration of applications within the Client and to the Server. TANGO is quite portable as it involves installation of a single plug-in which is mainly written in Java with a small machine specific C layer. TANGO itself is set up as an applet so that it can be dynamically downloaded to a general machine – currently Windows 95/NT and UNIX are supported. TANGO has a well defined API which allows general client applications to send and receive messages and so link their behavior in different instances on different clients. TANGO supports a simple security mechanism, general dynamic master-slave relationship and scalable multimedia support built on Netscape’s openDVE/LiveMedia standard with several different audio and video codecs. A key capability of TANGO is that it will support applications written in arbitrary languages including Java C and C++. This allowed us to build a reasonable performance 3D geographical information system (GIS) for the Rome CIV project by replacing an initial VRML version with a very much faster C++ (OpenInventor) system. This is collaboratized so that a 3D fly through on one machine is tracked on the other TANGO clients joining this session. TANGO has a relational database backend (currently Oracle but soon with JDBC, this can be a general system) which supports an archiving system for session replays. TANGOsim is based on the conceptually simple idea of replacing the session manager in TANGO by a general discrete event simulator. TANGO processes events represented as messages from the clients and generates the appropriate multicast to the collaborating members of session. TANGOsim has three essential additions: time is virtualized with in the TANGO limit, virtual time equaling real time; events can also be generated from a built in script ; and messages are all passed through filters which allow dynamic heterogeneous interactions between client applications. This enables a given session to support related but different views for each client which are optimized for the user’s needs and the available network and compute resource available. TANGOsim is currently built for a C2 application designed by Vanguard who are participating in this proposal. We have built several relevant support applications – including for C2 specifically, a weather applet as well as 2D and 3D GIS with multiple overlays allowed. These are augmented by generic services including chatboard, collaborative Web search, collaborative Web browser and multimedia mail. All these applications are collaboratized and can be scripted for interpretation by the simulation engine. We expect that both TANGOsim and other research and commercial systems will evolve over the next few months and will carefully evaluate TANGOsim both viewed as a basic Web collaboration system and in terms of requirements of the application areas. We will add multimedia annotation where we have limited experience in education (the course CPS615 taught by Fox this fall semester includes annotated audio clips recorded in real time). We will also add initial tools to support the assessment process where we build on HPCC performance measurement systems such as Illinois’s Pablo. Both of these enhancements will make use of the backend database where we can expect a typical tracking of the evolving Web standards with the Java Database Connection (JDBC) replacing our current Oracle WoW system. This application and base technology driven enhancement will continue but in the spirit of this proposal, we will reduce the level of effort in base TANGOsim in out years and focus more on applications and assessment. Enhancement areas we expect to need include support for mobile users, scalable service and immersive environments. The latter will build on our existing 3D GIS and track the several ongoing web experiments in this area – especially those in VRML 2 arena. Currently TANGOsim has a single central Java server and we can naturally exploit the Java Server efforts such as Jigsaw and Jeeves to replace this by a distributed set of collaborating servers. We will retain a sequential model for the event driven simulator as parallel versions of this have several very difficult issues. We have identified a special task(6) on performance measurement/monitoring and associated quality of service. These issues will be essential in using TANGOsim for all application areas and need special attention which will be led by the Maryland group. They have developed special methods of monitoring network performance which separate latency and bandwidth and recognize their intrinsic variability and are based on a study of 45 Web hosts accessed from four distinct sites. We will study application level adaptivity using special TANGOsim QOS API’s which are fed from the computer and network performance characterization tools which will developed from the Maryland research. Particularly interesting are the applications using images (2D and 3D GIS), audio and video where our applications are designed to be able to use general codecs. In ref. 6 of sec III, we show how the best wavelet compression level changes according to available client CPU and network bandwidth available. We will build into TANGO, the ability to aggressively cache information and also to migrate services (as in a digital server for instance) to improve performance. The C2 application will extend our work from Rome Laboratory and will focus on the mix of simulation and collaboration needed by C2 developers who wish to rapid prototype and evaluate the capability of the resultant system. Fox was a member of a recent DARPA sponsored National Council Study of Computing and Communication for Crisis Management and it would attractive to enhance our current application in the civilian area as many C2 problems actually mix both military and civilian authorities and there is a critical need for better software tools to be used by FEMA and other state and local teams. The Web technology approach of TANGOsim is very appropriate for crisis management as it is inherently broadly deployable and has flexible support spanning low end and high end devices. In health care, we intend at least two distinct applications. One will focus on a typical training scenario which will naturally use collaboration and simulation capabilities of TANGOsim. This involves experts, residents in training, observers and sets of test cases such as pathology images. The experts discuss cases and guide others with remarks aimed at clarifying the decision making process. In the second example, we consider team health care which will use involve both spontaneous collaboration and the asynchronous case represented by later expert consultants. We understand that the traditional view of Telemedicine as video conferencing with exchange of medical data is limited and adhere to Warner’s view of it as “Interventional Informatics” with a complex collaborative information rich environment typical of C2. Annotation will be very important in this application as it will allow initial participants such as the nurses and primary care physician to summarize sessions for later use by the specialists. Here we expect the TANGO backend database to be critical in allowing a flexible hierarchical view of a multimedia session We believe that collaboration systems such as TANGOsim will be critical in distant education. Up to now we have successfully used the Web in many educational projects including Syracuse’s use of the Internet to offer an International Certificate in Computational Science to Harbin in Northern China. This used Web technology (a replicated Web server) to disseminate information but just electronic mail for collaboration. On the other hand, K-12 students on our ATM network report collaboration (using simple non Web SGI and Apple tools) as the most exciting opportunity. TANGO and Habanero typify the opportunity to add Web based collaborative component to the rather well understood Web dissemination technologies in education. Naturally this collaboration involves teachers, students and parents(observers) in both synchronous and asynchronous modes. We currently intend experiments using TANGOsim at K-12, Undergraduate and graduate (continuing education) levels. There are clear needs for perhaps 20-100 fully interactive students and many more observers. The whole future of Universities and Schools could depend on the success or failures of experiments such as these! Thus it will be very important to understand the functionality and performance requirements for TANGOsim that will lead to good learning environments. Note DoD has many schools as well as major training needs for which these applications are directly relevant. However no funding is called for in this proposal except for the comparative assessment activity. The assessment process itself will need new tools and a good visual collaborative environment. We will log selected sessions fully in the database. Further basic message traffic and application use will generate event logs in the Pablo SDDF format. We already have a good set of Java applets to visualize such data which we built for performance monitoring of HPCC applications. We will extend these with analysis tools to summarize key performance bottlenecks. We will use standard assessment techniques with Tango requesting users to fill in forms after each session and suitably harassing ongoing users who do not regularly do this! We expect that an important deliverable from the contract will be a study as to successful approaches to evaluation of both the appropriateness of computer driven collaboration and the necessary services. For instance in education, does one need (digital talking heads) video conferencing or will good audio suffice? The final task includes general integration but also presentation of demonstrations of the technology on a regular basis. Our good linkage to NSF and DoD ATM networks should allow these to have broad participation. We will use the scripting capability of TANGOsim to prepare tutorials for all applications demonstrating the key services and how they were used. This will both aid the understanding of the general value of this technology but also of evaluation conclusions. H. Comparison with other ongoing research We believe that TANGOsim is competitive with or ahead of all comparable Web collaboration schemes. For instance comparing to the earlier Habanero from NCSA, TANGOsim is purely applet based and so downloadable from any client; has a built in discrete event simulator; comes with session logging to an attached web-linked database. In addition, TANGOsim already supports different, task specific views of the same basic infrastructure and has built-in mechanisms for secure access. TANGOsim also supports applications written in any language including C C++ and Java. The use of C++ allows us to package a fast 3D Geographical Information System enhancing basic VRML version. There are several non Web technology based collaborative systems which are more robust and functional than TANGOsim in certain capabilities. However we built TANGOsim out of open Web subsystems because only this will give the rich range of services required for future Intelligent Collaboration and Visualization environments. Our team has already built many Web technology applications in the chosen three areas and in particular TANGOsim was built to support a Web environment for rapid prototyping of C2 applications and already has been demonstrated with an interesting if not fully developed C2 application. Our team contains experts in the chosen areas – C2, education and Healthcare -- who are invited to the major conferences and maintain their leading edge position. I. List of key personnel and concise summary of their qualifications. 1. Geoffrey Charles Fox. Fox is Director of NPAC and Professor of Computer Science and Physics and an internationally recognized expert in the use of parallel architectures and the development of concurrent algorithms. He has published 300 papers and 3 books. He led a major project to develop first prototype high performance Fortran (Fortran90D) compilers with language independent runtime. He has always emphasized the role of applications in driving and validating technology. This is illustrated by his recent book "Parallel Computing Works" which describes the use of HPCC technologies in 50 significant application examples. Fox directs InfoMall, which is focused on accelerating the introduction of high speed communications and parallel computing into New York State industry and developing the corresponding software and systems industry. Much of this activity is centered on NYNET with ISDN and ATM connectivity throughout the state including schools where Fox is leading developments of new K-12 applications that exploit Web technology. With Rome Laboratory CIV project, NPAC has developed the Web based command and control application indicating how this COTS technology can be effectively used in DoD applications. 2. Joel Saltz Joel Saltz is Associate Professor of Computer Science at the University of Maryland, College Park, and Associate Professor of Pathology at Johns Hopkins Medical School. He is director of the High Performance Systems Software Laboratory, and of the Center for Computer Science in Medicine. Saltz leads a research group that is dedicated to developing applications driven systems software. He has published over 100 papers in a variety of areas in systems software including tools and compilers for distributed and parallel machines, databases, and the use of performance monitoring and program migration to optimize performance of distributed Internet based programs. His applications work includes the development of performance optimization and ontology generation methods to facilitate medical database interoperability. He is also developing Web based software to support multimedia computerized medical records along with software to support clinical conferencing. 3. Marek Podgorny Marek Podgorny is Research Director of NPAC at Syracuse University and teaches in Syracuse’s new Internet Systems Continuing Education program. He has established his research record as a theoretical and computational physicist with over 60 publications in this field. He joined NPAC in 1991 and led there a number of projects, including implementation of NYNET, one of the first operational wide-area ATM networks, parallel database evaluation and integration project, and video-on-demand project. Currently, Podgorny is a technical lead of the CIV project sponsored by Rome Laboratory. The CIV project developed TANGO and TANGOsim and now it applies this technology for telemedicine, C2, and education. In the video on demand project, Podgorny designed, developed and implemented a complete digital video storage, retrieval, and delivery system sup porting text-based indexing. Podgorny's fields of expertise include high performance and multimedia networking, relational and multimedia databases, digital video storage and transport systems, collaboratory systems, advanced Web technology, and large-scale system integration. J. Discussion of proposer’s previous accomplishments and work 1)NPAC brings expertise in general Web technologies and as exemplified by TANGOsim, novel approaches to collaborative environments. The latter work is funded through the “Collaborative Interaction and Visualization” project (CIV) of Rome Laboratory to which Vanguard is a subcontractor. Important related expertise and novel technology was also developed in a two year Video on Demand project for Rome which was just completed. NPAC has extensive expertise on ATM and other modern networking and was recently awarded a vBNS connection by NSF which will link to NYNET – an ATM network linking Rome, NPAC and Cornell as well as local Schools. DARPA’s Defense Sciences Office recently selected “The Grok-Box” for initial start up funding as part of BAA 96-24. This is based NPAC’s work with the “Institute of Interventional Informatics” on new Web based interfaces to maximize perceptualization for both disabled and unimpaired users. NPAC has focused on the development of new approaches to education and is funded by the State of New York, the University, and NSF in this area. We are currently studying the integration of TANGO into existing Web education resources where previously we had used conventional collaboration technology such as SGI’s Inperson. NPAC’s practical emphasis is illustrated by Carrier Corporation whose web site features their product line stored in an Oracle database and queried through the Web; NPAC developed this capability and hosts this site as part of its InfoMall State funded technology transfer activity. 2)John Hopkins has a world famous medical school and Joel Saltz has started there a significant Informatics activity. This is based on novel datamining ideas (Maryland has developed these as part of several projects with the Earth and Space Science part of NASA) on the distributed database formed by migrating existing systems to Web linked Oracle. Maryland also brings recent expertise in adaptive approaches to optimizing application level system performance by a combination of program migration, caching and carefully interpreted monitoring. Maryland also has a recent vBNS award from NSF. 3)Vanguard Research Inc.(VRI) is a small business in Fairfax, Virginia specializing in C3I and automated systems and technologies. Its major customers included USSTRATCOM for computer system development, Ballistic Missile Organization (BMDO) where VRI is the primary SETA support for BM/C3 and as SETA to the National Test Facility where VRI is intimately involved in the major Air Force initiatives in C3I, Open Systems and distributed simulation. VRI is the ongoing subcontractor to NPAC for the Rome Laboratory CIV project described above where VRI is responsible for the C2 application. Section III. Additional information -- References Subcontractor Addresses: Professor Joel Saltz: 1)Computer Science Department A.V. Williams Building University of Maryland College Park Maryland 20742 email: saltz@cs.umd.edu Phone: 301-405-2729 and 2)Department of Pathology The Johns Hopkins Medical Institutions 600 North Wolfe Street Baltimore, MD 21287 Phone: 410-614-5220 John F. Kantak Senior Vice President Vanguard Research Inc. 10400 Eaton Place Suite 450 Fairfax VA 22030 Email: jkantak@vriffx.com Phone 703-934-6300: Fax 703-273-9398 References: 1. Ranganathan, Anurag Acharya, Shamik Sharma, Joel Saltz, “Network Aware Mobile Programs”, to appear in the proceedings of USENIX'97. 2. K. Walczak, K. Olszewski, L. Beca, T. Jurga, G. Cheng, M. Podgorny, “TANGO - A Collaborative Environment for the World Wide Web”, unpublished (see http://www.npac.syr.edu/tango). 3. G.Fox, G.Cheng, M.Podgorny “Use of Web Collaboration Tools in Scientific Visualization and Computational Steering”, Invited Presentation at Minisymposium for 8th SIAM Conference on Parallel Processing for Scientific Computing. (Paper published in Procedings). 4. A summary of the many NPAC education resources and papers will be found at http://www.npac.syr.edu/users/gcf/sc96/ed.html. 5. http://www.npac.syr.edu/projects/cps615fall96 has some multimedia (audio) annotated lectures. 6. G. Lewandowski, R. Trzaska, M. Podgorny, G. Fox “Living SchoolBook Applications: Tradeoffs between Network Bandwidth and CPU Power”, http://www.npac.syr.edu/users/gcf/edperf.html. 7. Work on advanced perceptualization will be found at http://www.npac.syr.edu/projects/pulsar