NORTHEAST PARALLEL ARCHITECTURES CENTER AT SYRACUSE UNIVERSITY 
Multipurpose, Internet, Desktop Videoconferencing System  

Introduction

In recent years many videoconferencing systems emerged on software market. They offer different audio and video quality and require different hardware capabilities. However none of them is moving toward enriching its functionality beyond the audio-video data exchange and integration into multiplatform, collaborative tool, enabling variety of applications in different domains. 

Therefore we undertook some activities in order to create such a system in Internet and Web environment. The conclusions presented in this article are results of specific approach based on idea of creating independently videoconferencing, real-time system and Web-oriented collaborative environment which were integrated afterwards. In real-time applications we paid particular attention to archiving capabilities including random access to stored conference data. All the solutions were implemented and tested on both SGI Indy IRIX and PC Windows 95/NT platforms. 

BuenaVista Conferencing System

Overview

The goals of this project are:  Initially such system was developed in NPAC based on InSoft's OpenDVE software development kit. Since there have been restrictions and limitations for OpenDVE-based conferencing system, we decided to develop our own conferencing software infrastructure. Reasons for creation NPAC Conferencing System (NCS), independent of OpenDVE:  First, NCS was implemented on SGI Indy workstation. However, after satisfied performance under IRIX 5.X operation system, NCS was ported successfully on PC Windows NT/95 platform. 

Conference management

A conference refers to a group of geographically dispersed nodes that are joined together and that are capable of exchanging audiographic and audiovisual information across various communication networks. Conference participants may have access to various types of media handling capabilities such as audio only (telephony), audio and data, audio and video, or audio, video and data. 

The conference in NCS is based on the notion of session. Session is a group of users on different machines connected in virtual many-to-many channel. So far there can only be one session on one machine an just one user on the machine can participate in conference. Many applications can be attached to the session. 

Conference participants

Conference participant can start new session and invite others. Invitation will be visible for remote user providing he has his Conference Manager running. He can also enable the Conference Engine "watchdog" function which automatically starts Conference Manager and displays incoming invitation. Remote user can accept or reject the invitation. If he accepts invitation he will become conference participant. From this moment all his NCS application will collaborate within the current session. Each conference participant is free to invite users to participate in conference. So far here is no other way to join the conference. 

Each participant has his own user identification number, unique within the session. Every conference message includes this user id. There is always one special conference participant - statekeeper. Although all participants hold all the information about the conference it is necessary to name one that provides information to the new participants and solve any state conflicts. Usually it is conference initiator but in some situations (e.g. initiator left) it may be another conference member. 

Conference applications

Conference application is every application which is attached to Conference Engine and collaborates within the session. Each participant can start conference application. Applications can be started from Conference Manager control panel or independently. If application is invoked from Conference Manager panel it will be started automatically on all participant machines. Also exiting the application from Conference Manager panel causes closing all participant applications. If application is started independently Conference Engine may reject the application trial to connect to the session. 

Architecture

The main assumption during NCS development was its modularity. Modular system structure enables easy and quick modification and improvement of some of its parts without affecting the others. Based on core modules it is possible to add new collaborative applications using existing NCS mechanisms. It is especially important in Internet environment where system context evolves dynamically. 

NPAC Conferencing System consists of two main modules. First module is the Conference Engine (CE) which contacts the remote hosts, processes control messages from them and passes the indications of events to the Conference Manager (CM). Conference Engine also processes the local messages from Conference Manager. To create Conference Manager the set of API functions is provided. There is also the API for the conference application management. 

 
Figure 1 NCS Architecture

Conference Engine

Conference Engine is the core of NCS system. It performs the following functions:  CE sends and receives all the conference control messages, however application data exchange is performed separately by the applications. It allows to avoid bottleneck in CE which would be intermediate point for all NCS messages in opposite case. It is particularly important for real-time applications for which this system was designed. Therefore applications keep copies of some necessary conference information (e.g. members addresses) and they are indicated of any changes in that data by the CE. It is particularly important for real-time applications for which this system was designed. 

Conference Engines on different hosts communicate with the UDP socket connections. Also communication with CM and local applications is done by the same type of connection and even by the same socket. To accomplish this special kind of control protocol is designed consisting information about message sender, receiver and type. 

NCS API

NCS API was created to facilitate development of NCS-based applications. Conference Engine and API provides the backbone for conferencing system. It allows flexible improvement of NCS, creation of variety new applications and enriching its functionality. According to type of application that is implemented there are two different kinds of API functions:  API functions perform all the message exchanging and offer easy to use callbacks mechanism to indicate of NCS events. They also enable acquiring all kinds of conference information that may be useful for application. 

Conference Manager

Conference Manager creates NCS user interface and interacts with user to manage the conference. It uses API session functions to:  Additionally it offers possibilities to starting and closing NCS application. Information about system application are included in easily configurable text file what ensure the required system flexibility. 

NCS Server

Server is optional part of NCS - system works properly without it. However, working with server is much more convenient, because server keeps track of all users with NCS started. This important information allows to avoid sending invitation to non-existing(from the NCS point of view) users. In the future functionality of the server will be extended of capabilities of giving other useful information about NCS users 

Applications

Video Tool 

Video Tool is an application implemented using NCS API over the backbone of Conference Engine and Conference Manager. Video Tool has the capabilities of:  Since it is the application designed for the Internet it adopts two very efficient video compressing techniques:  Both options provide very good picture quality. H.263 option takes less bandwidth but is more CPU intensive, thus for slower machines H.261 algorithm is recommended. Additional feature enables only INTRA frames compression which speed up coding process. However it causes significant increase of used bandwidth. 

Video Tool automatically recognize received video format and uses appropriate decompression method. All the video frames are captured and sent in QCIF format, however it is possible to switch on the expand mode where QCIF frames are extrapolated to CIF format. Application includes statistics module which provides information about bitrate, framerate and other connection parameters. 

Audio Tool 

Audio Tool is application created to enable voice communication between conference participants. It was designed only for the speech signal, however it is possible to transfer also low bandwidth music. Since application was intend for Internet, high sound quality was sacrificed for the bandwidth efficiency. Audio Tool adopts two compression options:  So far both option exchange 8000 samples per second sound, 16bits sound. Audio stream is coded and sent in the 100 ms samples chunks. Audio formats are automatically recognized and appropriate decompression method is chosen. 

Whiteboard

Overview

Whiteboard is very convenient collaborative tool that allows to share drawing among conference participants. It is possible to perform following operations: 

Implementation

Each drawing element is stored in memory as an "graphic object". Object consists of following informations:  Application creates and manages list of objects that can be also save and retrieved from a file. Each object type has its own drawing procedure. Objects are exchanged between applications using NCS mechanisms. Objects received are handled the same way as obtain as a user input. 

So far whiteboard was implemented only on SGI workstations based on Motif library. However porting it on PC platform is predicted, as well as improving application of clipboard capabilities. 

Reliable vs. Non-reliable protocol option

For the conferencing system data exchange there are two most significant factors affecting quality of service:  Obtaining both of them introduces trade-off situation-improving reliability results in more sophisticated protocols with packets resending and data acknowledgments what significantly increase messages delays. 

Therefore our conferencing system offers both options. Each conference application can use TCP protocol for reliable messages transfer or UDP protocol for real-time messages, not requiring high reliability (e.g. audio and video). It is also possible to use combined mode and transfer ordinary messages with UDP protocol and priority messages with TCP. This approach allows to overcome the obstacles mentioned in previous paragraph and keeping the advantages of both options. 

Additionally for non-reliable mode we built protocol covering UDP, offering capabilities suitable for real-time applications. It includes: 

Java API

Our conferencing system was written in C++. However the only constraint for conference application is to meet protocol requirements. Hence it can be created in any language supporting TCP/UDP network capabilities. A huge advantage to application developer is existing API which enables creating applications without getting into complicated protocol issues. Initially such API was implemented only for C++ and C applications. Nevertheless to prove that it is possible for any other language and to provide tool to create such application in winning enormous popularity in recent years Java we created API for this environment. 

As an example of BuenaVista collaborative application written in Java we successfully ported Tango chat application to our conferencing system. Therefore we can say that our system is in a sense language independent. 

Tango collaborative environment

Overview

TANGO is an integration platform which enables building Web-based collaborative environments. The system provides the means of fast integration of Web- and non-Web-applications into a multi-user collaborative environment. The main functionality provided by the system consists of session management, communication between collaborating applications, user authentication and authorization and event logging. Our application of this Java/WWW-based collaborative framework is focused on military command and control, Internet distance education and remote collaboration. 

Technology Used in this project:

Integration with Web-oriented Collaboration System 

Both presented above systems have many unique features enabling collaboration in Internet environment. However, none of them cover full scope of possible collaboration aspects. Tango is Web-oriented system based on idea of connecting applications through central server. This solution gives better control over management and maintenance of collaboration process and enables multisession mode. It also allows adding new applications easier, with less work overhead (e.g. Java applets). Nevertheless, it does not support efficient enough real-time applications. Central server that is big advantage concerning mentioned above features, becomes bottleneck and decreases performance of such applications, not meeting their needs. Web-independent, "out-of-browser" system is also more reliable and more suitable for this specific requirements. 

Taking into account all those features, we concluded that the best solution would be integrated system consisted of both Tango and NCS. Certainly both systems can work separately matching the particular needs of some users, but combined hybrid is a powerful tool based on state-of-the-art technologies, offering rich functionality necessary in modern desktop collaboration environment. 

Obviously, reaching this goal is connected with variety of different problems. Both systems are based on opposite approaches. Their architecture, communication channels and protocols are totally distinct. 

Implementation

To obtain seamless integration following activities were undertaken: 
Figure 2 (a) NCS-user interaction (b) NCS-Tango integration

The main problem of integration was linking multisession Tango with one-session BuenaVista. It was overcome allowing just one BuenaVista session per machine and creating simple user interface enabling BuenaVista applications management. 

Archiving capabilities

Using videoconference system it would be very convenient to store sequences of events occurred during the session. Then it could be retrieved, replayed or searched, enriching the collaboration process. NCS includes archiving capabilities enabling retrieving session events in real-time mode. 

Extensive advantage of NCS archiving system is its application independence. Internal system mechanisms provide the means for storing and retrieving application data seamlessly for conference application developer. Therefore, any new NCS application supports archiving functionality without any extra effort from its developer. 

Architecture

Figure 3Archiving System Structure

NCS archiving subsystem consists of following modules:  Recording and retrieving process is presented on figure 3, archiving capabilities management is illustrated by figure 4. 

Figure 4 Archiving capabilities management 

Functionality

Archiving system functionality goes beyond simple storage and retrieving data from disc. It also allows random access to session events, acquiring information about session participants, date, discussed subjects. We predict to organize conference files in a database enabling searching for particular session data (topic, members, date), arranging data into groups etc.. 

Another possibility offered by archiving subsystem is "answering machine" functionality. If for some reasons it is impossible to contact someone immediately you can leave him audio-video message that can be retrieved in appropriate time. 

Conclusions

Conferencing system described in this article goes beyond functionality offered in other systems created so far. It presents flexibility needed for modern collaborative tool and abilities not just to exchange information but also to manage them. 

In the future we plan to add new applications, port it to new platforms (Sun, Mac) and enrich its archiving capabilities to fully integrate it with database system. 


Written in April, 1997 by Tomasz Stachowiak, stach@npac.syr.edu