NORTHEAST
PARALLEL ARCHITECTURES CENTER AT SYRACUSE UNIVERSITY
Multipurpose,
Internet, Desktop Videoconferencing System
Introduction
In recent years many videoconferencing systems emerged on software market.
They offer different audio and video quality and require different hardware
capabilities. However none of them is moving toward enriching its functionality
beyond the audio-video data exchange and integration into multiplatform,
collaborative tool, enabling variety of applications in different domains.
Therefore we undertook some activities in order to create such a system
in Internet and Web environment. The conclusions presented in this article
are results of specific approach based on idea of creating independently
videoconferencing, real-time system and Web-oriented collaborative environment
which were integrated afterwards. In real-time applications we paid particular
attention to archiving capabilities including random access to stored conference
data. All the solutions were implemented and tested on both SGI Indy IRIX
and PC Windows 95/NT platforms.
BuenaVista Conferencing System
Overview
The goals of this project are:
-
create infrastructure for collaborative, distributed, real-time applications
(video, audio)
-
implement multiplatform, modular conferencing system for Internet,
compliant to existing standards (T.120, H.320)
-
create development environment for improving and adding collaborative
components
Initially such system was developed in NPAC based on InSoft's OpenDVE software
development kit. Since there have been restrictions and limitations for
OpenDVE-based conferencing system, we decided to develop our own conferencing
software infrastructure. Reasons for creation NPAC Conferencing System
(NCS), independent of OpenDVE:
-
full control over data transmission process - especially important
for real-time applications(video, audio)
-
ability to porting the system on different platforms
-
ability to create system compliant to conference standards(T.120, H.320)
-
possibility of implementing standard real-time data transportation
protocols(RTP, RSVP)
-
ability to integrate NCS seamlessly with other NPAC projects(e.g. Tango)
-
integrate the multimedia conferencing tools into the World Wide Web
environment
First, NCS was implemented on SGI Indy workstation. However, after satisfied
performance under IRIX 5.X operation system, NCS was ported successfully
on PC Windows NT/95 platform.
Conference management
A conference refers to a group of geographically dispersed nodes
that are joined together and that are capable of exchanging audiographic
and audiovisual information across various communication networks. Conference
participants may have access to various types of media handling capabilities
such as audio only (telephony), audio and data, audio and video, or audio,
video and data.
The conference in NCS is based on the notion of session. Session is a group
of users on different machines connected in virtual many-to-many channel.
So far there can only be one session on one machine an just one user on
the machine can participate in conference. Many applications can be attached
to the session.
Conference participants
Conference participant can start new session and invite others. Invitation
will be visible for remote user providing he has his Conference Manager
running. He can also enable the Conference Engine "watchdog"
function which automatically starts Conference Manager and displays incoming
invitation. Remote user can accept or reject the invitation. If he accepts
invitation he will become conference participant. From this moment all
his NCS application will collaborate within the current session. Each conference
participant is free to invite users to participate in conference. So far
here is no other way to join the conference.
Each participant has his own user identification number, unique within
the session. Every conference message includes this user id. There is always
one special conference participant - statekeeper. Although all participants
hold all the information about the conference it is necessary to name one
that provides information to the new participants and solve any state conflicts.
Usually it is conference initiator but in some situations (e.g. initiator
left) it may be another conference member.
Conference applications
Conference application is every application which is attached to Conference
Engine and collaborates within the session. Each participant can start
conference application. Applications can be started from Conference Manager
control panel or independently. If application is invoked from Conference
Manager panel it will be started automatically on all participant machines.
Also exiting the application from Conference Manager panel causes closing
all participant applications. If application is started independently Conference
Engine may reject the application trial to connect to the session.
Architecture
The main assumption during NCS development was its modularity. Modular
system structure enables easy and quick modification and improvement of
some of its parts without affecting the others. Based on core modules it
is possible to add new collaborative applications using existing NCS mechanisms.
It is especially important in Internet environment where system context
evolves dynamically.
NPAC Conferencing System consists of two main modules. First module is
the Conference Engine (CE) which contacts the remote hosts, processes control
messages from them and passes the indications of events to the Conference
Manager (CM). Conference Engine also processes the local messages from
Conference Manager. To create Conference Manager the set of API functions
is provided. There is also the API for the conference application management.
Figure 1 NCS Architecture
Conference Engine
Conference Engine is the core of NCS system. It performs the following
functions:
-
keeps the information about conference members and applications
-
receives the requests and sends the indications to Conference Manager
and conference applications
-
exchange the control messages with other conference members
CE sends and receives all the conference control messages, however application
data exchange is performed separately by the applications. It allows to
avoid bottleneck in CE which would be intermediate point for all NCS messages
in opposite case. It is particularly important for real-time applications
for which this system was designed. Therefore applications keep copies
of some necessary conference information (e.g. members addresses) and they
are indicated of any changes in that data by the CE. It is particularly
important for real-time applications for which this system was designed.
Conference Engines on different hosts communicate with the UDP socket connections.
Also communication with CM and local applications is done by the same type
of connection and even by the same socket. To accomplish this special kind
of control protocol is designed consisting information about message sender,
receiver and type.
NCS API
NCS API was created to facilitate development of NCS-based applications.
Conference Engine and API provides the backbone for conferencing system.
It allows flexible improvement of NCS, creation of variety new applications
and enriching its functionality. According to type of application that
is implemented there are two different kinds of API functions:
-
session API - for development of conference management applications
(Conference Manager)
-
application API - for development of ordinary conference applications
(Video Tool, Whiteboard)
API functions perform all the message exchanging and offer easy to use
callbacks mechanism to indicate of NCS events. They also enable acquiring
all kinds of conference information that may be useful for application.
Conference Manager
Conference Manager creates NCS user interface and interacts with user to
manage the conference. It uses API session functions to:
-
create and destroy session,
-
inviting conference members,
-
accepting or rejecting conference invitations.
Additionally it offers possibilities to starting and closing NCS application.
Information about system application are included in easily configurable
text file what ensure the required system flexibility.
NCS Server
Server is optional part of NCS - system works properly without it. However,
working with server is much more convenient, because server keeps track
of all users with NCS started. This important information allows to avoid
sending invitation to non-existing(from the NCS point of view) users. In
the future functionality of the server will be extended of capabilities
of giving other useful information about NCS users
Applications
Video Tool
Video Tool is an application implemented using NCS API over the backbone
of Conference Engine and Conference Manager. Video Tool has the capabilities
of:
-
capturing, compressing and sending video stream to all conference participants
-
receiving, decompressing and displaying video from other participants
Since it is the application designed for the Internet it adopts two very
efficient video compressing techniques:
-
H.261 - Standford University codec, improved and adapted for the needs
of videoconferencing
-
H.263 - Telenor's codec, improved and adapted for the needs of videoconferencing
by Janusz Bulawa
Both options provide very good picture quality. H.263 option takes less
bandwidth but is more CPU intensive, thus for slower machines H.261 algorithm
is recommended. Additional feature enables only INTRA frames compression
which speed up coding process. However it causes significant increase of
used bandwidth.
Video Tool automatically recognize received video format and uses appropriate
decompression method. All the video frames are captured and sent in QCIF
format, however it is possible to switch on the expand mode where QCIF
frames are extrapolated to CIF format. Application includes statistics
module which provides information about bitrate, framerate and other connection
parameters.
Audio Tool
Audio Tool is application created to enable voice communication between
conference participants. It was designed only for the speech signal, however
it is possible to transfer also low bandwidth music. Since application
was intend for Internet, high sound quality was sacrificed for the bandwidth
efficiency. Audio Tool adopts two compression options:
-
GSM 06.10 - used in digital cellular telephony,
very bandwidth efficient but designed just for the speech transmission.
-
Intel/DVI ADPCM - takes more bandwidth but offers much
better quality, good enough even for the music.
So far both option exchange 8000 samples per second sound, 16bits sound.
Audio stream is coded and sent in the 100 ms samples chunks. Audio formats
are automatically recognized and appropriate decompression method is chosen.
Whiteboard
Overview
Whiteboard is very convenient collaborative tool that allows to share drawing
among conference participants. It is possible to perform following operations:
-
drawing lines and basic geometrical shapes
-
choosing drawing colors
-
choosing line width
-
displaying text
-
saving and retrieving drawing from a file
Implementation
Each drawing element is stored in memory as an "graphic object".
Object consists of following informations:
-
object type
-
object position
-
additional location
Application creates and manages list of objects that can be also save and
retrieved from a file. Each object type has its own drawing procedure.
Objects are exchanged between applications using NCS mechanisms. Objects
received are handled the same way as obtain as a user input.
So far whiteboard was implemented only on SGI workstations based on Motif
library. However porting it on PC platform is predicted, as well as improving
application of clipboard capabilities.
Reliable vs. Non-reliable protocol option
For the conferencing system data exchange there are two most significant
factors affecting quality of service:
-
reliable transfer
-
messages delays
Obtaining both of them introduces trade-off situation-improving reliability
results in more sophisticated protocols with packets resending and data
acknowledgments what significantly increase messages delays.
Therefore our conferencing system offers both options. Each conference
application can use TCP protocol for reliable messages transfer or UDP
protocol for real-time messages, not requiring high reliability (e.g. audio
and video). It is also possible to use combined mode and transfer ordinary
messages with UDP protocol and priority messages with TCP. This approach
allows to overcome the obstacles mentioned in previous paragraph and keeping
the advantages of both options.
Additionally for non-reliable mode we built protocol covering UDP, offering
capabilities suitable for real-time applications. It includes:
-
packets time-stamping for received messages time management
-
packets numbering for handling lost and misplaced messages
-
session ID for identifying non-session messages
-
message type for handling only appropriate packets
Java API
Our conferencing system was written in C++. However the only constraint
for conference application is to meet protocol requirements. Hence it can
be created in any language supporting TCP/UDP network capabilities. A huge
advantage to application developer is existing API which enables creating
applications without getting into complicated protocol issues. Initially
such API was implemented only for C++ and C applications. Nevertheless
to prove that it is possible for any other language and to provide tool
to create such application in winning enormous popularity in recent years
Java we created API for this environment.
As an example of BuenaVista collaborative application written in Java we
successfully ported Tango chat application to our conferencing system.
Therefore we can say that our system is in a sense language independent.
Tango collaborative environment
Overview
TANGO is an integration platform which enables building Web-based collaborative
environments. The system provides the means of fast integration of Web-
and non-Web-applications into a multi-user collaborative environment. The
main functionality provided by the system consists of session management,
communication between collaborating applications, user authentication and
authorization and event logging. Our application of this Java/WWW-based
collaborative framework is focused on military command and control, Internet
distance education and remote collaboration.
Technology Used in this project:
-
Client-server architecture and seamlessly integrated into the World
Wide Web
-
Key server and client components are written in Java for multi-threaded
Java collaborative server management,
-
session control and a Web user interface.
-
Netscape Navigator 3.0 as client collaboration interface and working
environment
-
Netscape plugin as part of system componets on clients
-
Use of Oracle and JDBC for user authentication and authorization
-
Netscape LiveConnect for inter-applet communications
-
Simple APIs in Java applet, C and Java to port any application from
single-user mode into master-slave or
-
multi-user environment
-
Available on multiple platforms - SGI, SUN and PC
-
Current Tango applications include:
-
Line-based and image-based on-line chat
-
Whiteboard
-
Audio Conferencing
-
Collaborative Netscape Navigator
-
Collaborative Weather Viewer
-
Teaching demos from physics and math (Hopfield and Cross-Product)
-
Collaborative Visible Human
-
Multiple-Choice Exam Tool
-
Collaborative Web Search
-
Collaborative Tetris Game
-
WebWisdom (a Collaborative Presentation Framework)
Integration with Web-oriented Collaboration System
Both presented above systems have many unique features enabling collaboration
in Internet environment. However, none of them cover full scope of possible
collaboration aspects. Tango is Web-oriented system based on idea of connecting
applications through central server. This solution gives better control
over management and maintenance of collaboration process and enables multisession
mode. It also allows adding new applications easier, with less work overhead
(e.g. Java applets). Nevertheless, it does not support efficient enough
real-time applications. Central server that is big advantage concerning
mentioned above features, becomes bottleneck and decreases performance
of such applications, not meeting their needs. Web-independent, "out-of-browser"
system is also more reliable and more suitable for this specific requirements.
Taking into account all those features, we concluded that the best solution
would be integrated system consisted of both Tango and NCS. Certainly both
systems can work separately matching the particular needs of some users,
but combined hybrid is a powerful tool based on state-of-the-art technologies,
offering rich functionality necessary in modern desktop collaboration environment.
Obviously, reaching this goal is connected with variety of different problems.
Both systems are based on opposite approaches. Their architecture, communication
channels and protocols are totally distinct.
Implementation
To obtain seamless integration following activities were undertaken:
-
removing existing graphical user interface
-
agreeing message exchange protocol with Tango developers
-
installing Tango communication unit instead of GUI
Figure 2 (a) NCS-user interaction (b) NCS-Tango integration
The main problem of integration was linking multisession Tango with one-session
BuenaVista. It was overcome allowing just one BuenaVista session per machine
and creating simple user interface enabling BuenaVista applications management.
Archiving capabilities
Using videoconference system it would be very convenient to store sequences
of events occurred during the session. Then it could be retrieved, replayed
or searched, enriching the collaboration process. NCS includes archiving
capabilities enabling retrieving session events in real-time mode.
Extensive advantage of NCS archiving system is its application independence.
Internal system mechanisms provide the means for storing and retrieving
application data seamlessly for conference application developer. Therefore,
any new NCS application supports archiving functionality without any extra
effort from its developer.
Architecture
Figure 3Archiving System Structure
NCS archiving subsystem consists of following modules:
-
Recorder-intermediate point between NCS applications and file
storage. Recorder multiplexes messages from different applications adding
their unique identification numbers, inserts time stamp for every message
and store them to the file.
-
Retriever-performs similar functions in opposing direction.
After retrieving a message it schedules its sending time based on time
stamps, waits until scheduled moment and send data to appropriate application
using its identifier.
-
NCS application API-improved of sending messages to recorder
capabilities and switching on/off database mode upon NCS engine request
(sending messages to the database all the time would be wasteful and would
affect application performance)
-
NCS engine-enriched in receiving and handling Conference Manager
requests for database mode functionality. Handling those requests includes:
starting recorder or retriever, requesting database mode from applications,
transmitting priority aplication data to recorder.
-
NCS session API-enables switching between recording, retrieving
or normal working mode.
Recording and retrieving process is presented on figure 3, archiving capabilities
management is illustrated by figure 4.
Figure 4 Archiving capabilities management
Functionality
Archiving system functionality goes beyond simple storage and retrieving
data from disc. It also allows random access to session events, acquiring
information about session participants, date, discussed subjects. We predict
to organize conference files in a database enabling searching for particular
session data (topic, members, date), arranging data into groups etc..
Another possibility offered by archiving subsystem is "answering machine"
functionality. If for some reasons it is impossible to contact someone
immediately you can leave him audio-video message that can be retrieved
in appropriate time.
Conclusions
Conferencing system described in this article goes beyond functionality
offered in other systems created so far. It presents flexibility needed
for modern collaborative tool and abilities not just to exchange information
but also to manage them.
In the future we plan to add new applications, port it to new platforms
(Sun, Mac) and enrich its archiving capabilities to fully integrate it
with database system.
Written in April, 1997 by Tomasz
Stachowiak, stach@npac.syr.edu