| David E. Bernholdt*, Geoffrey C. Fox
Roman Markowski, Nancy J. McCracken Marek Podgorny, Thomas R. Scavo Northeast Parallel Architectures Center 111 College Place Syracuse, NY 13244 http://www.npac.syr.edu/ |
| Qutaibah Malluhi, Debasis Mitra
Department of Computer Science Jackson State University P.O. Box 18839 Jackson, MS 39217 http://dragon.jsums.edu/ |
| * Author for Correspondence: bernhold@npac.syr.edu |
Many academic courses at Syracuse University and elsewhere use technology to enhance the learning experience. For example, commercial textbooks often include CD-ROMs complete with databases, online resources, utilities, and hot links to related material on the World Wide Web. Mailing lists and bulletin board systems (such as USENET) are commonly used to make timely course-related announcements and as a medium to facilitate asynchronous group discussions and administer technical forums. Course material is often published on web servers, which students are encouraged to access at their leisure (that is, asynchronously).
There appears to be no shortage of such course material on the web (especially in the technical fields), although the quality of such material varies greatly. What is clearly missing, however, is a vehicle to deliver educational material synchronously, that is, in real time, over the Internet using web-based technologies. Although some "distance learning" courses use web chat tools to communicate synchronously, there appears to be few (if any) systems that deliver real-time multimedia content in an authentic, two-way interactive format.
The computational science education group at the Northeast Parallel
Architectures Center (NPAC) has developed a huge repository of online course
material, including lectures, tutorials, and programming examples in various
languages. We believe, however, that a significant majority of students require
regular and sustained interaction (i.e., synchronous learning activities)
involving teachers and other learners, in addition to asynchronous learning
materials. Hence, our interest in the TANGO Interactive System, which was used
to deliver
The first offering of Jackson State University (JSU) course
The goal of
http://www.npac.syr.edu/projects/cps606spring98/for a recent offering of
Students in
http://www.npac.syr.edu/projects/jsuspring98/
students link to resources covering HTML, CGI (including Perl), and Java programming. These pages contain links to lectures, tutorials, documentation, and numerous examples.
The course also included online mentoring by instructors at Syracuse with help from two JSU supervisors who monitored student progress. Students were assigned weekly homework assignments, which they submitted via web pages. Grades were checked using a password protected, online grading system.
The initial class meeting was a traditional face-to-face meeting of
instructors and students. During this meeting, instructors reviewed the syllabus
and students were introduced to the TANGO software. (Little or no time was spent
on web browser technology since almost all students were familiar with this.)
Students were asked to connect to an online database application and complete a
sign-up form (
Besides the initial face-to-face meeting, two other such meetings were held during the semester: one shortly after we began the unit on Java (about two-fifths of the way through the course) and another during final week. The second meeting helped ease the transition from one course topic (Perl) to another (Java) and also served to strengthen the personal relationships between instructors and students. The final face-to-face meeting was the final class meeting of the semester. During this class, students presented their final projects, which gave closure to the course.
At the beginning of the term, a majordomo mailing list was created. The mailing list facilitated asynchronous discussion of course-related topics such as assignments and programming details not covered during lecture. In addition to the mailing list, a "Post Office" page was created:
http://www.npac.syr.edu/projects/jsuspring98/postoffice.html
The Post Office is a more visual, browser-based approach to e-mail (see Figure 1). It is generated by script from information obtained from the online sign-up form.
Students were encouraged to use e-mail as their primary communication medium. This was largely successful, but we found that students felt more comfortable using e-mail for individual conversations with instructors, rather than group e-mail discussions (via the mailing list). Even though the instructors used the mailing list regularly, the students did not (and could not be persuaded to do so). The reasons for this are not clear.
Example programs are an integral part of learning any programming language. These examples must be carefully designed to illustrate the essential features of the language without introducing unnecessary detail. To facilitate the showing of examples, a dual-panel presentation window was designed in which the code appears in the bottom panel and the output of the code simultaneously appears in the top panel. See, for example,
http://www.npac.syr.edu/projects/tutorials/Java/examples1.1/AWT/or
http://www.npac.syr.edu/projects/tutorials/Java/examples1.1/HelloWorld/
which utilizes a three-panel display. Screen shots of these programming
examples are shown in Figure 2 and Figure 3, respectively. (Note: If your
browser has trouble displaying the above examples, it is probably because it
does not support
Notice that the code is line numbered, which greatly simplifies online explanation of the example program. The annotated programs also include embedded hot links (shown in blue) to related files such as supporting documentation. These links are used by the student when browsing the example asynchronously or by the teacher to facilitate online presentation of the example.
The course home page is a record of both student and course progress. Lectures, for example, are added to the lecture page on a weekly basis. The lecture foils also have optional "addons", that is, links to related examples and background material. If a student misses a lecture, he or she can consult the lecture page for missed material. This is especially important in a distance-learning course where access to instructors is limited.
Assignments are also posted on the course home page. Any student with web access can find out immediately what assignments have been given and which are coming due. Moreover, each assignment includes links to background material that the assignment presupposes.
As a student completes his or her assignments, links are added to their individual homework page. (The creation of this homework page is the first assignment a student receives in the course.) On or after the assignment due date, a TA or instructor reads each student's homework page and grades the assignment. The grades are entered into a secure online database, which a student accesses with a browser. The backend of this online database system is an Oracle database server. The frontend is a custom web application with many security and administrative features. This database system has been used successfully in several courses taught by NPAC, both on and off-campus.
From the student's point of view, the database provides an immediate answer to the perennial question: "What is my grade in this course?" From an administrative point of view, the database centralizes student records and automates certain routine tasks. For example, an instructor may easily produce summary statistics at mid-term or at course's end. Student tracking is thus simplified, and because the system is web-based, the information is readily available to instructors at both SU and JSU.
TANGO Interactive is a Java-based web collaboratory developed at NPAC. It is implemented with standard Internet technologies and protocols, and runs inside an ordinary Netscape browser window. Although TANGO was originally designed to support collaborative workgroups, in this project it was used to synchronously deliver course materials stored in an otherwise asynchronous repository.
The primary TANGO window is called the control application (CA). From the CA participants have access to many tools including:
At the beginning of each class, the instructor and the student all start up a copy of the CA (see Figure 4) on their workstation by clicking a browser link. The instructor initiates a "session" of each tool that is to be used in class and "joins" each student to the session. This starts up a window on the student's workstation where they can see each action of the instructor with that tool.
WebWisdom is a presentation tool for showing lecture slides or foils (see Figure 5.) The system includes tools that convert a source document prepared with PowerPoint or Persuasion into WebWisdom format. The WebWisdom database consists of over 400 foilsets and 17,000 foils. A foil may have an "addon", which is a link (or links) to supporting material such as online documentation or example programs.
The SharedBrowser is used to "push" learning material onto client screens. It is similar to an ordinary browser window: there is a textfield for typing URLs, a "Back" button, and a history list. When the "master" (that is, the user doing the "pushing") loads a web page into the SharedBrowser, that web page is automatically and simultaneously displayed in all client browsers. Clients may activate links, scroll the window, or otherwise interact with the page as usual, but as the master SharedBrowser loads a new page, that page is automatically loaded into all client browsers. In this way, an instructor may show examples or a student may demonstrate a project.
One of the first things we noticed when working with TANGO for teaching purposes was that the instructor often had the urge to "point" at portions of slides or lines of code. (Evidently, this is an important, almost unconscious mechanism for conveying information in a traditional classroom.) Sophisticated pointing mechanisms, such as tracking the instructor's mouse on student workstations, were not available while these courses were being taught. In the short term, however, WebWisdom was equipped with a "highlighter" that the instructor used to emphasize bulleted material on a foil. For the same reason, sample programs have been line numbered (automatically, by a Perl script developed specifically for this purpose), which facilitates verbal descriptions of the code. (Note: a new version of WebWisdom has an instructor pointer arrow.)
The WhiteBoard is a handy tool for drawing small amounts of information on-the-fly, say, a code fragment or a simple diagram. This WhiteBoard was full duplex, that is, any number of clients may write on it simultaneously. Although such a tool is useful for collaborative work, in the instructional setting, we would have preferred for only the instructor to be able to draw on the whiteboard. The Tango architecture allows both sorts of control mechanisms to be implemented. The version that we used was fairly simple; a new version of the WhiteBoard has recently been released and is currently being evaluated.
TANGO's Chat tool was an indispensable part of the synchronous learning process. At least one chat window (See Figure 8) was on every participant's desktop at all times. Note that in Tango, it is possible to open multiple sessions of each tool, so that we could have one chat session for the instructor and students to discuss class content, and another one for the instructor and support staff to discuss real-time technical issues.
BuenaVista (BV) is a multi-platform, audio/video conferencing system developed at NPAC.
BuenaVista requires limited but consistent bandwidth. Detailed network requirements will be outlined in the following section.
Audio conferencing capabilities are crucial for teachers to give lectures and explanations, while students need audio to ask questions and engage in problem solving activities. To facilitate the latter, a RaiseHand tool was devised, whereby students could signal their desire to ask a question.
In an effort to make lectures more interactive, microphones were installed on each PC in the lab at JSU, which we had hoped would allow students to easily ask questions. We immediately discovered, however, that this setup did not work since feedback from the loudspeaker system made lecturing impossible. Thus the microphones were disconnected and audio was consolidated on a single workstation.
Finally, we should mention that it is easy to join remote clients to a TANGO session. The master of any given tool can force-join any or all TANGO users. We were worried that this feature might be abused, but we didn't have any problems with it. Indeed, we found it to be one of TANGO's premiere session-management features.
Of the TANGO tools discussed in the previous section, all but BuenaVista require minimal bandwidth. Audio and video, on the other hand, involve real-time, two-way interactivity, which make them sensitive to network delay and jitter.
BuenaVista uses standard codecs, namely, H.263 and H.261 for video, and GSM (Global System for Mobile communication), PCM (Pulse Code Modulation), and ADPCM (Adaptive Differential PCM) for audio. These have been optimized to ensure the highest possible transfer rates. For LAN applications, BV also supports high bandwidth, high quality video formats. Minimum bandwidth and delay requirements for these protocols are given below:
In general, to provide effective multimedia content delivery, the network must possess the following operational characteristics at acceptable levels:
In our experience, the three most significant factors affecting quality of
service are:
In any event, to minimize network performance problems and better utilize available bandwidth, servers were installed in both Syracuse and Jackson. Specifically, we
The speakerphone was used in case of an Internet brownout or a system crash. It is difficult to conduct a class over the telephone, but in a pinch, it's better than nothing!
Instead of sending the full text of a web document, TANGO would (in essence) transmit a simple URL to participants in Jackson. This URL pointed to an identical document on the JSU server. This dual-server architecture (See Figure 9) had minimal bandwidth requirements and was extremely efficient. However, the technical costs to administer such a setup were considerable, so we are investigating the possibility of replacing the second server with a CD-ROM, which will put all course materials directly on each student's desktop.
Early in the fall course, we relied on an ordinary Internet connection between SU and JSU. This connection was provided by NYSERNet, Sprint, ICP, and BBNplanet (a GTE affiliate). Typical performance over this connection was:
It was not uncommon to experience 6% packet loss (one NYSERNet router, in
particular---the line between 144.228.20.110 and 169.130.1.93---was consistently
losing packets). On any given class day, network performance could be quite
different in both directions. As an extreme example, on Thursday,
The network was gradually improved during the fall semester by:
Typical performance over the Internet/DREN connection provided by NYSERNet, Sprint, ICP, and DREN was:
with 10% packet loss mainly on ICP routers. This problem was due to the configuration of router 192.157.69.55/138.18.253.1, which (according to NYSERNet) is under DREN control. Packets from NPAC to JSU travel along one path (which is fast and shows no packet loss) while packets from JSU arrive at NPAC along another path (which is slow with high packet loss). The solution was to redirect packets (using static routing) in such a way that the same path was traveled in both directions.
Initial experiments with DREN were not at all promising. After switching from
an ordinary Internet connection to DREN, the performance was dreadful (
During the spring semester the situation changed again. Packets generally
traveled from SU to JSU along a path with speed
In summary,
Overall, the courses were successful, that is, almost all of the students submitted their homework assignments, participated in e-mail discussions, completed a final project, and received a passing grade in the course. There were some difficulties, however, which caused the online lecturing process to be less successful than an actual classroom lecture course.
First of all, our experience indicates that current Internet bandwidth does
not always support real-time video. Although smart compression algorithms can
sometimes compensate for poor Internet connections, we found that these were not
enough. Audio, on the other hand, worked relatively well. In approximately five
of the
During the fall semester,
Some people felt that the TANGO user interface, where each tool appears in a separate window, is too cumbersome for instructional use. This was not seen as a problem by the JSU students themselves, who were quite good at manipulating multiple windows, but we still feel that an integrated design may be more attractive for instructional use.
One of the obvious differences between traditional classroom lecturing and distance lecturing is the interaction between the instructor and the students in the classroom. One of the aspects of this is the "bonding" process whereby the personal connection between the instructor and the students encourages the students to work on the course. Since we did not have video throughout much of the year, the instructor(s) would made personal visits to the class at three times during the semester. The students felt that this was very important in to the quality of the class. When we did have video of the instructor transmitted to the students, they felt that that was a great improvement over just the audio. On the other hand, when a picture of the students in the lab was transmitted to the instructor during the class, it was only a marginal help to the instructor. The instructor could still not see enough detail of the students' expressions and body language to receive that instantaneous feedback about whether students are understanding the lectures.
In our lab setup, the students primarily used chat instead of audio to ask questions. The students did regularly type questions into the chat window and participate in class. However, there was again the drawback of a delay between asking the question and receiving the answer, which was sometimes distracting.
The distance setting also requires the instructor to make an adjustment of lecturing style. The most significant thing is to make short, direct sentences stating the points of the lecture. A slightly slower speaking rate is also helpful in allowing the audio to be clear. In particular, the audience felt that it was confusing if the instructor made digressions on a related topic. This is a bit of a dilemma, since it is often the digression or interjection of a humorous story, etc., that can liven up an otherwise dull topic in the classroom, and these techniques apparently don't work as well at a distance.
Having one or more instructors on the receiving end of a distance-learning course is a definite plus. Indeed, the JSU instructors:
In addition to the instructors, the JSU lab technician was important to the overall success of the course. In particular, at the beginning of the course when the software was being installed and the network was being configured, the technician's role was indispensable. Once the system stabilized, the focus switched to the course content and the JSU instructor's role became more important.
In fact, we discovered that a stable networking environment to support distance learning via TANGO can be achieved in a couple of weeks. (In time, we may be able to cut this in half.) Once stabilized, this environment should not be altered unless absolutely necessary since a lengthy disruption in the middle of the semester is undesirable, especially in a distance-learning course where contact between instructor and student is limited.
At the end of the fall semester, the students were given a short
questionnaire, which included questions on the perceived value of the various
instructional components used in the course. (The results of this questionnaire
are summarized in
We suspected that these comments were a reflection of the network
difficulties experienced early in the semester. To test this hypothesis, a
follow-up questionnaire was administered to the spring section of
A new
online course evaluation was written and administered at the end of the
spring semester in
Jackson State University course
We learned much by this experience, but there are still many unanswered questions. For instance, during our site visits we observed students quickly losing interest in remote lectures delivered via one-way audio links. Why is this? Is it due to the relatively passive nature of the lectures? If so, improved two-way audio and video should increase the perceived quality of the remote lectures. It will be interesting to see if this is the case.
Also, we may be underestimating the importance of visual cues and feedback
mechanisms inherent in the learning process. The lack of eye contact between
instructor and student may be hampering the overall learning experience.
Unfortunately, given the network connection available to us in the fall semester
1997, there was little we could do to test this hypothesis. During the spring
semester, however, the bandwidth between NPAC and JSU was typically in the
neighborhood of
In general, we must develop criteria for critically evaluating the inherent differences between traditional classroom lectures and remote lectures broadcast over a network. An understanding of these differences will drive the development process. In particular, we need a reliable estimate of the cost/benefit ratio associated with real-time video.
As a direct result of our experiences, TANGO is currently undergoing rapid development. The following enhancements to TANGO are being considered at this time:
Other improvements to the overall delivery system have been proposed:
Recall that SU instructors made three face-to-face visits to JSU during a semester. Although these visits were very helpful, they were of course expensive. Our hope is that improvements in the technology (video, in particular) will alleviate the need for numerous on-site visits.
It remains an important open question whether or not current Internet bandwidth will support two-way audio and video to the extent that students and teachers feel as comfortable with remote lectures as they do with traditional classroom lectures.
This work was funded (in part) by the DoD High Performance Computing Modernization Program at the U.S. Army Corps of Engineers Waterways Experiment Station (CEWES) Major Shared Resource Center through Programming Environment and Training (PET) through contract(s) DAHC94-96-C-002 with Nichols Research Corporation.
The major technology Tango Interactive was initially developed with funding from Rome Laboratory. Curriculum development was largely funded by the College of Engineering and Computer Science at Syracuse University.