\setcounter{equation}{0}
\setcounter{figure}{0}

\chapter{Case Studies of Integrating World Wide Web and  
Distributed Computing}
\markright{APPENDIX D. CASE STUDIES IN CHAPTER 5}

This appendix details three application case studies used in
Chapter 5 to demonstrate the integration of distributed WWW
and centralized high-performance parallel relational
database servers in a client-server framework.
The three case studies are chosen from the email-based WWW collaboration system,
full-text information
retrieval, and financial modeling. For
each application we describe a prototype system that illustrates
system configuration, software integration,
performance, functionality, and other issues. Some background information
about parallel Oracle 7 RDBMS software and the IBM SP2 are also given 
to help a reader understand the 
approaches we adopted.

\section{Parallel Oracle 7 RDBMS Servers on IBM SP2}
Oracle 7 on an IBM SP2 includes these components:
\begin{enumerate}
\item
An IBM RS/6000 SP system, including multiple processor nodes, a 
high-performance switch (HPS), and a control workstation
\item
 IBM Virtual Shared Disks (VSDs) with data striping (HSD)
\item
Oracle Unix Distributed Lock Manager (DLM)
\item Oracle Parallel Server
\end{enumerate}

Figure \ref{oracle-sp2} shows the relationships among
Oracle instances, VSDs and the DLM on an IBM SP2 system.

\begin{figure}
\centerline{\epsfxsize=450pt \epsfbox[60 290 550 740]{appendix/oracle1.eps}}
%%BoundingBox: 53 40 560 753
\caption{Parallel Oracle7 Server Architecture on an IBM SP2}
\label{oracle-sp2}
\end{figure}

\subsection{The IBM SP2}
The IBM SP2 is a distributed memory MIMD system that is scalable from
4 to 512 processor nodes. Each node is a RISC/6000 processor with
at least 64MB of memory, 2GB of direct access storage devices (DASD),
and an Ethernet adapter for connecting to a RISC/6000 control workstation.

The high-performance switch (HPS) provides a message-passing network
connecting the processor nodes. It is used for message passing among
nodes by the Virtual Shared Disk system, the Distributed Lock Manage,
and the Oracle parallel query option. The HPS supports the TCP/IP network
protocol.

An external LAN connects the SP2 nodes to a computer network, making
them accessible from any network workstation over the LAN or WAN.

\subsection{Virtual Shared Disks}
A VSD gives each node access to disks attached to remote nodes as if
there were local disks. All database files, redo log files, and
control files of the Oracle parallel server must be stored
on VSDs. Oracle instances on different nodes use
VSDs to share access to disks transparently.

Since a VSD allows multiple nodes simultaneous access to devices,
the Oracle7 parallel server must provide its own distributed synchronization
mechanisms to insure data integrity. This is done by the
Distributed Lock Manager.

The data striping device HSD is a device driver layer above
the VSD layer. It lets Oracle7 server stripe data across
physical disks in multiple VSDs to reduce I/O bottlenecks
and hot spots. With data striping, one I/O request reads or writes
blocks of data on each of several separate VSDs. This automatically
distributes or partitions data across physical disks and notes, which
improves performance of the underlying disks drives.

\subsection{Oracle Unix Distributed Lock Manager (DLM)}

The Oracle Parallel Server on the SP2 makes use of a distributed
lock manager (DLM) to coordinate access to data and other resources
by processes running on different SP2 nodes. The DLM synchronizes
the sharing of resources such as data blocks, and ensures the 
consistency of modified data, effectively providing
uninterrupted, transparent, and reliable data sharing for Oracle
instances.

The DLM maintains data consistency by
limiting disk access with distributed
locks. Each instance of the Oracle Parallel Server
uses the DLM to communicate with other instances and to
coordinate modifications of database locks.

The DLM lock services are implemented
as shared-memory, user-level service calls. It distributes
lock requests and lock data structure databases throughout
the nodes. Deadlock detection is also distributed.

Each resource is associated with a master node, and a directory
node keeps track of the associations between resources and
master nodes. A lock request from a DLM client is sent to the lock
manager process running on the master node for that resource. Every node
maintains information in its shared memory about client processes,
resources, and distributed locks on resources associated with that
node.

\begin{figure}
\centerline{\epsfxsize=450pt \epsfbox[60 480 550 720]{appendix/oracle2.eps}}
%%BoundingBox: 53 40 560 753
\caption{The DLM Architecture}
\label{dlm}
\end{figure}

Figure \ref{dlm} illustrates the architecture of the Oracle UNIX
DLM. The distributed lock database of the DLM consists of the
collection of all lock information in every node's
local shared memory. The local shared memory also provides
communications between the DLM background
processes (the DLM daemon process and the DLM monitor
process).

In Figure \ref{dlm}, the DLM client is any process that requests a
lock to access a shared resource. The DLM client on Node A sends
a lock request directly to the remote DLM daemon process on Node B (arrow 1). The
DLM daemon on Node B then processes the remote request and sends
the result back to the DLM daemon on Node A (arrow 2); and
the DLM daemon on Node A signals the DLM client about the result (arrow 3).

\subsection{The Oracle7 Parallel Server (OPS)}

The OPS gives users on SP2 multiple nodes access to a single
database located on a pool of shared disks. All users on all clients
within the SP2 system can run transactions concurrently
against this database. It
allows multiple instances of the Oracle database server to run concurrently and
independently. Each runs on its own processing node with its own 
Shared Global Area (SGA) memory for a database buffers cache 
and its own set of Oracle7 background processes (including
the system monitor, process monitor, and database write) 
for backup and recovery. Any number of
Oracle7 instances can access the same Oracle7 
database files and control files on disk, which
collectively form an Oracle7 Parallel Server. 

Features of the Oracle Parallel Server on SP2 include:
\begin{enumerate}
\item
Data reliability through Parallel Cache Management using the DLM
\item Transaction isolation and concurrency control
\item Online recovery for high availability
\item Centralized administration of a single database
\end{enumerate}

Oracle7 on the SP2 with the parallel query option also provides the
following functions:
\begin{enumerate}
\item
Parallel query
\item
Parallel load
\item
Parallel index creation
\item
Parallel backup and recovery
\end{enumerate}

Further reading about Parallel Oracle7 on the IBM SP2 can be found
in \cite{oracle7-sp2}.

\section{A Web and Parallel RDBMS-Based Information 
System for Internet Collaboration}

\subsection{Motivation}
Information distribution/sharing and information exchange/remote-collaboration
are two major Internet infomation services that have served the Internet community
for many years. Popular systems for text-based 
information distribution and sharing services
are FTP, Gopher, WAIS, ARCHIE etc.
These systems and the established protocol standards were developed to
satisfy particular, yet basic, needs for data access, distribution and searching
among distributed hosts over wide area networks. For example, FTP is used to
transfer data files from one machine to another and Gopher for publishing plain
text files. The WWW system has integrated almost all the major Internet
information distribution protocols into a single unified
networking navigation environment. 
The integration approach by WWW is to retain existing data transfer protocols
and server management, while replacing the particular client programs
that were previously developed for accessing those non-HTTPD servers.
A Web browser becomes a universal
client program that can access information space on different
kinds of information servers.

On the other hand, there are three major Internet services for 
remote collaboration and information exchange: 
mailing list, USENET news, and chat systems.

A mailing list is a group of email addresses that can all be reached by 
sending messages to a single address/server. Mail sent to the mail server
is redistributed to all subscribers using the basic Internet email services.
The list of addressees is maintained on a central mail server running
an automated mailing list program such as Majordomo, ListServ, and
ListProc.

The USENET news system was invented to reduce/improve the amount of resources
consumed in the mailing list approach. 
Articles are now stored in a central repository
on a receiving host instead of in each subscriber's mailbox. 
A Network News Transfer Protocol
(NNTP) \cite{nntp} was developed for the distribution, inquiry, retrieval, and posting
of news articles using a reliable stream (such as TCP) client-server model. 
For news distribution, the USENET system also uses a data propagation and
replication mechanism \cite{rfc850} to reduce network traffic congestion
and the overhead of network bandwidth over wide area networks by caching
recently arrived and retrieved news articles on a hierarchical intermediate
news server that mediates regional and local requests. An article is usually
posted first on a local news server. That machine then forwards it to all its
neighboring news servers. The sites receiving the incoming article
stores it locally and in turn forwards the article to all their interested
neighbors. This process continues until the entire USENET network has
seen the article. To read articles in a newsgroup, a user connects
to a local news server using a client news reader program.

While the mailing list and the USENET news system use slightly different 
distribution mechanisms, they adopt the same text format in electronic mail
and use an asynchronized message-passing model for text-oriented
collaboration. Mail is  usually stored on a server one piece per file, and
each consists of a series of heading text containing the 
{\sl name:value} pairs of sender's
identification, time, email reply path, subject, category, and other attributes
as specified in \cite{rfc822}. Another 
popular collaborative system is Internet Relay Chat (IRC) \cite{irc} 
for text-based teleconferencing over wide area networks. IRC protocol
uses a synchronized message passing model for real-time message delivery and
multiplexing. The execution model of IRC is also built on generic
client-server architecture in which a single chat server forms
a central point for chat clients to
connect to and exchange messages in real time over wide-area networks. 
IRC messages are a stream of characters terminated by a CR-LF pair.

While the three systems are developed for particular needs 
of Internet collaboration, they share several common features that
are also the common ground for our integration:

\begin{enumerate}
\item
Text-based messages as data content. Email format is the basic
data entity that is used to conceptually and physically carry
nformation. The major body of a message is free-text with
associated attributes, including sender identity, time,
subject text and receiver identity.

\item
Client-server model as execution architecture. Server management
and data transfer protocol are the key components and software
development effort in building these systems. 
Client programs can be quite different on different
platforms and there are usually various user interfaces for different
operating system environments. There is no unified client GUI that can
facilitate all three collaborative modes. 

\item
Major functionality of these systems is focused on simple server management,
reliable message distribution and standard protocol design. They lack 
sophisticated search and archiving capability as found in advanced RDBMS-based
information systems. They rely on file systems to store and index data thus
the logical representation of the collaboration information is almost the same as
physical representation. Browsing through the message list is the meta-index 
approach to locating interest messages. 

\item
Data propagation and caching, as well as server replication,
are common approaches to leverage server performance 
and network traffic requirements imposed
on wide area networks. 

\end{enumerate}

We expect the three conventional collaboration systems will continue to have
widespread use in various applications for many years to come.
We believe the WWW  provides opportunities to 
integrate them into a single unified framework. WWW is not only 
an integrated environment to access and distribute
various information resources on the Internet, it can also be a
generic interface and framework that integrates various existing and new
Internet collaboration tools for remote group interaction and interactive
information exchange. Instead of developing new collaborative systems
within the new WWW environment, our approach is to integrate
existing Internet systems and to develop a new Web-top software layer
that provides an integrated access path to underlying non-WWW software
systems. In this case study we build a powerful WWW collaborative
information system that integrates and enhances basic functionality of
the three conventional Internet collaboration systems.
We use a parallel Web and RDBMS server technology as a centralized 
high-performance collaborative server to store, synchronize,
and distribute email-based messages over the Internet. Built on this 
collaborative server is an RDBMS-based information management system
that serves as a search and archive engine for USENET news, a mailing list and
a chat system. This single server can facilitate either asynchronized USENET news 
and mailing articles or synchronized chat messages. 
On the client side, we develop HTML-based
interface that can be embedded in any Web browser for basic collaborative
functions, including read/navigate, search,
compose, post, reply and receive messages. Sophisticated real-time search 
capabilities that are not possible in conventional systems
are implemented, such as search by keyword, by date, by sender, etc.
The integration approach used in this case study is on both client and server
sides. On the client side, a unified GUI is developed so that any one of
the three collaborative modes can be switched to at any time
of collaboration by participants. On the server side, the same data format of
email used by all three systems is stored in a single Oracle database
and the server provides support for both SQL-based search in the
collaborative information space and concurrent control in the collaborative
threads. Overall, this prototype system provides support of
bulletin board type for the USENET news and mailing list, and 
for the real-time IRC type
of email-based chat.

\subsection{System Configuration}
\begin{figure}
\centerline{\epsfxsize=440pt \epsfbox[75 85 550 680]{appendix/asknpac.eps}}
%%BoundingBox: 53 40 560 753
\caption{Configuration of the Integrated Internet Collaborative Server}
\label{asknpac}
\end{figure}

Our system is powered on a 4-node IBM SP2 parallel machine. Parallel 
Oracle7 RDBMS software is used on the SP2 as a back-end search engine and
database server. Appendix D.1 describes in detail 
this parallel database software system.

As shown in Figure \ref{asknpac}, 
key system components in the configuration include:
\begin{enumerate}
\item
{\em Parallel HTTPD servers}.
Each server is an NCSA HTTPD server.
The parallel Web server consists of multiple independent
HTTPD servers, each running on a separate IBM SP2 node. Each SP2 node
has a unique IP address that is accessible to Internet via a Web
client/server. Through a CGI to an Oracle RDBMS instance also
running on each
SP2 node, each Web server has access to the entire database content
that is shared among all the Oracle instances. In this configuration,
the HTTPD server becomes an Oracle client application that routes
the request from Web clients to the Oracle server.

Parallelism is obtained by distributing Web client requests and hyperlinks
to multiple Web servers sharing the same database. This is done by
a simple ``round-robin'' load-balance algorithm implemented on a master
HTTPD server. Initially, all clients are given a single URL that points to
the master Web server; for example, {\sl www.merlin} in Figure \ref{asknpac}. This URL is
the entry CGI program to use/access this information system. Because 
each client's identification, including
client hostname and access time gathered from each HTTP connection, 
is logged in the database, 
all the URLs generated as a result of the initial request 
are redirected
to the other Web servers in the parallel system, based on a round-robin
URL assignment for concurrent users. Thus the user's subsequent accesses
are confined in a Web/Oracle server that may be different from other concurrent
users. Therefore, database queries originated from concurrent Web clients
will be handled by different SP2 nodes simultaneously to achieve a roughly
linear performance scalability.
\item
{\em Parallel Oracle7 servers}.
Along with an HTTPD server, each SP2 node runs an identical Oracle
RDBMS instance that operates on a shared database. This Oracle instance
is a powerful data processing engine that back-ends the Web server
to support concurrent simple or complex SQL-based 
queries and database transactions requested from the Web server.
Each Oracle instance handles queries independently from the other
Oracle instances. With the dynamic load-balancing scheme in
distributing URLs among all the Web/Oracle servers, the parallel Oracle
server is expected to serve a large number of concurrent Internet users
without serious server performance degradation.

\item 
{\em A Shared Oracle database}.
The parallel Oracle on an IBM SP2 uses a parallel file system called
Virtual Shared Disk (VSD) to make the distributed disks local to each 
SP2 node appear to be a single logical file system. This software
automatically handles remote disk access without performance
degradation (via a high-performance
switch with a sustained bandwidth of 40MB/s). On top of the VSD system
is another file system named Hashed Shared Disk (HSD). HSD transparently
stripes database table content evenly across multiple VSDs
to minimize disk contention and to alleviate disk bottleneck. 
Therefore, the Oracle database is identical on all SP2 nodes, and
the aggregate database I/O performance far exceeds the speed of accessing
data from a single disk. It provides an infrastructure to host a
very large volume of data with high-performance I/O.

\item
{\em Data gathering programs}.
While the Web servers mostly provide a WWW-based search/navigation interface
for Internet users to access this collaborative server and database, we rely on
another set of programs to interface with the existing Internet
collaboration facilities. They are small gateway programs whose tasks are
to gather in real-time messages that occurr on conventional USENET news servers
and mailing list servers and convert them
into the Oracle database. For USENET news,
a program periodically fetches newly arrived articles on a local news server via NNTP.
For subscribers of particular mailing lists, the system has a program
that periodically incorporates incoming emails into its mailbox and converts/inserts
them into the Oracle database. Note that this gathering process must run
in real-time while the server is serving the WWW interface. Updating a database
usually involves a significant amount of concurrency control (e.g., table locks).
To reduce its impact on overall system performance, 
we devote a particular
SP2 node to carry out this operation, for example, node 4 in 
Figure \ref{asknpac}.

\end{enumerate}

\subsection{Search Interfaces}
\begin{figure}
\centerline{\epsfxsize=380pt \epsfbox[80 150 550 770]{appendix/asknpac/all.ps}}
%%BoundingBox: 13 83 600 709
\caption{Search Interfaces for the USENET Newsgroup and Mailing list Database}
\label{all}
\end{figure}

\begin{figure}
\centerline{\epsfxsize=360pt \epsfbox[90 260 550 770]{appendix/asknpac/frame.ps}}
%%BoundingBox: 10 291 602 786
\caption{A Frame-Based Search Interface}
\label{frame}
\end{figure}

\begin{figure}
\centerline{\epsfxsize=490pt \epsfbox[90 370 550 720]{appendix/asknpac/form.ps}}
%%BoundingBox: 120 399 494 785
\caption{A Form-Based Search Interface}
\label{form}
\end{figure}

\begin{figure}
\centerline{\epsfxsize=490pt \epsfbox[80 370 550 720]{appendix/asknpac/hypermail.ps}}
%%BoundingBox: 104 389 520 785
\caption{A Hypermail-Like Search Interface}
\label{hypermail}
\end{figure}

\begin{figure}
\centerline{\epsfxsize=360pt \epsfbox[90 260 550 770]{appendix/asknpac/calendar.ps}}
%%BoundingBox: 13 362 603 780
\caption{A Calendar-based Search Interface}
\label{calendar}
\end{figure}

\subsubsection{USENET Newsgroup and Mailing List}
Access to the newsgroup and mailing-list database use the 
same WWW-based search interfaces.
To satisfy different users' needs,
we have developed several different search interfaces
that allow different search methods and styles (Figure \ref{all}), 
while they all provide base functions of search-by-keyword, search-by-listing, 
search-by-sender, search-by-date, etc.

The frame-based search interface (Figure \ref{frame})
assumes the client Web browser can support frames in an HTML page.
It combines a meta-index search frame that consists of multiple
search options with a message-body display on the same page to
allow tight navigation of search results.

The form-based search interface (Figure \ref{form})
ultilizes HTML form constructs to allow very specific queries for
each search option.

The line-based search interface (Figure \ref{hypermail})
is modeled after a popular WWW mail-archive system HyperMail \cite{hypermail}.
Almost any Web browser can use this interface without support of
enhanced HTML constructs such as frame, table, and form.

The calendar-based interface (Figure \ref{calendar}) 
combines features in the other three
interfaces. In additon to the index and search widgets
used in the other interfaces, it uses a calendar as the 
meta-index to group monthly and weekly
messages for each newsgroup or mailing list. 

Detailed functionality of the prototype can be found
at ASKNPAC \cite{asknpac}.

%\subsubsection{Chat and Bulletin Board}

\section {WWW Search Systems Using 
SQL*TextRetrieval and Parallel Server for Unstructured Data}

\subsection{Components of a WWW search system}

\begin{figure}
\centerline{\epsfxsize=420pt \epsfbox[11 300 602 720]{appendix/figure1.ps}}
%%BoundingBox: 53 40 560 753
\caption{Components of a Web Search System}
\label{web-search}
\end{figure}

A Web search system is a facility to locate/discover relevant information 
contained in the WWW URLs. The search domain can either be the whole WWW 
information space (including httpd, ftp, gopher, wais and USENET news servers) 
on the Internet or HTML documents on a single web site.

As shown in Figure \ref{web-search}, a Web search system generally has 
four major components: a gathering subsystem, an indexing 
subsystem, a search engine, and a Web-based search interface.

\subsubsection{Web Gathering Subsystem}

A Web Gathering subsystem's major function is to gather web pages/files 
from either remote or local WWW servers. It may involve either the whole 
Internet ``Web Space'' which includes information on HTTPD, FTP, GOPHER, WAIS and 
USENET news servers or HTML documents on a single Web server. This gathering task 
is usually carried out automatically by a relatively small program sometimes called 
``Web robot'' or ``Web Spider'' or ``Web Agent'' whose functionality can be briefly 
described as follows:
\begin{enumerate}
\item
Starting from a single URL, it requests a web page from the remote web site. 
Upon receiving the full page, it parses and adds all the URLs in this page 
to a queue for further gathering. 
\item
For the web page from (1), according to the indexing rules predefined in the 
indexing subsystem and the search engine, it parses the page and filters only 
the information needed by the indexing subsystem. 
\item
After (2) is done, it gets a new URL from the queue, checks if the new URL is 
already indexed and proceeds with (1) again to repeat the gathering process.
\end{enumerate}

In summary, a web robot continuously requests files from remote web sites and 
parses and filters each file for indexing and further gathering.

It is up to the gathering subsystem to decide where, what, and when 
the information will be cached/indexed/updated into the text database for web 
search. It determines the information content and volume and, to a degree, 
the performance of a web search system. It can also have significant impact 
on the accuracy of a web search system. Different web systems may use different 
rules and approaches. Major issues and design parameters for the gathering subsystem are:
\begin{enumerate}
\item
From where to gather: 
\begin{enumerate}
\item all or only HTTPD,FTP,GOPHER,WAIS,NEWS server 
\item for a single server, all or first N levels of URLs in the 'hyperlinks tree' 
\item web pages in single or multiple text formats of HTML, Plain Text, Latex, etc.
\end{enumerate}

\item  
What to index:
\begin{enumerate}
\item all words or parts of 'significant' keywords in a web page, i.e., full-text or keywords only 
\item attributes of a file, such as last update date, title, subtitles, size, outline etc. 
\item controlled by size/lines, e.g., first 10 lines of a text file.
\end{enumerate}

\item
When to gather/index/update:
\begin{enumerate}
\item the same as or a separate indexing database from the current one used by web search interface choice 
\item real time (i.e. online indexing) or batch during nights.
\end{enumerate}
\item
Performance of a gather: number of  files that can be gathered/indexed per day/hour
\end{enumerate}


\subsubsection {Indexing Subsystem}

The indexing subsystem defines how contents gathered 
from remote web sites is internally stored and managed in 
a text database in order to efficiently and effectively 
support searching. It depends on the logical data structure 
of text entities and the physical organization/layout of textual 
indexes in the database. The indexing subsystem is tightly coupled 
with the search engine as its sole purpose is to speed up the free 
text search/query process conducted by the search engine. A common 
approach is to build an ``inverted index''  for all keywords in a document.

The major decisions to be made for the indexing subsystems are:
\begin{enumerate}
\item
What compression scheme will be used to store the text and its indexes. 
\item
Do we keep both original text and indexes in the database, or 
just indexes? In order to support certain advanced search 
capabilities such as phrase search 
and proximity search, the original text must be stored 
together with its indexes. This will increase the space consumption 
of the text database. 
\item  
Index modes: real-time indexing, batch indexing, or incremental 
indexing. Real-time indexing allows updates, inserts or deletes of 
text in/from the existing text database at any time, which will automatically 
modify the associated indexes, without affecting the simultaneous text retrieval 
on the index database. Batch indexing only updates index in a batch mode after a 
bulk of texts has been modified/loaded. Incremental indexing allows the indexing 
process to be done incrementally, as only documents that have changed since 
the last indexing run are indexed. Thus, adding new text will not affect the 
existing indexes in the database.
\item  
Case sensitivity, symbols, or numbers. Support of case sensitivity and symbols 
in the index will increase the space requirements.
\end{enumerate}

Performance of a web search system will largely depend on how good its 
index scheme is. A good index scheme should have moderate storage requirements,  
while significantly speeding up the query processing.

\subsubsection {Search Engine Subsystem}

The search engine subsystem accepts queries from a Web-based search interface and 
schedules, partitions, and executes them against the indexed database to locate URLs 
and their associated attributes that satisfy the query criteria. It also deals with the 
I/O, caching, assembling, sorting,  and performance-related system activity. Based on 
what content is indexed and on the scheme used in the indexing subsystem, it implements/supports 
different search algorithms to provide various search capabilities of a text 
retrieval system. Together with the previous two subsystems, the search engine 
determines what basic and advanced search capability a system can support. It
is critical for overall system performance.

Most Web search systems currently online are keyword-based search systems. 
The major search functions include:

Basic search capability:
\begin{enumerate}
\item  Keywords with logical operators and their combinations, including ``and,'' ``or,'' ``not''
\item  Regular expression of keywords, including single/multiple wildcard matching
\item  keyword expansion: stemming, fuzzy, soundex, expansions based on a thesaurus (such as synonyms)
\item  Ranking of query results 
\item  Case sensitive/insensitive
\end{enumerate}

Advanced search capability: 
\begin{enumerate}
\item  Summarizing --- summarize a document 
\item  Similarity search - search ``similar'' documents to a particular document 
\item  Phrase search 
\item  Proximity search --- specify words distance between keywords
\end{enumerate}

\subsubsection {A Web-Based Search Interface}

Any information retrieval system has the two core subsystems,
``indexing'' and ``search engine,'' discussed above. They represent base technology developed in 
information processing literature and are used directly in the Web search systems. 
What is unique and different in a web search system, compared to a traditional 
text retrieval system, lies in the ``Web gathering'' and ``Web search interface'' 
subsystems. These two subsystems are also the major focus of the development work 
we have undertaken, as we rely on Oracle's RDBMS and SQL*TextRetrieval 
technology to build the ``search engine'' and ``indexing'' subsystems.

A conventional text retrieval system usually has its own search interface 
and also allows developers to build new search interfaces to access the 
text database. For example, SQL*TextRetrieval provides both form-based and 
C-based APIs. Most text retrieval systems use a client-server model as its 
fundamental model to build access interface to end-users. Therefore, the 
client-server model used in the WWW makes it seamless and natural to integrate 
a text retrieval system into a web search system, where a web client becomes 
a client of a text retrieval system. Due to the stateless (or sessionless) 
feature inherent to the web client-server model, new techniques are 
required in a web search interface that are quite different from those 
used in a traditional search interface design.

The major issues in developing a web search interface are:
\begin {enumerate}
\item
Integration of a web server and the back-end search engine, i.e., 
interaction between a web server and the search engine, to efficiently 
facilitate the single transaction path: 
 query requests --- query processing --- query results. While interaction between 
a Web client and a Web server is handled by the common http protocol, the mechanism 
of  the interaction between a Web server and a search engine may vary in 
different Web search systems. Because a Web search system is designed to 
handle a large number of requests (which are, essentially, OLTP read-only 
transactions) simultaneously, and a heavy load is assumed on the server side, 
high-performance server technology is usually required to support scalable 
CPU and I/O processing. Another important consideration is how the system 
handles simultaneous search and database updates/indexing in real time.  
 Most current Web search systems use some very limited ``parallel processing'' 
techniques and replication technology to handle performance scalability issues. 
For instance, a common approach popularly used in Lycos, InfoSeek, OpenText etc. 
is to use more than one workstation, each running a Web server and a local 
database. The local database on each machine is identical across all the 
machines, replicated from the machine dedicated to gather/index new Web pages. 
The replication (thus the updates) of the local database usually is done once a 
day during the night (some once a week/month). As far as we can see, this is a far 
cry from modern database replication techniques. 
 
\item
Sessionless Web connections towards session-oriented database 
transactions on the Web search interface. In many situations, 
session-oriented memorizing capability is required for Web search 
activity. For example, due to networking bandwidth and cache/memory constraints of the 
client machine, one query may generate a copious  output that 
is too costly to send back all the way down to the display of the  client's 
browser.
One possible approach is to store all the results on the 
server and provide navigation buttons (next, previous, etc.) to guide the 
user in going through the query results in subsets. Because each user may 
issue a different query and the navigation requires more than one Web 
transaction, the Web search system must ``remember'' the first query's 
results for each concurrent client. This means that a user's Web session 
includes the first query and successive navigation transactions. In the early 
days of Web search systems, the only way to solve this problem 
was to use a ``Max hits'' button for the first query so that the system restricted 
the maximum number of query results to be returned to the client. Currently, 
session-oriented transaction are becoming a common practice in most Web search systems.
 
\item  
Query refining. This capability implies that after a query is issued whose 
search domain may be the whole indexed database, the new query uses the 
results returned in the previous query as its search domain. In that way, 
one can keep refining the search and finally find/narrow down to the 
required search targets.
 
\item Support of the ``natural language like query''. Most Web search systems 
support only keywords and their regular expressions to form a query. 
A good Web search system should also allow users to type in queries 
using natural language. This function requires additional processing and 
analyzing of the query input from the search interface.
\end {enumerate}

\subsection{Oracle SQL*TextRetrieval}

Traditional RDBMS systems are developed to deal with well-structured 
information entities such as business data. Information represented 
in WWW space is usually in the form of free-format with little or no 
predefined structure such as HTML pages, program source code, and 
plain-ASCII articles. Because HTML is a very loosely defined markup 
language, an HTML page can be loosely or even poorly formatted. For example, 
not every page has a title (as enclosed by <title> and </title> tag in HTML),
 although it is obligatory and acceptable by any Web browser. Although 
documents in free-text form can be stored in tables of an RDBMS server, 
the search for the content of documents without predefined search 
terms (words, phrases, or concepts) requires a serial scan of the table 
and an exact pattern matching the text column. Performance of such a 
search is not acceptable. The standard indexing techniques used in Oracle 
RDBMS cannot help in this case, as the full-text documents are stored 
in single or multiple table columns of character type. A new indexing facility, 
together with its text search utility, must be developed to solve the performance 
issue in free text searching. SQL*TextRetrieval is one such solutions developed by 
Oracle Corp. SQL*TR indexes both text contained in the multimedia documents and 
in the textual description of the data. The OPS allows users to perform a free text 
search that returns a set of documents or database rows that match the 
selection criteria. Because SQL*TR is a software layer built on top of the 
core Oracle7 RDBMS engine, it uses SQL queries as the standard method and 
query language for retrieving both textual information and structured data. 
It can take full advantage of Oracle7 server features such as concurrency 
control, recovery, data integrity, security, backup, and server management. 
Users can combine searching of structured information, (such as title, author, 
size, date of a Web page) with free text searching. Among many utilities provided 
in the SQL*TR package \cite{sqltr1} to build a fully functional yet traditional 
information retrieval system, the indexing facility and text retrieval API 
(in C) are most relevant to building a WWW search system. 
Readers are refered to \cite{sqltr1,sqltr2} for details of SQL*TR.

\subsection {A Prototype WWW Search System}

Our approach to building a Web search system was to develop a 
high-performance and intelligent Web-gathering subsystem and to build 
a flexible and customizable HTML form-based Web search interface while 
employing parallel Oracle RDBMS technology for the search engine and server management, 
and SQL*TextRetrieval for the full-text indexing engine. 
Because the SQL*TR package was originally not developed for WWW information 
retrieval, much work has also been devoted to interfacing the gathering subsystem 
with SQL*TR, and to integrating the Web search interface with the parallel 
Oracle7 RDBMS server on an IBM SP2 parallel machine.

Figure \ref{Web-config} is the system diagram of the Web-gathering subsystem 
developed for our Web search system. It consists of three major components: 
a loader, a checker and one or more gatherers, each of which is an individual process. 
The fundamental philosophy behind this design is that the gathering 
program must be both high performance  (the potential performance 
bottleneck of the limited networking bandwidth over the wide area 
network should not become a bottleneck for the gathering process; 
the full advantage of an RDBMS server in concurrency control and 
caching should be utilized to build such an RDBMS-based Web agent) 
and intelligent. (It should detect the URLs that have different symbolic 
names but point to the same document, the Web robot convention \cite{robot} 
should be followed, and it should be extensible to gather domain/topic 
specific information.)


\begin{figure}
\centerline{\epsfxsize=420pt \epsfbox[11 140 602 720]{appendix/figure2.ps}}
%%BoundingBox: 31 206 582 783
\caption{An RDBMS-based Web robot system}
\label{Web-config}
\end{figure}


During the gathering process the loader and the checker are connected to 
the RDBMS server where tables consisting of the text and index database 
are stored. A set of temporary tables are also created to store URLs and 
hosts used to identify URLs that are new (to be gathered), pending, being 
processed, or waiting to be updated in the gathering processing by the gatherers and the checker. Concurrency control of accessing or updating these tables by the checker and the loader is managed by the RDBMS server. To sustain the maximum networking bandwidth, more than one gatherer can be started to interact with Web servers simultaneously via HTTP to get the bulk of documents in HTML. Each light-weighted gatherer parses the document to identify URLs to be considered for further gathering and a couple of attributes associated with the document, including URL, size, date, title, number of URLs and inline images, outline (first 256 characters), etc.

Major functions of the loader include: 
\begin{enumerate}
\item
Fetch new URLs to be gathered/updated from the Oracle server and pass them to gatherers upon request. 
\item
Invoke SQL*TR ``rtloa'' process to load/update and/or index incrementally documents 
when they become available (in batch).
\end{enumerate}

The checker is responsible for checking URLs: 
\begin{enumerate}
\item
Validate URLs to be gathered according to several filtering conditions:
\begin{enumerate}

\item the disallowed URLs specified in the ``robot.tx'' on a host 
\item URLs with all the attributes (size, title, outline etc.) are identical yet may have different 
hostnames (alias) or paths (symbolic links) 
\item URLs found to be non-existed/removed in the previous visits of the gatherer (dead links) 
\item URLs being processed by the loader or one of the gatherers
\end{enumerate}
\item
Filter out URLs not to be gathered according to some predefined conditions, such as URLs for server statistics. 
\item
Filter in only URLs to be gathered according to predefined conditions, such as the online HTML books 
(http://www.infomall.org/npac/pcw/*), NPAC Web server (http://www.npac.syr.edu/*), or any WWW server 
(http://*,ftp://* etc).
\end{enumerate}

Currently the information used by the checker to do the filtering is only 
the URL itself and the URL/host tables in the database. 
To make the gathering subsystem more intelligent, to be able to gather/index domain-specific 
information and such, the architecture design of our system is developed to be extensible/open. 
For example, another module ``filter'' between the gatherer and the loader can be added 
that filters documents (not URLs) based on the content of the documents.


Figure \ref{Web-interface1} is the main search interface of the prototyping system where one 
of the indexed WWW databases is the NPAC official Web server. 
Other databases currently indexed include three online HTML books in parallel computing

Built directly on top of the text retrieval in SQL*TR, this HTML 
form-based interface allows users to have the following search options:
\begin{enumerate}
\item
Logical operations of keywords --- AND/OR (``all''/``any''), NOT (``contain''/``not contain'') 
\item 
Word expansions:
\begin{enumerate}
\item word stemming --- expanding a keyword based on its stem and indexed words 
having the same stem (wildcard matching on either end)  
\item fuzzy or soundex --- words similarly spelled or phonetically similar
\item synonyms --- same or nearly same meaning as other words predefined in the 
same synonym ring of a thesaurus. We converted a public domain Roget Thesaurus  
into our database that has about 6000 synonym rings
\item search keywords to be treated as case sensitive/insensitive
\end{enumerate}
\item  Phrase search --- exact matching of a phrase
\item  Proximity search --- two words with relative location to each other
\end{enumerate}

\begin{figure}
\centerline{\epsfxsize=400pt \epsfbox[60 180 550 770]{appendix/figure3.ps}}
%%BoundingBox: 53 40 560 753
\caption{The search interface of the NPAC Web search system}
\label{Web-interface1}
\end{figure}


\begin{figure}
\centerline{\epsfxsize=400pt \epsfbox[50 130 550 770]{appendix/figure4.ps}}
%%BoundingBox: 53 40 560 753
\caption{Display of search results and an interface of the navigation and refining}
\label{Web-interface2}
\end{figure}

The interface also provides access to a gateway to view all indexed words and stoplist 
words currently in the database. This provides a simple server management layer.

As shown in Figure \ref{Web-interface2}, search results are returned 
to the client's Web browser one subset per page. Included in each result page are:
\begin{enumerate}
\item A set of navigation buttons (``next'', ``previous'', ``top'' and ``end'') to allow the user 
to browse through all the query results page by page. SQL*TR's Stored Query Expressions 
(SQE), together with a sequence, is used to first store the query and its hitlist 
(i.e., identification of matching documents) and later restore them when 
these buttons are activated to request additional query results in 
successive Web client-server connections. The use of SQE prevents the 
initial query from rerunning. Note that the hitlist stored by SQE is 
in a highly compressed bitmap form so that the temporary space consumed 
should not be an issue.

\item A message area indicating search keywords and their expansions, 
and total number of matching documents found.

\item The line items for matching documents that contain the search keywords, 
each of which is represented by a set of attributes associated with the 
document. The title of each document is hyperlinked back to the original 
URL for quick browsing of the full document source, while the URL attribute 
links to another query to pull out the document text from the database and 
make the search keywords to be highlighted in the display.

\item A similar search interface to allow the user to refine the previous 
query so that the search domain of a new query will be the output of the 
previous query and not the whole database. This is again implemented by SQL*TR's SQE.
\end{enumerate}

By using word expansions we attempt to broaden the search domain to 
improve search accuracy limited by keywords search; while using 
the ``query refining'' we support gradual narrowing of the search 
domain, which may be over-extended by the word expansions.


