Design Goals
DOGMA's overall design goals are to provide an easy-to-use, object-oriented system with usefully high performance that can perform well in both Internet and intranet settings. DOGMA is targeted towards applications where systems such as PVM and MPI are now used, but adds the capability of harnessing the power of both heterogenous nodes and nodes across the Internet. DOGMA seeks to be both a platform for further research into parallel programming with Java, and a tool capable of producing useful results.
Toward this end there are several requirements which DOGMA seeks to meet in both the runtime system and in its programming interface:
Runtime system design goals
DOGMA is built using Java Remote Method Invocation as a foundation. RMI allows Java objects to seamlessly interact with objects residing in separate Java Virtual Machines (JVM's), however, the programmer is required to handle issues such as object location etc. at a low level.
The Distributed Java Machine (DJM) layer allows several JVM's to act as one distributed machine while freeing the programmer from needing to worry about the location of objects, the names of nodes, the location of nodes, etc. This layer also allows application binary code be served via one or more http servers so that applications do not need to be installed on the local file system of every cluster included in the system.
In addition, DOGMA includes an MPI implementation written entirely in Java.
Further, Distributed Object Groups allow method invocation, data partitioning, communication, etc. on large groups of objects without the programmer needing to worry about the exact number of nodes used etc.
As in Mentat, nodes are organized into families and clusters. Families of nodes are nodes that have a similar configuration and architecture. Clusters of nodes are nodes that are located near each other physically.
System topology is configured dynamically in three ways:
2 - The system topology database may be updated at runtime. This is done by grouping families and clusters into groups known as "configurations" each managed by a ConfigurationManager. When a ConfigurationManager is made active, it contacts DJMManager and informs it of all families, clusters, and nodes that are in the ConfigurationManager's configuration. When a ConfigurationManager is shutdown it removes all knowledge of its configuration from DJMManager.
3 - Web browsers can effectively become nodes in the system. Web browsers can load the NodeConApplet which starts a NodeManager running in the browser. This allows users to run applications by simply browsing into the system and running the desired application, or to donate computational power by browsing into the system simply to help.
In this way ConfigManagers provide modularity. They free the DJMManager
from needing to know the configuration of the system when it is started.
They also handle all architecture specific issues, freeing DJMManager from
any system specific functions. In this way arbitrary clusters of
nodes can be added to and removed from the system dynamically without requiring
them to all use browsers.
MPIJ is based on the MPI C++ bindings as much as possible.
MPIJ applications are written by importing mpij.* and extending the class MPIApplication. They are then run from the system command prompt by using the runMPI or runMPIP command.
Prototype versions under Windows NT have been able to achieve excellent bandwidth enabling MPIJ applications to beat native applications (PVM3.4 and WINMPICH) when communication becomes an issue (usually around 4 nodes).
A DOG is organized according to a data configuration scheme. The scheme defines how data will be partitioned, group elements will communicate, what type of coordinates are used to define group element position, etc.
DOG features include:
ElementImpl
Each element in an object group is derived from the data configuration scheme's extension of this class. This gives group elements common capabilities such as peer communication.
Class reloading - Java currently has no standard way of loading a new version of a class into a JVM. It is possible to work around this by using a custom class loader, but this is not a clean approach. The effect of this on DOGMA is that the system must be brought down in order to replace classes that have been loaded with new versions. Sun is addressing this issue, and DOGMA will be able to replace classes at runtime as soon as a new JDK allows this cleanly.
Native threads - DOGMA is heavily multithreaded, and will perform better as native threaded JVM's become more common.
Others - As Java matures, compilers and just-in-time compilers
should be able to generate code with speeds very close to that of native
C++ even for somewhat large and complex applications like DOGMA.
Also, RMI and object serialization are likely to improve in future JDK
releases.
Future DOGMA enhancements
Object Placement Enhancements
There is much research to be done in the area of object placement for an application. DOGMA currently places objects on nodes which appear to have most computational power (where power = speed*(1-load)) available. However, there are many factors such as network bandwidth, network latency, and an application's data configuration scheme that are not captured by this model. Future research will investigate more sophisticated object placement algorithms such as performance surface analysis.
Dynamic Data Partitioning Enhancements
Designing an efficient dynamic data partitioning algorithm is closely related to designing an efficient object placement algorithm. As mentioned earlier, the current dynamic data partitioning algorithm has realized significant improvements in performance from simply taking node load and node speed into account. In the future more sophisticated algorithms may be able to yield yet more performance especially on long running applications.
Object migration
Future versions of DOGMA will feature object migration. Object migration is very useful for several reasons. Among these reasons are:
DOGMA's current implementation is targeted for use of a maximum of several hundred nodes spread across an intranet or the Internet. This is due to the fact that DOGMA is centralized at the DJM layer since only one DJMManager may exist in a given system. Future versions of DOGMA will allow multiple DJMManagers in a manner somewhat similar to SuperWeb's brokers (while still retaining DOGMA's cluster management capability/philosophy).
Fault tolerance
Current fault tolerance is limited to the system level. Faulty nodes are detected and removed from the system. There is much research to be done in both system and application level fault tolerance.
Security
Internet security is nearly a contradiction in terms. Fortunately, Java makes many security issues approachable that would not be if security were not designed into the language.
Current DOGMA security is divided into two main classes -
Cluster based nodes: On these nodes, users must have login privilege.
Security for these users is simply a relaxing of the RMISecurityManager
enough to allow the system to function.
Browser/screen saver based nodes: On these nodes DOGMA runs as an applet
with all the usual sandbox restrictions. This allows users to safely
donate CPU cycles to DOGMA without needing to worry about a malicious program
harming their system.
With JDK 1.2 nearly released, more advanced security implementations
are possible, and future versions of DOGMA will address security much more
thoroughly.
Created 10/97. Last modified 9/98