\subsubsection{Measurements}

For our measurements, we chose to test the Mersenne primality for all
119 prime exponents between 4000 and 5000. The reason for selecting this range
is that on the one hand, we tried to make numbers large enough to simulate the true working
conditions of the application, and on the other hand, we wanted to keep them small
enough to be able to complete our set of measurements in a reasonable amount of time.

We conducted the experiments on a heterogeneous computing
environment consisting of various architectures: Sun UltraSparc (both
single and multiprocessor), Sparc5, Pentium PC, and a 64-node Meiko CS-2. We used the
JDK (versions 1.0.2 and 1.1.2 without the just-in-time 
compiler\footnote{The Solaris JIT compiler from Sun was
  available to us in a beta version. However, at present the Mersenne
  application will not run on this compiler due to a numerical bug
  which we hope will be fixed in the near future.}) 
on all platforms. 

%We started by measuring  the total 
%execution time on each architecture
%separately. Figure~\ref{fig:one_machine}, shows the total time needed
%to test all the prime numbers less than 3000 on a single
%machine. The tests were performed using the Javelin infrastructure,
%therefore the reported times include all overheads. 


%\begin{figure}[htbp]
%\centerline{\psfig{figure=figs/OneMachine.eps,width=4.2in,clip=}}
%\caption{\em {Mersenne primality test for all prime exponents less
%than 3000. The tested architectures were 90 MHz PowerMac, 166 MHz
%Pentium (running both Solaris and Windows 95), Sun UltraSparc 2, and SGI MIPS R4600.}}
%\label{fig:one_machine}
%\end{figure}

The first set of measurements was performed on clusters of identical
machines. Figure~\ref{fig:speedup} presents the speedup curves for
test runs on a 64-node Meiko CS-2\footnote{The nodes on a CS-2 are
  Sparc 10 processors running an enhanced version of Solaris.} (left)
and for a cluster of 8 Sun UltraSparcs (right). In both cases, the
speedup was close to linear as long as the ratio of job size to number
of processors was big enough. For large numbers of processors
communication through a single router becomes a bottleneck. In a more
realistic setting where huge primes are checked we do not expect this
to be a problem since the computation will be extremely
coarse-grained. For our tests, we chose a strategy where the biggest
tasks (large amount of computation) were enqueued at the beginning of
the task queue, thus ensuring that the hosts that participate first
get the largest amount of work.  This led to an even
task distribution. 

\begin{figure}[htbp]
\centerline{\psfig{figure=figs/meiko.eps,width=2.6in,clip=} \hskip1cm
\psfig{figure=figs/ultra.eps,width=2.6in,clip=}}
\caption{\em {Speedup curves for primality test on a 64 node Meiko
CS-2 (Sparc 10 processors) and on a cluster of 8 UltraSparcs.}}
\label{fig:speedup}
\end{figure}

Next, we tested our prototype on a heterogeneous environment
consisting of 16 PCs, 8 UltraSparcs and one Sparc5. All machines were
running Solaris 2.5.1. The execution time for 32 Meiko nodes is given
for comparison. We chose an uneven distribution of platforms to
reflect a possible real situation of computing resources registered at
a broker at some moment. The broker should distribute the tasks such
that resource utilization is close to optimal. Figure~\ref{fig:hetero}
shows total execution times for different combinations of the
previously mentioned architectures. In our case, the PCs are slightly
faster than the UltraSparcs. The combined resources of all machines
showed an improvement, but not linear, due to the
saturation of the router and the big difference in computing speed of
the various hosts. There are three basic approaches to solve this
problem: the client could give out more than one number at a time,
thus reducing the number of messages, or more than one router could be
used, or hosts themselves could become clients for other hosts,
leading to a work-stealing approach as in our previous raytracer
application.

\begin{figure}[htbp]
\centerline{\psfig{figure=figs/various.eps,width=3.4in,clip=}}
\caption{\em {Primality test on a heterogeneous platform consisting of
Pentiums, Sparcs and UltraSparcs.}}
\label{fig:hetero}
\end{figure}


We also measured the difference between the C and Java versions of our
Mersenne primality test program. Figure~\ref{fig:comparison} shows the
execution times to test a Mersenne prime ($2^{4253} - 1$) for a 166
MHz Pentium and an UltraSparc 2. Here, the Pentium PC was running
Windows 95, and we used Microsoft Internet Explorer as the test
platform. The Explorer already has a just-in-time compiler. We tested
the C version of the program and the Java version with and without
just-in-time compilation. As we can see, the performance of
just-in-time compilers becomes much closer to C (in the case of the
Pentium processor, the C version is only 50\% faster than the Java
compiled version).  Newly released JIT
compilers for Java are already having a dramatic impact on the
execution time (here, a factor 8 improvement in the case of the
166 MHz Pentium).

\begin{figure}[htbp]
\centerline{\psfig{figure=figs/C_versus_Java.eps,width=3.4in,clip=}}
\caption{\em {Comparison of C and Java versions of the Mersenne
primality test program for one large prime: $2^{4253} - 1$.}}
\label{fig:comparison}
\end{figure}
