Subject: [Fwd: FMA support in Java]

From: Ronald F Boisvert <[email-masked]>
Date: Tue, 11 May 1999 09:20:18 -0400

Attached are some thoughts regarding support of FMAs in Java.  These have arisin
from discussions among some JGF members.  We would like to have your opinion on
the attached draft proposals.  These are just in the early formative stages, so
we would appreciate it if you did not circulate this document outside of this mail 
group until they are further solidified.  Thanks.

Ron Boisvert

-------- Original Message --------
Subject: FMA support in Java
Date: Fri, 7 May 1999 11:04:23 -0700 (PDT)
From: Tim Lindholm <[email-masked]>
Reply-To: Tim Lindholm <[email-masked]>
To: [email-masked], [email-masked], [email-masked],[email-masked],
[email-masked], [email-masked],[email-masked], [email-masked],
[email-masked],[email-masked], [email-masked], [email-masked],[email-masked],
[email-masked], [email-masked]
CC: [email-masked], [email-masked]

Hi folks,

Marc Snir and I have been looking around for a proposal that would
satisfy IBM's desire to be able to use FMA implicitly as well as
explicitly.  Marc produced the following proposal and has asked me to
forward it after giving it a sanity check.  Rather than take more time
suggesting tweaks (I've already sat on this far too long, for which I
apologize...), I'll list what are I think the key issues involved,
and close with some related thoughts after Marc's proposal.

I think that the high-level issues around Marc's proposal include (1)
whether it is atteptable to have some mode of Java floating point in
which the significand can vary unpredictably; (2) whether it is
acceptable to make it possible to fuse mul-adds that are not evident in
the source; and (3) whether the likelihood of tying several loosenings
beyond Java's default floating point to a single mode makes that mode
too unpredictable to be useful.

========== Marc's proposal ==========================================

We would like to solicit your opinion of the following proposal for
support of additional floating point formats and of fused-multiply
adds.  The proposal may be within the realm of the possible for
the next JVM release from Sun, as it requires little additional
implementation work. It seems to achieve what IBM is after, namely
allowing the use of fused-multiply-adds, with no major code changes.
But it also has some drawbacks. A quick (one week) turnaround is highly
desirable.

-------------

The current JVM 1.2 spec supports two double formats: IEEE format with
53 bit mantissa, 12 bit exponent; and a default format, that has
exactly the same mantissa, but may have a larger exponent.  Internal
values (not visible to the user) can be held in the default format, and
converted to the IEEE format when assigned to program variables. If a
method is declared "strict", then the use of large exponents is
disallowed.  A similar design is used  for floats.  This design enables
Intel to take (partial) advantage of its 80 bit floating point
registers, using the mode where only 53 bit mantissas are used in
floating point operations.  It does not allow Power PC to take
advantage of the fused multiply add, and does not allow the use of full
sized mantissas in the 80 bit Intel floating point registers.

Java Grande has discussed several proposals to remediate this problem.
To be adopted, a proposal should

1. require limited implementation effort, and negligible implementation
effort on platforms that are not affected by current Java floating
point restrictions.  

2. introduce only limited platform dependence (similar to the platform
dependence due to the default/strictfp design).

3. not worsen accuracy 

4. improve performance of codes without required significant,
platform-dependent, code rewrite.

The proposal outlined below seems to achieve these four goals. An
"extended" floating point mode is added, where larger mantissas are
allowed, in the same circumstances where larger exponents are allowed
in JVM 1.2: internal values can be stored with more digits of precision
than required by IEEE format, but need be converted back to IEEE format
when assigned to program variables.  This solution would allow (i) the
use of fused multiply adds, where the intermediary output from the
multiplication is not rounded.  (ii) the promotion of floats to
doubles, on machines where double arithmetic is faster (iii) the use of
full 80 bit precision on Intel machines -- at least for internal
values.

We can have two versions of this design:  V1. The use of the "extended"
format would be allowed only for methods (and classes) that were
explicitly tagged with the keyword "extendedfp" in the source.  The
byte file will contain an extendedfp flag. Otherwise, the rules of JVM
1.2 apply. We then have two floating point modes: strictfp and
extendedfp (in addition to the default mode).  V2. "extended" format
can be used in default mode -- no new keyword.

Note that the user can always prohibit the use of extended format by
explicitly naming each internal value (replace x=a*b+c with x1=a*b;
x=x1+c), by using strictfp, or, if V1 is adopted, just by avoiding the
use of the extendedfp attribute.  Note, too, that the implementation
effort is minimal, for implementations that do not take advantage of
the extended format: the Java compilers need generate a new flag, but
this flag may be ingored by java interpreters and JITs.

A downside of this design is that it is unlikely that Sun will add new
keywords and new floating point modes. Thus, either (i) we agree
upfront what extendedfp entails (does it allow usage of associativity,
as proposed with "associativefp"?), or (ii) we agree that extendedfp
may add new relaxations, over time, or (iii) we agree that none will be
added. (i) is unlikely, due to short time.

======== Some comments from Tim =====================================

I haven't been able to envision a proposal that I think is workable
(short of the trivial but relatively clumsy addition of fma methods to
the Math libraries).

I think that I finally understand the original JG proposal for fused
mac support.  The critical issue there is that if you have fused macs
turned on then you get them consistently.  You might not get the same
answer as you would have with fused mac off, but your answer will be
reproducible.  If you don't want that variance across implementations
or platforms you can turn off fused macs, again consistently.  There
remains to be argued whether JG thinks that fused macs can only be used
where they can be tied to the source.  I think that must be so, else
you can't retain reproducibility in the presence of JITs and other
runtime code generation technologies that may create and destroy
opportunities to fuse mul-adds as they optimize.

But tying fused mac to source means some sort of annotation to binary
classes that indicates mul-adds that can be fused.  Although such a
mechanism could be provided, as Java is actually used a straightforward
mechanism will not produce the desired effect.  People regularly run
with old binary classes, which would not have the annotations, and
might with classes built with compilers that don't annotate (if we
don't mandate the compiler changes).  Source mul-adds cannot be
inferred from JVM instructions alone.  And we can't just tell people to
recompile their apps with the new compiler in order to get all their
classes annotated consistently.  We can't refuse to fuse if we detect
old classes, as they may be loaded dynamically after fusing has taken
place, and we can't dynamically unfuse in that case, as that would lead
to inconsistency within the run.

In short, I don't yet see how to get the consistency of fused mac use
that JG appears to desire.  BTW, as a proposal I like this, I just
don't see how to implement it.

Proposals such as Marc's that rely on programmer directives like
floating-point mode modifiers don't suffer from the same problem.  In
those proposals the programmer specifies where mul-adds are to be
fused.  Those are then fused consistently, and there are few surprises.
Old binaries haven't been marked as fusing, so never fuse, and there
are no surprises.

But this path leads to either many fairly specific floating-point modes
(e.g. "fmafp"), which is unlikely to be acceptable for the Java
language, or fewer modes that each do several things (like Marc's
"extendedfp").  I think there is some chance that we would be able
to add a single new keyword that loosens floating point execution, but
I doubt we would be able to add a lot of them.  So there would be
pressure over time to throw loosened mantissas, FMA, associative FP,
and the kitchen sink onto our looser mode.  I would be surprised if
that was acceptable to JG.

-- Tim