Plea for IEEE help (Was: Problem with Decimal Values)

[Da...@lotus.com]

Scott Boag wrote:
>Shouldn't the number be rounded to the nearest representable number with
>the given precision?
>0.10123456789234567893,
>0.10123456789234568 (old)
>0.1012345678923457 (new)

>OK, yes, you would seem to be right.  Does anybody have an opinion about
>this?  Should the test be changed?  I think this will give us different
>output than XT or SAXON?

>Is this really just a formatting issue?  If I do:
>  "0.10123456789234567893 * 0.10123456789234567893",
>how consistent will the actual numeric result be?

This is all foisted off to the IEEE-754 standard. Unfortunately for all
of us, IEEE standards are not available for free download anywhere, as
far as I have been able to determine. I guess they see selling of
documents as a revenue stream. Is there anyone out there in Xalan-Dev
land who has already purchased the document? If so, could you give us
some practical guidelines on precision in decimal representation?

When I wrote the tests in question (string132-135), I thought that I
should go past the limit to ensure that we behave stably. Another
approach would be to cut back the tests in the "conf" group to just
the number of digits that are *guaranteed* to be generated reliably
per IEEE-754, which will probably vary with the precise numeric
value. I wouldn't be surprised if a power of 2 can be stretched to
more places than an arbitrary number. Similar tests in the "stress"
group would be the ones to go out into the unreliable zone.

This splitting of tests can be done if we get a good reading of
IEEE-754, preferably with a full citation of the applicable
provisions.
.................David Marston

Re: Plea for IEEE help (Was: Problem with Decimal Values)

[James Melton <ja...@cylogix.com>]

It would be great to have the actual standard. Anyone?

I worked from various second-hand resources on the web that all seemed
to agree with each other:
http://babbage.cs.qc.edu/courses/cs341/IEEE-754references.html
http://www.scri.fsu.edu/~jac/MAD3401/Backgrnd/ieee.html
http://www.psc.edu/general/software/packages/ieee/ieee.html
http://cch.loria.fr/documentation/divers/IEEE754/numerical_comp_guide/goldberg1.ps

As long as we don't convert back and forth between double and String
during calculations this code should not screw up the essential
IEEE754-ness of our results. i.e. "0.10123456789234567893 *
0.10123456789234567893" should be as good as it ever was. I'll do a
couple more tests though to make sure :)  Are you interested in
incorporating these kind of tests?

Jim.

David_Marston@lotus.com wrote:
> 
> >Is this really just a formatting issue?  If I do:
> >  "0.10123456789234567893 * 0.10123456789234567893",
> >how consistent will the actual numeric result be?
> 
> Is there anyone out there in Xalan-Dev
> land who has already purchased the document? If so, could you give us
> some practical guidelines on precision in decimal representation?
> 

____________________________________________________________
James Melton                 CyLogix
609.750.5190                 609.750.5100
james.melton@cylogix.com     www.cylogix.com