You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@xalan.apache.org by bu...@apache.org on 2003/07/12 01:04:46 UTC
DO NOT REPLY [Bug 21534] New: - Floating point output incorrect

DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG 
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://nagoya.apache.org/bugzilla/show_bug.cgi?id=21534>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND 
INSERTED IN THE BUG DATABASE.

http://nagoya.apache.org/bugzilla/show_bug.cgi?id=21534

Floating point output incorrect

           Summary: Floating point output incorrect
           Product: XalanC
           Version: 1.5
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: Minor
          Priority: Other
         Component: XalanC
        AssignedTo: xalan-dev@xml.apache.org
        ReportedBy: mhoyt@ca.ibm.com


Not sure if these should be logged as seperate bugs, but the source of the 
errors has been traced to limitations of the sprintf/"format string lookup 
table" of the DomStringHelper::DoubleToDOMString() method.

(1) From conformance test 'string134' (simplified, source xml is irrelevant):
  <xsl:template match="/">
    <xsl:value-of select="987654321.012345"/>
  </xsl:template>

Produces output:
  987654321.01234496

Why? The result number is the full decimal value of the binary approximation. 
In this case, the input's fraction only has 6 digits but the conversion method 
defaults to at least 10 digits of precision, resulting in the invalid output of 
extra values from the binary approximation.  The output precision should be the 
same as the original value's.

(2) From conformance test 'string135' (simplified, source xml is irrelevant):
  <xsl:template match="/">
    <xsl:value-of select=".000000000000000000000000000000000000123456789"/>
  </xsl:template>

Produces output:
  0

Why? The conversion method will output 0 for values < 1.0e-35.  According to 
the the XPath spec a number can be any double-precision 64-bit format IEEE 754 
number.  This all means values > 1.0e-323 should be displayed.  Currently, the 
conversion function is limited to creating strings with numbers containing at 
most 35 digits of precision.

XalanJ 2.5.1, Saxon 6.5.2 and MSXSL 3.0 behave correctly in these cases.