You are viewing a plain text version of this content. The canonical link for it is here.
Posted to c-dev@axis.apache.org by "Fred Preston (JIRA)" <ax...@ws.apache.org> on 2005/09/28 12:51:47 UTC

[jira] Created: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.

The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
-----------------------------------------------------------------------------------

         Key: AXISCPP-838
         URL: http://issues.apache.org/jira/browse/AXISCPP-838
     Project: Axis-C++
        Type: Bug
  Components: Serialization  
 Environment: n/a
    Reporter: Fred Preston


Using the UnitTest_XSD_float test.  If you change the code as follows:-

Was
[40]// Test non-nillable element
[41]	    xsd__float result = ws->asNonNillableElement((xsd__float)35.353588);

Now
[40]// Test non-nillable element
[41]	    xsd__float result = ws->asNonNillableElement((xsd__float)555.555);

Then on the wire you get:-
<nonNillableElement>555.554993</nonNillableElement>


-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Re: [jira] Created: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.

Posted by Fred Preston <PR...@uk.ibm.com>.
Hi Manohar,
        This is obviously to do with rounding errors in the floating point 
conversion.  There will always be some numbers that will not interpret 
exactly to a given length because of the way that the number is held 
internally.  I would need to work out what 555.555 looked like as a 
floating point number, but I'm sure it will not be able to be represented 
exactly within the limits of a float type.  Thus a certain amount of 
accuracy is lost.  It could be that we are compounding errors somewhere or 
we need to look at how we store and recover numbers.  We may also need to 
look at how much accuracy is 'actually' available in floating point 
numbers.

Regards,

Fred Preston.





Manohar K Chintala <cm...@in.ibm.com>
30/09/2005 09:11
Please respond to "Apache AXIS C Developers List"
 
        To:     "Apache AXIS C Developers List" <ax...@ws.apache.org>
        cc: 
        Subject:        Re: [jira] Created: (AXISCPP-838) The 
serialisation of 555.555 as a float produces a value of 555.554993 on the 
wire.

 


I've observed that when result is displayed using "%.5f" this problem
occurs. If result is displayed using "%.3f" the output is 555.555, and for
"%.4f" the output is proper 555.5550. Also, this problem happens with 
1,5,7
and 9 numbers when only. i.e. when the values given like  555.111 ,
555.555, 555.777 and 555.999 this serialization problem occurs and for
other numbers it works fine.

Regards
Manohar



 
             "Fred Preston 
             (JIRA)" 
             <axis-c-dev@ws.ap                                          To 

             ache.org>                 axis-c-dev@ws.apache.org 
                                                                        cc 

             09/28/2005 04:21 
             PM                                                    Subject 

                                       [jira] Created: (AXISCPP-838) The 
                                       serialisation of 555.555 as a float 

             Please respond to         produces a value of 555.554993 on 
              "Apache AXIS C           the wire. 
             Developers List" 
 
 
 
 
 




The serialisation of 555.555 as a float produces a value of 555.554993 on
the wire.
-----------------------------------------------------------------------------------


         Key: AXISCPP-838
         URL: http://issues.apache.org/jira/browse/AXISCPP-838
     Project: Axis-C++
        Type: Bug
  Components: Serialization
 Environment: n/a
    Reporter: Fred Preston


Using the UnitTest_XSD_float test.  If you change the code as follows:-

Was
[40]// Test non-nillable element
[41]             xsd__float result =
ws->asNonNillableElement((xsd__float)35.353588);

Now
[40]// Test non-nillable element
[41]             xsd__float result =
ws->asNonNillableElement((xsd__float)555.555);

Then on the wire you get:-
<nonNillableElement>555.554993</nonNillableElement>


--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira





Re: [jira] Created: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.

Posted by Manohar K Chintala <cm...@in.ibm.com>.
I've observed that when result is displayed using "%.5f" this problem
occurs. If result is displayed using "%.3f" the output is 555.555, and for
"%.4f" the output is proper 555.5550. Also, this problem happens with 1,5,7
and 9 numbers when only. i.e. when the values given like  555.111 ,
555.555, 555.777 and 555.999 this serialization problem occurs and for
other numbers it works fine.

Regards
Manohar



                                                                           
             "Fred Preston                                                 
             (JIRA)"                                                       
             <axis-c-dev@ws.ap                                          To 
             ache.org>                 axis-c-dev@ws.apache.org            
                                                                        cc 
             09/28/2005 04:21                                              
             PM                                                    Subject 
                                       [jira] Created: (AXISCPP-838) The   
                                       serialisation of 555.555 as a float 
             Please respond to         produces a value of 555.554993 on   
              "Apache AXIS C           the wire.                           
             Developers List"                                              
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           




The serialisation of 555.555 as a float produces a value of 555.554993 on
the wire.
-----------------------------------------------------------------------------------


         Key: AXISCPP-838
         URL: http://issues.apache.org/jira/browse/AXISCPP-838
     Project: Axis-C++
        Type: Bug
  Components: Serialization
 Environment: n/a
    Reporter: Fred Preston


Using the UnitTest_XSD_float test.  If you change the code as follows:-

Was
[40]// Test non-nillable element
[41]             xsd__float result =
ws->asNonNillableElement((xsd__float)35.353588);

Now
[40]// Test non-nillable element
[41]             xsd__float result =
ws->asNonNillableElement((xsd__float)555.555);

Then on the wire you get:-
<nonNillableElement>555.554993</nonNillableElement>


--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira




[jira] Assigned: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.

Posted by "Fred Preston (JIRA)" <ax...@ws.apache.org>.
     [ http://issues.apache.org/jira/browse/AXISCPP-838?page=all ]

Fred Preston reassigned AXISCPP-838:
------------------------------------

    Assign To: Fred Preston

> The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
> -----------------------------------------------------------------------------------
>
>          Key: AXISCPP-838
>          URL: http://issues.apache.org/jira/browse/AXISCPP-838
>      Project: Axis-C++
>         Type: Bug
>   Components: Serialization
>  Environment: n/a
>     Reporter: Fred Preston
>     Assignee: Fred Preston

>
> Using the UnitTest_XSD_float test.  If you change the code as follows:-
> Was
> [40]// Test non-nillable element
> [41]	    xsd__float result = ws->asNonNillableElement((xsd__float)35.353588);
> Now
> [40]// Test non-nillable element
> [41]	    xsd__float result = ws->asNonNillableElement((xsd__float)555.555);
> Then on the wire you get:-
> <nonNillableElement>555.554993</nonNillableElement>

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


[jira] Resolved: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.

Posted by "Fred Preston (JIRA)" <ax...@ws.apache.org>.
     [ http://issues.apache.org/jira/browse/AXISCPP-838?page=all ]
     
Fred Preston resolved AXISCPP-838:
----------------------------------

    Resolution: Fixed

Replaced %f with %.6g in float.cpp and %f with %.10g in double.cpp.

> The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
> -----------------------------------------------------------------------------------
>
>          Key: AXISCPP-838
>          URL: http://issues.apache.org/jira/browse/AXISCPP-838
>      Project: Axis-C++
>         Type: Bug
>   Components: Serialization
>  Environment: n/a
>     Reporter: Fred Preston
>     Assignee: Fred Preston

>
> Using the UnitTest_XSD_float test.  If you change the code as follows:-
> Was
> [40]// Test non-nillable element
> [41]	    xsd__float result = ws->asNonNillableElement((xsd__float)35.353588);
> Now
> [40]// Test non-nillable element
> [41]	    xsd__float result = ws->asNonNillableElement((xsd__float)555.555);
> Then on the wire you get:-
> <nonNillableElement>555.554993</nonNillableElement>

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


[jira] Commented: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.

Posted by "Fred Preston (JIRA)" <ax...@ws.apache.org>.
    [ http://issues.apache.org/jira/browse/AXISCPP-838?page=comments#action_12358351 ] 

Fred Preston commented on AXISCPP-838:
--------------------------------------

Precision problems with floats
------------------------------------------
Serialisation
-----------------
Before trying to resolve anything, I though I would do some investigation into floats...  I first created lots of float values around the 555.555 problem area to see how this number was being represented...

Code                                                                       "%f" Produced     "%.7g" Produced  "%.8g" Produced
xsd__float      myFloat00 = 555.5 + 0.0001;           555.500122           555.5001           555.50012
xsd__float      myFloat01 = 555.55 + 0.0001;         555.550110           555.5501           555.55011
xsd__float      myFloat02 = 555.555;                       555.554993           555.555              555.55499
xsd__float      myFloat020 = 555.555 + 0.1;           555.655029           555.655              555.65503
xsd__float      myFloat021 = 555.555 + 0.01;         555.565002           555.565              555.565
xsd__float      myFloat022 = 555.555 + 0.001;       555.556030           555.556             555.55603
xsd__float      myFloat023 = 555.555 + 0.0001;    555.555115           555.5551           555.55511
xsd__float      myFloat024 = 555.555 + 0.00001;  555.554993           555.555             555.55499
xsd__float      myFloat03 = 555.5555 + 0.0001;    555.555603           555.5556           555.5556
xsd__float      myFloat04 = 555.55555 + 0.0001;  555.555664           555.5557           555.55566
xsd__float      myFloat05 = 555.556 + 0.0001;       555.556091           555.5561           555.55609
xsd__float      myFloat06 = -555.5 - 0.0001;          -555.500122         -555.5001           -555.50012
xsd__float      myFloat07 = -555.55 - 0.0001;        -555.550110         -555.5501           -555.55011
xsd__float      myFloat08 = -555.555 - 0.0001;     -555.555115         -555.5551            -555.55511
xsd__float      myFloat09 = -555.5555 - 0.0001;   -555.555603         -555.5556           -555.5556
xsd__float      myFloat10 = -555.55555 - 0.0001; -555.555664         -555.5557           -555.55566
xsd__float      myFloat11 = -555.556 - 0.0001;      -555.556091         -555.5561           -555.55609
xsd__float      myFloat12 = 555.5;                             555.500000          555.5                   555.5
xsd__float      myFloat13 = 555.55;           (5)           555.55
                                                                         (6)           555.55
                                                                         (7)           555.55
                                                                         (8)           555.54999
xsd__float      myFloat14 = 555.555;         (6)           555.555
                                                                         (7)           555.555
                                                                         (8)           555.55499
xsd__float      myFloat15 = 555.5555;       (7)           555.5555
xsd__float      myFloat16 = 555.55499;     (6)           555.555            <- This has rounded up rather than truncated.
                                                                          (7)           555.555            <- This has rounded up rather than truncated.
                                                                         (8)           555.55499
xsd__float      myFloat17 = 555.5499;       (6)           555.55            <- This has rounded up rather than truncated.
                                                                          (7)           555.5499
                                                                          (8)           555.54993

The best guaranteed precision without any rounding appears to be 5 digits and with rounding 6 (although I would need to confirm this).

Reading through the IEEE numeric standards, types have the following precision:-
float =  sign bit, 8  bit exponent, 23 bit mantissa
double =  sign bit, 11 bit exponent, 52 bit mantissa
long double = sign bit, 15 bit exponent, 64 bit mantissa

For float this would imply a precision of 1/2^22 (or 0.00000011920928955078125) or a maximum of 6 significant digits.  This backs up what I've seen experimentally.  I have also seen that 555.55499 (myFloat16) was rounded up by the precision specification "%.6g" to 555.555 and not truncated to 555.554.

Bits Decimal
00   0.5
01   0.25
02   0.125
03   0.0625
04   0.03125
05   0.015625
06   0.0078125
07   0.00390625
08   0.001953125
09   0.0009765625
10   0.00048828125
11   0.000244140625
12   0.0001220703125
13   0.00006103515625
14   0.000030517578125
15   0.0000152587890625
16   0.00000762939453125
17   0.000003814697265625
18   0.0000019073486328125
19   0.00000095367431640625
20   0.000000476837158203125
21   0.0000002384185791015625
22   0.00000011920928955078125

My recommendation is that for serialisation of a float (this may also be necessary for double and decimal), we use the following logic...

char * FloatToThreeDecimalPlaces( xsd__float myFloat)
{
    int                 sign = 1;

    if( myFloat < 0)
    {
        sign = -1;
        myFloat = -myFloat;
    }

    int                 whole = (int) myFloat;
    int                  fraction = (int) ((myFloat - (xsd__float) whole + (xsd__float) 0.00005) * (xsd__float) 1000.0);
    static char   szNumber[16];

    sprintf( szNumber, "%d.%03d", whole * sign, fraction);

    char *    pZero = szNumber + strlen( szNumber) - 1;

    while( *pZero != '.' && *pZero == '0')
    {
        *pZero = '\0';
        pZero--;
    }

    if( *pZero == '.')
    {
        *pZero = '\0';
    }

    return szNumber;
}

Deserialisation
---------------------
We are using 'strtod' which appears to be working satisfactorily.  It should be noted though that there will be some problems converting the output (which is double) to a float!


> The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
> -----------------------------------------------------------------------------------
>
>          Key: AXISCPP-838
>          URL: http://issues.apache.org/jira/browse/AXISCPP-838
>      Project: Axis-C++
>         Type: Bug
>   Components: Serialization
>  Environment: n/a
>     Reporter: Fred Preston
>     Assignee: Fred Preston

>
> Using the UnitTest_XSD_float test.  If you change the code as follows:-
> Was
> [40]// Test non-nillable element
> [41]	    xsd__float result = ws->asNonNillableElement((xsd__float)35.353588);
> Now
> [40]// Test non-nillable element
> [41]	    xsd__float result = ws->asNonNillableElement((xsd__float)555.555);
> Then on the wire you get:-
> <nonNillableElement>555.554993</nonNillableElement>

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


[jira] Closed: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.

Posted by "Fred Preston (JIRA)" <ax...@ws.apache.org>.
     [ http://issues.apache.org/jira/browse/AXISCPP-838?page=all ]
     
Fred Preston closed AXISCPP-838:
--------------------------------


> The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
> -----------------------------------------------------------------------------------
>
>          Key: AXISCPP-838
>          URL: http://issues.apache.org/jira/browse/AXISCPP-838
>      Project: Axis-C++
>         Type: Bug
>   Components: Serialization
>  Environment: n/a
>     Reporter: Fred Preston
>     Assignee: Fred Preston

>
> Using the UnitTest_XSD_float test.  If you change the code as follows:-
> Was
> [40]// Test non-nillable element
> [41]	    xsd__float result = ws->asNonNillableElement((xsd__float)35.353588);
> Now
> [40]// Test non-nillable element
> [41]	    xsd__float result = ws->asNonNillableElement((xsd__float)555.555);
> Then on the wire you get:-
> <nonNillableElement>555.554993</nonNillableElement>

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


[jira] Commented: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.

Posted by "Fred Preston (JIRA)" <ax...@ws.apache.org>.
    [ http://issues.apache.org/jira/browse/AXISCPP-838?page=comments#action_12358364 ] 

Fred Preston commented on AXISCPP-838:
--------------------------------------

A post script to the above comment is the following:-

When you use:-

xsd__float myFloat = 555.555;

This will be represented in the float type as 555.55499.  Thus, if your code prints out myFloat using:-

printf( "%f\n", myFloat);

you will get:-

555.55499

But, we have said that float only has an accuracy of 6 digits, so we have to use the correct formatting to get the correct result.  To do this, replace %f with %.6g for a float and %.10g for a double.  Thus, if we have:-

printf( "%.6g", myFloat);

you will get:-

555.555

Giving the correct result to the proper resolution.


> The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
> -----------------------------------------------------------------------------------
>
>          Key: AXISCPP-838
>          URL: http://issues.apache.org/jira/browse/AXISCPP-838
>      Project: Axis-C++
>         Type: Bug
>   Components: Serialization
>  Environment: n/a
>     Reporter: Fred Preston
>     Assignee: Fred Preston

>
> Using the UnitTest_XSD_float test.  If you change the code as follows:-
> Was
> [40]// Test non-nillable element
> [41]	    xsd__float result = ws->asNonNillableElement((xsd__float)35.353588);
> Now
> [40]// Test non-nillable element
> [41]	    xsd__float result = ws->asNonNillableElement((xsd__float)555.555);
> Then on the wire you get:-
> <nonNillableElement>555.554993</nonNillableElement>

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


[jira] Commented: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.

Posted by "Fred Preston (JIRA)" <ax...@ws.apache.org>.
    [ http://issues.apache.org/jira/browse/AXISCPP-838?page=comments#action_12358360 ] 

Fred Preston commented on AXISCPP-838:
--------------------------------------

I have tried to compromise on this.  The XML spec for floats refers back to the IEEE specifications.  This is defined as follows:-
float               = sign bit, 8  bit exponent, 23 bit mantissa
double          = sign bit, 11 bit exponent, 52 bit mantissa
long double = sign bit, 15 bit exponent, 64 bit mantissa

Float should be able to support 6 significant digits without rounding errors.  Thus 555.555 will be supported, but 555.5549 will not (although it may be able to accuately represent this number).  By replacing

    AxisSprintf (serializedValue, 80, "%f", *value);

with

    AxisSprintf (serializedValue, 80, "%.6g", *value);

in float.cpp, xsd__float should cope with 6 significant digits, but will round up if more digits are supplied.  For example,

555.555 will be serialised as 555.555 but 555.5549 will also be serialised as 555.555.

This needs to be raised as a note, but this is a restriction of the XML/IEEE specification.

> The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
> -----------------------------------------------------------------------------------
>
>          Key: AXISCPP-838
>          URL: http://issues.apache.org/jira/browse/AXISCPP-838
>      Project: Axis-C++
>         Type: Bug
>   Components: Serialization
>  Environment: n/a
>     Reporter: Fred Preston
>     Assignee: Fred Preston

>
> Using the UnitTest_XSD_float test.  If you change the code as follows:-
> Was
> [40]// Test non-nillable element
> [41]	    xsd__float result = ws->asNonNillableElement((xsd__float)35.353588);
> Now
> [40]// Test non-nillable element
> [41]	    xsd__float result = ws->asNonNillableElement((xsd__float)555.555);
> Then on the wire you get:-
> <nonNillableElement>555.554993</nonNillableElement>

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira