You are viewing a plain text version of this content. The canonical link for it is here.
Posted to c-dev@axis.apache.org by "Fred Preston (JIRA)" <ax...@ws.apache.org> on 2005/09/28 12:51:47 UTC
[jira] Created: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
-----------------------------------------------------------------------------------
Key: AXISCPP-838
URL: http://issues.apache.org/jira/browse/AXISCPP-838
Project: Axis-C++
Type: Bug
Components: Serialization
Environment: n/a
Reporter: Fred Preston
Using the UnitTest_XSD_float test. If you change the code as follows:-
Was
[40]// Test non-nillable element
[41] xsd__float result = ws->asNonNillableElement((xsd__float)35.353588);
Now
[40]// Test non-nillable element
[41] xsd__float result = ws->asNonNillableElement((xsd__float)555.555);
Then on the wire you get:-
<nonNillableElement>555.554993</nonNillableElement>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
Re: [jira] Created: (AXISCPP-838) The serialisation of 555.555 as a float
produces a value of 555.554993 on the wire.
Posted by Fred Preston <PR...@uk.ibm.com>.
Hi Manohar,
This is obviously to do with rounding errors in the floating point
conversion. There will always be some numbers that will not interpret
exactly to a given length because of the way that the number is held
internally. I would need to work out what 555.555 looked like as a
floating point number, but I'm sure it will not be able to be represented
exactly within the limits of a float type. Thus a certain amount of
accuracy is lost. It could be that we are compounding errors somewhere or
we need to look at how we store and recover numbers. We may also need to
look at how much accuracy is 'actually' available in floating point
numbers.
Regards,
Fred Preston.
Manohar K Chintala <cm...@in.ibm.com>
30/09/2005 09:11
Please respond to "Apache AXIS C Developers List"
To: "Apache AXIS C Developers List" <ax...@ws.apache.org>
cc:
Subject: Re: [jira] Created: (AXISCPP-838) The
serialisation of 555.555 as a float produces a value of 555.554993 on the
wire.
I've observed that when result is displayed using "%.5f" this problem
occurs. If result is displayed using "%.3f" the output is 555.555, and for
"%.4f" the output is proper 555.5550. Also, this problem happens with
1,5,7
and 9 numbers when only. i.e. when the values given like 555.111 ,
555.555, 555.777 and 555.999 this serialization problem occurs and for
other numbers it works fine.
Regards
Manohar
"Fred Preston
(JIRA)"
<axis-c-dev@ws.ap To
ache.org> axis-c-dev@ws.apache.org
cc
09/28/2005 04:21
PM Subject
[jira] Created: (AXISCPP-838) The
serialisation of 555.555 as a float
Please respond to produces a value of 555.554993 on
"Apache AXIS C the wire.
Developers List"
The serialisation of 555.555 as a float produces a value of 555.554993 on
the wire.
-----------------------------------------------------------------------------------
Key: AXISCPP-838
URL: http://issues.apache.org/jira/browse/AXISCPP-838
Project: Axis-C++
Type: Bug
Components: Serialization
Environment: n/a
Reporter: Fred Preston
Using the UnitTest_XSD_float test. If you change the code as follows:-
Was
[40]// Test non-nillable element
[41] xsd__float result =
ws->asNonNillableElement((xsd__float)35.353588);
Now
[40]// Test non-nillable element
[41] xsd__float result =
ws->asNonNillableElement((xsd__float)555.555);
Then on the wire you get:-
<nonNillableElement>555.554993</nonNillableElement>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
Re: [jira] Created: (AXISCPP-838) The serialisation of 555.555 as a float
produces a value of 555.554993 on the wire.
Posted by Manohar K Chintala <cm...@in.ibm.com>.
I've observed that when result is displayed using "%.5f" this problem
occurs. If result is displayed using "%.3f" the output is 555.555, and for
"%.4f" the output is proper 555.5550. Also, this problem happens with 1,5,7
and 9 numbers when only. i.e. when the values given like 555.111 ,
555.555, 555.777 and 555.999 this serialization problem occurs and for
other numbers it works fine.
Regards
Manohar
"Fred Preston
(JIRA)"
<axis-c-dev@ws.ap To
ache.org> axis-c-dev@ws.apache.org
cc
09/28/2005 04:21
PM Subject
[jira] Created: (AXISCPP-838) The
serialisation of 555.555 as a float
Please respond to produces a value of 555.554993 on
"Apache AXIS C the wire.
Developers List"
The serialisation of 555.555 as a float produces a value of 555.554993 on
the wire.
-----------------------------------------------------------------------------------
Key: AXISCPP-838
URL: http://issues.apache.org/jira/browse/AXISCPP-838
Project: Axis-C++
Type: Bug
Components: Serialization
Environment: n/a
Reporter: Fred Preston
Using the UnitTest_XSD_float test. If you change the code as follows:-
Was
[40]// Test non-nillable element
[41] xsd__float result =
ws->asNonNillableElement((xsd__float)35.353588);
Now
[40]// Test non-nillable element
[41] xsd__float result =
ws->asNonNillableElement((xsd__float)555.555);
Then on the wire you get:-
<nonNillableElement>555.554993</nonNillableElement>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Assigned: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
Posted by "Fred Preston (JIRA)" <ax...@ws.apache.org>.
[ http://issues.apache.org/jira/browse/AXISCPP-838?page=all ]
Fred Preston reassigned AXISCPP-838:
------------------------------------
Assign To: Fred Preston
> The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
> -----------------------------------------------------------------------------------
>
> Key: AXISCPP-838
> URL: http://issues.apache.org/jira/browse/AXISCPP-838
> Project: Axis-C++
> Type: Bug
> Components: Serialization
> Environment: n/a
> Reporter: Fred Preston
> Assignee: Fred Preston
>
> Using the UnitTest_XSD_float test. If you change the code as follows:-
> Was
> [40]// Test non-nillable element
> [41] xsd__float result = ws->asNonNillableElement((xsd__float)35.353588);
> Now
> [40]// Test non-nillable element
> [41] xsd__float result = ws->asNonNillableElement((xsd__float)555.555);
> Then on the wire you get:-
> <nonNillableElement>555.554993</nonNillableElement>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Resolved: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
Posted by "Fred Preston (JIRA)" <ax...@ws.apache.org>.
[ http://issues.apache.org/jira/browse/AXISCPP-838?page=all ]
Fred Preston resolved AXISCPP-838:
----------------------------------
Resolution: Fixed
Replaced %f with %.6g in float.cpp and %f with %.10g in double.cpp.
> The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
> -----------------------------------------------------------------------------------
>
> Key: AXISCPP-838
> URL: http://issues.apache.org/jira/browse/AXISCPP-838
> Project: Axis-C++
> Type: Bug
> Components: Serialization
> Environment: n/a
> Reporter: Fred Preston
> Assignee: Fred Preston
>
> Using the UnitTest_XSD_float test. If you change the code as follows:-
> Was
> [40]// Test non-nillable element
> [41] xsd__float result = ws->asNonNillableElement((xsd__float)35.353588);
> Now
> [40]// Test non-nillable element
> [41] xsd__float result = ws->asNonNillableElement((xsd__float)555.555);
> Then on the wire you get:-
> <nonNillableElement>555.554993</nonNillableElement>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
Posted by "Fred Preston (JIRA)" <ax...@ws.apache.org>.
[ http://issues.apache.org/jira/browse/AXISCPP-838?page=comments#action_12358351 ]
Fred Preston commented on AXISCPP-838:
--------------------------------------
Precision problems with floats
------------------------------------------
Serialisation
-----------------
Before trying to resolve anything, I though I would do some investigation into floats... I first created lots of float values around the 555.555 problem area to see how this number was being represented...
Code "%f" Produced "%.7g" Produced "%.8g" Produced
xsd__float myFloat00 = 555.5 + 0.0001; 555.500122 555.5001 555.50012
xsd__float myFloat01 = 555.55 + 0.0001; 555.550110 555.5501 555.55011
xsd__float myFloat02 = 555.555; 555.554993 555.555 555.55499
xsd__float myFloat020 = 555.555 + 0.1; 555.655029 555.655 555.65503
xsd__float myFloat021 = 555.555 + 0.01; 555.565002 555.565 555.565
xsd__float myFloat022 = 555.555 + 0.001; 555.556030 555.556 555.55603
xsd__float myFloat023 = 555.555 + 0.0001; 555.555115 555.5551 555.55511
xsd__float myFloat024 = 555.555 + 0.00001; 555.554993 555.555 555.55499
xsd__float myFloat03 = 555.5555 + 0.0001; 555.555603 555.5556 555.5556
xsd__float myFloat04 = 555.55555 + 0.0001; 555.555664 555.5557 555.55566
xsd__float myFloat05 = 555.556 + 0.0001; 555.556091 555.5561 555.55609
xsd__float myFloat06 = -555.5 - 0.0001; -555.500122 -555.5001 -555.50012
xsd__float myFloat07 = -555.55 - 0.0001; -555.550110 -555.5501 -555.55011
xsd__float myFloat08 = -555.555 - 0.0001; -555.555115 -555.5551 -555.55511
xsd__float myFloat09 = -555.5555 - 0.0001; -555.555603 -555.5556 -555.5556
xsd__float myFloat10 = -555.55555 - 0.0001; -555.555664 -555.5557 -555.55566
xsd__float myFloat11 = -555.556 - 0.0001; -555.556091 -555.5561 -555.55609
xsd__float myFloat12 = 555.5; 555.500000 555.5 555.5
xsd__float myFloat13 = 555.55; (5) 555.55
(6) 555.55
(7) 555.55
(8) 555.54999
xsd__float myFloat14 = 555.555; (6) 555.555
(7) 555.555
(8) 555.55499
xsd__float myFloat15 = 555.5555; (7) 555.5555
xsd__float myFloat16 = 555.55499; (6) 555.555 <- This has rounded up rather than truncated.
(7) 555.555 <- This has rounded up rather than truncated.
(8) 555.55499
xsd__float myFloat17 = 555.5499; (6) 555.55 <- This has rounded up rather than truncated.
(7) 555.5499
(8) 555.54993
The best guaranteed precision without any rounding appears to be 5 digits and with rounding 6 (although I would need to confirm this).
Reading through the IEEE numeric standards, types have the following precision:-
float = sign bit, 8 bit exponent, 23 bit mantissa
double = sign bit, 11 bit exponent, 52 bit mantissa
long double = sign bit, 15 bit exponent, 64 bit mantissa
For float this would imply a precision of 1/2^22 (or 0.00000011920928955078125) or a maximum of 6 significant digits. This backs up what I've seen experimentally. I have also seen that 555.55499 (myFloat16) was rounded up by the precision specification "%.6g" to 555.555 and not truncated to 555.554.
Bits Decimal
00 0.5
01 0.25
02 0.125
03 0.0625
04 0.03125
05 0.015625
06 0.0078125
07 0.00390625
08 0.001953125
09 0.0009765625
10 0.00048828125
11 0.000244140625
12 0.0001220703125
13 0.00006103515625
14 0.000030517578125
15 0.0000152587890625
16 0.00000762939453125
17 0.000003814697265625
18 0.0000019073486328125
19 0.00000095367431640625
20 0.000000476837158203125
21 0.0000002384185791015625
22 0.00000011920928955078125
My recommendation is that for serialisation of a float (this may also be necessary for double and decimal), we use the following logic...
char * FloatToThreeDecimalPlaces( xsd__float myFloat)
{
int sign = 1;
if( myFloat < 0)
{
sign = -1;
myFloat = -myFloat;
}
int whole = (int) myFloat;
int fraction = (int) ((myFloat - (xsd__float) whole + (xsd__float) 0.00005) * (xsd__float) 1000.0);
static char szNumber[16];
sprintf( szNumber, "%d.%03d", whole * sign, fraction);
char * pZero = szNumber + strlen( szNumber) - 1;
while( *pZero != '.' && *pZero == '0')
{
*pZero = '\0';
pZero--;
}
if( *pZero == '.')
{
*pZero = '\0';
}
return szNumber;
}
Deserialisation
---------------------
We are using 'strtod' which appears to be working satisfactorily. It should be noted though that there will be some problems converting the output (which is double) to a float!
> The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
> -----------------------------------------------------------------------------------
>
> Key: AXISCPP-838
> URL: http://issues.apache.org/jira/browse/AXISCPP-838
> Project: Axis-C++
> Type: Bug
> Components: Serialization
> Environment: n/a
> Reporter: Fred Preston
> Assignee: Fred Preston
>
> Using the UnitTest_XSD_float test. If you change the code as follows:-
> Was
> [40]// Test non-nillable element
> [41] xsd__float result = ws->asNonNillableElement((xsd__float)35.353588);
> Now
> [40]// Test non-nillable element
> [41] xsd__float result = ws->asNonNillableElement((xsd__float)555.555);
> Then on the wire you get:-
> <nonNillableElement>555.554993</nonNillableElement>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Closed: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
Posted by "Fred Preston (JIRA)" <ax...@ws.apache.org>.
[ http://issues.apache.org/jira/browse/AXISCPP-838?page=all ]
Fred Preston closed AXISCPP-838:
--------------------------------
> The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
> -----------------------------------------------------------------------------------
>
> Key: AXISCPP-838
> URL: http://issues.apache.org/jira/browse/AXISCPP-838
> Project: Axis-C++
> Type: Bug
> Components: Serialization
> Environment: n/a
> Reporter: Fred Preston
> Assignee: Fred Preston
>
> Using the UnitTest_XSD_float test. If you change the code as follows:-
> Was
> [40]// Test non-nillable element
> [41] xsd__float result = ws->asNonNillableElement((xsd__float)35.353588);
> Now
> [40]// Test non-nillable element
> [41] xsd__float result = ws->asNonNillableElement((xsd__float)555.555);
> Then on the wire you get:-
> <nonNillableElement>555.554993</nonNillableElement>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
Posted by "Fred Preston (JIRA)" <ax...@ws.apache.org>.
[ http://issues.apache.org/jira/browse/AXISCPP-838?page=comments#action_12358364 ]
Fred Preston commented on AXISCPP-838:
--------------------------------------
A post script to the above comment is the following:-
When you use:-
xsd__float myFloat = 555.555;
This will be represented in the float type as 555.55499. Thus, if your code prints out myFloat using:-
printf( "%f\n", myFloat);
you will get:-
555.55499
But, we have said that float only has an accuracy of 6 digits, so we have to use the correct formatting to get the correct result. To do this, replace %f with %.6g for a float and %.10g for a double. Thus, if we have:-
printf( "%.6g", myFloat);
you will get:-
555.555
Giving the correct result to the proper resolution.
> The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
> -----------------------------------------------------------------------------------
>
> Key: AXISCPP-838
> URL: http://issues.apache.org/jira/browse/AXISCPP-838
> Project: Axis-C++
> Type: Bug
> Components: Serialization
> Environment: n/a
> Reporter: Fred Preston
> Assignee: Fred Preston
>
> Using the UnitTest_XSD_float test. If you change the code as follows:-
> Was
> [40]// Test non-nillable element
> [41] xsd__float result = ws->asNonNillableElement((xsd__float)35.353588);
> Now
> [40]// Test non-nillable element
> [41] xsd__float result = ws->asNonNillableElement((xsd__float)555.555);
> Then on the wire you get:-
> <nonNillableElement>555.554993</nonNillableElement>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
Posted by "Fred Preston (JIRA)" <ax...@ws.apache.org>.
[ http://issues.apache.org/jira/browse/AXISCPP-838?page=comments#action_12358360 ]
Fred Preston commented on AXISCPP-838:
--------------------------------------
I have tried to compromise on this. The XML spec for floats refers back to the IEEE specifications. This is defined as follows:-
float = sign bit, 8 bit exponent, 23 bit mantissa
double = sign bit, 11 bit exponent, 52 bit mantissa
long double = sign bit, 15 bit exponent, 64 bit mantissa
Float should be able to support 6 significant digits without rounding errors. Thus 555.555 will be supported, but 555.5549 will not (although it may be able to accuately represent this number). By replacing
AxisSprintf (serializedValue, 80, "%f", *value);
with
AxisSprintf (serializedValue, 80, "%.6g", *value);
in float.cpp, xsd__float should cope with 6 significant digits, but will round up if more digits are supplied. For example,
555.555 will be serialised as 555.555 but 555.5549 will also be serialised as 555.555.
This needs to be raised as a note, but this is a restriction of the XML/IEEE specification.
> The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
> -----------------------------------------------------------------------------------
>
> Key: AXISCPP-838
> URL: http://issues.apache.org/jira/browse/AXISCPP-838
> Project: Axis-C++
> Type: Bug
> Components: Serialization
> Environment: n/a
> Reporter: Fred Preston
> Assignee: Fred Preston
>
> Using the UnitTest_XSD_float test. If you change the code as follows:-
> Was
> [40]// Test non-nillable element
> [41] xsd__float result = ws->asNonNillableElement((xsd__float)35.353588);
> Now
> [40]// Test non-nillable element
> [41] xsd__float result = ws->asNonNillableElement((xsd__float)555.555);
> Then on the wire you get:-
> <nonNillableElement>555.554993</nonNillableElement>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira