You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@xalan.apache.org by bu...@apache.org on 2003/08/21 22:24:01 UTC
DO NOT REPLY [Bug 21534] -
Floating point output incorrect
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://nagoya.apache.org/bugzilla/show_bug.cgi?id=21534>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND
INSERTED IN THE BUG DATABASE.
http://nagoya.apache.org/bugzilla/show_bug.cgi?id=21534
Floating point output incorrect
------- Additional Comments From mhoyt@ca.ibm.com 2003-08-21 20:24 -------
After some investigation, there does not seem to be a quick and efficient fix
to this problem.
- The brute force approach would be to simply complete the entries in the table
i.e. add format strings for 1-9 digits precision and 36+ digits of precision
and continue the process of doing a linear search until the correct level of
precision is reached.
- A performance improvement on the above approach would be to implement a
binary search. E.g. start with 10 digits of precision, if 10 is sufficient -try
5 digits of precision, if not try 122 digits of precision, etc. This approach
would still require at least 2 comparisons though.
- Another solution would be to implement David M. Gay's dtoa which is an
efficient and accurate double to string conversion routine.
- The Java processor does not have these problems because it has a different
implementation and can rely on the system class library to convert the douuble
to the shortest unique value.