String conversion to Double

[Eldon Carman <ec...@ucr.edu>]

Not sure if we discussed this before, but it keeps coming up in our tests.

Several of the XQTS tests fail for extremely large or small value numbers
being converted to a double form a string. The issue lies in our custom
parse double code. (Its not a simple algorithm to convert a string to a
double without losing precision.) The cast function's argument is a byte
array that hold a UTF8 string. Java offers a function that parses a string
and returns a double. Using the java function would give us greater
precision than our current code but would require converting the byte array
to a string object.

Do you know of an parseDouble function that could be modified to fit our
use case?

Or do you think it would be ok to break our object avoidance rule in this
case for better precision?

Thanks
Preston

Re: String conversion to Double

[Till Westmann <ti...@apache.org>]

Hi Preston,

I think that we should move to the JDK code for casting to double and float. We are using it in other places (e.g. in the XQuery parser) and so if our own implementation produces different results, we are actually inconsistent within VXQuery.
If we find out at some point that this is a performance issue, we can re-visit this, but I think that won’t be soon.

Cheers,
Till


> On Mar 27, 2015, at 11:46 AM, Eldon Carman <ec...@ucr.edu> wrote:
> 
> Not sure if we discussed this before, but it keeps coming up in our tests.
> 
> Several of the XQTS tests fail for extremely large or small value numbers
> being converted to a double form a string. The issue lies in our custom
> parse double code. (Its not a simple algorithm to convert a string to a
> double without losing precision.) The cast function's argument is a byte
> array that hold a UTF8 string. Java offers a function that parses a string
> and returns a double. Using the java function would give us greater
> precision than our current code but would require converting the byte array
> to a string object.
> 
> Do you know of an parseDouble function that could be modified to fit our
> use case?
> 
> Or do you think it would be ok to break our object avoidance rule in this
> case for better precision?
> 
> Thanks
> Preston