You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@groovy.apache.org by "Tx. T" <tx...@yahoo.com> on 2017/03/19 14:03:46 UTC
float/double calculation bug ?
Any idea why the follow code "calc" returns the "70%" of the 330000 incorrectly?testing on: Groovy Version: 2.4.9 JVM: 1.8.0_112 Vendor: Oracle Corporation OS: Mac OS XMac OS X 10.12.3
groovy:000> def calc = { amount, ttl ->
groovy:001> double rtn
groovy:002> if (amount[-1] != '%') rtn = amount.toDouble()
groovy:003> else rtn = ttl / 100.0 * amount.replaceAll(/%\Z/, '').toFloat()
groovy:004>
groovy:004> rtn
groovy:005> }
===> groovysh_evaluate$_run_closure1@39fcbef6
groovy:000> calc("70%", 330000)
===> 230999.99999999997
groovy:000> calc("10%", 330000)
===> 33000.0
groovy:000> calc("20%", 330000)
===> 66000.0
groovy:000> calc("30%", 330000)
===> 99000.0
groovy:000> calc("40%", 330000)
===> 132000.0
groovy:000> calc("50%", 330000)
===> 165000.0
groovy:000> calc("60%", 330000)
===> 198000.0
groovy:000> calc("70%", 330000)
===> 230999.99999999997
groovy:000> calc("80%", 330000)
===> 264000.0
groovy:000> calc("90%", 330000)
===> 297000.0
groovy:000> calc("100%", 330000)
===> 330000.0
Re: float/double calculation bug ?
Posted by Edmond Kemokai <ek...@gmail.com>.
Seems like floating point approximation at runtime. Interestingly enough
your code works with exact result when I try it but any minor reformulation
ends up with the approximate result. Also when I replace your computation
with literal integers I get the exact result...floating point arithmetic
can get messy so you'll have to think about what you're trying to
accomplish and perhaps consider rounding up/down as needed...I would avoid
words like mantissa or significand but basically you could choose a tiny
number and set it as the limit for rounding up in terms of the difference
between your floating point result and the next integral/integer value...I
don't want to get back into numerical analysis so I'll stop :)
I am the developer of HiveMind, it is a JSR223 based web app platform for
Groovy and other JVM languages. You can access:
url: http://demo.crudzilla.com:7000/
login: developer/developer
I have created a test file in: /com/crudzilla/cloudTest/web/txt8888/test.ste
You can test the code by right clicking on the test.ste file and selecting
"Open In Browser".
I have put your code in there, feel free to play with it and try out the
platform as a whole.
On Sun, Mar 19, 2017 at 11:28 AM, Derek Visch <de...@gmail.com> wrote:
> Looks like floating point to me, what are you expecting?
>
> On Mar 19, 2017 10:04 AM, "Tx. T" <tx...@yahoo.com> wrote:
>
>> Any idea why the follow code "calc" returns the "70%" of the 330000 incorrectly?
>>
>> testing on: Groovy Version: 2.4.9 JVM: 1.8.0_112 Vendor: Oracle Corporation OS: Mac OS X
>>
>> Mac OS X 10.12.3
>>
>>
>>
>> groovy:000> def calc = { amount, ttl ->
>> groovy:001> double rtn
>> groovy:002> if (amount[-1] != '%') rtn = amount.toDouble()
>> groovy:003> else rtn = ttl / 100.0 * amount.replaceAll(/%\Z/, '').toFloat()
>> groovy:004>
>> groovy:004> rtn
>> groovy:005> }
>> ===> groovysh_evaluate$_run_closure1@39fcbef6*groovy:000> calc("70%", 330000)
>> ===> 230999.99999999997*
>> groovy:000> calc("10%", 330000)
>> ===> 33000.0
>> groovy:000> calc("20%", 330000)
>> ===> 66000.0
>> groovy:000> calc("30%", 330000)
>> ===> 99000.0
>> groovy:000> calc("40%", 330000)
>> ===> 132000.0
>> groovy:000> calc("50%", 330000)
>> ===> 165000.0
>> groovy:000> calc("60%", 330000)
>> ===> 198000.0*groovy:000> calc("70%", 330000)
>> ===> 230999.99999999997*
>> groovy:000> calc("80%", 330000)
>> ===> 264000.0
>> groovy:000> calc("90%", 330000)
>> ===> 297000.0
>> groovy:000> calc("100%", 330000)
>> ===> 330000.0
>>
>>
Re: float/double calculation bug ?
Posted by Søren Berg Glasius <so...@glasius.dk>.
You are running into Floating point rounding in the JVM, which is not a
Groovy bug.
Your code can be salvaged by using BigDecimal:
def calc = { amount, ttl ->
BigDecimal rtn
if (amount[-1] != '%') rtn = amount.toBigDecimal()
else rtn = ttl / 100.0 * amount.replaceAll(/%\Z/, '').toBigDecimal()
rtn
}
On Sun, 19 Mar 2017 at 17:16 Søren Berg Glasius <so...@glasius.dk> wrote:
Hi,
This discussion does not belong on the dev mailing list, but the user
mailing list (*users@groovy.apache.org <us...@groovy.apache.org>)*. Please
continue your very interesting discussion there :-) Thanks.
/Søren
On Sun, 19 Mar 2017 at 16:47 James Bond <sb...@gmail.com> wrote:
What happens if you use .toDouble() instead of .toFloat, and an explicitly
double 100.0 literal? Right now, your computation is taking an integer,
dividing by a float, then multiplying by another float. I suspect that
going double precision won't fix this in all cases (floating point math is,
after all, just an approximation), but at least you're processing the
values as precisely as you can.
On Sun, Mar 19, 2017 at 8:28 AM, Derek Visch <de...@gmail.com> wrote:
Looks like floating point to me, what are you expecting?
On Mar 19, 2017 10:04 AM, "Tx. T" <tx...@yahoo.com> wrote:
Any idea why the follow code "calc" returns the "70%" of the 330000 incorrectly?
testing on: Groovy Version: 2.4.9 JVM: 1.8.0_112 Vendor: Oracle
Corporation OS: Mac OS X
Mac OS X 10.12.3
groovy:000> def calc = { amount, ttl ->
groovy:001> double rtn
groovy:002> if (amount[-1] != '%') rtn = amount.toDouble()
groovy:003> else rtn = ttl / 100.0 * amount.replaceAll(/%\Z/,
'').toFloat()
groovy:004>
groovy:004> rtn
groovy:005> }
===> groovysh_evaluate$_run_closure1@39fcbef6*groovy:000> calc("70%", 330000)
===> 230999.99999999997*
groovy:000> calc("10%", 330000)
===> 33000.0
groovy:000> calc("20%", 330000)
===> 66000.0
groovy:000> calc("30%", 330000)
===> 99000.0
groovy:000> calc("40%", 330000)
===> 132000.0
groovy:000> calc("50%", 330000)
===> 165000.0
groovy:000> calc("60%", 330000)
===> 198000.0*groovy:000> calc("70%", 330000)
===> 230999.99999999997*
groovy:000> calc("80%", 330000)
===> 264000.0
groovy:000> calc("90%", 330000)
===> 297000.0
groovy:000> calc("100%", 330000)
===> 330000.0
--
Best regards / Med venlig hilsen,
Søren Berg Glasius
Hedevej 1, Gl. Rye, 8680 Ry, Denmark
Mobile: +45 40 44 91 88 <40%2044%2091%2088>, Skype: sbglasius
--- Press ESC once to quit - twice to save the changes.
--
Best regards / Med venlig hilsen,
Søren Berg Glasius
Hedevej 1, Gl. Rye, 8680 Ry, Denmark
Mobile: +45 40 44 91 88 <40%2044%2091%2088>, Skype: sbglasius
--- Press ESC once to quit - twice to save the changes.
Re: float/double calculation bug ?
Posted by Søren Berg Glasius <so...@glasius.dk>.
Hi,
This discussion does not belong on the dev mailing list, but the user
mailing list (*users@groovy.apache.org <us...@groovy.apache.org>)*. Please
continue your very interesting discussion there :-) Thanks.
/Søren
On Sun, 19 Mar 2017 at 16:47 James Bond <sb...@gmail.com> wrote:
What happens if you use .toDouble() instead of .toFloat, and an explicitly
double 100.0 literal? Right now, your computation is taking an integer,
dividing by a float, then multiplying by another float. I suspect that
going double precision won't fix this in all cases (floating point math is,
after all, just an approximation), but at least you're processing the
values as precisely as you can.
On Sun, Mar 19, 2017 at 8:28 AM, Derek Visch <de...@gmail.com> wrote:
Looks like floating point to me, what are you expecting?
On Mar 19, 2017 10:04 AM, "Tx. T" <tx...@yahoo.com> wrote:
Any idea why the follow code "calc" returns the "70%" of the 330000 incorrectly?
testing on: Groovy Version: 2.4.9 JVM: 1.8.0_112 Vendor: Oracle
Corporation OS: Mac OS X
Mac OS X 10.12.3
groovy:000> def calc = { amount, ttl ->
groovy:001> double rtn
groovy:002> if (amount[-1] != '%') rtn = amount.toDouble()
groovy:003> else rtn = ttl / 100.0 * amount.replaceAll(/%\Z/,
'').toFloat()
groovy:004>
groovy:004> rtn
groovy:005> }
===> groovysh_evaluate$_run_closure1@39fcbef6*groovy:000> calc("70%", 330000)
===> 230999.99999999997*
groovy:000> calc("10%", 330000)
===> 33000.0
groovy:000> calc("20%", 330000)
===> 66000.0
groovy:000> calc("30%", 330000)
===> 99000.0
groovy:000> calc("40%", 330000)
===> 132000.0
groovy:000> calc("50%", 330000)
===> 165000.0
groovy:000> calc("60%", 330000)
===> 198000.0*groovy:000> calc("70%", 330000)
===> 230999.99999999997*
groovy:000> calc("80%", 330000)
===> 264000.0
groovy:000> calc("90%", 330000)
===> 297000.0
groovy:000> calc("100%", 330000)
===> 330000.0
--
Best regards / Med venlig hilsen,
Søren Berg Glasius
Hedevej 1, Gl. Rye, 8680 Ry, Denmark
Mobile: +45 40 44 91 88, Skype: sbglasius
--- Press ESC once to quit - twice to save the changes.
Re: float/double calculation bug ?
Posted by Søren Berg Glasius <so...@glasius.dk>.
Hi,
This discussion does not belong on the dev mailing list, but the user
mailing list (*users@groovy.apache.org <us...@groovy.apache.org>)*. Please
continue your very interesting discussion there :-) Thanks.
/Søren
On Sun, 19 Mar 2017 at 16:47 James Bond <sb...@gmail.com> wrote:
What happens if you use .toDouble() instead of .toFloat, and an explicitly
double 100.0 literal? Right now, your computation is taking an integer,
dividing by a float, then multiplying by another float. I suspect that
going double precision won't fix this in all cases (floating point math is,
after all, just an approximation), but at least you're processing the
values as precisely as you can.
On Sun, Mar 19, 2017 at 8:28 AM, Derek Visch <de...@gmail.com> wrote:
Looks like floating point to me, what are you expecting?
On Mar 19, 2017 10:04 AM, "Tx. T" <tx...@yahoo.com> wrote:
Any idea why the follow code "calc" returns the "70%" of the 330000 incorrectly?
testing on: Groovy Version: 2.4.9 JVM: 1.8.0_112 Vendor: Oracle
Corporation OS: Mac OS X
Mac OS X 10.12.3
groovy:000> def calc = { amount, ttl ->
groovy:001> double rtn
groovy:002> if (amount[-1] != '%') rtn = amount.toDouble()
groovy:003> else rtn = ttl / 100.0 * amount.replaceAll(/%\Z/,
'').toFloat()
groovy:004>
groovy:004> rtn
groovy:005> }
===> groovysh_evaluate$_run_closure1@39fcbef6*groovy:000> calc("70%", 330000)
===> 230999.99999999997*
groovy:000> calc("10%", 330000)
===> 33000.0
groovy:000> calc("20%", 330000)
===> 66000.0
groovy:000> calc("30%", 330000)
===> 99000.0
groovy:000> calc("40%", 330000)
===> 132000.0
groovy:000> calc("50%", 330000)
===> 165000.0
groovy:000> calc("60%", 330000)
===> 198000.0*groovy:000> calc("70%", 330000)
===> 230999.99999999997*
groovy:000> calc("80%", 330000)
===> 264000.0
groovy:000> calc("90%", 330000)
===> 297000.0
groovy:000> calc("100%", 330000)
===> 330000.0
--
Best regards / Med venlig hilsen,
Søren Berg Glasius
Hedevej 1, Gl. Rye, 8680 Ry, Denmark
Mobile: +45 40 44 91 88, Skype: sbglasius
--- Press ESC once to quit - twice to save the changes.
Re: float/double calculation bug ?
Posted by James Bond <sb...@gmail.com>.
What happens if you use .toDouble() instead of .toFloat, and an explicitly
double 100.0 literal? Right now, your computation is taking an integer,
dividing by a float, then multiplying by another float. I suspect that
going double precision won't fix this in all cases (floating point math is,
after all, just an approximation), but at least you're processing the
values as precisely as you can.
On Sun, Mar 19, 2017 at 8:28 AM, Derek Visch <de...@gmail.com> wrote:
> Looks like floating point to me, what are you expecting?
>
> On Mar 19, 2017 10:04 AM, "Tx. T" <tx...@yahoo.com> wrote:
>
>> Any idea why the follow code "calc" returns the "70%" of the 330000 incorrectly?
>>
>> testing on: Groovy Version: 2.4.9 JVM: 1.8.0_112 Vendor: Oracle Corporation OS: Mac OS X
>>
>> Mac OS X 10.12.3
>>
>>
>>
>> groovy:000> def calc = { amount, ttl ->
>> groovy:001> double rtn
>> groovy:002> if (amount[-1] != '%') rtn = amount.toDouble()
>> groovy:003> else rtn = ttl / 100.0 * amount.replaceAll(/%\Z/, '').toFloat()
>> groovy:004>
>> groovy:004> rtn
>> groovy:005> }
>> ===> groovysh_evaluate$_run_closure1@39fcbef6*groovy:000> calc("70%", 330000)
>> ===> 230999.99999999997*
>> groovy:000> calc("10%", 330000)
>> ===> 33000.0
>> groovy:000> calc("20%", 330000)
>> ===> 66000.0
>> groovy:000> calc("30%", 330000)
>> ===> 99000.0
>> groovy:000> calc("40%", 330000)
>> ===> 132000.0
>> groovy:000> calc("50%", 330000)
>> ===> 165000.0
>> groovy:000> calc("60%", 330000)
>> ===> 198000.0*groovy:000> calc("70%", 330000)
>> ===> 230999.99999999997*
>> groovy:000> calc("80%", 330000)
>> ===> 264000.0
>> groovy:000> calc("90%", 330000)
>> ===> 297000.0
>> groovy:000> calc("100%", 330000)
>> ===> 330000.0
>>
>>
Re: float/double calculation bug ?
Posted by Derek Visch <de...@gmail.com>.
Looks like floating point to me, what are you expecting?
On Mar 19, 2017 10:04 AM, "Tx. T" <tx...@yahoo.com> wrote:
> Any idea why the follow code "calc" returns the "70%" of the 330000 incorrectly?
>
> testing on: Groovy Version: 2.4.9 JVM: 1.8.0_112 Vendor: Oracle Corporation OS: Mac OS X
>
> Mac OS X 10.12.3
>
>
>
> groovy:000> def calc = { amount, ttl ->
> groovy:001> double rtn
> groovy:002> if (amount[-1] != '%') rtn = amount.toDouble()
> groovy:003> else rtn = ttl / 100.0 * amount.replaceAll(/%\Z/, '').toFloat()
> groovy:004>
> groovy:004> rtn
> groovy:005> }
> ===> groovysh_evaluate$_run_closure1@39fcbef6*groovy:000> calc("70%", 330000)
> ===> 230999.99999999997*
> groovy:000> calc("10%", 330000)
> ===> 33000.0
> groovy:000> calc("20%", 330000)
> ===> 66000.0
> groovy:000> calc("30%", 330000)
> ===> 99000.0
> groovy:000> calc("40%", 330000)
> ===> 132000.0
> groovy:000> calc("50%", 330000)
> ===> 165000.0
> groovy:000> calc("60%", 330000)
> ===> 198000.0*groovy:000> calc("70%", 330000)
> ===> 230999.99999999997*
> groovy:000> calc("80%", 330000)
> ===> 264000.0
> groovy:000> calc("90%", 330000)
> ===> 297000.0
> groovy:000> calc("100%", 330000)
> ===> 330000.0
>
>