You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "Brian S. McCormick (JIRA)" <ji...@apache.org> on 2016/09/19 13:48:21 UTC
[jira] [Created] (MATH-1386) EPSILON value in
org.apache.commons.math.util.MathUtils seems like half of what it should
be
Brian S. McCormick created MATH-1386:
----------------------------------------
Summary: EPSILON value in org.apache.commons.math.util.MathUtils seems like half of what it should be
Key: MATH-1386
URL: https://issues.apache.org/jira/browse/MATH-1386
Project: Commons Math
Issue Type: Bug
Environment: win 7
Reporter: Brian S. McCormick
I've always used 1.1920929E-7 for dealing with float epsilon values. When dealing with double I've always used 2.220446049250313E-16 for epsilon.
You have it defined in org.apache.commons.math.util.MathUtils as 1.1102230246251565E-16 which is half what I think it should be.
I come up with these numbers using the following:
float fEps = Float.intBitsToFloat(Float.floatToIntBits(1f) + 1) - 1;
double dEps = Double.longBitsToDouble(Double.doubleToLongBits(1) + 1) - 1;
Am I correct? I don't really know. I do know that float epsilon in every legacy C/C++ compiler etc I have ever used is defined as about 1e-7 and this is the value using the formula for fEps above. When I started doing comps using doubles instead of floats I started using the formula for dEps above which looks to me like the equivalent for double numbers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)