You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "DB Tsai (JIRA)" <ji...@apache.org> on 2016/03/22 02:14:25 UTC
[jira] [Comment Edited] (SPARK-14053) Merge absTol and relTol into
one in MLlib tests
[ https://issues.apache.org/jira/browse/SPARK-14053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205514#comment-15205514 ]
DB Tsai edited comment on SPARK-14053 at 3/22/16 1:14 AM:
----------------------------------------------------------
This makes sense for me. We just need to document it properly. Also, the current code for comparing double is symmetric. We can do
If abs( y ) > eps / t && abs( x ) > eps / t test abs(y - x) < t * math.min(absX, absY)
else test abs(y - x) < eps
```
was (Author: dbtsai):
This makes sense for me. We just need to document it properly. Also, the current code for comparing double is symmetric. We can do
If abs(y) > eps / t && abs(x) > eps / t test abs(y - x) < t * math.min(absX, absY)
else test abs(y - x) < eps
```
> Merge absTol and relTol into one in MLlib tests
> -----------------------------------------------
>
> Key: SPARK-14053
> URL: https://issues.apache.org/jira/browse/SPARK-14053
> Project: Spark
> Issue Type: Improvement
> Components: ML, MLlib, Tests
> Affects Versions: 2.0.0
> Reporter: Xiangrui Meng
> Assignee: Xiangrui Meng
>
> We have absTol and relTol in MLlib tests to compare values with possible numerical differences. However, in most cases we should just use relTol. Many absTol are not used properly. See https://github.com/apache/spark/search?q=absTol. One corner case relTol doesn't handle is when the target value is 0. We can make the following change to relTol to solve the issue. Consider `x ~== y relTol t`.
> 1. If abs( y ) > eps / t, test abs(y - x) / abs( y ) < t,
> 2. else test abs(y - x) < eps
> where eps is a reasonably small value, e.g., 1e-14. Note that the transition is smooth at abs( y ) = eps / t.
> cc [~dbtsai]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org