[jira] [Commented] (SPARK-5101) Add common ML math functions

["Sean Owen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-5101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14265916#comment-14265916 ] 

Sean Owen commented on SPARK-5101:
----------------------------------

(Ah, very good point about overflow!)

> Add common ML math functions
> ----------------------------
>
>                 Key: SPARK-5101
>                 URL: https://issues.apache.org/jira/browse/SPARK-5101
>             Project: Spark
>          Issue Type: New Feature
>          Components: MLlib
>            Reporter: Xiangrui Meng
>            Assignee: DB Tsai
>            Priority: Minor
>
> We can add common ML math functions to MLlib. It may be a little tricky to implement those functions in a numerically stable way. For example,
> {code}
> math.log(1 + math.exp(x))
> {code}
> should be implemented as
> {code}
> if (x > 0) {
>   x + math.log1p(math.exp(-x))
> } else {
>   math.log1p(math.exp(x))
> }
> {code}
> It becomes hard to maintain if we have multiple copies of the correct implementation in the codebase. A good place for those functions could be `mllib.util.MathFunctions`.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org