You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "zhengruifeng (Jira)" <ji...@apache.org> on 2021/03/18 06:39:00 UTC
[jira] [Resolved] (SPARK-32060) Huber loss Convergence
[ https://issues.apache.org/jira/browse/SPARK-32060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhengruifeng resolved SPARK-32060.
----------------------------------
Resolution: Resolved
> Huber loss Convergence
> ----------------------
>
> Key: SPARK-32060
> URL: https://issues.apache.org/jira/browse/SPARK-32060
> Project: Spark
> Issue Type: Sub-task
> Components: ML
> Affects Versions: 3.1.0
> Reporter: zhengruifeng
> Priority: Minor
> Attachments: huber.xlsx
>
>
> |performace test in https://issues.apache.org/jira/browse/SPARK-31783,
> Huber loss seems start to diverge since 70 iters.
> {code:scala}
> for (size <- Seq(1, 4, 16, 64); iter <- Seq(10, 50, 100)) {
> Thread.sleep(10000)
> val hlir = new LinearRegression().setLoss("huber").setSolver("l-bfgs").setMaxIter(iter).setTol(0)
> val start = System.currentTimeMillis
> val model = hlir.setBlockSize(size).fit(df)
> val end = System.currentTimeMillis
> println((model.uid, size, iter, end - start, model.summary.objectiveHistory.last, model.summary.totalIterations, model.coefficients.toString.take(100)))
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org