You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2015/03/10 12:16:38 UTC
[jira] [Resolved] (SPARK-6234) 10% Performance regression with
Breeze upgrade
[ https://issues.apache.org/jira/browse/SPARK-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-6234.
------------------------------
Resolution: Not a Problem
The question is, is there a performance regression in "real" Spark? The speed of the examples does not matter, unless it were so incredibly slow as to be unusable on even small input.
I checked, and there are no usages of this squaredDistance function from Breeze outside the two example K-means implementations, and two test suites.
However there could be, somehow, other affected Breeze functions or other usages of this function from Breeze methods that we do call. It's things like norm, axpy and basic matrix/vector ops that we use. I didn't chase it down thoroughly, but I doubt they use squared-distance heavily if at all.
For now I don't think we have evidence of a (non-example, non-test) performance regression, and the examples / tests aren't noticeably harmed by this. I think this is NotAProblem for the moment.
However it would certainly be good to report this to Breeze since there was clearly a late change to several functions, and if it introduced a slow-down that's measurable to the tune of 10% that should be fixed. Other functions may be affected. We would want all of that into Spark eventually.
> 10% Performance regression with Breeze upgrade
> ----------------------------------------------
>
> Key: SPARK-6234
> URL: https://issues.apache.org/jira/browse/SPARK-6234
> Project: Spark
> Issue Type: Bug
> Reporter: Nishkam Ravi
>
> KMeans regresses by 10% with the Breeze upgrade from 0.10 to 0.11
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org