You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Joseph K. Bradley (JIRA)" <ji...@apache.org> on 2015/08/28 00:54:46 UTC
[jira] [Commented] (SPARK-10307) Fix regression in block matrix
multiply (1.4->1.5 regression)
[ https://issues.apache.org/jira/browse/SPARK-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14717704#comment-14717704 ]
Joseph K. Bradley commented on SPARK-10307:
-------------------------------------------
I tested this a number of times to try to reproduce the issue on branch-1.5. Weirdly, I reproduced it once, with running times:
{code}
"results":[{"time":79.313},{"time":82.344},{"time":77.169},{"time":63.269},{"time":86.671},{"time":79.732},{"time":76.208},{"time":91.78},{"time":73.738},{"time":56.931},{"time":75.267},{"time":75.316},{"time":63.639},{"time":66.429},{"time":67.172}]
{code}
But when I tried re-running on branch-1.5 a few times (on both RC1 and the most recent branch with updates post-RC21), I got times like this:
{code}
"results":[{"time":49.95},{"time":49.081},{"time":50.712},{"time":49.272},{"time":49.81},{"time":47.067},{"time":52.498},{"time":48.093},{"time":48.468},{"time":49.142},{"time":47.212},{"time":47.21},{"time":48.007},{"time":55.267},{"time":48.121}]
{code}
Note these were all on the same EC2 cluster.
So...I'd say there is no obvious regression. If something is wrong, then it's pretty subtle. I'll close this for now.
> Fix regression in block matrix multiply (1.4->1.5 regression)
> -------------------------------------------------------------
>
> Key: SPARK-10307
> URL: https://issues.apache.org/jira/browse/SPARK-10307
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Affects Versions: 1.5.0
> Reporter: Joseph K. Bradley
> Assignee: Joseph K. Bradley
> Priority: Critical
>
> Running spark-perf on the block-matrix-mult test (BlockMatrix.multiply), I found the running time increased from 50 sec to 80 sec. This was on the default test settings, on 16 r3.2xlarge workers on EC2, and with 15 trials, dropping the first 2.
> The only relevant changes I found are:
> * [https://github.com/apache/spark/commit/520ad44b17f72e6465bf990f64b4e289f8a83447]
> * [https://github.com/apache/spark/commit/99c40cd0d8465525cac34dfa373b81532ef3d719]
> I'm testing reverting each of those now.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org