You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Iurii Antykhovych (JIRA)" <ji...@apache.org> on 2017/07/20 21:23:00 UTC
[jira] [Comment Edited] (SPARK-21491) Performance enhancement:
eliminate creation of intermediate collections
[ https://issues.apache.org/jira/browse/SPARK-21491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095398#comment-16095398 ]
Iurii Antykhovych edited comment on SPARK-21491 at 7/20/17 9:22 PM:
--------------------------------------------------------------------
This is relevant to all scala versions starting from 2.8, it's in `scala.collection.breakOut`.
The problem with {{collection.map(...).toMap}} is in creation of intermediate collection of tuples, that is converted to map then;
that leads to performance degradation and excess object allocation.
The price of {{collection.breakOut}} is the code readability, it significantly suffers I guess, compared to {{.toMap}} method.
was (Author: sereneant):
This is relevant to all scala versions starting from 2.8, it's in `scala.collection.breakOut`.
The problem with `collection.map(...).toMap` is in creation of intermediate collection of tuples, that is converted to map then;
that leads to performance degradation and excess object allocation.
The price of `collection.breakOut` is the code readability, it significantly suffers I guess, compared to '.toMap' method.
> Performance enhancement: eliminate creation of intermediate collections
> -----------------------------------------------------------------------
>
> Key: SPARK-21491
> URL: https://issues.apache.org/jira/browse/SPARK-21491
> Project: Spark
> Issue Type: Improvement
> Components: GraphX
> Affects Versions: 2.2.0
> Reporter: Iurii Antykhovych
> Priority: Trivial
>
> Simple performance optimization in a few places of GraphX:
> {{Traversable.toMap}} can be replaced with {{collection.breakout}}.
> This would eliminate creation of an intermediate collection of tuples, see
> [Stack Overflow article|https://stackoverflow.com/questions/1715681/scala-2-8-breakout]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org