You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "uncleGen (JIRA)" <ji...@apache.org> on 2014/11/19 15:58:34 UTC

[jira] [Updated] (SPARK-3373) Filtering operations should optionally rebuild routing tables

     [ https://issues.apache.org/jira/browse/SPARK-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

uncleGen updated SPARK-3373:
----------------------------
    Target Version/s: 1.1.1, 1.2.0  (was: 1.1.0, 1.0.3)

> Filtering operations should optionally rebuild routing tables
> -------------------------------------------------------------
>
>                 Key: SPARK-3373
>                 URL: https://issues.apache.org/jira/browse/SPARK-3373
>             Project: Spark
>          Issue Type: Improvement
>          Components: GraphX
>    Affects Versions: 1.0.0, 1.0.2
>            Reporter: uncleGen
>            Priority: Minor
>
> Graph operations that filter the edges (subgraph, mask, groupEdges) currently reuse the existing routing table to avoid the shuffle which would be required to build a new one. However, this may be inefficient when the filtering is highly selective. Vertices will be sent to more partitions than necessary, and the extra routing information may take up excessive space.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org