You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "PEIYUAN SUN (Jira)" <ji...@apache.org> on 2022/06/12 13:54:00 UTC
[jira] [Updated] (SPARK-39452) Extend EdgePartition1D with Destination based Strategy
[ https://issues.apache.org/jira/browse/SPARK-39452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
PEIYUAN SUN updated SPARK-39452:
--------------------------------
Description:
Currently, the *[EdgePartition1D|https://github.com/apache/spark/blob/master/graphx/src/main/scala/org/apache/spark/graphx/PartitionStrategy.scala#L101]* use only the source vertex ID.
If doing topological like algorithm on graph, directly partition based on destination vertex ID would improve the performance significantly.
The proposal here is either adding a new case object like *EdgePartition1DDst* or update the *EdgePartition1D* such that user can configure the src or dst directly.
was:
Currently, the *[EdgePartition1D|https://github.com/apache/spark/blob/master/graphx/src/main/scala/org/apache/spark/graphx/PartitionStrategy.scala#L101]* use only the source vertex ID.
If doing topological like algorithm on graph, directly partition based on destination vertex ID would improve the performance significantly.
The proposal here is either adding a new case object like *EdgePartition1DDst* or ** update the ** *EdgePartition1D* such that user can configure the src or dst.{*}{*}
> Extend EdgePartition1D with Destination based Strategy
> ------------------------------------------------------
>
> Key: SPARK-39452
> URL: https://issues.apache.org/jira/browse/SPARK-39452
> Project: Spark
> Issue Type: New Feature
> Components: GraphX
> Affects Versions: 3.1.2, 3.2.1
> Reporter: PEIYUAN SUN
> Priority: Minor
> Labels: graph, partition
> Fix For: 3.3.0, 3.3.1
>
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> Currently, the *[EdgePartition1D|https://github.com/apache/spark/blob/master/graphx/src/main/scala/org/apache/spark/graphx/PartitionStrategy.scala#L101]* use only the source vertex ID.
> If doing topological like algorithm on graph, directly partition based on destination vertex ID would improve the performance significantly.
> The proposal here is either adding a new case object like *EdgePartition1DDst* or update the *EdgePartition1D* such that user can configure the src or dst directly.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org