You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by marymwu <gi...@git.apache.org> on 2018/07/16 10:14:55 UTC
[GitHub] spark pull request #21783: [SPARK-24799]A solution of dealing with data skew...
GitHub user marymwu opened a pull request:
https://github.com/apache/spark/pull/21783
[SPARK-24799]A solution of dealing with data skew in left,right,inner join
## What changes were proposed in this pull request?
For the left,right,inner join statment execution, this solution is mainling about to devide the partions where the data skew has occured into serveral partions with smaller data scale, in order to parallelly execute more tasks to increase effeciency.
## How was this patch tested?
Unit tests in DatasetSuite.scala
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marymwu/spark branch-2.3
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21783.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21783
----
commit 2a01c813b6ef7223a489a4bcda3c9e5feb899060
Author: wangsm9 <wa...@...>
Date: 2018-07-16T09:48:44Z
“data skew code for spark2.3
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21783: [SPARK-24799]A solution of dealing with data skew in lef...
Posted by wangyum <gi...@git.apache.org>.
Github user wangyum commented on the issue:
https://github.com/apache/spark/pull/21783
@marymwu Could you make a PR against `master` branch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21783: [SPARK-24799]A solution of dealing with data skew in lef...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21783
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21783: [SPARK-24799]A solution of dealing with data skew in lef...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21783
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21783: [SPARK-24799]A solution of dealing with data skew in lef...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21783
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org