You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by wujianping10043419 <gi...@git.apache.org> on 2018/02/07 11:35:11 UTC
[GitHub] spark pull request #20530: [SPARK-23349][SQL]ShuffleExchangeExec: Duplicate ...
GitHub user wujianping10043419 opened a pull request:
https://github.com/apache/spark/pull/20530
[SPARK-23349][SQL]ShuffleExchangeExec: Duplicate and redundant type determination for ShuffleManager Object
## What changes were proposed in this pull request?
org.apache.spark.sql.execution.exchange.ShuffleExchangeExec:
There is a nested "if or else" branch within the "needtocopyobjectsbeforguffle()" function.
The \<sortBasedShufffleOn\> condition in the first layer "if" has the same value as the \<bypassIsSupported\> condition in the second layer "if", that is, \<bypassIsSupported\> must be true when \<sortBasedShufffleOn\> is true.
In addition, the \<byPassMergeThreshold\> condition will be used in the second layer "if" and should not be calculated until needed.
It's to remove the \<bypassIsSupported\> condition in the second layer " if" and to move the \<bypassmergethreshold\> calculation backward
## How was this patch tested?
Existing tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wujianping10043419/spark-1 wujp-0107-home
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20530.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20530
----
commit db50cccd1e3fd34478fc72500917211f9b3bf800
Author: wujianping10043419 <wu...@...>
Date: 2018-02-07T10:59:50Z
[SPARK-23349][SQL]ShuffleExchangeExec: Duplicate and redundant type determination for ShuffleManager Object
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20530: [SPARK-23349][SQL]ShuffleExchangeExec: Duplicate and red...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20530
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20530: [SPARK-23349][SQL]ShuffleExchangeExec: Duplicate and red...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/20530
@wujianping10043419 this change is too trivial to merge, I am sorry. The main reason for not merging is because this will mess up the git blame for the given code, which is sometimes very useful to find out why we implemented something the way it is. Can you close this?
Thank you for your contribution.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20530: [SPARK-23349][SQL]ShuffleExchangeExec: Duplicate ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20530
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20530: [SPARK-23349][SQL]ShuffleExchangeExec: Duplicate and red...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20530
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org