You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by wujianping10043419 <gi...@git.apache.org> on 2018/02/07 08:41:08 UTC
[GitHub] spark pull request #20526: [SPARK-23349][SQL]ShuffleExchangeExec: Duplicate ...
GitHub user wujianping10043419 opened a pull request:
https://github.com/apache/spark/pull/20526
[SPARK-23349][SQL]ShuffleExchangeExec: Duplicate and redundant type determination for ShuffleManager Object
## What changes were proposed in this pull request?
org.apache.spark.sql.execution.exchange.ShuffleExchangeExec:
There is a nested "if or else" branch within the "needtocopyobjectsbeforguffle()" function.
The <sortBasedShufffleOn> condition in the first layer "if" has the same value as the <bypassIsSupported> condition in the second layer "if", that is, <bypassIsSupported> must be true when <sortBasedShufffleOn> is true.
In addition, the <byPassMergeThreshold> condition will be used in the second layer "if" and should not be calculated until needed.
It's to remove the <bypassIsSupported> condition in the second layer " if" and to move the <bypassmergethreshold> calculation backward
## How was this patch tested?
Existing tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wujianping10043419/spark-1 wujp-0107
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20526.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20526
----
commit 1c8b73ad9a662ca44f0bc2553b8c43130ab632d4
Author: wujianping10043419 <wu...@...>
Date: 2018-02-07T08:04:07Z
It's to remove the bypassmergethreshold condition in the second layer " if"
and to move the <bypassmergethreshold> calculation backward
Signed-off-by: wujianping10043419 <wu...@zte.com.cn>
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20526: [SPARK-23349][SQL]ShuffleExchangeExec: Duplicate and red...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20526
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20526: [SPARK-23349][SQL]ShuffleExchangeExec: Duplicate ...
Posted by wujianping10043419 <gi...@git.apache.org>.
Github user wujianping10043419 closed the pull request at:
https://github.com/apache/spark/pull/20526
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20526: [SPARK-23349][SQL]ShuffleExchangeExec: Duplicate and red...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20526
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20526: [SPARK-23349][SQL]ShuffleExchangeExec: Duplicate and red...
Posted by wujianping10043419 <gi...@git.apache.org>.
Github user wujianping10043419 commented on the issue:
https://github.com/apache/spark/pull/20526
Pom. XML file was modified incorrectly during code submission
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org