You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/09/07 23:40:30 UTC
[GitHub] [spark] manuzhang opened a new pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
manuzhang opened a new pull request #29665:
URL: https://github.com/apache/spark/pull/29665
This PR backports https://github.com/apache/spark/pull/29593 to branch-3.0
### What changes were proposed in this pull request?
Only copy tags to node with no tags when transforming plans.
### Why are the changes needed?
@cloud-fan [made a good point](https://github.com/apache/spark/pull/29593#discussion_r482013121) that it doesn't make sense to append tags to existing nodes when nodes are removed. That will cause such bugs as duplicate rows when deduplicating and repartitioning by the same column with AQE.
```
spark.range(10).union(spark.range(10)).createOrReplaceTempView("v1")
val df = spark.sql("select id from v1 group by id distribute by id")
println(df.collect().toArray.mkString(","))
println(df.queryExecution.executedPlan)
// With AQE
[4],[0],[3],[2],[1],[7],[6],[8],[5],[9],[4],[0],[3],[2],[1],[7],[6],[8],[5],[9]
AdaptiveSparkPlan(isFinalPlan=true)
+- CustomShuffleReader local
+- ShuffleQueryStage 0
+- Exchange hashpartitioning(id#183L, 10), true
+- *(3) HashAggregate(keys=[id#183L], functions=[], output=[id#183L])
+- Union
:- *(1) Range (0, 10, step=1, splits=2)
+- *(2) Range (0, 10, step=1, splits=2)
// Without AQE
[4],[7],[0],[6],[8],[3],[2],[5],[1],[9]
*(4) HashAggregate(keys=[id#206L], functions=[], output=[id#206L])
+- Exchange hashpartitioning(id#206L, 10), true
+- *(3) HashAggregate(keys=[id#206L], functions=[], output=[id#206L])
+- Union
:- *(1) Range (0, 10, step=1, splits=2)
+- *(2) Range (0, 10, step=1, splits=2)
```
It's too expensive to detect node removal so we make a compromise only to copy tags to node with no tags.
### Does this PR introduce any user-facing change?
Yes. Fix a bug.
### How was this patch tested?
Add test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688671325
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688853688
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688875002
thanks, merging to 3.0!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688671325
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688543739
**[Test build #128366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128366/testReport)** for PR 29665 at commit [`be84096`](https://github.com/apache/spark/commit/be84096d9f5f09a3dfe69c82b78c57434f1ee244).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688851695
**[Test build #128387 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128387/testReport)** for PR 29665 at commit [`6973697`](https://github.com/apache/spark/commit/6973697505874ebd0265af2bab8bcf098bab8dad).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688665254
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688578723
**[Test build #128374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128374/testReport)** for PR 29665 at commit [`15b1673`](https://github.com/apache/spark/commit/15b1673a205ccdbf68d1d32576a17ceed659590e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688545327
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan removed a comment on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
cloud-fan removed a comment on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688661483
label added.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688578723
**[Test build #128374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128374/testReport)** for PR 29665 at commit [`15b1673`](https://github.com/apache/spark/commit/15b1673a205ccdbf68d1d32576a17ceed659590e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688661455
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688853688
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688665262
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128374/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688664163
**[Test build #128374 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128374/testReport)** for PR 29665 at commit [`15b1673`](https://github.com/apache/spark/commit/15b1673a205ccdbf68d1d32576a17ceed659590e).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688545320
**[Test build #128366 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128366/testReport)** for PR 29665 at commit [`be84096`](https://github.com/apache/spark/commit/be84096d9f5f09a3dfe69c82b78c57434f1ee244).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688545327
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688545333
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128366/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan edited a comment on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
cloud-fan edited a comment on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688877500
@manuzhang can you create a new PR against master to add the new code updates from this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-689123081
Thank you, @cloud-fan and @manuzhang .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688577397
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688543739
**[Test build #128366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128366/testReport)** for PR 29665 at commit [`be84096`](https://github.com/apache/spark/commit/be84096d9f5f09a3dfe69c82b78c57434f1ee244).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688544056
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] manuzhang commented on a change in pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
manuzhang commented on a change in pull request #29665:
URL: https://github.com/apache/spark/pull/29665#discussion_r484700557
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala
##########
@@ -889,4 +889,18 @@ class AdaptiveQueryExecSuite
}
}
}
+
+ test("SPARK-32753: Only copy tags to node with no tags") {
+ withSQLConf(
+ SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> "true"
+ ) {
Review comment:
Sure. BTW, if this indentation style must be followed, should it be added to checkstyle test ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688650945
@manuzhang and @cloud-fan . If this is a kind of a correctness issue, could you add a label to JIRA, please?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #29665:
URL: https://github.com/apache/spark/pull/29665
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688674554
**[Test build #128387 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128387/testReport)** for PR 29665 at commit [`6973697`](https://github.com/apache/spark/commit/6973697505874ebd0265af2bab8bcf098bab8dad).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688674554
**[Test build #128387 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128387/testReport)** for PR 29665 at commit [`6973697`](https://github.com/apache/spark/commit/6973697505874ebd0265af2bab8bcf098bab8dad).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] manuzhang commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
manuzhang commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688924111
@dongjoon-hyun @cloud-fan thanks for review. https://github.com/apache/spark/pull/29682 has been opened as follow-up against master.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688577397
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29665:
URL: https://github.com/apache/spark/pull/29665#discussion_r484683489
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala
##########
@@ -889,4 +889,18 @@ class AdaptiveQueryExecSuite
}
}
}
+
+ test("SPARK-32753: Only copy tags to node with no tags") {
+ withSQLConf(
+ SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> "true"
+ ) {
+ spark.range(10).union(spark.range(10)).createOrReplaceTempView("v1")
Review comment:
Shall we use `withTempView("v1")`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29665:
URL: https://github.com/apache/spark/pull/29665#discussion_r484682143
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala
##########
@@ -889,4 +889,18 @@ class AdaptiveQueryExecSuite
}
}
}
+
+ test("SPARK-32753: Only copy tags to node with no tags") {
+ withSQLConf(
+ SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> "true"
+ ) {
Review comment:
Indentation?
```scala
withSQLConf(SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> "true") {
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688544056
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688665254
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #29665: [SPARK-32753][SQL][3.0] Only copy tags to node with no tags
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #29665:
URL: https://github.com/apache/spark/pull/29665#issuecomment-688877500
@manuzhang can you create a new PR against master to add the new code updates in this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org