You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/02/14 09:48:14 UTC
[GitHub] [spark] xuanyuanking opened a new pull request #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
xuanyuanking opened a new pull request #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579
### What changes were proposed in this pull request?
Define a new enumeration `LegacyBehaviorPolicy` in SQLConf, it will be used as the common value for result change configs.
### Why are the changes needed?
During API auditing for the 3.0 release, we found several new approaches that will change the results silently. For these features, we need a common three-value config.
### Does this PR introduce any user-facing change?
Yes, original config `spark.sql.legacy.ctePrecedence.enabled` change to `spark.sql.legacy.ctePrecedencePolicy`.
### How was this patch tested?
Existing UT.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586330708
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586329828
**[Test build #118414 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118414/testReport)** for PR 27579 at commit [`af7827c`](https://github.com/apache/spark/commit/af7827c9a910d5508da5327f1246737e64b4f72f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586188485
yea this looks clearer, +1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586187945
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-587048660
**[Test build #118565 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118565/testReport)** for PR 27579 at commit [`bc70f75`](https://github.com/apache/spark/commit/bc70f75d9cb2fdc19130874401e616a85861a94e).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#discussion_r379345161
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -2106,13 +2106,19 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
- val LEGACY_CTE_PRECEDENCE_ENABLED = buildConf("spark.sql.legacy.ctePrecedence.enabled")
+ object LegacyBehaviorPolicy extends Enumeration {
+ val EXCEPTION, LEGACY, NEW_BEHAVIOR = Value
Review comment:
Or `INNER_PRECEDENCE`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586187945
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#discussion_r380016560
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -2106,13 +2106,19 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
- val LEGACY_CTE_PRECEDENCE_ENABLED = buildConf("spark.sql.legacy.ctePrecedence.enabled")
+ object LegacyBehaviorPolicy extends Enumeration {
+ val EXCEPTION, LEGACY, CORRECTED = Value
+ }
+
+ val LEGACY_CTE_PRECEDENCE_POLICY = buildConf("spark.sql.legacy.ctePrecedencePolicy")
.internal()
- .doc("When true, outer CTE definitions takes precedence over inner definitions. If set to " +
- "false, inner CTE definitions take precedence. The default value is empty, " +
+ .doc("When LEGACY, outer CTE definitions takes precedence over inner definitions. If set to " +
+ "CORRECTED, inner CTE definitions take precedence. The default value is EXCEPTION, " +
"AnalysisException is thrown while name conflict is detected in nested CTE.")
Review comment:
let's add one more sentence: This config will be removed in future versions and CORRECTED will be the only behavior.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586864683
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118542/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#discussion_r379344993
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -2106,13 +2106,19 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
- val LEGACY_CTE_PRECEDENCE_ENABLED = buildConf("spark.sql.legacy.ctePrecedence.enabled")
+ object LegacyBehaviorPolicy extends Enumeration {
+ val EXCEPTION, LEGACY, NEW_BEHAVIOR = Value
Review comment:
`NEW_BEHAVIOR` sounds not a good name. `INNER_FIRST`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586187951
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23171/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586846567
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23297/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586912201
**[Test build #118565 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118565/testReport)** for PR 27579 at commit [`bc70f75`](https://github.com/apache/spark/commit/bc70f75d9cb2fdc19130874401e616a85861a94e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#discussion_r379385520
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -2106,13 +2106,19 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
- val LEGACY_CTE_PRECEDENCE_ENABLED = buildConf("spark.sql.legacy.ctePrecedence.enabled")
+ object LegacyBehaviorPolicy extends Enumeration {
+ val EXCEPTION, LEGACY, NEW_BEHAVIOR = Value
Review comment:
Thanks for the advice @viirya. Yes, both `INNER_FIRST` and `INNER_PRECEDENCE` are more accurate in this PR, but I also want to make this enum as a common config value, which can be reused in other similar issues. WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#discussion_r380016686
##########
File path: sql/core/src/test/resources/sql-tests/inputs/cte-nonlegacy.sql
##########
@@ -1,2 +1,2 @@
---SET spark.sql.legacy.ctePrecedence.enabled = false
+--SET spark.sql.legacy.ctePrecedencePolicy = new_behavior
Review comment:
corrected
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586912864
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-587239573
Thanks for the review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586330719
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118414/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586330708
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-587049718
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586846201
**[Test build #118542 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118542/testReport)** for PR 27579 at commit [`47edd88`](https://github.com/apache/spark/commit/47edd887fed7e3a833b1928955e19a3164c8283d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-587049718
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#discussion_r380084826
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -2106,13 +2106,19 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
- val LEGACY_CTE_PRECEDENCE_ENABLED = buildConf("spark.sql.legacy.ctePrecedence.enabled")
+ object LegacyBehaviorPolicy extends Enumeration {
+ val EXCEPTION, LEGACY, CORRECTED = Value
+ }
+
+ val LEGACY_CTE_PRECEDENCE_POLICY = buildConf("spark.sql.legacy.ctePrecedencePolicy")
.internal()
- .doc("When true, outer CTE definitions takes precedence over inner definitions. If set to " +
- "false, inner CTE definitions take precedence. The default value is empty, " +
+ .doc("When LEGACY, outer CTE definitions takes precedence over inner definitions. If set to " +
+ "CORRECTED, inner CTE definitions take precedence. The default value is EXCEPTION, " +
"AnalysisException is thrown while name conflict is detected in nested CTE.")
Review comment:
Thanks, added in bc70f75.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-587049726
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118565/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586186789
**[Test build #118414 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118414/testReport)** for PR 27579 at commit [`af7827c`](https://github.com/apache/spark/commit/af7827c9a910d5508da5327f1246737e64b4f72f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586912201
**[Test build #118565 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118565/testReport)** for PR 27579 at commit [`bc70f75`](https://github.com/apache/spark/commit/bc70f75d9cb2fdc19130874401e616a85861a94e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586864666
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#discussion_r379391745
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CTESubstitution.scala
##########
@@ -54,8 +54,8 @@ object CTESubstitution extends Rule[LogicalPlan] {
case (cteName, _) =>
if (cteNames.contains(cteName)) {
throw new AnalysisException(s"Name $cteName is ambiguous in nested CTE. " +
- s"Please set ${LEGACY_CTE_PRECEDENCE_ENABLED.key} to false so that name defined " +
- "in inner CTE takes precedence. See more details in SPARK-28228.")
+ s"Please set ${LEGACY_CTE_PRECEDENCE_POLICY.key} to NEW_BEHAVIOR so that name " +
+ "defined in inner CTE takes precedence. See more details in SPARK-28228.")
Review comment:
since it's enum now, maybe also explain what will happen if set it to legacy.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#discussion_r379344993
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -2106,13 +2106,19 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
- val LEGACY_CTE_PRECEDENCE_ENABLED = buildConf("spark.sql.legacy.ctePrecedence.enabled")
+ object LegacyBehaviorPolicy extends Enumeration {
+ val EXCEPTION, LEGACY, NEW_BEHAVIOR = Value
Review comment:
`NEW_BEHAVIOR` sounds not a good name. `INNERFIRST`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586912872
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23321/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586912864
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586846560
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586846201
**[Test build #118542 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118542/testReport)** for PR 27579 at commit [`47edd88`](https://github.com/apache/spark/commit/47edd887fed7e3a833b1928955e19a3164c8283d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586846560
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on issue #27579: [SPARK-30829][SQL] Define
LegacyBehaviorPolicy enumeration as the common value for result change
configs
Posted by GitBox <gi...@apache.org>.
viirya commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586190532
+1 this is better.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-587049726
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118565/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586186789
**[Test build #118414 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118414/testReport)** for PR 27579 at commit [`af7827c`](https://github.com/apache/spark/commit/af7827c9a910d5508da5327f1246737e64b4f72f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#discussion_r380009637
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -2106,13 +2106,19 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
- val LEGACY_CTE_PRECEDENCE_ENABLED = buildConf("spark.sql.legacy.ctePrecedence.enabled")
+ object LegacyBehaviorPolicy extends Enumeration {
+ val EXCEPTION, LEGACY, NEW_BEHAVIOR = Value
Review comment:
Thanks, change to CORRECTED in 47edd88
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586864683
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118542/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586187951
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23171/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#discussion_r380084947
##########
File path: sql/core/src/test/resources/sql-tests/inputs/cte-nonlegacy.sql
##########
@@ -1,2 +1,2 @@
---SET spark.sql.legacy.ctePrecedence.enabled = false
+--SET spark.sql.legacy.ctePrecedencePolicy = new_behavior
Review comment:
Ah yes, done in bc70f75.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#discussion_r379423136
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -2106,13 +2106,19 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
- val LEGACY_CTE_PRECEDENCE_ENABLED = buildConf("spark.sql.legacy.ctePrecedence.enabled")
+ object LegacyBehaviorPolicy extends Enumeration {
+ val EXCEPTION, LEGACY, NEW_BEHAVIOR = Value
Review comment:
ok.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586330719
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118414/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#discussion_r379391569
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -2106,13 +2106,19 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
- val LEGACY_CTE_PRECEDENCE_ENABLED = buildConf("spark.sql.legacy.ctePrecedence.enabled")
+ object LegacyBehaviorPolicy extends Enumeration {
+ val EXCEPTION, LEGACY, NEW_BEHAVIOR = Value
Review comment:
how about `CORRECTED` to indicate that it's the corrected behavior?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586846567
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23297/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#discussion_r380009687
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CTESubstitution.scala
##########
@@ -54,8 +54,8 @@ object CTESubstitution extends Rule[LogicalPlan] {
case (cteName, _) =>
if (cteNames.contains(cteName)) {
throw new AnalysisException(s"Name $cteName is ambiguous in nested CTE. " +
- s"Please set ${LEGACY_CTE_PRECEDENCE_ENABLED.key} to false so that name defined " +
- "in inner CTE takes precedence. See more details in SPARK-28228.")
+ s"Please set ${LEGACY_CTE_PRECEDENCE_POLICY.key} to NEW_BEHAVIOR so that name " +
+ "defined in inner CTE takes precedence. See more details in SPARK-28228.")
Review comment:
Sure, done in 47edd88.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586912872
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23321/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586864082
**[Test build #118542 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118542/testReport)** for PR 27579 at commit [`47edd88`](https://github.com/apache/spark/commit/47edd887fed7e3a833b1928955e19a3164c8283d).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-587081627
thanks, merging to master/3.0!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on issue #27579: [SPARK-30829][SQL]
Define LegacyBehaviorPolicy enumeration as the common value for result
change configs
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586185277
cc @viirya @dongjoon-hyun @cloud-fan
After further thinking, it's strange in #27454 to use `Option[Boolean]` to present a three-value config, especially we have other similar works in the future(e.g. #27537)
In this PR, I propose `LegacyBehaviorPolicy` as the common config value for these similar issues. Please take a look. Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27579:
[SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common
value for result change configs
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27579: [SPARK-30829][SQL] Define LegacyBehaviorPolicy enumeration as the common value for result change configs
URL: https://github.com/apache/spark/pull/27579#issuecomment-586864666
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org