You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/06/09 03:56:01 UTC
[GitHub] [spark] linhongliu-db opened a new pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
linhongliu-db opened a new pull request #32832:
URL: https://github.com/apache/spark/pull/32832
### What changes were proposed in this pull request?
As described in #32831, Spark has compatible issues when querying a view created by an
older version. The root cause is that Spark changed the auto-generated alias name. To avoid
this in the future, we could ask the user to specify explicit column names when creating
a view.
### Why are the changes needed?
Avoid compatible issue when querying a view
### Does this PR introduce _any_ user-facing change?
Yes. User will get error when running query below after this change
```
CREATE OR REPLACE VIEW v AS SELECT CAST(t.a AS INT), to_date(t.b, 'yyyyMMdd') FROM t
```
### How was this patch tested?
not yet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-857661540
**[Test build #139572 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139572/testReport)** for PR 32832 at commit [`58af84f`](https://github.com/apache/spark/commit/58af84f12d12d7b59f5eb5fbf267533709eb7ccb).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872003261
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44996/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-857362781
**[Test build #139531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139531/testReport)** for PR 32832 at commit [`251ca26`](https://github.com/apache/spark/commit/251ca26be44fca19d0bef17e99be27775043c68f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32832:
URL: https://github.com/apache/spark/pull/32832#discussion_r657622253
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -432,10 +432,26 @@ class Analyzer(override val catalogManager: CatalogManager)
case e if !e.resolved => u
case g: Generator => MultiAlias(g, Nil)
case c @ Cast(ne: NamedExpression, _, _, _) => Alias(c, ne.name)()
- case e: ExtractValue => Alias(e, toPrettySQL(e))()
+ case e: ExtractValue =>
+ val onlyExtract = e.collect {
+ case _: ExtractValue => true
+ case _ => false
+ }.forall(a => a)
Review comment:
let's write a small function for it:
```
def extractOnly(e: Expression): Boolean = e match {
case _: ExtractValue => e.children.forall(extractOnly)
case _: Literal => true
case _: Attribute => true
case _ => false
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32832:
URL: https://github.com/apache/spark/pull/32832#discussion_r661988654
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala
##########
@@ -910,4 +910,45 @@ abstract class SQLViewSuite extends QueryTest with SQLTestUtils {
}
}
}
+
+ test("SPARK-35686: error out for creating view with auto gen alias") {
Review comment:
shall we put the tests in `PersistedViewTestSuite`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-867348592
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872323277
**[Test build #140513 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140513/testReport)** for PR 32832 at commit [`751a55e`](https://github.com/apache/spark/commit/751a55e65033e9bb2c279acee6a300e6d61c6317).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-870967074
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44902/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872211236
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-867311810
**[Test build #140239 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140239/testReport)** for PR 32832 at commit [`86673c0`](https://github.com/apache/spark/commit/86673c04748c964eda42d94266d4d1f6fcbe6c53).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-867348592
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on a change in pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #32832:
URL: https://github.com/apache/spark/pull/32832#discussion_r658436593
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -432,10 +432,26 @@ class Analyzer(override val catalogManager: CatalogManager)
case e if !e.resolved => u
case g: Generator => MultiAlias(g, Nil)
case c @ Cast(ne: NamedExpression, _, _, _) => Alias(c, ne.name)()
- case e: ExtractValue => Alias(e, toPrettySQL(e))()
+ case e: ExtractValue =>
+ val onlyExtract = e.collect {
+ case _: ExtractValue => true
+ case _ => false
+ }.forall(a => a)
+ if (onlyExtract) {
+ Alias(e, toPrettySQL(e))()
+ } else {
+ val meta = new MetadataBuilder()
+ .putString("autoGeneratedAlias", "true")
Review comment:
could we change this to something like `__unstableAlias`? because for example, the alias for `extractOnly` is also auto-generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872005977
**[Test build #140499 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140499/testReport)** for PR 32832 at commit [`69d5912`](https://github.com/apache/spark/commit/69d5912cf035d0a25ec99ee62c109eb9c1d770da).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-867312984
**[Test build #140240 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140240/testReport)** for PR 32832 at commit [`dfce0fe`](https://github.com/apache/spark/commit/dfce0fe053524fa31cdd01fd292743644f2e3227).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32832:
URL: https://github.com/apache/spark/pull/32832#discussion_r659609709
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala
##########
@@ -910,4 +910,45 @@ abstract class SQLViewSuite extends QueryTest with SQLTestUtils {
}
}
}
+
+ test("SPARK-35686: error out for creating view with auto gen alias") {
+ withView("v") {
+ val e = intercept[AnalysisException] {
+ sql("CREATE VIEW v AS SELECT count(*) FROM VALUES (1), (2), (3) t(a)")
+ }
+ assert(e.getMessage.contains("without explicitly assigning an alias"))
+ sql("CREATE VIEW v AS SELECT count(*) AS cnt FROM VALUES (1), (2), (3) t(a)")
+ checkAnswer(sql("SELECT * FROM v"), Seq(Row(3)))
+ }
+ }
+
+ test("SPARK-35686: error out for creating view with auto gen alias - nested type") {
+ withView("v") {
+ val e = intercept[AnalysisException] {
+ sql("CREATE VIEW v AS SELECT array(1, 2, 3)[0]")
Review comment:
This is not a very interesting case. How about testing a real nested case like `SELECT * FROM (SELECT a + b ...)`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-870825342
**[Test build #140379 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140379/testReport)** for PR 32832 at commit [`a98c833`](https://github.com/apache/spark/commit/a98c833828892d0b8d5cc40b41664f9be959de14).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-868379883
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140317/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-867350755
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140239/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on a change in pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #32832:
URL: https://github.com/apache/spark/pull/32832#discussion_r661660462
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -424,6 +424,12 @@ class Analyzer(override val catalogManager: CatalogManager)
*/
object ResolveAliases extends Rule[LogicalPlan] {
private def assignAliases(exprs: Seq[NamedExpression]) = {
+ def extractOnly(e: Expression): Boolean = e match {
+ case _: ExtractValue => e.children.forall(extractOnly)
+ case _: Literal => true
Review comment:
@cloud-fan I don't think the alias of literal will be changed. And in order to keep current test cases passing. I prefer allowing the literal for non ExtractValue as well. Is this OK?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872127964
**[Test build #140513 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140513/testReport)** for PR 32832 at commit [`751a55e`](https://github.com/apache/spark/commit/751a55e65033e9bb2c279acee6a300e6d61c6317).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-857699406
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44099/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on a change in pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #32832:
URL: https://github.com/apache/spark/pull/32832#discussion_r662049294
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala
##########
@@ -910,4 +910,45 @@ abstract class SQLViewSuite extends QueryTest with SQLTestUtils {
}
}
}
+
+ test("SPARK-35686: error out for creating view with auto gen alias") {
Review comment:
currently the SQLViewTestSuite only supports run a case against all 3 types. it's not able to specify a single view type. Maybe we can start to unify the `SQLViewSuite` and `SQLViewTestSuite`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32832:
URL: https://github.com/apache/spark/pull/32832#discussion_r657623564
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala
##########
@@ -534,6 +535,14 @@ object ViewHelper extends SQLConfHelper with Logging {
}
}
+ def verifyAutoGeneratedAliasesNotExists(child: LogicalPlan, name: TableIdentifier): Unit = {
+ child.expressions.foreach {
Review comment:
We should check `child.output`, instead of expressions. e.g.
```
SELECT * FROM (SELECT a + b FROM t)
```
The root plan here is a `Project` with `AttributeReference` only, and the check here can't catch it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-857699377
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44099/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872126414
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45013/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-867329401
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44768/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-868437496
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44849/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-857699406
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44099/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-868379883
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140317/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-867329401
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44768/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-868402421
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44849/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-857365152
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44055/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-867329383
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44768/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-857364878
**[Test build #139531 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139531/testReport)** for PR 32832 at commit [`251ca26`](https://github.com/apache/spark/commit/251ca26be44fca19d0bef17e99be27775043c68f).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-870981650
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44902/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872348801
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140513/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-871989387
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44996/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872171368
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45013/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32832:
URL: https://github.com/apache/spark/pull/32832#discussion_r659609709
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala
##########
@@ -910,4 +910,45 @@ abstract class SQLViewSuite extends QueryTest with SQLTestUtils {
}
}
}
+
+ test("SPARK-35686: error out for creating view with auto gen alias") {
+ withView("v") {
+ val e = intercept[AnalysisException] {
+ sql("CREATE VIEW v AS SELECT count(*) FROM VALUES (1), (2), (3) t(a)")
+ }
+ assert(e.getMessage.contains("without explicitly assigning an alias"))
+ sql("CREATE VIEW v AS SELECT count(*) AS cnt FROM VALUES (1), (2), (3) t(a)")
+ checkAnswer(sql("SELECT * FROM v"), Seq(Row(3)))
+ }
+ }
+
+ test("SPARK-35686: error out for creating view with auto gen alias - nested type") {
+ withView("v") {
+ val e = intercept[AnalysisException] {
+ sql("CREATE VIEW v AS SELECT array(1, 2, 3)[0]")
Review comment:
This is not a very interesting case. How about testing `SELECT * FROM (SELECT a + b ...)`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32832:
URL: https://github.com/apache/spark/pull/32832#discussion_r657622410
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -432,10 +432,26 @@ class Analyzer(override val catalogManager: CatalogManager)
case e if !e.resolved => u
case g: Generator => MultiAlias(g, Nil)
case c @ Cast(ne: NamedExpression, _, _, _) => Alias(c, ne.name)()
- case e: ExtractValue => Alias(e, toPrettySQL(e))()
+ case e: ExtractValue =>
+ val onlyExtract = e.collect {
+ case _: ExtractValue => true
+ case _ => false
+ }.forall(a => a)
+ if (onlyExtract) {
+ Alias(e, toPrettySQL(e))()
+ } else {
+ val meta = new MetadataBuilder()
+ .putString("autoGeneratedAlias", "true")
Review comment:
it's an internal metadata, let's name it `__ autoGeneratedAlias `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-867346076
**[Test build #140240 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140240/testReport)** for PR 32832 at commit [`dfce0fe`](https://github.com/apache/spark/commit/dfce0fe053524fa31cdd01fd292743644f2e3227).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `public class MergedBlockMetaRequest extends AbstractMessage implements RequestMessage `
* `public class MergedBlockMetaSuccess extends AbstractResponseMessage `
* `public abstract class AbstractFetchShuffleBlocks extends BlockTransferMessage `
* `public class FetchShuffleBlockChunks extends AbstractFetchShuffleBlocks `
* `public class FetchShuffleBlocks extends AbstractFetchShuffleBlocks `
* `case class ShuffleBlockInfo(shuffleId: Int, mapId: Long) `
* `case class ShuffleMergedDataBlockId(appId: String, shuffleId: Int, reduceId: Int) extends BlockId `
* `case class ShuffleMergedIndexBlockId(`
* `case class ShuffleMergedMetaBlockId(`
* ` class AvroSchemaHelper(avroSchema: Schema, avroPath: Seq[String]) `
* `class DecimalOps(FractionalOps):`
* `class IntegralExtensionOps(IntegralOps):`
* `class FractionalExtensionOps(FractionalOps):`
* `class StringExtensionOps(StringOps):`
* `class GroupBy(Generic[T_Frame], metaclass=ABCMeta):`
* `class DataFrameGroupBy(GroupBy[DataFrame]):`
* `class SeriesGroupBy(GroupBy[Series]):`
* `class SparkIndexOpsMethods(Generic[T_IndexOps], metaclass=ABCMeta):`
* `class SparkSeriesMethods(SparkIndexOpsMethods[\"ps.Series\"]):`
* `class SparkIndexMethods(SparkIndexOpsMethods[\"ps.Index\"]):`
* `class KubernetesLocalDiskShuffleDataIO(sparkConf: SparkConf) extends ShuffleDataIO `
* `class KubernetesLocalDiskShuffleExecutorComponents(sparkConf: SparkConf)`
* `case class TempResolvedColumn(child: Expression, nameParts: Seq[String]) extends UnaryExpression`
* `case class Cast(`
* `class ExpressionContainmentOrdering extends Ordering[Expression] `
* `case class GetTimestampWithoutTZ(`
* `case class ParseToTimestampWithoutTZ(`
* `case class MakeDTInterval(`
* `case class MakeYMInterval(years: Expression, months: Expression)`
* `trait InvokeLike extends Expression with NonSQLExpression with ImplicitCastInputTypes `
* `case class LateralSubquery(`
* `case class LateralJoin(`
* `case class DayTimeIntervalType(startField: Byte, endField: Byte) extends AtomicType `
* `case class YearMonthIntervalType(startField: Byte, endField: Byte) extends AtomicType `
* `final class ParquetReadState `
* `public class ParquetVectorUpdaterFactory `
* `case class CommandResult(`
* `case class CommandResultExec(`
* `case class MergingSessionsExec(`
* `class MergingSessionsIterator(`
* `class RocksDBFileManager(`
* `case class StateStoreCustomSumMetric(name: String, desc: String) extends StateStoreCustomMetric `
* `case class StateStoreCustomSizeMetric(name: String, desc: String) extends StateStoreCustomMetric `
* `case class StateStoreCustomTimingMetric(name: String, desc: String) extends StateStoreCustomMetric `
* `trait TestGroupState[S] extends GroupState[S] `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872202815
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45026/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-868327194
**[Test build #140317 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140317/testReport)** for PR 32832 at commit [`bb3b750`](https://github.com/apache/spark/commit/bb3b7504a12113bfea6e8b398766f60c1d733f09).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872151980
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45013/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #32832:
URL: https://github.com/apache/spark/pull/32832
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872004123
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140484/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32832:
URL: https://github.com/apache/spark/pull/32832#discussion_r661940720
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -424,6 +424,12 @@ class Analyzer(override val catalogManager: CatalogManager)
*/
object ResolveAliases extends Rule[LogicalPlan] {
private def assignAliases(exprs: Seq[NamedExpression]) = {
+ def extractOnly(e: Expression): Boolean = e match {
+ case _: ExtractValue => e.children.forall(extractOnly)
+ case _: Literal => true
Review comment:
sure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-870985364
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44902/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-857661606
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139572/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872127964
**[Test build #140513 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140513/testReport)** for PR 32832 at commit [`751a55e`](https://github.com/apache/spark/commit/751a55e65033e9bb2c279acee6a300e6d61c6317).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872003261
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44996/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-871968715
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44996/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-868437496
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44849/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32832:
URL: https://github.com/apache/spark/pull/32832#discussion_r659949686
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -424,6 +424,12 @@ class Analyzer(override val catalogManager: CatalogManager)
*/
object ResolveAliases extends Rule[LogicalPlan] {
private def assignAliases(exprs: Seq[NamedExpression]) = {
+ def extractOnly(e: Expression): Boolean = e match {
+ case _: ExtractValue => e.children.forall(extractOnly)
+ case _: Literal => true
Review comment:
or maybe we shouldn't allow literal here either
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-867312984
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32832:
URL: https://github.com/apache/spark/pull/32832#discussion_r659948827
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -424,6 +424,12 @@ class Analyzer(override val catalogManager: CatalogManager)
*/
object ResolveAliases extends Rule[LogicalPlan] {
private def assignAliases(exprs: Seq[NamedExpression]) = {
+ def extractOnly(e: Expression): Boolean = e match {
+ case _: ExtractValue => e.children.forall(extractOnly)
+ case _: Literal => true
Review comment:
since we allow literal here, maybe we should allow it in the non `ExtractValue` case as well
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -424,6 +424,12 @@ class Analyzer(override val catalogManager: CatalogManager)
*/
object ResolveAliases extends Rule[LogicalPlan] {
private def assignAliases(exprs: Seq[NamedExpression]) = {
+ def extractOnly(e: Expression): Boolean = e match {
+ case _: ExtractValue => e.children.forall(extractOnly)
+ case _: Literal => true
Review comment:
or maybe we shouldn't allow literal here either
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-857657092
**[Test build #139572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139572/testReport)** for PR 32832 at commit [`58af84f`](https://github.com/apache/spark/commit/58af84f12d12d7b59f5eb5fbf267533709eb7ccb).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872207962
**[Test build #140499 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140499/testReport)** for PR 32832 at commit [`69d5912`](https://github.com/apache/spark/commit/69d5912cf035d0a25ec99ee62c109eb9c1d770da).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-868420400
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44849/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-867326642
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44768/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872003906
**[Test build #140484 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140484/testReport)** for PR 32832 at commit [`da3998c`](https://github.com/apache/spark/commit/da3998cb719a5938ca026af5930082c42c04e48a).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-857364900
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139531/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-867350090
**[Test build #140239 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140239/testReport)** for PR 32832 at commit [`86673c0`](https://github.com/apache/spark/commit/86673c04748c964eda42d94266d4d1f6fcbe6c53).
* This patch **fails Spark unit tests**.
* This patch **does not merge cleanly**.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-867330556
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44769/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-857364900
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139531/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-857365152
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44055/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-857657092
**[Test build #139572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139572/testReport)** for PR 32832 at commit [`58af84f`](https://github.com/apache/spark/commit/58af84f12d12d7b59f5eb5fbf267533709eb7ccb).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872005977
**[Test build #140499 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140499/testReport)** for PR 32832 at commit [`69d5912`](https://github.com/apache/spark/commit/69d5912cf035d0a25ec99ee62c109eb9c1d770da).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-871937014
**[Test build #140484 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140484/testReport)** for PR 32832 at commit [`da3998c`](https://github.com/apache/spark/commit/da3998cb719a5938ca026af5930082c42c04e48a).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872217246
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-871937014
**[Test build #140484 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140484/testReport)** for PR 32832 at commit [`da3998c`](https://github.com/apache/spark/commit/da3998cb719a5938ca026af5930082c42c04e48a).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872004123
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140484/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-870933102
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140379/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872211239
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-857362781
**[Test build #139531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139531/testReport)** for PR 32832 at commit [`251ca26`](https://github.com/apache/spark/commit/251ca26be44fca19d0bef17e99be27775043c68f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-857365132
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44055/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-870985364
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44902/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32832:
URL: https://github.com/apache/spark/pull/32832#discussion_r659948827
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -424,6 +424,12 @@ class Analyzer(override val catalogManager: CatalogManager)
*/
object ResolveAliases extends Rule[LogicalPlan] {
private def assignAliases(exprs: Seq[NamedExpression]) = {
+ def extractOnly(e: Expression): Boolean = e match {
+ case _: ExtractValue => e.children.forall(extractOnly)
+ case _: Literal => true
Review comment:
since we allow literal here, maybe we should allow it in the non `ExtractValue` case as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-867333835
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44769/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-868379351
**[Test build #140317 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140317/testReport)** for PR 32832 at commit [`bb3b750`](https://github.com/apache/spark/commit/bb3b7504a12113bfea6e8b398766f60c1d733f09).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-870933102
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140379/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32832:
URL: https://github.com/apache/spark/pull/32832#discussion_r655098545
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -432,10 +432,22 @@ class Analyzer(override val catalogManager: CatalogManager)
case e if !e.resolved => u
case g: Generator => MultiAlias(g, Nil)
case c @ Cast(ne: NamedExpression, _, _) => Alias(c, ne.name)()
- case e: ExtractValue => Alias(e, toPrettySQL(e))()
+ case e: ExtractValue =>
Review comment:
We can allow an expression tree that contains only `ExtractValue` and attributes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872171368
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45013/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-857661606
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139572/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-868327194
**[Test build #140317 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140317/testReport)** for PR 32832 at commit [`bb3b750`](https://github.com/apache/spark/commit/bb3b7504a12113bfea6e8b398766f60c1d733f09).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [SPARK-35686][SQL] Not allow using auto-generated alias when creating view
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32832:
URL: https://github.com/apache/spark/pull/32832#issuecomment-872348801
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140513/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org