You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by maropu <gi...@git.apache.org> on 2017/04/18 01:09:45 UTC
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
GitHub user maropu opened a pull request:
https://github.com/apache/spark/pull/17666
[SPARK-20311][SQL] Support aliases for table value functions
## What changes were proposed in this pull request?
This pr added parsing rules to support aliases in table value functions.
## How was this patch tested?
Added tests in `PlanParserSuite`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/maropu/spark SPARK-20311
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/17666.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #17666
----
commit a611a13ceef3b1f3431355a9caf1965bc5793e55
Author: Takeshi Yamamuro <ya...@apache.org>
Date: 2017-04-18T00:39:26Z
Support aliases for table value functions
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17666
thanks, merging to master/2.2!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76650/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76615/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/17666
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #76604 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76604/testReport)** for PR 17666 at commit [`399d823`](https://github.com/apache/spark/commit/399d823b13719c7623013164f284cb6668c88218).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #76615 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76615/testReport)** for PR 17666 at commit [`81bef3b`](https://github.com/apache/spark/commit/81bef3ba21cb0c3e4b36f3fc492d9ab3a3124829).
* This patch **fails SparkR unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #76604 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76604/testReport)** for PR 17666 at commit [`399d823`](https://github.com/apache/spark/commit/399d823b13719c7623013164f284cb6668c88218).
* This patch **fails SparkR unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76604/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/17666
ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r111955189
--- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -472,11 +472,11 @@ identifierComment
;
relationPrimary
- : tableIdentifier sample? (AS? strictIdentifier)? #tableName
- | '(' queryNoWith ')' sample? (AS? strictIdentifier)? #aliasedQuery
- | '(' relation ')' sample? (AS? strictIdentifier)? #aliasedRelation
- | inlineTable #inlineTableDefault2
- | identifier '(' (expression (',' expression)*)? ')' #tableValuedFunction
+ : tableIdentifier sample? (AS? strictIdentifier)? #tableName
+ | '(' queryNoWith ')' sample? (AS? strictIdentifier)? #aliasedQuery
+ | '(' relation ')' sample? (AS? strictIdentifier)? #aliasedRelation
+ | inlineTable #inlineTableDefault2
+ | identifier '(' (expression (',' expression)*)? ')' (AS? identifier identifierList?)? #tableValuedFunction
--- End diff --
oh, it seems I misunderstood what you pointed out. You meant should we need to support a query like `SELECT * FROM [[tvf]] AS t(a, b, ...)` in this pr? Yea, I know we currently support `range` only as a table value function though, I also think it'd be better to put a more general rule in this file. So +1 for keeping this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #76608 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76608/testReport)** for PR 17666 at commit [`625dbda`](https://github.com/apache/spark/commit/625dbda3aab90922d6301f044dc90746d2ffb238).
* This patch **fails SparkR unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #75889 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75889/testReport)** for PR 17666 at commit [`539a9e8`](https://github.com/apache/spark/commit/539a9e82aa306b0f4a5fa0cd32121732e9e3ef15).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class UnresolvedTableValuedFunction(`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17666
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r115361494
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala ---
@@ -57,19 +57,21 @@ object ResolveTableValuedFunctions extends Rule[LogicalPlan] {
* A TVF maps argument lists to resolver functions that accept those arguments. Using a map
* here allows for function overloading.
*/
- private type TVF = Map[ArgumentList, Seq[Any] => LogicalPlan]
+ private type TVF = Map[ArgumentList, (UnresolvedTableValuedFunction, Seq[Any]) => LogicalPlan]
/**
* TVF builder.
*/
- private def tvf(args: (String, DataType)*)(pf: PartialFunction[Seq[Any], LogicalPlan])
- : (ArgumentList, Seq[Any] => LogicalPlan) = {
+ private def tvf(args: (String, DataType)*)(
+ pf: PartialFunction[(UnresolvedTableValuedFunction, Seq[Any]), LogicalPlan])
--- End diff --
I also think that we should separate the aliasing from constructing the table valued function
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r115317931
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala ---
@@ -78,37 +80,52 @@ object ResolveTableValuedFunctions extends Rule[LogicalPlan] {
private val builtinFunctions: Map[String, TVF] = Map(
"range" -> Map(
/* range(end) */
- tvf("end" -> LongType) { case Seq(end: Long) =>
- Range(0, end, 1, None)
+ tvf("end" -> LongType) { case (tvf, args @ Seq(end: Long)) =>
+ validateInputDimension(tvf, 1)
+ Range(0, end, 1, None, tvf.outputNames.headOption)
},
/* range(start, end) */
- tvf("start" -> LongType, "end" -> LongType) { case Seq(start: Long, end: Long) =>
- Range(start, end, 1, None)
+ tvf("start" -> LongType, "end" -> LongType) {
+ case (tvf, args @ Seq(start: Long, end: Long)) =>
+ validateInputDimension(tvf, 1)
+ Range(start, end, 1, None, tvf.outputNames.headOption)
},
/* range(start, end, step) */
tvf("start" -> LongType, "end" -> LongType, "step" -> LongType) {
- case Seq(start: Long, end: Long, step: Long) =>
- Range(start, end, step, None)
+ case (tvf, args @ Seq(start: Long, end: Long, step: Long)) =>
+ validateInputDimension(tvf, 1)
+ Range(start, end, step, None, tvf.outputNames.headOption)
},
/* range(start, end, step, numPartitions) */
tvf("start" -> LongType, "end" -> LongType, "step" -> LongType,
"numPartitions" -> IntegerType) {
- case Seq(start: Long, end: Long, step: Long, numPartitions: Int) =>
- Range(start, end, step, Some(numPartitions))
+ case (tvf, args @ Seq(start: Long, end: Long, step: Long, numPartitions: Int)) =>
+ validateInputDimension(tvf, 1)
+ Range(start, end, step, Some(numPartitions), tvf.outputNames.headOption)
})
)
+ private def validateInputDimension(tvf: UnresolvedTableValuedFunction, expectedNumCols: Int)
--- End diff --
Conceptually, `expectedNumCols` needs to be part of `type/class TVF`. This is the num of TVF's output arguments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r115357687
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ---
@@ -498,12 +498,16 @@ case class Sort(
/** Factory for constructing new `Range` nodes. */
object Range {
- def apply(start: Long, end: Long, step: Long, numSlices: Option[Int]): Range = {
- val output = StructType(StructField("id", LongType, nullable = false) :: Nil).toAttributes
+
+ def apply(start: Long, end: Long, step: Long, numSlices: Option[Int], outputName: Option[String])
--- End diff --
Why make this an option? It is not optional right? The next function should either call Range(start, end, step, Some(numSlices), "id") or this function should have a default parameter.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/17666
@maropu Sorry. I think this PR introduces a regression.
```
scala> spark.sql("select * from range(1, 10) cross join range(1, 10)").explain
== Physical Plan ==
org.apache.spark.sql.AnalysisException: Detected cartesian product for INNER join between logical plans
Range (1, 10, step=1, splits=None)
and
Range (1, 10, step=1, splits=None)
Join condition is missing or trivial.
Use the CROSS JOIN syntax to allow cartesian products between these relations.;
```
I think we are taking the cross as the alias.
I reverted your change locally and the query worked. I am attaching the expected analyzed plan below.
```
scala> spark.sql("select * from range(1, 10) cross join range(1, 10)").queryExecution.analyzed
res1: org.apache.spark.sql.catalyst.plans.logical.LogicalPlan =
Project [id#8L, id#9L]
+- Join Cross
:- Range (1, 10, step=1, splits=None)
+- Range (1, 10, step=1, splits=None)
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r115318006
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala ---
@@ -78,37 +80,52 @@ object ResolveTableValuedFunctions extends Rule[LogicalPlan] {
private val builtinFunctions: Map[String, TVF] = Map(
"range" -> Map(
/* range(end) */
- tvf("end" -> LongType) { case Seq(end: Long) =>
- Range(0, end, 1, None)
+ tvf("end" -> LongType) { case (tvf, args @ Seq(end: Long)) =>
+ validateInputDimension(tvf, 1)
+ Range(0, end, 1, None, tvf.outputNames.headOption)
},
/* range(start, end) */
- tvf("start" -> LongType, "end" -> LongType) { case Seq(start: Long, end: Long) =>
- Range(start, end, 1, None)
+ tvf("start" -> LongType, "end" -> LongType) {
+ case (tvf, args @ Seq(start: Long, end: Long)) =>
+ validateInputDimension(tvf, 1)
+ Range(start, end, 1, None, tvf.outputNames.headOption)
},
/* range(start, end, step) */
tvf("start" -> LongType, "end" -> LongType, "step" -> LongType) {
- case Seq(start: Long, end: Long, step: Long) =>
- Range(start, end, step, None)
+ case (tvf, args @ Seq(start: Long, end: Long, step: Long)) =>
+ validateInputDimension(tvf, 1)
+ Range(start, end, step, None, tvf.outputNames.headOption)
},
/* range(start, end, step, numPartitions) */
tvf("start" -> LongType, "end" -> LongType, "step" -> LongType,
"numPartitions" -> IntegerType) {
- case Seq(start: Long, end: Long, step: Long, numPartitions: Int) =>
- Range(start, end, step, Some(numPartitions))
+ case (tvf, args @ Seq(start: Long, end: Long, step: Long, numPartitions: Int)) =>
+ validateInputDimension(tvf, 1)
+ Range(start, end, step, Some(numPartitions), tvf.outputNames.headOption)
})
)
+ private def validateInputDimension(tvf: UnresolvedTableValuedFunction, expectedNumCols: Int)
--- End diff --
BTW, we need to add comments to this function.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r115297323
--- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -472,15 +472,19 @@ identifierComment
;
relationPrimary
- : tableIdentifier sample? (AS? strictIdentifier)? #tableName
- | '(' queryNoWith ')' sample? (AS? strictIdentifier)? #aliasedQuery
- | '(' relation ')' sample? (AS? strictIdentifier)? #aliasedRelation
- | inlineTable #inlineTableDefault2
- | identifier '(' (expression (',' expression)*)? ')' #tableValuedFunction
+ : tableIdentifier sample? (AS? strictIdentifier)? #tableName
+ | '(' queryNoWith ')' sample? (AS? strictIdentifier)? #aliasedQuery
+ | '(' relation ')' sample? (AS? strictIdentifier)? #aliasedRelation
+ | inlineTable #inlineTableDefault2
+ | identifier '(' (expression (',' expression)*)? ')' tableAlias #tableValuedFunction
--- End diff --
Please follow [`inlineTable`](https://github.com/maropu/spark/blob/8d3a03712a4b3f55e60819cb8b6b3ec3454e12ab/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4#L482-L484) to add a separate rule.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75876/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #75888 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75888/testReport)** for PR 17666 at commit [`539a9e8`](https://github.com/apache/spark/commit/539a9e82aa306b0f4a5fa0cd32121732e9e3ef15).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r111951527
--- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -472,11 +472,11 @@ identifierComment
;
relationPrimary
- : tableIdentifier sample? (AS? strictIdentifier)? #tableName
- | '(' queryNoWith ')' sample? (AS? strictIdentifier)? #aliasedQuery
- | '(' relation ')' sample? (AS? strictIdentifier)? #aliasedRelation
- | inlineTable #inlineTableDefault2
- | identifier '(' (expression (',' expression)*)? ')' #tableValuedFunction
+ : tableIdentifier sample? (AS? strictIdentifier)? #tableName
+ | '(' queryNoWith ')' sample? (AS? strictIdentifier)? #aliasedQuery
+ | '(' relation ')' sample? (AS? strictIdentifier)? #aliasedRelation
+ | inlineTable #inlineTableDefault2
+ | identifier '(' (expression (',' expression)*)? ')' (AS? identifier identifierList?)? #tableValuedFunction
--- End diff --
I missed the point. okay, I'll reconsider this. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #75889 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75889/testReport)** for PR 17666 at commit [`539a9e8`](https://github.com/apache/spark/commit/539a9e82aa306b0f4a5fa0cd32121732e9e3ef15).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r111955658
--- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -472,11 +472,11 @@ identifierComment
;
relationPrimary
- : tableIdentifier sample? (AS? strictIdentifier)? #tableName
- | '(' queryNoWith ')' sample? (AS? strictIdentifier)? #aliasedQuery
- | '(' relation ')' sample? (AS? strictIdentifier)? #aliasedRelation
- | inlineTable #inlineTableDefault2
- | identifier '(' (expression (',' expression)*)? ')' #tableValuedFunction
+ : tableIdentifier sample? (AS? strictIdentifier)? #tableName
+ | '(' queryNoWith ')' sample? (AS? strictIdentifier)? #aliasedQuery
+ | '(' relation ')' sample? (AS? strictIdentifier)? #aliasedRelation
+ | inlineTable #inlineTableDefault2
+ | identifier '(' (expression (',' expression)*)? ')' (AS? identifier identifierList?)? #tableValuedFunction
--- End diff --
Then, I'll update this pr to separate this rule and share it with the inline table rule.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #75903 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75903/testReport)** for PR 17666 at commit [`8d3a037`](https://github.com/apache/spark/commit/8d3a03712a4b3f55e60819cb8b6b3ec3454e12ab).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r111952174
--- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -472,11 +472,11 @@ identifierComment
;
relationPrimary
- : tableIdentifier sample? (AS? strictIdentifier)? #tableName
- | '(' queryNoWith ')' sample? (AS? strictIdentifier)? #aliasedQuery
- | '(' relation ')' sample? (AS? strictIdentifier)? #aliasedRelation
- | inlineTable #inlineTableDefault2
- | identifier '(' (expression (',' expression)*)? ')' #tableValuedFunction
+ : tableIdentifier sample? (AS? strictIdentifier)? #tableName
+ | '(' queryNoWith ')' sample? (AS? strictIdentifier)? #aliasedQuery
+ | '(' relation ')' sample? (AS? strictIdentifier)? #aliasedRelation
+ | inlineTable #inlineTableDefault2
+ | identifier '(' (expression (',' expression)*)? ')' (AS? identifier identifierList?)? #tableValuedFunction
--- End diff --
It makes sense to add this. Lets keep the multi-alias for now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r115296724
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala ---
@@ -69,7 +69,10 @@ case class UnresolvedInlineTable(
* select * from range(10);
--- End diff --
Please update this example.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/17666
@hvanhovell Could you check again?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/17666
Please rebase to pick up fix for the R tests.
Though again, I'm not sure why it is running R tests for this PR - is the change detection logic broken somehow?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75903/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #75876 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75876/testReport)** for PR 17666 at commit [`a611a13`](https://github.com/apache/spark/commit/a611a13ceef3b1f3431355a9caf1965bc5793e55).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r115374896
--- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -472,15 +472,19 @@ identifierComment
;
relationPrimary
- : tableIdentifier sample? (AS? strictIdentifier)? #tableName
- | '(' queryNoWith ')' sample? (AS? strictIdentifier)? #aliasedQuery
- | '(' relation ')' sample? (AS? strictIdentifier)? #aliasedRelation
- | inlineTable #inlineTableDefault2
- | identifier '(' (expression (',' expression)*)? ')' #tableValuedFunction
+ : tableIdentifier sample? (AS? strictIdentifier)? #tableName
+ | '(' queryNoWith ')' sample? (AS? strictIdentifier)? #aliasedQuery
+ | '(' relation ')' sample? (AS? strictIdentifier)? #aliasedRelation
+ | inlineTable #inlineTableDefault2
+ | identifier '(' (expression (',' expression)*)? ')' tableAlias #tableValuedFunction
--- End diff --
ok
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/17666
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r111951160
--- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -472,11 +472,11 @@ identifierComment
;
relationPrimary
- : tableIdentifier sample? (AS? strictIdentifier)? #tableName
- | '(' queryNoWith ')' sample? (AS? strictIdentifier)? #aliasedQuery
- | '(' relation ')' sample? (AS? strictIdentifier)? #aliasedRelation
- | inlineTable #inlineTableDefault2
- | identifier '(' (expression (',' expression)*)? ')' #tableValuedFunction
+ : tableIdentifier sample? (AS? strictIdentifier)? #tableName
+ | '(' queryNoWith ')' sample? (AS? strictIdentifier)? #aliasedQuery
+ | '(' relation ')' sample? (AS? strictIdentifier)? #aliasedRelation
+ | inlineTable #inlineTableDefault2
+ | identifier '(' (expression (',' expression)*)? ')' (AS? identifier identifierList?)? #tableValuedFunction
--- End diff --
Why did you add the multi-alias anyway?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r115375400
--- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala ---
@@ -441,4 +440,15 @@ class AnalysisSuite extends AnalysisTest with ShouldMatchers {
checkAnalysis(SubqueryAlias("tbl", testRelation).as("tbl2"), testRelation)
}
+
+ test("SPARK-20311 range(N) as alias") {
+ def rangeWithAliases(outputNames: Seq[String]): LogicalPlan = {
+ SubqueryAlias("t", UnresolvedTableValuedFunction("range", Literal(7) :: Nil, outputNames))
--- End diff --
ok, I'll add more tests.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r115320568
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala ---
@@ -78,37 +80,52 @@ object ResolveTableValuedFunctions extends Rule[LogicalPlan] {
private val builtinFunctions: Map[String, TVF] = Map(
"range" -> Map(
/* range(end) */
- tvf("end" -> LongType) { case Seq(end: Long) =>
- Range(0, end, 1, None)
+ tvf("end" -> LongType) { case (tvf, args @ Seq(end: Long)) =>
+ validateInputDimension(tvf, 1)
+ Range(0, end, 1, None, tvf.outputNames.headOption)
--- End diff --
Instead of changing the output of `Range`, I think we can simply add a `Project` above Range for column alias, like what we do in the Dataframe API: `spark.range(100).toDF("i")`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/17666
I am going to revert this PR from master and branch-2.2.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/17666
LGTM.
Thank you! @maropu
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75888/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r115375162
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala ---
@@ -57,19 +57,21 @@ object ResolveTableValuedFunctions extends Rule[LogicalPlan] {
* A TVF maps argument lists to resolver functions that accept those arguments. Using a map
* here allows for function overloading.
*/
- private type TVF = Map[ArgumentList, Seq[Any] => LogicalPlan]
+ private type TVF = Map[ArgumentList, (UnresolvedTableValuedFunction, Seq[Any]) => LogicalPlan]
/**
* TVF builder.
*/
- private def tvf(args: (String, DataType)*)(pf: PartialFunction[Seq[Any], LogicalPlan])
- : (ArgumentList, Seq[Any] => LogicalPlan) = {
+ private def tvf(args: (String, DataType)*)(
+ pf: PartialFunction[(UnresolvedTableValuedFunction, Seq[Any]), LogicalPlan])
--- End diff --
ok, I'll re-consider the signature.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #76608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76608/testReport)** for PR 17666 at commit [`625dbda`](https://github.com/apache/spark/commit/625dbda3aab90922d6301f044dc90746d2ffb238).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/17666
I have reverted this change from both master and branch-2.2. I have reopened the jira.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/17666
@yhuai okay, thanks for letting me know! I'll make a new pr to fix.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/17666
@hvanhovell ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/17666
ok, thanks! no, I think I do not touch on that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #76602 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76602/testReport)** for PR 17666 at commit [`f494e41`](https://github.com/apache/spark/commit/f494e417557539369c2a5c6ee472d9697937a587).
* This patch passes all tests.
* This patch **does not merge cleanly**.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #75876 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75876/testReport)** for PR 17666 at commit [`a611a13`](https://github.com/apache/spark/commit/a611a13ceef3b1f3431355a9caf1965bc5793e55).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class UnresolvedTableValuedFunction(`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r115394355
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala ---
@@ -78,37 +80,52 @@ object ResolveTableValuedFunctions extends Rule[LogicalPlan] {
private val builtinFunctions: Map[String, TVF] = Map(
"range" -> Map(
/* range(end) */
- tvf("end" -> LongType) { case Seq(end: Long) =>
- Range(0, end, 1, None)
+ tvf("end" -> LongType) { case (tvf, args @ Seq(end: Long)) =>
+ validateInputDimension(tvf, 1)
+ Range(0, end, 1, None, tvf.outputNames.headOption)
--- End diff --
great suggestion! I could make it simpler (https://github.com/apache/spark/pull/17666/files#diff-e2023817ad13201adeb45b74e99d29f8R129). Thought?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r115358211
--- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala ---
@@ -441,4 +440,15 @@ class AnalysisSuite extends AnalysisTest with ShouldMatchers {
checkAnalysis(SubqueryAlias("tbl", testRelation).as("tbl2"), testRelation)
}
+
+ test("SPARK-20311 range(N) as alias") {
+ def rangeWithAliases(outputNames: Seq[String]): LogicalPlan = {
+ SubqueryAlias("t", UnresolvedTableValuedFunction("range", Literal(7) :: Nil, outputNames))
--- End diff --
Should we also test different cases?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r115361167
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala ---
@@ -57,19 +57,21 @@ object ResolveTableValuedFunctions extends Rule[LogicalPlan] {
* A TVF maps argument lists to resolver functions that accept those arguments. Using a map
* here allows for function overloading.
*/
- private type TVF = Map[ArgumentList, Seq[Any] => LogicalPlan]
+ private type TVF = Map[ArgumentList, (UnresolvedTableValuedFunction, Seq[Any]) => LogicalPlan]
/**
* TVF builder.
*/
- private def tvf(args: (String, DataType)*)(pf: PartialFunction[Seq[Any], LogicalPlan])
- : (ArgumentList, Seq[Any] => LogicalPlan) = {
+ private def tvf(args: (String, DataType)*)(
+ pf: PartialFunction[(UnresolvedTableValuedFunction, Seq[Any]), LogicalPlan])
--- End diff --
Hmmm... this signature is kind of complex. Can we try to use some kind of class/case class that encapsulates this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #76615 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76615/testReport)** for PR 17666 at commit [`81bef3b`](https://github.com/apache/spark/commit/81bef3ba21cb0c3e4b36f3fc492d9ab3a3124829).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76608/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/17666
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r115375986
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ---
@@ -498,12 +498,16 @@ case class Sort(
/** Factory for constructing new `Range` nodes. */
object Range {
- def apply(start: Long, end: Long, step: Long, numSlices: Option[Int]): Range = {
- val output = StructType(StructField("id", LongType, nullable = false) :: Nil).toAttributes
+
+ def apply(start: Long, end: Long, step: Long, numSlices: Option[Int], outputName: Option[String])
--- End diff --
yea, ok. I'll update. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76602/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r115394532
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala ---
@@ -57,19 +57,21 @@ object ResolveTableValuedFunctions extends Rule[LogicalPlan] {
* A TVF maps argument lists to resolver functions that accept those arguments. Using a map
* here allows for function overloading.
*/
- private type TVF = Map[ArgumentList, Seq[Any] => LogicalPlan]
+ private type TVF = Map[ArgumentList, (UnresolvedTableValuedFunction, Seq[Any]) => LogicalPlan]
/**
* TVF builder.
*/
- private def tvf(args: (String, DataType)*)(pf: PartialFunction[Seq[Any], LogicalPlan])
- : (ArgumentList, Seq[Any] => LogicalPlan) = {
+ private def tvf(args: (String, DataType)*)(
+ pf: PartialFunction[(UnresolvedTableValuedFunction, Seq[Any]), LogicalPlan])
--- End diff --
I a bit worry though, this fix satisfies your intention? https://github.com/apache/spark/pull/17666/commits/625dbda3aab90922d6301f044dc90746d2ffb238
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #76650 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76650/testReport)** for PR 17666 at commit [`81bef3b`](https://github.com/apache/spark/commit/81bef3ba21cb0c3e4b36f3fc492d9ab3a3124829).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/17666
@hvanhovell ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #76602 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76602/testReport)** for PR 17666 at commit [`f494e41`](https://github.com/apache/spark/commit/f494e417557539369c2a5c6ee472d9697937a587).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75889/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #76650 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76650/testReport)** for PR 17666 at commit [`81bef3b`](https://github.com/apache/spark/commit/81bef3ba21cb0c3e4b36f3fc492d9ab3a3124829).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17666
**[Test build #75903 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75903/testReport)** for PR 17666 at commit [`8d3a037`](https://github.com/apache/spark/commit/8d3a03712a4b3f55e60819cb8b6b3ec3454e12ab).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r115394606
--- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala ---
@@ -441,4 +440,15 @@ class AnalysisSuite extends AnalysisTest with ShouldMatchers {
checkAnalysis(SubqueryAlias("tbl", testRelation).as("tbl2"), testRelation)
}
+
+ test("SPARK-20311 range(N) as alias") {
+ def rangeWithAliases(outputNames: Seq[String]): LogicalPlan = {
+ SubqueryAlias("t", UnresolvedTableValuedFunction("range", Literal(7) :: Nil, outputNames))
--- End diff --
I a bit worry though, this fix satisfies your intention? https://github.com/apache/spark/pull/17666/commits/625dbda3aab90922d6301f044dc90746d2ffb238
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17666#discussion_r111950902
--- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -472,11 +472,11 @@ identifierComment
;
relationPrimary
- : tableIdentifier sample? (AS? strictIdentifier)? #tableName
- | '(' queryNoWith ')' sample? (AS? strictIdentifier)? #aliasedQuery
- | '(' relation ')' sample? (AS? strictIdentifier)? #aliasedRelation
- | inlineTable #inlineTableDefault2
- | identifier '(' (expression (',' expression)*)? ')' #tableValuedFunction
+ : tableIdentifier sample? (AS? strictIdentifier)? #tableName
+ | '(' queryNoWith ')' sample? (AS? strictIdentifier)? #aliasedQuery
+ | '(' relation ')' sample? (AS? strictIdentifier)? #aliasedRelation
+ | inlineTable #inlineTableDefault2
+ | identifier '(' (expression (',' expression)*)? ')' (AS? identifier identifierList?)? #tableValuedFunction
--- End diff --
Perhaps we should put the multi-alias in a seperate rule? Since it is also used by inline table.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17666: [SPARK-20311][SQL] Support aliases for table value funct...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17666
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org