You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/06/29 16:53:05 UTC
[GitHub] [spark] GuoPhilipse opened a new pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
GuoPhilipse opened a new pull request #28951:
URL: https://github.com/apache/spark/pull/28951
### What changes were proposed in this pull request?
fix error exception messages during exceptions on Union and set operations
### Why are the changes needed?
Union and set operations can only be performed on tables with the compatible column types,while when we have more than two column, the warning messages will have wrong column index.More details on in SPARK-32131
### Does this PR introduce _any_ user-facing change?
NO
### How was this patch tested?
unit test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dilipbiswal commented on a change in pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
dilipbiswal commented on a change in pull request #28951:
URL: https://github.com/apache/spark/pull/28951#discussion_r447426485
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala
##########
@@ -831,4 +831,77 @@ class AnalysisSuite extends AnalysisTest with Matchers {
}
}
}
+
+ test("SPARK-32131 Fix wrong column index when we have more than two columns" +
+ " during union and set operations" ) {
+ val firstTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", FloatType)())
+
+ val secondTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", TimestampType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", FloatType)())
+
+ val thirdTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", TimestampType)(),
+ AttributeReference("d", FloatType)())
+
+ val fourthTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", TimestampType)())
+
+ val a1 = firstTable.output(0)
Review comment:
@GuoPhilipse Variables a1, b1, c1, d1 not used ? Were you planning to use it in the test later ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #28951:
URL: https://github.com/apache/spark/pull/28951#discussion_r447476067
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala
##########
@@ -831,4 +831,57 @@ class AnalysisSuite extends AnalysisTest with Matchers {
}
}
}
+
+ test("SPARK-32131 Fix wrong column index when we have more than two columns" +
Review comment:
nit: `SPARK-32131` -> `SPARK-32131:`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28951: [SPARK-32131][SQL] Fix AnalysisException messages at UNION/EXCEPT/MINUS operations
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-652225009
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #28951: [SPARK-32131][SQL] Fix AnalysisException messages at UNION/EXCEPT/MINUS operations
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-652224404
Thank you all! This lands at `master/3.0/2.4`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] GuoPhilipse commented on a change in pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
GuoPhilipse commented on a change in pull request #28951:
URL: https://github.com/apache/spark/pull/28951#discussion_r447512748
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala
##########
@@ -831,4 +831,77 @@ class AnalysisSuite extends AnalysisTest with Matchers {
}
}
}
+
+ test("SPARK-32131 Fix wrong column index when we have more than two columns" +
+ " during union and set operations" ) {
+ val firstTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", FloatType)())
+
+ val secondTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", TimestampType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", FloatType)())
+
+ val thirdTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", TimestampType)(),
+ AttributeReference("d", FloatType)())
+
+ val fourthTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", TimestampType)())
+
+ val a1 = firstTable.output(0)
+ val b1 = firstTable.output(1)
+ val c1 = firstTable.output(2)
+ val d1 = firstTable.output(3)
+
+ val a2 = secondTable.output(0)
+ val b2 = secondTable.output(1)
+ val c2 = secondTable.output(2)
+ val d2 = secondTable.output(3)
+
+ val a3 = thirdTable.output(0)
+ val b3 = thirdTable.output(1)
+ val c3 = thirdTable.output(2)
+ val d3 = thirdTable.output(3)
+
+ val a4 = fourthTable.output(0)
Review comment:
Thanks @dilipbiswal for your review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-651610683
nit: `test2` not used in the PR description?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28951: [SPARK-32131][SQL] Fix AnalysisException messages at UNION/EXCEPT/MINUS operations
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-652225009
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] GuoPhilipse commented on a change in pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
GuoPhilipse commented on a change in pull request #28951:
URL: https://github.com/apache/spark/pull/28951#discussion_r447512274
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala
##########
@@ -831,4 +831,57 @@ class AnalysisSuite extends AnalysisTest with Matchers {
}
}
}
+
+ test("SPARK-32131 Fix wrong column index when we have more than two columns" +
Review comment:
Thanks maropu ,btw, I did not see the test build start to build, do you know why?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-652111401
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] GuoPhilipse commented on pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
GuoPhilipse commented on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-651692515
retest this please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28951: [SPARK-32131][SQL] Fix AnalysisException messages at UNION/EXCEPT/MINUS operations
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-652109787
**[Test build #124704 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124704/testReport)** for PR 28951 at commit [`984c652`](https://github.com/apache/spark/commit/984c65275bdf68bf8ff1f9ce3c91696a90e65439).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-652111401
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #28951:
URL: https://github.com/apache/spark/pull/28951#discussion_r447475628
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
##########
@@ -337,7 +337,8 @@ trait CheckAnalysis extends PredicateHelper {
def ordinalNumber(i: Int): String = i match {
case 0 => "first"
case 1 => "second"
- case i => s"${i}th"
+ case 2 => "third"
+ case i => s"${i + 1}th"
Review comment:
hehehe, nice catch ;)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] GuoPhilipse commented on pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
GuoPhilipse commented on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-651632797
> nit: `test2` not used in the PR description?
have updated in the PR, it is a normal case ,it is used for comparing between the normal and abnormal cases.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-652109787
**[Test build #124704 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124704/testReport)** for PR 28951 at commit [`984c652`](https://github.com/apache/spark/commit/984c65275bdf68bf8ff1f9ce3c91696a90e65439).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] holdenk commented on pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
holdenk commented on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-651415959
Good catch. LGTM but I'll leave it for a bit of a SQL committer has any thoughts.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #28951: [SPARK-32131][SQL] Fix AnalysisException messages at UNION/EXCEPT/MINUS operations
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #28951:
URL: https://github.com/apache/spark/pull/28951
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-652108311
ok to test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dilipbiswal commented on a change in pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
dilipbiswal commented on a change in pull request #28951:
URL: https://github.com/apache/spark/pull/28951#discussion_r447426571
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala
##########
@@ -831,4 +831,77 @@ class AnalysisSuite extends AnalysisTest with Matchers {
}
}
}
+
+ test("SPARK-32131 Fix wrong column index when we have more than two columns" +
+ " during union and set operations" ) {
+ val firstTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", FloatType)())
+
+ val secondTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", TimestampType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", FloatType)())
+
+ val thirdTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", TimestampType)(),
+ AttributeReference("d", FloatType)())
+
+ val fourthTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", TimestampType)())
+
+ val a1 = firstTable.output(0)
+ val b1 = firstTable.output(1)
+ val c1 = firstTable.output(2)
+ val d1 = firstTable.output(3)
+
+ val a2 = secondTable.output(0)
+ val b2 = secondTable.output(1)
+ val c2 = secondTable.output(2)
+ val d2 = secondTable.output(3)
+
+ val a3 = thirdTable.output(0)
Review comment:
ditto
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala
##########
@@ -831,4 +831,77 @@ class AnalysisSuite extends AnalysisTest with Matchers {
}
}
}
+
+ test("SPARK-32131 Fix wrong column index when we have more than two columns" +
+ " during union and set operations" ) {
+ val firstTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", FloatType)())
+
+ val secondTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", TimestampType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", FloatType)())
+
+ val thirdTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", TimestampType)(),
+ AttributeReference("d", FloatType)())
+
+ val fourthTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", TimestampType)())
+
+ val a1 = firstTable.output(0)
+ val b1 = firstTable.output(1)
+ val c1 = firstTable.output(2)
+ val d1 = firstTable.output(3)
+
+ val a2 = secondTable.output(0)
+ val b2 = secondTable.output(1)
+ val c2 = secondTable.output(2)
+ val d2 = secondTable.output(3)
+
+ val a3 = thirdTable.output(0)
+ val b3 = thirdTable.output(1)
+ val c3 = thirdTable.output(2)
+ val d3 = thirdTable.output(3)
+
+ val a4 = fourthTable.output(0)
Review comment:
ditto
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] GuoPhilipse commented on a change in pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
GuoPhilipse commented on a change in pull request #28951:
URL: https://github.com/apache/spark/pull/28951#discussion_r447448239
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala
##########
@@ -831,4 +831,77 @@ class AnalysisSuite extends AnalysisTest with Matchers {
}
}
}
+
+ test("SPARK-32131 Fix wrong column index when we have more than two columns" +
+ " during union and set operations" ) {
+ val firstTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", FloatType)())
+
+ val secondTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", TimestampType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", FloatType)())
+
+ val thirdTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", TimestampType)(),
+ AttributeReference("d", FloatType)())
+
+ val fourthTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", TimestampType)())
+
+ val a1 = firstTable.output(0)
Review comment:
em,,,let me remove it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] GuoPhilipse commented on pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
GuoPhilipse commented on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-651759412
It seesm i cannot trigger the test build, @dilipbiswal @maropu @HyukjinKwon @holdenk ,do you have any ideas?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dilipbiswal commented on pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
dilipbiswal commented on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-651648804
LGTM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dilipbiswal commented on a change in pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
dilipbiswal commented on a change in pull request #28951:
URL: https://github.com/apache/spark/pull/28951#discussion_r447426527
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala
##########
@@ -831,4 +831,77 @@ class AnalysisSuite extends AnalysisTest with Matchers {
}
}
}
+
+ test("SPARK-32131 Fix wrong column index when we have more than two columns" +
+ " during union and set operations" ) {
+ val firstTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", FloatType)())
+
+ val secondTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", TimestampType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", FloatType)())
+
+ val thirdTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", TimestampType)(),
+ AttributeReference("d", FloatType)())
+
+ val fourthTable = LocalRelation(
+ AttributeReference("a", StringType)(),
+ AttributeReference("b", DoubleType)(),
+ AttributeReference("c", IntegerType)(),
+ AttributeReference("d", TimestampType)())
+
+ val a1 = firstTable.output(0)
+ val b1 = firstTable.output(1)
+ val c1 = firstTable.output(2)
+ val d1 = firstTable.output(3)
+
+ val a2 = secondTable.output(0)
Review comment:
ditto
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28951: [SPARK-32131][SQL] Fix AnalysisException messages at UNION/EXCEPT/MINUS operations
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-652223150
**[Test build #124704 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124704/testReport)** for PR 28951 at commit [`984c652`](https://github.com/apache/spark/commit/984c65275bdf68bf8ff1f9ce3c91696a90e65439).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-651901057
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #28951: [SPARK-32131][SQL] union and set operations have wrong exception infomation
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #28951:
URL: https://github.com/apache/spark/pull/28951#issuecomment-651512938
ok to test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org