You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by xubo245 <gi...@git.apache.org> on 2018/02/22 10:22:18 UTC
[GitHub] carbondata pull request #1990: [CARBONDATA-2195] Add new test case for parti...
GitHub user xubo245 opened a pull request:
https://github.com/apache/carbondata/pull/1990
[CARBONDATA-2195] Add new test case for partition feature and optimize old test case
Add new test case for partition feature and optimize old test case
This PR only contains test case
During optimize test case, find a bug and fix it in: https://github.com/apache/carbondata/pull/1989
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [ ] Any interfaces changed?
- [ ] Any backward compatibility impacted?
- [ ] Document update required?
- [ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xubo245/carbondata StandardPartition
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/1990.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1990
----
commit c09e2ecd0c361dde26879f10b710194fc2b3863e
Author: xubo245 <60...@...>
Date: 2018-02-22T10:18:35Z
[CARBONDATA-2195] Add new test case for partition feature and optimize old test case
----
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1990
Please review it. @ravipesala @jackylk @QiangCai @chenliang613 @sraghunandan @kumarvishal09
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1990
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3853/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4057/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2945/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3005/
---
[GitHub] carbondata pull request #1990: [WIP][CARBONDATA-2195] Add new test case for ...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 closed the pull request at:
https://github.com/apache/carbondata/pull/1990
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2572/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3816/
---
[GitHub] carbondata pull request #1990: [CARBONDATA-2195] Add new test case for parti...
Posted by anubhav100 <gi...@git.apache.org>.
Github user anubhav100 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1990#discussion_r172473473
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableQueryTestCase.scala ---
@@ -256,9 +256,11 @@ test("Creation of partition table should fail if the colname in table schema and
""".stripMargin)
sql("insert into partitionTable select 1,'huawei','abc'")
checkAnswer(sql("show partitions partitionTable"), Seq(Row("email=abc")))
- intercept[Exception]{
+ val e = intercept[AnalysisException]{
sql("alter table partitionTable PARTITION (email='abc') rename to PARTITION (email='def)")
}
+ //TODO: error message is improper
+ assert(e.getMessage.contains("failure: identifier matching regex"))
--- End diff --
@xubo245
sql("alter table partitionTable PARTITION (email='abc') rename to PARTITION (email='def)")
closing single quote is missing after (email='def)") it is going to pass anyhow
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3032/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1990
retest sdv please
---
[GitHub] carbondata pull request #1990: [CARBONDATA-2195] Add new test case for parti...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1990#discussion_r173708980
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableQueryTestCase.scala ---
@@ -189,16 +189,15 @@ class StandardPartitionTableQueryTestCase extends QueryTest with BeforeAndAfterA
test("badrecords on partition column") {
sql("create table badrecordsPartition(intField1 int, stringField1 string) partitioned by (intField2 int) stored by 'carbondata'")
sql(s"load data local inpath '$resourcesPath/data_partition_badrecords.csv' into table badrecordsPartition options('bad_records_action'='force')")
- sql("select count(*) from badrecordsPartition").show()
checkAnswer(sql("select count(*) cnt from badrecordsPartition where intfield2 is null"), Seq(Row(9)))
checkAnswer(sql("select count(*) cnt from badrecordsPartition where intfield2 is not null"), Seq(Row(2)))
}
test("badrecords fail on partition column") {
sql("create table badrecordsPartitionfail(intField1 int, stringField1 string) partitioned by (intField2 int) stored by 'carbondata'")
+ //TODO: check message
--- End diff --
this issue need to be discussed, should throw BadRecordFoundException and check message. you can check the ignore test case
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4096/
---
[GitHub] carbondata pull request #1990: [CARBONDATA-2195] Add new test case for parti...
Posted by akashrn5 <gi...@git.apache.org>.
Github user akashrn5 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1990#discussion_r173064349
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionBadRecordLoggerTest.scala ---
@@ -53,8 +47,7 @@ class StandardPartitionBadRecordLoggerTest extends QueryTest with BeforeAndAfter
" ',', 'QUOTECHAR'= '\"')")
checkAnswer(
sql("select count(*) from sales"),
- Seq(Row(2)
--- End diff --
can move checkAnswer to single line in all places
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4356/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1990
@jackylk @QiangCai Please review it
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2957/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4249/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3359/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4192/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4373/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4276/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1990
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3767/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1990
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3844/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1990
retest sdv please
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2797/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4043/
---
[GitHub] carbondata pull request #1990: [CARBONDATA-2195] Add new test case for parti...
Posted by anubhav100 <gi...@git.apache.org>.
Github user anubhav100 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1990#discussion_r172472894
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableQueryTestCase.scala ---
@@ -256,9 +256,11 @@ test("Creation of partition table should fail if the colname in table schema and
""".stripMargin)
sql("insert into partitionTable select 1,'huawei','abc'")
checkAnswer(sql("show partitions partitionTable"), Seq(Row("email=abc")))
- intercept[Exception]{
+ val e = intercept[AnalysisException]{
--- End diff --
@xubo245
sql("alter table partitionTable PARTITION (email='abc') rename to PARTITION (email='def)")
closing single quote is missing after sql("alter table partitionTable PARTITION (email='def) rename to PARTITION (email='def)") it is going to pass anyhow
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1990
retest this please
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2583/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1990
Please review it. @ravipesala @jackylk @QiangCai @chenliang613 @sraghunandan @kumarvishal09
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1990
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3890/
---
[GitHub] carbondata pull request #1990: [CARBONDATA-2195] Add new test case for parti...
Posted by anubhav100 <gi...@git.apache.org>.
Github user anubhav100 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1990#discussion_r172795113
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableQueryTestCase.scala ---
@@ -243,11 +242,12 @@ class StandardPartitionTableQueryTestCase extends QueryTest with BeforeAndAfterA
}
-test("Creation of partition table should fail if the colname in table schema and partition column is same even if both are case sensitive"){
- intercept[Exception]{
- sql("CREATE TABLE uniqdata_char2(name char,id int) partitioned by (NAME char)stored by 'carbondata' ")
+ test("Creation of partition table should fail if the colname in table schema and partition column is same even if both are case sensitive") {
+ //TODO: error message is improper, it is different between spark2.1 and spark2.2
+ intercept[AnalysisException] {
+ sql("CREATE TABLE uniqdata_char2(name char,id int) partitioned by (NAME char)stored by 'carbondata' ")
--- End diff --
i fixed this test case with proper exception checking logic for both spark version and comments in pr #2034
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1990
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3753/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1990
retest this please
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1990
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3606/
---
[GitHub] carbondata pull request #1990: [CARBONDATA-2195] Add new test case for parti...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1990#discussion_r173706497
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionBadRecordLoggerTest.scala ---
@@ -53,8 +47,7 @@ class StandardPartitionBadRecordLoggerTest extends QueryTest with BeforeAndAfter
" ',', 'QUOTECHAR'= '\"')")
checkAnswer(
sql("select count(*) from sales"),
- Seq(Row(2)
--- End diff --
you mean in other class?
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4037/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4585/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1990
@anubhav100 How to handle this PR?
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1990
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4457/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2791/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1990
@jackylk Done, please check
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3134/
---
[GitHub] carbondata pull request #1990: [CARBONDATA-2195] Add new test case for parti...
Posted by anubhav100 <gi...@git.apache.org>.
Github user anubhav100 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1990#discussion_r170196522
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableQueryTestCase.scala ---
@@ -242,11 +242,13 @@ class StandardPartitionTableQueryTestCase extends QueryTest with BeforeAndAfterA
}
-test("Creation of partition table should fail if the colname in table schema and partition column is same even if both are case sensitive"){
- intercept[Exception]{
- sql("CREATE TABLE uniqdata_char2(name char,id int) partitioned by (NAME char)stored by 'carbondata' ")
+ test("Creation of partition table should fail if the colname in table schema and partition column is same even if both are case sensitive") {
+ val e = intercept[AnalysisException] {
+ sql("CREATE TABLE uniqdata_char2(name char,id int) partitioned by (NAME char)stored by 'carbondata' ")
+ }
+ //TODO: error message is improper
+ assert(e.getMessage.contains("DataType char is not supported"))
--- End diff --
@xubo245 spark 2.1.0 supports creating table even if char data type is given with out digits
scala> spark.sql("create table id(id char)");
18/02/23 14:12:23 WARN HiveMetaStore: Location: file:/home/anubhav/Documents/phatak/spark-2.1/bin/spark-warehouse/id specified for non-external table:id
res0: org.apache.spark.sql.DataFrame = []
but not in spark 2.2.1
cala> spark.sql("create table id(id char)");
18/02/23 14:22:05 WARN ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
org.apache.spark.sql.catalyst.parser.ParseException:
DataType char is not supported.(line 1, pos 19)
so to make sure that test case works fine for both spark version you can change the create table ddl to
sql("CREATE TABLE uniqdata_char2(name char,id int) partitioned by (NAME char(1))stored by 'carbondata' ")
the check for exception message
Operation not allowed: Partition columns should not be specified in the schema: ["name"](line 1, pos 62)
otherwise it will be passed for spark 2.1 .0 but not for spark 2.2.1
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1990
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4082/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4202/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3827/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:
https://github.com/apache/carbondata/pull/1990
please rebase
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3975/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2811/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1990
Please review it. @ravipesala @jackylk @QiangCai @chenliang613 @sraghunandan @kumarvishal09 @kunal642
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3963/
---
[GitHub] carbondata pull request #1990: [CARBONDATA-2195] Add new test case for parti...
Posted by akashrn5 <gi...@git.apache.org>.
Github user akashrn5 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1990#discussion_r173067874
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableQueryTestCase.scala ---
@@ -189,16 +189,15 @@ class StandardPartitionTableQueryTestCase extends QueryTest with BeforeAndAfterA
test("badrecords on partition column") {
sql("create table badrecordsPartition(intField1 int, stringField1 string) partitioned by (intField2 int) stored by 'carbondata'")
sql(s"load data local inpath '$resourcesPath/data_partition_badrecords.csv' into table badrecordsPartition options('bad_records_action'='force')")
- sql("select count(*) from badrecordsPartition").show()
checkAnswer(sql("select count(*) cnt from badrecordsPartition where intfield2 is null"), Seq(Row(9)))
checkAnswer(sql("select count(*) cnt from badrecordsPartition where intfield2 is not null"), Seq(Row(2)))
}
test("badrecords fail on partition column") {
sql("create table badrecordsPartitionfail(intField1 int, stringField1 string) partitioned by (intField2 int) stored by 'carbondata'")
+ //TODO: check message
--- End diff --
you can remove todo and check for the standard bad record message from exception
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1990
retest this please
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2730/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3122/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5173/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1990
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2852/
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1990
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3731/
---
[GitHub] carbondata pull request #1990: [CARBONDATA-2195] Add new test case for parti...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1990#discussion_r173709334
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableQueryTestCase.scala ---
@@ -243,11 +242,12 @@ class StandardPartitionTableQueryTestCase extends QueryTest with BeforeAndAfterA
}
-test("Creation of partition table should fail if the colname in table schema and partition column is same even if both are case sensitive"){
- intercept[Exception]{
- sql("CREATE TABLE uniqdata_char2(name char,id int) partitioned by (NAME char)stored by 'carbondata' ")
+ test("Creation of partition table should fail if the colname in table schema and partition column is same even if both are case sensitive") {
+ //TODO: error message is improper, it is different between spark2.1 and spark2.2
+ intercept[AnalysisException] {
+ sql("CREATE TABLE uniqdata_char2(name char,id int) partitioned by (NAME char)stored by 'carbondata' ")
--- End diff --
ok
---
[GitHub] carbondata pull request #1990: [CARBONDATA-2195] Add new test case for parti...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1990#discussion_r172508765
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableQueryTestCase.scala ---
@@ -256,9 +256,11 @@ test("Creation of partition table should fail if the colname in table schema and
""".stripMargin)
sql("insert into partitionTable select 1,'huawei','abc'")
checkAnswer(sql("show partitions partitionTable"), Seq(Row("email=abc")))
- intercept[Exception]{
+ val e = intercept[AnalysisException]{
sql("alter table partitionTable PARTITION (email='abc') rename to PARTITION (email='def)")
}
+ //TODO: error message is improper
+ assert(e.getMessage.contains("failure: identifier matching regex"))
--- End diff --
ok, fixed it. Thanks!
---
[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1990
@jackylk CI pass, please check
---