You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by xubo245 <gi...@git.apache.org> on 2018/01/24 09:47:39 UTC
[GitHub] carbondata pull request #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests...
GitHub user xubo245 opened a pull request:
https://github.com/apache/carbondata/pull/1857
[WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add test cases for base pre-aggregate table
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [ ] Any interfaces changed?
No
- [ ] Any backward compatibility impacted?
NO
- [ ] Document update required?
No
- [ ] Testing done
add test case for r base pre-aggregate table
in package org.apache.carbondata.integration.spark.testsuite.preaggregate
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
refer [CARBONDATA-1516]
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xubo245/carbondata PreAggregateTestBase
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/1857.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1857
----
commit 75c5fb32fff1166bede4440f4b6fff6c9e7fcd21
Author: xubo245 <60...@...>
Date: 2018-01-24T09:45:00Z
[WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add test cases for base pre-aggregate table
----
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3589/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2960/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3855/
---
[GitHub] carbondata issue #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add t...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3074/
---
[GitHub] carbondata issue #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add t...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1903/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2355/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
retest this please
---
[GitHub] carbondata pull request #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add...
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1857#discussion_r178037845
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/preaggregate/TestPreAggregateLoad.scala ---
@@ -415,8 +444,661 @@ test("check load and select for avg double datatype") {
sql(s"LOAD DATA LOCAL INPATH '$testData' into table maintable")
sql(s"LOAD DATA LOCAL INPATH '$testData' into table maintable")
val rows = sql("select age,avg(age) from maintable group by age").collect()
- sql("create datamap maintbl_douoble on table maintable using 'preaggregate' as select avg(age) from maintable group by age")
+ sql("create datamap maintbl_double on table maintable using 'preaggregate' as select avg(age) from maintable group by age")
checkAnswer(sql("select age,avg(age) from maintable group by age"), rows)
+ sql("drop table if exists maintable ")
+ }
+
+ def testFunction(): Unit = {
+ // check answer
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_sum"),
+ Seq(Row(1, 31), Row(2, 27), Row(3, 70), Row(4, 55)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_avg"),
+ Seq(Row(1, 31, 1), Row(2, 27, 1), Row(3, 70, 2), Row(4, 55, 2)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_count"),
+ Seq(Row(1, 1), Row(2, 1), Row(3, 2), Row(4, 2)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_min"),
+ Seq(Row(1, 31), Row(2, 27), Row(3, 35), Row(4, 26)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_max"),
+ Seq(Row(1, 31), Row(2, 27), Row(3, 35), Row(4, 29)))
+
+ testPreAggTable()
+ }
+
+ def testPreAggTable(): Unit = {
+ // check select and match or not match pre-aggregate table
+ checkPreAggTable(sql("SELECT id, SUM(age) FROM main_table GROUP BY id"),
--- End diff --
These scenarios are already covered in TestPreAggregateTableSelection
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on the issue:
https://github.com/apache/carbondata/pull/1857
LGTM
---
[GitHub] carbondata issue #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add t...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2304/
---
[GitHub] carbondata issue #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add t...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3543/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3444/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2409/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
Any one can help to review it? Please @ravipesala @jackylk @QiangCai @chenliang613 @sraghunandan @kumarvishal09
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2434/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
retest this please
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:
https://github.com/apache/carbondata/pull/1857
LGTM. SDV failure is not due to this PR, will fix it later
---
[GitHub] carbondata issue #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add t...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
retest this please
---
[GitHub] carbondata pull request #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1857#discussion_r178040942
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/preaggregate/TestPreAggregateLoad.scala ---
@@ -415,8 +444,661 @@ test("check load and select for avg double datatype") {
sql(s"LOAD DATA LOCAL INPATH '$testData' into table maintable")
sql(s"LOAD DATA LOCAL INPATH '$testData' into table maintable")
val rows = sql("select age,avg(age) from maintable group by age").collect()
- sql("create datamap maintbl_douoble on table maintable using 'preaggregate' as select avg(age) from maintable group by age")
+ sql("create datamap maintbl_double on table maintable using 'preaggregate' as select avg(age) from maintable group by age")
checkAnswer(sql("select age,avg(age) from maintable group by age"), rows)
+ sql("drop table if exists maintable ")
+ }
+
+ def testFunction(): Unit = {
+ // check answer
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_sum"),
+ Seq(Row(1, 31), Row(2, 27), Row(3, 70), Row(4, 55)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_avg"),
+ Seq(Row(1, 31, 1), Row(2, 27, 1), Row(3, 70, 2), Row(4, 55, 2)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_count"),
+ Seq(Row(1, 1), Row(2, 1), Row(3, 2), Row(4, 2)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_min"),
+ Seq(Row(1, 31), Row(2, 27), Row(3, 35), Row(4, 26)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_max"),
+ Seq(Row(1, 31), Row(2, 27), Row(3, 35), Row(4, 29)))
+
+ testPreAggTable()
+ }
+
+ def testPreAggTable(): Unit = {
+ // check select and match or not match pre-aggregate table
+ checkPreAggTable(sql("SELECT id, SUM(age) FROM main_table GROUP BY id"),
--- End diff --
test different data type, also can remove if necessary
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3008/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3364/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4039/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2814/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
@jackylk @kumarvishal09 Please review it.
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
retest sdv please
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4190/
---
[GitHub] carbondata issue #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add t...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3385/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4059/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4175/
---
[GitHub] carbondata pull request #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add...
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1857#discussion_r178033617
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/preaggregate/TestPreAggCreateCommand.scala ---
@@ -55,12 +57,33 @@ class TestPreAggCreateCommand extends QueryTest with BeforeAndAfterAll {
}
test("test pre agg create table 2") {
+ dropDataMaps("PreAggMain", "preagg2")
sql("create datamap preagg2 on table PreAggMain using 'preaggregate' as select a as a1,sum(b) as udfsum from PreAggMain group by a")
checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg2"), true, "preaggmain_a")
checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg2"), true, "preaggmain_b_sum")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg2"), false, "preaggmain_a1")
sql("drop datamap preagg2 on table PreAggMain")
}
+ test("test pre agg create table 3") {
+ dropDataMaps("PreAggMain", "preagg3")
+ sql("create datamap preagg3 on table PreAggMain using 'preaggregate' as select a,sum(b) as sum from PreAggMain group by a")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg3"), true, "preaggmain_a")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg3"), true, "preaggmain_b_sum")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg3"), false, "preaggmain_sum")
+ sql("drop datamap preagg3 on table PreAggMain")
+ }
+
+ test("test pre agg create table 4") {
+ dropDataMaps("PreAggMain", "preagg4")
+ sql("create datamap preagg4 on table PreAggMain using 'preaggregate' as select a as a1,sum(b) as sum from PreAggMain group by a")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg4"), true, "preaggmain_a")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg4"), true, "preaggmain_b_sum")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg4"), false, "preaggmain_a1")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg4"), false, "preaggmain_sum")
--- End diff --
this check can be moved to "test pre agg create table 2"
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4173/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4360/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3770/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4205/
---
[GitHub] carbondata pull request #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add...
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1857#discussion_r178037427
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/preaggregate/TestPreAggregateIUDSuite.scala ---
@@ -0,0 +1,206 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.integration.spark.testsuite.preaggregate
+
+import org.apache.spark.sql.Row
+import org.apache.spark.util.SparkUtil4Test
+import org.scalatest.{BeforeAndAfterAll, BeforeAndAfterEach}
+
+import org.apache.carbondata.core.constants.CarbonCommonConstants
+import org.apache.carbondata.core.util.CarbonProperties
+import org.apache.carbondata.spark.util.SparkQueryTest
+
+class TestPreAggregateIUDSuite extends SparkQueryTest with BeforeAndAfterAll with BeforeAndAfterEach {
--- End diff --
insert scenario is already handled in TestPreAggregateLoad.
Same with Update and Delete. No need to add
---
[GitHub] carbondata issue #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add t...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3536/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
@jackylk @sraghunandan @kumarvishal09 Please review it
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3437/
---
[GitHub] carbondata issue #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add t...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1860/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3593/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4187/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
Anyone can help to review it. @ravipesala @jackylk @QiangCai @chenliang613 @sraghunandan @kumarvishal09
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4084/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3674/
---
[GitHub] carbondata issue #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add t...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3383/
---
[GitHub] carbondata issue #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add t...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3136/
---
[GitHub] carbondata pull request #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add...
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1857#discussion_r178033489
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/preaggregate/TestPreAggCreateCommand.scala ---
@@ -55,12 +57,33 @@ class TestPreAggCreateCommand extends QueryTest with BeforeAndAfterAll {
}
test("test pre agg create table 2") {
+ dropDataMaps("PreAggMain", "preagg2")
sql("create datamap preagg2 on table PreAggMain using 'preaggregate' as select a as a1,sum(b) as udfsum from PreAggMain group by a")
checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg2"), true, "preaggmain_a")
checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg2"), true, "preaggmain_b_sum")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg2"), false, "preaggmain_a1")
sql("drop datamap preagg2 on table PreAggMain")
}
+ test("test pre agg create table 3") {
+ dropDataMaps("PreAggMain", "preagg3")
+ sql("create datamap preagg3 on table PreAggMain using 'preaggregate' as select a,sum(b) as sum from PreAggMain group by a")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg3"), true, "preaggmain_a")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg3"), true, "preaggmain_b_sum")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg3"), false, "preaggmain_sum")
--- End diff --
this check can be moved to "test pre agg create table 2"
---
[GitHub] carbondata issue #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add t...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2300/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
retest this please
---
[GitHub] carbondata issue #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add t...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3378/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
retest this please
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2723/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4183/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2793/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2357/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3725/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4170/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3437/
---
[GitHub] carbondata issue #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add t...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2306/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3595/
---
[GitHub] carbondata pull request #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1857#discussion_r178035445
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/preaggregate/TestPreAggCreateCommand.scala ---
@@ -55,12 +57,33 @@ class TestPreAggCreateCommand extends QueryTest with BeforeAndAfterAll {
}
test("test pre agg create table 2") {
+ dropDataMaps("PreAggMain", "preagg2")
sql("create datamap preagg2 on table PreAggMain using 'preaggregate' as select a as a1,sum(b) as udfsum from PreAggMain group by a")
checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg2"), true, "preaggmain_a")
checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg2"), true, "preaggmain_b_sum")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg2"), false, "preaggmain_a1")
sql("drop datamap preagg2 on table PreAggMain")
}
+ test("test pre agg create table 3") {
+ dropDataMaps("PreAggMain", "preagg3")
+ sql("create datamap preagg3 on table PreAggMain using 'preaggregate' as select a,sum(b) as sum from PreAggMain group by a")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg3"), true, "preaggmain_a")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg3"), true, "preaggmain_b_sum")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg3"), false, "preaggmain_sum")
+ sql("drop datamap preagg3 on table PreAggMain")
+ }
+
+ test("test pre agg create table 4") {
+ dropDataMaps("PreAggMain", "preagg4")
+ sql("create datamap preagg4 on table PreAggMain using 'preaggregate' as select a as a1,sum(b) as sum from PreAggMain group by a")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg4"), true, "preaggmain_a")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg4"), true, "preaggmain_b_sum")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg4"), false, "preaggmain_a1")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg4"), false, "preaggmain_sum")
--- End diff --
ok
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3845/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
retest sdv please
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4252/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4663/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
retest this please
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4588/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
rebase again
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
@jackylk Please review it
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2946/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3648/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3756/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
retest sdv please
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4290/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
Please review it @kunal642
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3046/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
retest sdv please
---
[GitHub] carbondata issue #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add t...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3541/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3439/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2351/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3445/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3841/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
retest sdv please
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3446/
---
[GitHub] carbondata pull request #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1857#discussion_r178035599
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/preaggregate/TestPreAggCreateCommand.scala ---
@@ -69,10 +92,30 @@ class TestPreAggCreateCommand extends QueryTest with BeforeAndAfterAll {
sql("drop datamap preagg11 on table PreAggMain1")
}
+ test("test pre agg create table 6") {
+ sql("create datamap preagg12 on table PreAggMain1 using 'preaggregate' as select a as a1,sum(b) from PreAggMain1 group by a")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain1_preagg12"), true, "preaggmain1_a")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain1_preagg12"), true, "preaggmain1_b_sum")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain1_preagg12"), false, "preaggmain1_a1")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain1_preagg12"), true, "DICTIONARY")
+ sql("drop datamap preagg12 on table PreAggMain1")
+ }
+
+ test("test pre agg create table 7") {
--- End diff --
ok
---
[GitHub] carbondata pull request #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1857#discussion_r178035059
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/preaggregate/TestPreAggCreateCommand.scala ---
@@ -55,12 +57,33 @@ class TestPreAggCreateCommand extends QueryTest with BeforeAndAfterAll {
}
test("test pre agg create table 2") {
+ dropDataMaps("PreAggMain", "preagg2")
sql("create datamap preagg2 on table PreAggMain using 'preaggregate' as select a as a1,sum(b) as udfsum from PreAggMain group by a")
checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg2"), true, "preaggmain_a")
checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg2"), true, "preaggmain_b_sum")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg2"), false, "preaggmain_a1")
sql("drop datamap preagg2 on table PreAggMain")
}
+ test("test pre agg create table 3") {
+ dropDataMaps("PreAggMain", "preagg3")
+ sql("create datamap preagg3 on table PreAggMain using 'preaggregate' as select a,sum(b) as sum from PreAggMain group by a")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg3"), true, "preaggmain_a")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg3"), true, "preaggmain_b_sum")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain_preagg3"), false, "preaggmain_sum")
--- End diff --
ok
---
[GitHub] carbondata issue #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests] Add t...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3091/
---
[GitHub] carbondata pull request #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1857#discussion_r178039781
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/preaggregate/TestPreAggregateIUDSuite.scala ---
@@ -0,0 +1,206 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.integration.spark.testsuite.preaggregate
+
+import org.apache.spark.sql.Row
+import org.apache.spark.util.SparkUtil4Test
+import org.scalatest.{BeforeAndAfterAll, BeforeAndAfterEach}
+
+import org.apache.carbondata.core.constants.CarbonCommonConstants
+import org.apache.carbondata.core.util.CarbonProperties
+import org.apache.carbondata.spark.util.SparkQueryTest
+
+class TestPreAggregateIUDSuite extends SparkQueryTest with BeforeAndAfterAll with BeforeAndAfterEach {
--- End diff --
ok. remove insert.
Where is Update and Delete?
---
[GitHub] carbondata pull request #1857: [WIP][CARBONDATA-2073][CARBONDATA-1516][Tests...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1857#discussion_r166269467
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/preaggregate/TestPreAggregateLoad.scala ---
@@ -412,8 +430,467 @@ test("check load and select for avg double datatype") {
sql(s"LOAD DATA LOCAL INPATH '$testData' into table maintable")
sql(s"LOAD DATA LOCAL INPATH '$testData' into table maintable")
val rows = sql("select age,avg(age) from maintable group by age").collect()
- sql("create datamap maintbl_douoble on table maintable using 'preaggregate' as select avg(age) from maintable group by age")
+ sql("create datamap maintbl_double on table maintable using 'preaggregate' as select avg(age) from maintable group by age")
checkAnswer(sql("select age,avg(age) from maintable group by age"), rows)
+ sql("drop table if exists maintable ")
+ }
+
+ def testFunction(): Unit = {
+ // check answer
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_sum"),
+ Seq(Row(1, 31), Row(2, 27), Row(3, 70), Row(4, 55)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_avg"),
+ Seq(Row(1, 31, 1), Row(2, 27, 1), Row(3, 70, 2), Row(4, 55, 2)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_count"),
+ Seq(Row(1, 1), Row(2, 1), Row(3, 2), Row(4, 2)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_min"),
+ Seq(Row(1, 31), Row(2, 27), Row(3, 35), Row(4, 26)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_max"),
+ Seq(Row(1, 31), Row(2, 27), Row(3, 35), Row(4, 29)))
+
+ // check select and match or not match pre-aggregate table
+ checkPreAggTable(sql("SELECT id, SUM(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_sum")
+ checkPreAggTable(sql("SELECT id, SUM(age) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_avg", "main_table")
+
+ checkPreAggTable(sql("SELECT id, AVG(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_avg")
+ checkPreAggTable(sql("SELECT id, AVG(age) from main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT id, COUNT(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_count")
+ checkPreAggTable(sql("SELECT id, COUNT(age) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT id, MIN(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_min")
+ checkPreAggTable(sql("SELECT id, MIN(age) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT id, MAX(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_max")
+ checkPreAggTable(sql("SELECT id, MAX(age) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ // sub query should match pre-aggregate table
+ checkPreAggTable(sql("SELECT SUM(age) FROM main_table"),
+ true, "main_table_preagg_sum")
+ checkPreAggTable(sql("SELECT SUM(age) FROM main_table"),
+ false, "main_table_preagg_avg", "main_table")
+
+ checkPreAggTable(sql("SELECT AVG(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_avg")
+ checkPreAggTable(sql("SELECT AVG(age) from main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT COUNT(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_count")
+ checkPreAggTable(sql("SELECT COUNT(age) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT MIN(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_min")
+ checkPreAggTable(sql("SELECT MIN(age) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT MAX(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_max")
+ checkPreAggTable(sql("SELECT MAX(age) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+ }
+
+ test("test load into main table with pre-aggregate table: double") {
+ sql(
+ """
+ | CREATE TABLE main_table(
+ | id INT,
+ | name STRING,
+ | city STRING,
+ | age DOUBLE)
+ | STORED BY 'org.apache.carbondata.format'
+ """.stripMargin)
+
+ createAllAggregateTables("main_table")
+ sql(s"LOAD DATA LOCAL INPATH '$testData' INTO TABLE main_table")
+
+ testFunction()
+ }
+
+ test("test load into main table with pre-aggregate table: short") {
+ sql(
+ """
+ | CREATE TABLE main_table(
+ | id INT,
+ | name STRING,
+ | city STRING,
+ | age SHORT)
+ | STORED BY 'org.apache.carbondata.format'
+ """.stripMargin)
+ createAllAggregateTables("main_table")
+ sql(s"LOAD DATA LOCAL INPATH '$testData' INTO TABLE main_table")
+
+ testFunction()
+ }
+
+ test("test load into main table with pre-aggregate table: float") {
+ sql(
+ """
+ | CREATE TABLE main_table(
+ | id INT,
+ | name STRING,
+ | city STRING,
+ | age FLOAT)
+ | STORED BY 'org.apache.carbondata.format'
+ """.stripMargin)
+ createAllAggregateTables("main_table")
+ sql(s"LOAD DATA LOCAL INPATH '$testData' INTO TABLE main_table")
+
+ testFunction()
+ }
+
+ test("test load into main table with pre-aggregate table: bigint") {
+ sql(
+ """
+ | CREATE TABLE main_table(
+ | id INT,
+ | name STRING,
+ | city STRING,
+ | age BIGINT)
+ | STORED BY 'org.apache.carbondata.format'
+ """.stripMargin)
+ createAllAggregateTables("main_table")
+ sql(s"LOAD DATA LOCAL INPATH '$testData' INTO TABLE main_table")
+
+ testFunction()
+
+ }
+
+ test("test load into main table with pre-aggregate table: string") {
+ sql(
+ """
+ | CREATE TABLE main_table(
+ | id INT,
+ | name STRING,
+ | city STRING,
+ | age STRING)
+ | STORED BY 'org.apache.carbondata.format'
+ """.stripMargin)
+ createAllAggregateTables("main_table")
+
+ sql(s"LOAD DATA LOCAL INPATH '$testData' INTO TABLE main_table")
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_sum"),
+ Seq(Row(1, 31), Row(2, 27), Row(3, 70), Row(4, 55)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_avg"),
+ Seq(Row(1, 31, 1), Row(2, 27, 1), Row(3, 70, 2), Row(4, 55, 2)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_count"),
+ Seq(Row(1, 1), Row(2, 1), Row(3, 2), Row(4, 2)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_min"),
+ Seq(Row(1, "31"), Row(2, "27"), Row(3, "35"), Row(4, "26")))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_max"),
+ Seq(Row(1, "31"), Row(2, "27"), Row(3, "35"), Row(4, "29")))
+
+ // check select and match or not match pre-aggregate table
+ checkPreAggTable(sql("SELECT id, SUM(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_sum")
+ checkPreAggTable(sql("SELECT id, SUM(age) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_avg", "main_table")
+
+ checkPreAggTable(sql("SELECT id, AVG(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_avg")
+ checkPreAggTable(sql("SELECT id, AVG(age) from main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT id, COUNT(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_count")
+ checkPreAggTable(sql("SELECT id, COUNT(age) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT id, MIN(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_min")
+ checkPreAggTable(sql("SELECT id, MIN(age) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT id, MAX(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_max")
+ checkPreAggTable(sql("SELECT id, MAX(age) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ // sub query should match pre-aggregate table
+ checkPreAggTable(sql("SELECT SUM(age) FROM main_table"),
+ true, "main_table_preagg_sum")
+ checkPreAggTable(sql("SELECT SUM(age) FROM main_table"),
+ false, "main_table_preagg_avg", "main_table")
+
+ checkPreAggTable(sql("SELECT AVG(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_avg")
+ checkPreAggTable(sql("SELECT AVG(age) from main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT COUNT(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_count")
+ checkPreAggTable(sql("SELECT COUNT(age) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT MIN(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_min")
+ checkPreAggTable(sql("SELECT MIN(age) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT MAX(age) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_max")
+ checkPreAggTable(sql("SELECT MAX(age) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+ }
+
+ test("test load into main table with pre-aggregate table: sum string column") {
+ sql(
+ """
+ | CREATE TABLE main_table(
+ | id INT,
+ | name STRING,
+ | city STRING,
+ | age STRING)
+ | STORED BY 'org.apache.carbondata.format'
+ """.stripMargin)
+ createAllAggregateTables("main_table", "name")
+ sql(s"LOAD DATA LOCAL INPATH '$testData' INTO TABLE main_table")
+
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_sum"),
+ Seq(Row(1, null), Row(2, null), Row(3, null), Row(4, null)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_avg"),
+ Seq(Row(1, null, 0), Row(2, null, 0), Row(3, null, 0), Row(4, null, 0)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_count"),
+ Seq(Row(1, 1), Row(2, 1), Row(3, 2), Row(4, 2)))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_min"),
+ Seq(Row(1, "david"), Row(2, "eason"), Row(3, "jarry"), Row(4, "kunal")))
+ checkAnswer(sql(s"SELECT * FROM main_table_preagg_max"),
+ Seq(Row(1, "david"), Row(2, "eason"), Row(3, "jarry"), Row(4, "vishal")))
+
+ // check select and match or not match pre-aggregate table
+ checkPreAggTable(sql("SELECT id, SUM(name) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_sum")
+ checkPreAggTable(sql("SELECT id, SUM(name) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_avg", "main_table")
+
+ checkPreAggTable(sql("SELECT id, AVG(name) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_avg")
+ checkPreAggTable(sql("SELECT id, AVG(name) from main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT id, COUNT(name) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_count")
+ checkPreAggTable(sql("SELECT id, COUNT(name) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT id, MIN(name) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_min")
+ checkPreAggTable(sql("SELECT id, MIN(name) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT id, MAX(name) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_max")
+ checkPreAggTable(sql("SELECT id, MAX(name) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ // sub query should match pre-aggregate table
+ checkPreAggTable(sql("SELECT SUM(name) FROM main_table"),
+ true, "main_table_preagg_sum")
+ checkPreAggTable(sql("SELECT SUM(name) FROM main_table"),
+ false, "main_table_preagg_avg", "main_table")
+
+ checkPreAggTable(sql("SELECT AVG(name) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_avg")
+ checkPreAggTable(sql("SELECT AVG(name) from main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT COUNT(name) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_count")
+ checkPreAggTable(sql("SELECT COUNT(name) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT MIN(name) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_min")
+ checkPreAggTable(sql("SELECT MIN(name) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+
+ checkPreAggTable(sql("SELECT MAX(name) FROM main_table GROUP BY id"),
+ true, "main_table_preagg_max")
+ checkPreAggTable(sql("SELECT MAX(name) FROM main_table GROUP BY id"),
+ false, "main_table_preagg_sum", "main_table")
+ }
+
+ test("test whether all segments are loaded into pre-aggregate table if segments are set on main table 2") {
+ sql("DROP TABLE IF EXISTS segmaintable")
+ sql(
+ """
+ | CREATE TABLE segmaintable(
+ | id INT,
+ | name STRING,
+ | city STRING,
+ | age INT)
+ | STORED BY 'org.apache.carbondata.format'
+ """.stripMargin)
+ sql(s"INSERT INTO segmaintable VALUES(1, 'xyz', 'bengaluru', 26)")
+
+ sql("set carbon.input.segments.default.segmaintable=0")
+ checkAnswer(sql(s"SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ Seq(Row(1, 26)))
+ sql(
+ s"""
+ | CREATE DATAMAP preagg_sum
+ | ON TABLE segmaintable
+ | USING 'preaggregate'
+ | AS SELECT id, SUM(age)
+ | FROM segmaintable
+ | GROUP BY id
+ """.stripMargin)
+ checkPreAggTable(sql("SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ false, "segmaintable_preagg_sum")
+
+ sql("reset")
+ checkAnswer(sql("SELECT * FROM segmaintable_preagg_sum"), Seq(Row(1, 26)))
+ checkPreAggTable(sql("SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ true, "segmaintable_preagg_sum")
+ }
+
+
+ test("test whether all segments are loaded into pre-aggregate table if segments are set on main table 3") {
+ sql("DROP TABLE IF EXISTS segmaintable")
+ sql(
+ """
+ | CREATE TABLE segmaintable(
+ | id INT,
+ | name STRING,
+ | city STRING,
+ | age INT)
+ | STORED BY 'org.apache.carbondata.format'
+ """.stripMargin)
+ sql(s"INSERT INTO segmaintable VALUES(1, 'xyz', 'bengaluru', 26)")
+
+ sql("set carbon.input.segments.default.segmaintable=0")
+ checkAnswer(sql(s"SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ Seq(Row(1, 26)))
+ sql(
+ s"""
+ | CREATE DATAMAP preagg_sum
+ | ON TABLE segmaintable
+ | USING 'preaggregate'
+ | AS SELECT id, SUM(age)
+ | FROM segmaintable
+ | GROUP BY id
+ """.stripMargin)
+ sql(s"INSERT INTO segmaintable VALUES(1, 'xyz', 'bengaluru', 26)")
+
+ sql("reset")
+ checkAnswer(sql("SELECT * FROM segmaintable_preagg_sum"), Seq(Row(1, 26), Row(1, 26)))
+ checkPreAggTable(sql("SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ true, "segmaintable_preagg_sum")
+ }
+
+ test("test whether all segments are loaded into pre-aggregate table if segments are set on main table 4") {
+ sql("DROP TABLE IF EXISTS segmaintable")
+ sql(
+ """
+ | CREATE TABLE segmaintable(
+ | id INT,
+ | name STRING,
+ | city STRING,
+ | age INT)
+ | STORED BY 'org.apache.carbondata.format'
+ """.stripMargin)
+ sql(s"INSERT INTO segmaintable VALUES(1, 'xyz', 'bengaluru', 26)")
+ sql(s"INSERT INTO segmaintable VALUES(1, 'xyz', 'bengaluru', 26)")
+
+ // check value before set segments
+ checkAnswer(sql(s"SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ Seq(Row(1, 52)))
+
+ sql("set carbon.input.segments.default.segmaintable=0")
+ // check value after set segments
+ checkAnswer(sql(s"SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ Seq(Row(1, 26)))
+
+ sql(
+ s"""
+ | CREATE DATAMAP preagg_sum
+ | ON TABLE segmaintable
+ | USING 'preaggregate'
+ | AS SELECT id, SUM(age)
+ | FROM segmaintable
+ | GROUP BY id
+ """.stripMargin)
+ sql(s"INSERT INTO segmaintable VALUES(1, 'xyz', 'bengaluru', 26)")
+
+ checkAnswer(sql("SELECT * FROM segmaintable_preagg_sum"), Seq(Row(1, 52), Row(1, 26)))
+ checkAnswer(sql(s"SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ Seq(Row(1, 26)))
+ checkPreAggTable(sql("SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ false, "segmaintable_preagg_sum")
+
+ // reset
+ sql("reset")
+ checkAnswer(sql(s"SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ Seq(Row(1, 78)))
+ checkPreAggTable(sql("SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ true, "segmaintable_preagg_sum")
+ }
+
+
+ test("test whether all segments are loaded into pre-aggregate table if segments are set on main table 5") {
+ sql("DROP TABLE IF EXISTS segmaintable")
+ sql(
+ """
+ | CREATE TABLE segmaintable(
+ | id INT,
+ | name STRING,
+ | city STRING,
+ | age INT)
+ | STORED BY 'org.apache.carbondata.format'
+ """.stripMargin)
+ sql(s"INSERT INTO segmaintable VALUES(1, 'xyz', 'bengaluru', 26)")
+ sql(s"INSERT INTO segmaintable VALUES(1, 'xyz', 'bengaluru', 26)")
+
+ // check value before set segments
+ checkAnswer(sql(s"SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ Seq(Row(1, 52)))
+
+ sql("set carbon.input.segments.default.segmaintable=0")
+ // check value after set segments
+ checkAnswer(sql(s"SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ Seq(Row(1, 26)))
+
+ sql(
+ s"""
+ | CREATE DATAMAP preagg_sum
+ | ON TABLE segmaintable
+ | USING 'preaggregate'
+ | AS SELECT id, SUM(age)
+ | FROM segmaintable
+ | GROUP BY id
+ """.stripMargin)
+ sql(s"INSERT INTO segmaintable VALUES(1, 'xyz', 'bengaluru', 26)")
+
+ checkAnswer(sql("SELECT * FROM segmaintable_preagg_sum"), Seq(Row(1, 52), Row(1, 26)))
+ checkAnswer(sql(s"SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ Seq(Row(1, 26)))
+ checkPreAggTable(sql("SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ false, "segmaintable_preagg_sum")
+
+ // set *
+ sql("set carbon.input.segments.default.segmaintable=*")
+ checkAnswer(sql(s"SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ Seq(Row(1, 78)))
+ checkPreAggTable(sql("SELECT id, SUM(age) FROM segmaintable GROUP BY id"),
+ false, "segmaintable_preagg_sum")
--- End diff --
we should support match pre aggregate table when sql("set carbon.input.segments.default.segmaintable=*")
---
[GitHub] carbondata pull request #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1857#discussion_r178048887
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/preaggregate/TestPreAggregateIUDSuite.scala ---
@@ -0,0 +1,206 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.integration.spark.testsuite.preaggregate
+
+import org.apache.spark.sql.Row
+import org.apache.spark.util.SparkUtil4Test
+import org.scalatest.{BeforeAndAfterAll, BeforeAndAfterEach}
+
+import org.apache.carbondata.core.constants.CarbonCommonConstants
+import org.apache.carbondata.core.util.CarbonProperties
+import org.apache.carbondata.spark.util.SparkQueryTest
+
+class TestPreAggregateIUDSuite extends SparkQueryTest with BeforeAndAfterAll with BeforeAndAfterEach {
--- End diff --
OK, I find. and remove the test case: TestPreAggregateIUDSuite.scala
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4174/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3433/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2942/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1857
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3485/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/1857
rebase again
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3968/
---
[GitHub] carbondata pull request #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add...
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1857#discussion_r178033753
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/preaggregate/TestPreAggCreateCommand.scala ---
@@ -69,10 +92,30 @@ class TestPreAggCreateCommand extends QueryTest with BeforeAndAfterAll {
sql("drop datamap preagg11 on table PreAggMain1")
}
+ test("test pre agg create table 6") {
+ sql("create datamap preagg12 on table PreAggMain1 using 'preaggregate' as select a as a1,sum(b) from PreAggMain1 group by a")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain1_preagg12"), true, "preaggmain1_a")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain1_preagg12"), true, "preaggmain1_b_sum")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain1_preagg12"), false, "preaggmain1_a1")
+ checkExistence(sql("DESCRIBE FORMATTED PreAggMain1_preagg12"), true, "DICTIONARY")
+ sql("drop datamap preagg12 on table PreAggMain1")
+ }
+
+ test("test pre agg create table 7") {
--- End diff --
cab be merged into "test pre agg create table 6"
---
[GitHub] carbondata pull request #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add...
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1857#discussion_r178036919
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/preaggregate/TestPreAggregateIUDSuite.scala ---
@@ -0,0 +1,206 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.integration.spark.testsuite.preaggregate
+
+import org.apache.spark.sql.Row
+import org.apache.spark.util.SparkUtil4Test
+import org.scalatest.{BeforeAndAfterAll, BeforeAndAfterEach}
+
+import org.apache.carbondata.core.constants.CarbonCommonConstants
+import org.apache.carbondata.core.util.CarbonProperties
+import org.apache.carbondata.spark.util.SparkQueryTest
+
+class TestPreAggregateIUDSuite extends SparkQueryTest with BeforeAndAfterAll with BeforeAndAfterEach {
+
+ val testData = s"$resourcesPath/sample.csv"
+ val p1 = CarbonProperties.getInstance()
+ .getProperty(CarbonCommonConstants.COMPACTION_SEGMENT_LEVEL_THRESHOLD,
+ CarbonCommonConstants.DEFAULT_SEGMENT_LEVEL_THRESHOLD)
+
+ override def beforeAll(): Unit = {
+ CarbonProperties.getInstance()
+ .addProperty(CarbonCommonConstants.ENABLE_AUTO_LOAD_MERGE, "false")
+ .addProperty(CarbonCommonConstants.COMPACTION_SEGMENT_LEVEL_THRESHOLD,
+ CarbonCommonConstants.DEFAULT_SEGMENT_LEVEL_THRESHOLD)
+
+ SparkUtil4Test.createTaskMockUp(sqlContext)
+ sql("DROP TABLE IF EXISTS maintable")
+ }
+
+ override protected def afterAll(): Unit = {
+ CarbonProperties.getInstance()
+ .addProperty(CarbonCommonConstants.ENABLE_AUTO_LOAD_MERGE,
+ CarbonCommonConstants.DEFAULT_ENABLE_AUTO_LOAD_MERGE)
+ .addProperty(CarbonCommonConstants.COMPACTION_SEGMENT_LEVEL_THRESHOLD, p1)
+ sql("DROP TABLE IF EXISTS y ")
+ sql("DROP TABLE IF EXISTS maintable")
+ sql("DROP TABLE IF EXISTS maintbl")
+ sql("DROP TABLE IF EXISTS main_table")
+ }
+
+ override protected def beforeEach(): Unit = {
+ sql("DROP TABLE IF EXISTS main_table")
+ sql("DROP TABLE IF EXISTS segmaintable")
+ }
+
+
+ test("test pre-aggregate IUD function 1: support insert after create pre aggregate table") {
--- End diff --
insert scenario is already handled in TestPreAggregateLoad
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3126/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4374/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3135/
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4660/
---
[GitHub] carbondata pull request #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/1857
---
[GitHub] carbondata issue #1857: [CARBONDATA-2073][CARBONDATA-1516][Tests] Add test c...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1857
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4673/
---