You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by "Babulal (JIRA)" <ji...@apache.org> on 2018/05/26 19:10:00 UTC
[jira] [Updated] (CARBONDATA-2544) [MV] Wrong data displayed with
Filter
[ https://issues.apache.org/jira/browse/CARBONDATA-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Babulal updated CARBONDATA-2544:
--------------------------------
Description:
spark.sql("drop table if exists test1")
spark.sql("create table test1( name string,country string,age int,salary int) stored by 'carbondata' ")
spark.sql("insert into test1 select 'name1','USA',12,23")
spark.sql("create datamap datamv2 using 'mv' as select country,sum(salary) from test1 group by country").show()
spark.sql("rebuild datamap datamv2")
spark.sql("select country,sum(salary) from test1 group by country").show(200,false)
+--------+----------+
|country|sum(salary)|
+--------+----------+
|USA|23|
+--------+----------+
spark.sql("select country,sum(salary) from test1 where country='USA' group by country").show(200,false)
+--------+----------+
|country|sum(salary)|
+--------+----------+
+--------+----------+
This is because, select query formation is wrong , filter value is changed to lowercase
2018-05-27 00:20:16 INFO CarbonSparkSqlParser:54 - Parsing command: select preAGG() as preAgg, gen_subsumer_0.`country`, gen_subsumer_0.`sum(salary)` as `sum(salary)`
from
(select test1.`country`, sum(cast(test1.`salary` as bigint)) as `sum(salary)`
from
test1
group by test1.`country`) gen_subsumer_0
where
(gen_subsumer_0.`country` = 'usa')
was:
spark.sql("drop table if exists test1")
spark.sql("create table test1( name string,country string,age int,salary int) stored by 'carbondata' ")
spark.sql("insert into test1 select 'name1','USA',12,23")
spark.sql("create datamap datamv2 using 'mv' as select country,sum(salary) from test1 group by country").show()
spark.sql("rebuild datamap datamv2")
spark.sql("select country,sum(salary) from test1 group by country").show(200,false)
+-------+-----------+
|country|sum(salary)|
+-------+-----------+
|USA |23 |
+-------+-----------+
spark.sql("select country,sum(salary) from test1 where country='USA' group by country").show(200,false)
+-------+-----------+
|country|sum(salary)|
+-------+-----------+
+-------+-----------+
This is because, select query is wrong filter value is changed to lowercase
2018-05-27 00:20:16 INFO CarbonSparkSqlParser:54 - Parsing command: select preAGG() as preAgg, gen_subsumer_0.`country`, gen_subsumer_0.`sum(salary)` as `sum(salary)`
from
(select test1.`country`, sum(cast(test1.`salary` as bigint)) as `sum(salary)`
from
test1
group by test1.`country`) gen_subsumer_0
where
(gen_subsumer_0.`country` = 'usa')
> [MV] Wrong data displayed with Filter
> --------------------------------------
>
> Key: CARBONDATA-2544
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2544
> Project: CarbonData
> Issue Type: Bug
> Reporter: Babulal
> Priority: Major
>
> spark.sql("drop table if exists test1")
> spark.sql("create table test1( name string,country string,age int,salary int) stored by 'carbondata' ")
> spark.sql("insert into test1 select 'name1','USA',12,23")
> spark.sql("create datamap datamv2 using 'mv' as select country,sum(salary) from test1 group by country").show()
> spark.sql("rebuild datamap datamv2")
> spark.sql("select country,sum(salary) from test1 group by country").show(200,false)
> +--------+----------+
> |country|sum(salary)|
> +--------+----------+
> |USA|23|
> +--------+----------+
>
> spark.sql("select country,sum(salary) from test1 where country='USA' group by country").show(200,false)
> +--------+----------+
> |country|sum(salary)|
> +--------+----------+
> +--------+----------+
>
> This is because, select query formation is wrong , filter value is changed to lowercase
> 2018-05-27 00:20:16 INFO CarbonSparkSqlParser:54 - Parsing command: select preAGG() as preAgg, gen_subsumer_0.`country`, gen_subsumer_0.`sum(salary)` as `sum(salary)`
> from
> (select test1.`country`, sum(cast(test1.`salary` as bigint)) as `sum(salary)`
> from
> test1
> group by test1.`country`) gen_subsumer_0
> where
> (gen_subsumer_0.`country` = 'usa')
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)