You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@carbondata.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/08/01 13:53:20 UTC
[jira] [Commented] (CARBONDATA-107) Remove unnecessary ConverToSafe
in spark planner
[ https://issues.apache.org/jira/browse/CARBONDATA-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15402062#comment-15402062 ]
ASF GitHub Bot commented on CARBONDATA-107:
-------------------------------------------
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-carbondata/pull/54
> Remove unnecessary ConverToSafe in spark planner
> ------------------------------------------------
>
> Key: CARBONDATA-107
> URL: https://issues.apache.org/jira/browse/CARBONDATA-107
> Project: CarbonData
> Issue Type: Improvement
> Components: carbon-spark
> Affects Versions: Apache CarbonData 0.1.0-incubating
> Reporter: Jacky Li
> Fix For: Apache CarbonData 0.1.0-incubating
>
>
> Query:
> select ch, sum(c) from (select ch,count(1) as c from t2 group by ch) temp where c > 1 group by ch
> Output plan looks like:
> == Physical Plan ==
> Limit 21
> ConvertToSafe
> CarbonDictionaryDecoder [CarbonDecoderRelation(Map(word#39 -> word#39, ch#40 -> ch#40, value#41 -> value#41),CarbonDatasourceRelation(`default`.`t1`,None))], ExcludeProfile(ArrayBuffer(#103)), CarbonAliasDecoderRelation()
> ConvertToSafe
> TungstenAggregate(key=[ch#40], functions=[(sum(c#101L),mode=Final,isDistinct=false)], output=[ch#40,_c1#102L])
> TungstenAggregate(key=[ch#40], functions=[(sum(c#101L),mode=Partial,isDistinct=false)], output=[ch#40,currentSum#122L])
> Filter (c#101L > FakeCarbonCast(1 as bigint))
> CarbonDictionaryDecoder [CarbonDecoderRelation(Map(word#39 -> word#39, ch#40 -> ch#40, value#41 -> value#41),CarbonDatasourceRelation(`default`.`t1`,None))], IncludeProfile(ArrayBuffer(#103)), CarbonAliasDecoderRelation()
> ConvertToSafe
> TungstenAggregate(key=[ch#40], functions=[(count(1),mode=Final,isDistinct=false)], output=[ch#40,c#101L])
> TungstenExchange hashpartitioning(ch#40)
> TungstenAggregate(key=[ch#40], functions=[(count(1),mode=Partial,isDistinct=false)], output=[ch#40,currentCount#126L])
> Project [ch#40]
> ConvertToSafe
> CarbonScan [ch#40], (CarbonRelation default, t1, CarbonMetaData(ArrayBuffer(word, ch),ArrayBuffer(value),org.carbondata.core.carbon.metadata.schema.table.CarbonTable@52d54ca2,DictionaryMap(Map(word -> true, ch -> true))), TableMeta(default_t1,/Users/jackylk/code/incubator-carbondata/target/store,org.carbondata.core.carbon.metadata.schema.table.CarbonTable@52d54ca2,Partitioner(org.carbondata.spark.partition.api.impl.SampleDataPartitionerImpl,[Ljava.lang.String;@62a877f4,1,[Ljava.lang.String;@3c180da5)), None), true
> There are unnecessary ConvertToSafe before CarbonDictionaryDecoder.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)