You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2018/03/18 08:28:00 UTC
[jira] [Commented] (KYLIN-3284) Refactor all OLAPRel
computeSelfCost
[ https://issues.apache.org/jira/browse/KYLIN-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403917#comment-16403917 ]
ASF subversion and git services commented on KYLIN-3284:
--------------------------------------------------------
Commit 4e0bb70f72e6fcc11bacd7d8ede996ede5449560 in kylin's branch refs/heads/sync from [~hn5092]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=4e0bb70 ]
KYLIN-3284 Refactor all OLAPRel computeSelfCost
> Refactor all OLAPRel computeSelfCost
> ------------------------------------
>
> Key: KYLIN-3284
> URL: https://issues.apache.org/jira/browse/KYLIN-3284
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
> Affects Versions: v2.3.0
> Reporter: yiming.xu
> Assignee: yiming.xu
> Priority: Major
>
> for queries like:
> select lstg_format_name from test_kylin_fact order by case
> when 1=1 then
> cal_dt
> ELSE
> seller_id
> end
> the expected logical plan after volcano is:
> {code:java}
> OLAPToEnumerableConverter
> OLAPLimitRel(ctx=[], fetch=[50000])
> OLAPSortRel(sort0=[$1], dir0=[ASC], ctx=[])
> OLAPProjectRel(LSTG_FORMAT_NAME=[$3], EXPR$1=[CASE(=(1, 1), $2, CAST($7):DATE)], ctx=[])
> OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]])
> {code}
> however the actual is:
> {code:java}
> EnumerableSort(sort0=[$1], dir0=[ASC])
> OLAPToEnumerableConverter
> OLAPSortRel(sort0=[$1], dir0=[ASC], ctx=[])
> OLAPProjectRel(LSTG_FORMAT_NAME=[$3], EXPR$1=[CASE(=(1, 1), $2, CAST($7):DATE)], ctx=[])
> OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]])
> {code}
> This is the RelOptPlanner:
> {code:java}
> Sets:
> Set#0, type: RecordType(BIGINT TRANS_ID, BIGINT ORDER_ID, DATE CAL_DT, VARCHAR(256) LSTG_FORMAT_NAME, BIGINT LEAF_CATEG_ID, INTEGER LSTG_SITE_ID, SMALLINT SLR_SEGMENT_CD, INTEGER SELLER_ID, DECIMAL(19, 4) PRICE, INTEGER ITEM_COUNT, VARCHAR(256) TEST_COUNT_DISTINCT_BITMAP, DECIMAL(19, 4) DEAL_AMOUNT, INTEGER DEAL_YEAR, VARCHAR(256) LEFTJOIN_BUYER_ID_AND_COUNTRY_NAME, VARCHAR(256) LEFTJOIN_SELLER_ID_AND_COUNTRY_NAME, VARCHAR(256) LEFTJOIN_BUYER_COUNTRY_ABBR, VARCHAR(256) LEFTJOIN_SELLER_COUNTRY_ABBR, BIGINT _KY_COUNT__, INTEGER _KY_SUM_TEST_KYLIN_FACT_ITEM_COUNT_, DECIMAL(19, 4) _KY_SUM_TEST_KYLIN_FACT_PRICE_, DECIMAL(19, 4) _KY_MIN_TEST_KYLIN_FACT_PRICE_, DECIMAL(19, 4) _KY_MAX_TEST_KYLIN_FACT_PRICE_, ANY _KY_COUNT_DISTINCT_TEST_KYLIN_FACT_SELLER_ID_, ANY _KY_COUNT_DISTINCT_TEST_KYLIN_FACT_LSTG_FORMAT_NAME_TEST_KYLIN_FACT_SELLER_ID_, ANY _KY_COUNT_DISTINCT_TEST_KYLIN_FACT_TEST_COUNT_DISTINCT_BITMAP_, DECIMAL(19, 4) _KY_SUM_TEST_KYLIN_FACT_DEAL_AMOUNT_, ANY _KY_PERCENTILE_APPROX_TEST_KYLIN_FACT_PRICE_)
> rel#140:Subset#0.OLAP.[], best=rel#131, importance=0.7290000000000001
> rel#131:OLAPTableScan.OLAP.[](table=[DEFAULT, TEST_KYLIN_FACT],ctx=,fields=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]), rowcount=100.0, cumulative cost={5.0 rows, 5.050000000000001 cpu, 0.0 io}
> rel#179:OLAPSortRel.OLAP.[2](input=rel#140:Subset#0.OLAP.[],sort0=$2,dir0=ASC,ctx=), rowcount=100.0, cumulative cost={2491.7919004335695 rows, 10.05 cpu, 0.0 io}
> rel#169:Subset#0.NONE.[2], best=null, importance=0.7290000000000001
> rel#167:LogicalSort.NONE.[2](input=rel#140:Subset#0.OLAP.[],sort0=$2,dir0=ASC), rowcount=100.0, cumulative cost={inf}
> rel#171:Subset#0.ENUMERABLE.[2], best=rel#182, importance=0.81
> rel#181:EnumerableSort.ENUMERABLE.[2](input=rel#177:Subset#0.ENUMERABLE.[],sort0=$2,dir0=ASC), rowcount=100.0, cumulative cost={1.0E100 rows, 110.05 cpu, 0.0 io}
> rel#182:OLAPToEnumerableConverter.ENUMERABLE.[2](input=rel#180:Subset#0.OLAP.[2]), rowcount=100.0, cumulative cost={1.0E100 rows, 10.05 cpu, 0.0 io}
> rel#173:Subset#0.OLAP.[1], best=null, importance=0.6561
> rel#177:Subset#0.ENUMERABLE.[], best=rel#182, importance=0.7290000000000001
> rel#181:EnumerableSort.ENUMERABLE.[2](input=rel#177:Subset#0.ENUMERABLE.[],sort0=$2,dir0=ASC), rowcount=100.0, cumulative cost={1.0E100 rows, 110.05 cpu, 0.0 io}
> rel#182:OLAPToEnumerableConverter.ENUMERABLE.[2](input=rel#180:Subset#0.OLAP.[2]), rowcount=100.0, cumulative cost={1.0E100 rows, 10.05 cpu, 0.0 io}
> rel#185:OLAPToEnumerableConverter.ENUMERABLE.[](input=rel#140:Subset#0.OLAP.[]), rowcount=100.0, cumulative cost={1.0E100 rows, 5.050000000000001 cpu, 0.0 io}
> rel#180:Subset#0.OLAP.[2], best=rel#179, importance=0.6561
> rel#179:OLAPSortRel.OLAP.[2](input=rel#140:Subset#0.OLAP.[],sort0=$2,dir0=ASC,ctx=), rowcount=100.0, cumulative cost={2491.7919004335695 rows, 10.05 cpu, 0.0 io}
> Set#1, type: RecordType(VARCHAR(256) LSTG_FORMAT_NAME, DATE EXPR$1)
> rel#142:Subset#1.NONE.[], best=null, importance=0.81
> rel#141:LogicalProject.NONE.[](input=rel#140:Subset#0.OLAP.[],LSTG_FORMAT_NAME=$3,EXPR$1=CASE(=(1, 1), $2, CAST($7):DATE)), importance=0.0, rowcount=100.0, cumulative cost={inf}
> rel#143:LogicalSort.NONE.[1](input=rel#142:Subset#1.NONE.[],sort0=$1,dir0=ASC), rowcount=100.0, cumulative cost={inf}
> rel#163:LogicalProject.NONE.[](input=rel#140:Subset#0.OLAP.[],LSTG_FORMAT_NAME=$3,EXPR$1=$2), rowcount=100.0, cumulative cost={inf}
> rel#170:LogicalProject.NONE.[1](input=rel#169:Subset#0.NONE.[2],LSTG_FORMAT_NAME=$3,EXPR$1=$2), rowcount=100.0, cumulative cost={inf}
> rel#149:Subset#1.ENUMERABLE.[], best=rel#157, importance=0.9
> rel#148:AbstractConverter.ENUMERABLE.[1](input=rel#152:Subset#1.NONE.[1],convention=ENUMERABLE,sort=[1]), rowcount=100.0, cumulative cost={inf}
> rel#150:EnumerableSort.ENUMERABLE.[1](input=rel#149:Subset#1.ENUMERABLE.[],sort0=$1,dir0=ASC), rowcount=100.0, cumulative cost={1.0E100 rows, 130.05 cpu, 0.0 io}
> rel#153:AbstractConverter.ENUMERABLE.[1](input=rel#149:Subset#1.ENUMERABLE.[],convention=ENUMERABLE,sort=[1]), rowcount=100.0, cumulative cost={inf}
> rel#157:OLAPToEnumerableConverter.ENUMERABLE.[1](input=rel#156:Subset#1.OLAP.[1]), rowcount=100.0, cumulative cost={1.0E100 rows, 30.05 cpu, 0.0 io}
> rel#162:OLAPToEnumerableConverter.ENUMERABLE.[](input=rel#154:Subset#1.OLAP.[]), rowcount=100.0, cumulative cost={1.0E100 rows, 25.05 cpu, 0.0 io}
> rel#172:EnumerableProject.ENUMERABLE.[[1]](input=rel#171:Subset#0.ENUMERABLE.[2],LSTG_FORMAT_NAME=$3,EXPR$1=$2), rowcount=100.0, cumulative cost={1.0E100 rows, 210.05 cpu, 0.0 io}
> rel#178:EnumerableProject.ENUMERABLE.[](input=rel#177:Subset#0.ENUMERABLE.[],LSTG_FORMAT_NAME=$3,EXPR$1=$2), rowcount=100.0, cumulative cost={1.0E100 rows, 210.05 cpu, 0.0 io}
> rel#151:Subset#1.ENUMERABLE.[1], best=rel#150, importance=1.0
> rel#148:AbstractConverter.ENUMERABLE.[1](input=rel#152:Subset#1.NONE.[1],convention=ENUMERABLE,sort=[1]), rowcount=100.0, cumulative cost={inf}
> rel#150:EnumerableSort.ENUMERABLE.[1](input=rel#149:Subset#1.ENUMERABLE.[],sort0=$1,dir0=ASC), rowcount=100.0, cumulative cost={1.0E100 rows, 130.05 cpu, 0.0 io}
> rel#153:AbstractConverter.ENUMERABLE.[1](input=rel#149:Subset#1.ENUMERABLE.[],convention=ENUMERABLE,sort=[1]), rowcount=100.0, cumulative cost={inf}
> rel#157:OLAPToEnumerableConverter.ENUMERABLE.[1](input=rel#156:Subset#1.OLAP.[1]), rowcount=100.0, cumulative cost={1.0E100 rows, 30.05 cpu, 0.0 io}
> rel#172:EnumerableProject.ENUMERABLE.[[1]](input=rel#171:Subset#0.ENUMERABLE.[2],LSTG_FORMAT_NAME=$3,EXPR$1=$2), rowcount=100.0, cumulative cost={1.0E100 rows, 210.05 cpu, 0.0 io}
> rel#152:Subset#1.NONE.[1], best=null, importance=0.9
> rel#143:LogicalSort.NONE.[1](input=rel#142:Subset#1.NONE.[],sort0=$1,dir0=ASC), rowcount=100.0, cumulative cost={inf}
> rel#170:LogicalProject.NONE.[1](input=rel#169:Subset#0.NONE.[2],LSTG_FORMAT_NAME=$3,EXPR$1=$2), rowcount=100.0, cumulative cost={inf}
> rel#154:Subset#1.OLAP.[], best=rel#160, importance=0.7290000000000001
> rel#155:OLAPSortRel.OLAP.[1](input=rel#154:Subset#1.OLAP.[],sort0=$1,dir0=ASC,ctx=), rowcount=100.0, cumulative cost={199.2068074395237 rows, 30.05 cpu, 0.0 io}
> rel#160:OLAPProjectRel.OLAP.[](input=rel#140:Subset#0.OLAP.[],LSTG_FORMAT_NAME=$3,EXPR$1=CASE(=(1, 1), $2, CAST($7):DATE),ctx=), rowcount=100.0, cumulative cost={15.0 rows, 25.05 cpu, 0.0 io}
> rel#164:OLAPProjectRel.OLAP.[](input=rel#140:Subset#0.OLAP.[],LSTG_FORMAT_NAME=$3,EXPR$1=$2,ctx=), rowcount=100.0, cumulative cost={15.0 rows, 25.05 cpu, 0.0 io}
> rel#174:OLAPProjectRel.OLAP.[1](input=rel#173:Subset#0.OLAP.[1],LSTG_FORMAT_NAME=$3,EXPR$1=$2,ctx=), rowcount=100.0, cumulative cost={inf}
> rel#156:Subset#1.OLAP.[1], best=rel#155, importance=0.81
> rel#155:OLAPSortRel.OLAP.[1](input=rel#154:Subset#1.OLAP.[],sort0=$1,dir0=ASC,ctx=), rowcount=100.0, cumulative cost={199.2068074395237 rows, 30.05 cpu, 0.0 io}
> rel#174:OLAPProjectRel.OLAP.[1](input=rel#173:Subset#0.OLAP.[1],LSTG_FORMAT_NAME=$3,EXPR$1=$2,ctx=), rowcount=100.0, cumulative cost={inf}
> {code}
> The best root rel is rel#151:Subset#1.ENUMERABLE.[1] : rel#150:EnumerableSort.ENUMERABLE ,because it cost equal rel#157:OLAPToEnumerableConverter.ENUMERABLE. The OLAPToEnumerableConverter cost is so big that the details are neglected.We should refactor the cost system.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)