You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "Shaofeng SHI (JIRA)" <ji...@apache.org> on 2018/11/15 13:14:00 UTC
[jira] [Assigned] (KYLIN-3636) in kylin 2.4.1 and 2.5.1 topn
aggregation query caused exception
[ https://issues.apache.org/jira/browse/KYLIN-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shaofeng SHI reassigned KYLIN-3636:
-----------------------------------
Assignee: Shaofeng SHI
Fix Version/s: v2.6.0
> in kylin 2.4.1 and 2.5.1 topn aggregation query caused exception
> ----------------------------------------------------------------
>
> Key: KYLIN-3636
> URL: https://issues.apache.org/jira/browse/KYLIN-3636
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: v2.4.1
> Reporter: KANG-SEN LU
> Assignee: Shaofeng SHI
> Priority: Major
> Fix For: v2.6.0
>
>
> Hi, ShaoFeng:
>
> Thanks for the reply. I missed this email and not responded earlier, I am sorry.
>
> I tried to reproduce this problem with the sample database, and it did not happen.
>
> So I am hoping by collecting enough "clue", someone can figure out why this problem occurred.
>
> --------------------------
> I issued the following query at the sample project to exercise the topn aggregation:
>
> select seller_id, SUM(price) as total from kylin_sales group by seller_id order by total limit 5;
>
> With my own added debugging, I saw the following log in the kylin.log: (the query worked OK).
>
> 2018-10-16 16:18:19,963 INFO [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90] model.FunctionDesc:59 : KSL22222, getRewriteFieldName=_KY_SUM_KYLIN_SALES_PRICE_
> 2018-10-16 16:18:19,963 INFO [kylin-coproc--pool12-t1] v2.CubeHBaseEndpointRPC:217 : Query-a747f16f-4b12-cc97-08d2-9b45c27a529f: send request to the init region server anovadata4.anovadata.local on table ANOVA_KYLIN_25X_K758MEAWJG
> 2018-10-16 16:18:19,963 INFO [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90] topn.TopNMeasureType:399 : KSL888: in TopNMeasureType.java, sumFieldName= _KY_SUM_KYLIN_SALES_PRICE_
>
>
> When I was executing my project query, I issued the following select statement:
>
> SELECT ZETTICSDW.A_VL_HOURLY_V.IMSIID \"ZETTICSDW_A_VL_HOURLY_V_IMSIID\", SUM(ZETTICSDW.A_VL_HOURLY_V.SIG_EVENT_COUNT) \"vl_aggs_model___USERS_BY_ERROR_3XX\" FROM ZETTICSDW.A_VL_HOURLY_V inner JOIN ZETTICSDW.T_VL_TRANSACTION_RULE_V ON ( ZETTICSDW.A_VL_HOURLY_V.CAUSE_CODE_KEY = ZETTICSDW.T_VL_TRANSACTION_RULE_V.CAUSE_CODE_KEY AND ZETTICSDW.A_VL_HOURLY_V.REASON_CODE_KEY = ZETTICSDW.T_VL_TRANSACTION_RULE_V.REASON_CODE_KEY AND ZETTICSDW.A_VL_HOURLY_V.TRANSACTION_TYPE_KEY = ZETTICSDW.T_VL_TRANSACTION_RULE_V.TRANSACTION_TYPE_KEY)
> WHERE ((ZETTICSDW.A_VL_HOURLY_V.THEDATE = '20180209') AND ((ZETTICSDW.A_VL_HOURLY_V.THEHOUR >= '02') AND (ZETTICSDW.A_VL_HOURLY_V.THEHOUR <= '03'))) AND ZETTICSDW.T_VL_TRANSACTION_RULE_V.DISPLAY_STRING LIKE '%+3%'
> GROUP BY ZETTICSDW.A_VL_HOURLY_V.IMSIID
> ORDER BY \"vl_aggs_model___USERS_BY_ERROR_3XX\"
> LIMIT 25
>
> An exception occurred within the method "private ColumnRowType buildColumnRowType()" of "query/src/main/java/org/apache/kylin/query/relnode/OLAPTableScan.java".
>
> if (columns.size() != rowType.getFieldCount()) {
> throw new IllegalStateException("RowType=" + rowType.getFieldCount() + ", ColumnRowType=" + columns.size());
> }
>
> It printed "RowType=133, ColumnRowType=132".
> The RowType list contains one extra column name: "ANY _KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_".
>
>
> I think this has something to do with the bug fix "KYLIN-3359 Support sum(expression) if possible".
>
> After this bug fix was submitted, I noticed that a lot of column name was added into rowType like "_KY_SUM_XXX".
>
> This strange column name "_KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_" is very similar.
>
> I also found that this extra column name, only existed in RowType but not in ColumnRowType, was added in the method "public void implementRewrite(RewriteImplementor implementor)" within "OLAPJoinRel.java".
>
> With my own debug statement, I saw this debug text in kylin.log:
>
> relnode.OLAPJoinRel:362 : KSL54: newField= #132: _KY_SUM_1_6735969a_SIG_EVENT_COUNT_ ANY
>
>
> I hope someone with a deeper technical knowledge in kylin query engine can figure out what was causing the problem I have seen.
>
> Thanks again.
>
> Kang-sen
>
> *From:* ShaoFeng Shi [[mailto:shaofengshi@apache.org]]
> *Sent:* Friday, October 05, 2018 9:59 PM
> *To:* user <[user@kylin.apache.org|mailto:user@kylin.apache.org]>
> *Subject:* Re: any body see topn in kylin 2.5.1 working?
>
> Hi Kang-Sen,
>
> Didn't see this; Can you reproduce the problem with the sample cube?
>
> Kang-Sen Lu <[klu@anovadata.com|mailto:klu@anovadata.com]> 于2018年10月6日周六 上午12:21写道：
> I have been running kylin 2.2.1 with topn aggregation OK.
>
> Recently, I upgraded to kylin 2.4.1 and 2.5.1, in both releases I am having query problem related to topn support.
>
> The problem is an exception occurred with the following error message in kylin.log:
>
> Caused by: java.lang.IllegalStateException: RowType=133, ColumnRowType=132
> at org.apache.kylin.query.relnode.OLAPJoinRel.buildColumnRowType(OLAPJoinRel.java:223)
> at org.apache.kylin.query.relnode.OLAPJoinRel.implementRewrite(OLAPJoinRel.java:359)
> at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
> at org.apache.kylin.query.relnode.OLAPFilterRel.implementRewrite(OLAPFilterRel.java:163)
> at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
> at org.apache.kylin.query.relnode.OLAPProjectRel.implementRewrite(OLAPProjectRel.java:228)
> at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
> at org.apache.kylin.query.relnode.OLAPAggregateRel.implementRewrite(OLAPAggregateRel.java:402)
> at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
> at org.apache.kylin.query.relnode.OLAPSortRel.implementRewrite(OLAPSortRel.java:86)
> at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
> at org.apache.kylin.query.relnode.OLAPLimitRel.implementRewrite(OLAPLimitRel.java:109)
> at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
> at org.apache.kylin.query.relnode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:100)
> at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:108)
> at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:92)
> at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1281)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:331)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:228)
> at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:801)
> at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:656)
> at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:618)
> at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:214)
> at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)
> at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
> at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
> ... 84 more
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)