You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "liyang (JIRA)" <ji...@apache.org> on 2016/07/09 08:37:10 UTC
[jira] [Updated] (KYLIN-1852) cube routing problem with distinct
count
[ https://issues.apache.org/jira/browse/KYLIN-1852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
liyang updated KYLIN-1852:
--------------------------
Summary: cube routing problem with distinct count (was: 在有distinct count时，cube 选择出错)
> cube routing problem with distinct count
> ----------------------------------------
>
> Key: KYLIN-1852
> URL: https://issues.apache.org/jira/browse/KYLIN-1852
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: v1.5.2
> Environment: CentOS release 6.6
> Reporter: bing huang
> Assignee: liyang
> Labels: cube, distinct, route
>
> 问题描述：
> 同一个model 建立两个cube, 在这称为cube1 与 cube2. 两个cube有相同的measure, 除了cube2多一个distinct_count measure , 这儿称为
> a 。
> 查询时使用 select count(distinct a) from table1. 这个时候查询出错，提示Can't create EnumerableAggregate! 日志中查询选择的cube
> 为cube1 而不是cube2。 以下为详细日志：
>
> 2016-07-05 21:18:57,839 INFO [http-bio-7070-exec-2] schema.OLAPSchemaFactory:135 : Schema json:{
> "version": "1.0",
> "defaultSchema": "DEFAULT",
> "schemas": [
> {
> "type": "custom",
> "name": "DEFAULT",
> "factory": "org.apache.kylin.query.schema.OLAPSchemaFactory",
> "operand": {
> "project": "TEST"
> },
> "functions": [
> {
> name: 'MASSIN',
> className: 'org.apache.kylin.query.udf.MassInUDF'
> }
> ]
> }
> ]
> }
> 2016-07-05 21:18:58,701 DEBUG [http-bio-7070-exec-2] service.QueryService:288 : getting table metas
> 2016-07-05 21:18:58,817 DEBUG [http-bio-7070-exec-2] service.QueryService:306 : getting column metas
> 2016-07-05 21:18:58,920 DEBUG [http-bio-7070-exec-2] service.QueryService:320 : done column metas
> 2016-07-05 21:19:18,991 INFO [http-bio-7070-exec-5] controller.QueryController:174 : Using project: test
> 2016-07-05 21:19:18,995 INFO [http-bio-7070-exec-5] controller.QueryController:175 : The original query: select count(DISTINCT order_title) from hwb_alipay_trade
> 2016-07-05 21:19:19,009 INFO [http-bio-7070-exec-5] service.QueryService:264 : The corrected query: select count(DISTINCT order_title) from hwb_alipay_trade
> LIMIT 50000
> 2016-07-05 21:19:19,968 INFO [http-bio-7070-exec-5] routing.QueryRouter:48 : The project manager's reference is org.apache.kylin.metadata.project.ProjectManager@7fab0bdb
> 2016-07-05 21:19:19,970 INFO [http-bio-7070-exec-5] routing.QueryRouter:60 : Find candidates by table DEFAULT.HWB_ALIPAY_TRADE and project=TEST : org.apache.kylin.query.routing.Candidate@5a6373b7,org.apache.kylin.query.routing.Candidate@7db14ef3
> 2016-07-05 21:19:19,974 INFO [http-bio-7070-exec-5] routing.QueryRouter:49 : Applying rule: class org.apache.kylin.query.routing.rules.RemoveUncapableRealizationsRule, realizations before: [hwb_alipay_cube_distinct(CUBE),hwb_alipay_cube(CUBE)], realizations after: [hwb_alipay_cube_distinct(CUBE),hwb_alipay_cube(CUBE)]
> 2016-07-05 21:19:19,975 INFO [http-bio-7070-exec-5] routing.QueryRouter:49 : Applying rule: class org.apache.kylin.query.routing.rules.RealizationSortRule, realizations before: [hwb_alipay_cube_distinct(CUBE),hwb_alipay_cube(CUBE)], realizations after: [hwb_alipay_cube(CUBE),hwb_alipay_cube_distinct(CUBE)]
> 2016-07-05 21:19:19,975 INFO [http-bio-7070-exec-5] routing.QueryRouter:85 : Adjust DimensionAsMeasure for FunctionDesc [expression=COUNT_DISTINCT, parameter=ParameterDesc [type=column, value=ORDER_TITLE, nextParam=null], returnType=null]
> 2016-07-05 21:19:19,976 INFO [http-bio-7070-exec-5] routing.QueryRouter:72 : The realizations remaining: [hwb_alipay_cube(CUBE),hwb_alipay_cube_distinct(CUBE)] And the final chosen one is the first one
> 2016-07-05 21:19:19,987 ERROR [http-bio-7070-exec-5] controller.QueryController:209 : Exception when execute sql
> java.sql.SQLException: Error while executing SQL "select count(DISTINCT order_title) from hwb_alipay_trade
> LIMIT 50000": Can't create EnumerableAggregate!
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:143)
> at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:186)
> at org.apache.kylin.rest.service.QueryService.execute(QueryService.java:355)
> at org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:271)
> at org.apache.kylin.rest.service.QueryService.query(QueryService.java:119)
> at org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke(<generated>)
> at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
> at org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$4398392b.query(<generated>)
> at org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:192)
> at org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
> at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
> at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
> at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
> at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
> at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
> at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
> at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
> at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
> at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:789)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
> at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:91)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
> at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
> at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
> at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:195)
> at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:266)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
> at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
> at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074)
> at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
> at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalStateException: Can't create EnumerableAggregate!
> at org.apache.kylin.query.relnode.OLAPAggregateRel.implementEnumerable(OLAPAggregateRel.java:371)
> at org.apache.kylin.query.relnode.OLAPRel$JavaImplementor.createEnumerable(OLAPRel.java:159)
> at org.apache.kylin.query.relnode.OLAPRel$JavaImplementor.createEnumerable(OLAPRel.java:155)
> at org.apache.kylin.query.relnode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:99)
> at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:102)
> at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:92)
> at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1171)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:297)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:196)
> at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:721)
> at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:588)
> at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:558)
> at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:214)
> at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:573)
> at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:571)
> at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:135)
> ... 80 more
> Caused by: org.apache.calcite.rel.InvalidRelException: distinct aggregation not supported
> at org.apache.calcite.adapter.enumerable.EnumerableAggregate.<init>(EnumerableAggregate.java:71)
> at org.apache.kylin.query.relnode.OLAPAggregateRel.implementEnumerable(OLAPAggregateRel.java:368)
> ... 95 more
> 2016-07-05 21:19:19,993 INFO [http-bio-7070-exec-5] service.QueryService:248 :
> ==========================[QUERY]===============================
> SQL: select count(DISTINCT order_title) from hwb_alipay_trade
> User: ADMIN
> Success: false
> Duration: 0.0
> Project: test
> Realization Names: [hwb_alipay_cube]
> Cuboid Ids: []
> Total scan count: 0
> Result row count: 0
> Accept Partial: true
> Is Partial Result: false
> Hit Exception Cache: false
> Storage cache used: false
> Message: Error while executing SQL "select count(DISTINCT order_title) from hwb_alipay_trade LIMIT 50000": Can't create EnumerableAggregate!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)