You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "hongbin ma (JIRA)" <ji...@apache.org> on 2016/07/24 12:47:20 UTC
[jira] [Commented] (KYLIN-1809) kylin1.5.2 query count distinct
with condition "in" will cause timeout .
[ https://issues.apache.org/jira/browse/KYLIN-1809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15391049#comment-15391049 ]
hongbin ma commented on KYLIN-1809:
-----------------------------------
hi [~dixingxing@yeah.net]
can you please help me to narrow down the scope?
seems you're blaming the error for the IN clause, do you have any evidence to support that?
will the following query succeed?(simply removing the IN clause)
select dt,sum(pv),count(distinct device_id) from olap.olap_app_flw_platform t1
left join olap.olap_dim_city_all t2 on t1.city_id=t2.id
where dt ='2016-05-29'
group by dt
> kylin1.5.2 query count distinct with condition "in" will cause timeout .
> --------------------------------------------------------------------------
>
> Key: KYLIN-1809
> URL: https://issues.apache.org/jira/browse/KYLIN-1809
> Project: Kylin
> Issue Type: Bug
> Affects Versions: v1.5.2
> Environment: hbase1.1.2 hadoop 2.4.1
> Reporter: Xingxing Di
> Assignee: hongbin ma
>
> Kylin1.5.2 query count distinct with condition "in" will cause exception .
> Note that , Kylin1.5 has no such problem.
> select dt,sum(pv),count(distinct device_id) from olap.olap_app_flw_platform t1
> left join olap.olap_dim_city_all t2 on t1.city_id=t2.id
> where dt ='2016-05-29' and t2.city_id in (110100,440100,440300,310100,330100)
> group by dt
> The error stack:
> 2016-06-21 09:28:59,995 ERROR [http-bio-8070-exec-2] controller.QueryController:209 : Exception when execute sql
> java.sql.SQLException: Error while executing SQL "select t2.city_name,sum(pv),count(distinct device_id) from olap.olap_app_flw_platform t1
> left join olap.olap_dim_city_all t2 on t1.city_id=t2.id
> where dt='2016-06-18' and t2.city_id in (110100,440100,440300,310100,330100)
> group by t2.city_name
> LIMIT 50000": Timeout visiting cube!
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:143)
> at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:186)
> at org.apache.kylin.rest.service.QueryService.execute(QueryService.java:361)
> at org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:273)
> at org.apache.kylin.rest.service.QueryService.query(QueryService.java:121)
> at org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke(<generated>)
> at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
> at org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$c462a47d.query(<generated>)
> at org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:192)
> at org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94)
> at sun.reflect.GeneratedMethodAccessor107.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
> at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
> at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
> at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
> at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
> at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
> at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
> at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
> at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
> at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:789)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
> at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:201)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:91)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
> at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
> at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
> at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
> at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:195)
> at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:266)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
> at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
> at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074)
> at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
> at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: Timeout visiting cube!
> at org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$ExpectedSizeIterator.next(CubeHBaseEndpointRPC.java:127)
> at org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$ExpectedSizeIterator.next(CubeHBaseEndpointRPC.java:81)
> at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
> at com.google.common.collect.Iterators$6.hasNext(Iterators.java:583)
> at org.apache.kylin.storage.hbase.cube.v2.SequentialCubeTupleIterator.hasNext(SequentialCubeTupleIterator.java:96)
> at org.apache.kylin.storage.hbase.cube.v2.SequentialCubeTupleIterator.hasNext(SequentialCubeTupleIterator.java:109)
> at org.apache.kylin.storage.hbase.cube.v2.SequentialCubeTupleIterator.hasNext(SequentialCubeTupleIterator.java:109)
> at org.apache.kylin.storage.hbase.cube.v2.SequentialCubeTupleIterator.hasNext(SequentialCubeTupleIterator.java:109)
> at org.apache.kylin.storage.hbase.cube.v2.SequentialCubeTupleIterator.hasNext(SequentialCubeTupleIterator.java:109)
> at org.apache.kylin.storage.hbase.cube.v2.SequentialCubeTupleIterator.hasNext(SequentialCubeTupleIterator.java:109)
> at org.apache.kylin.storage.hbase.cube.v2.SequentialCubeTupleIterator.hasNext(SequentialCubeTupleIterator.java:109)
> at org.apache.kylin.query.enumerator.OLAPEnumerator.moveNext(OLAPEnumerator.java:74)
> at Baz$1$1.moveNext(Unknown Source)
> at org.apache.calcite.linq4j.EnumerableDefaults.groupBy_(EnumerableDefaults.java:819)
> at org.apache.calcite.linq4j.EnumerableDefaults.groupBy(EnumerableDefaults.java:754)
> at org.apache.calcite.linq4j.DefaultEnumerable.groupBy(DefaultEnumerable.java:302)
> at Baz.bind(Unknown Source)
> at org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:326)
> at org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:281)
> at org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:545)
> at org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:536)
> at org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:187)
> at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:65)
> at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:44)
> at org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:566)
> at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:578)
> at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:571)
> at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:135)
> ... 79 more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)