Re: issue with superset and kylin date column

[Sachin Aggarwal <di...@gmail.com>]

Hi yongjie zhao,

I am not able to see ur screen shot. Do you mean to say that CAL_DATE
column as DATE in List Table Column tab.
I have already added CAL_DATE  AS DATE in superset and selected "Is
temporal" flag also as yes.

kylinpy Version:
pip list |grep kylinpy

*kylinpy*          1.5.2



On Thu, Sep 6, 2018 at 8:51 PM yongjie zhao <yo...@gmail.com> wrote:

> hi, Sachin.
> you can set dimension  *cal_date is DATE* type in superset column. like
> attachment show. What version kylinpy do you use?
>
>
>
>
> On Wed, Sep 5, 2018 at 8:10 PM, Sachin Aggarwal <
> different.sachin@gmail.com> wrote:
>
>> Hello,
>>
>> *Problem Overview:* I am trying to connect superset with kylin. In
>> superset
>> all slices are failing. Once I set "Main Dttm Col" to my cal_date (date
>> column in yyyy-mm-dd format). In my debugging I found that my table have
>> date as yyyy-mm-dd but superset is sending me date in form of yyyy-mm-dd
>> hh:mm:ss. To solve the issue I have added *SUBSTRING('2018-01-01
>> 10:26:50',1,10) as *Expression for my column cal_date. Then superset have
>> sent new query to kylin which have where cal_date > SUBSTRING('2018-01-01
>> 10:26:50',1,10) in filter condition. But now kylin fails with this query
>> as
>> Explained below.
>>
>> In my table `demotable`. I have a column `cal_date` which is of date type.
>> when I use string value in where clause, it works perfectly fine as shown
>> in below query.
>>
>>    - select  * from demotable where cal_date > '2018-01-01' limit 1;
>>
>> but when I use SUBSTRING() funtion in select it works fine
>>
>>    - select  SUBSTRING('2018-01-01 10:26:50',1,10) from demotable
>>
>> but when we use SUBSTRING in where clause than it does not work it returns
>> with follwing error
>>
>>    - select  * from demotable where cal_date > SUBSTRING('2018-01-01
>>    10:26:50',1,10) limit 1;
>>
>> null while executing SQL: "select * from demotable where cal_date >
>> SUBSTRING('2018-01-01 10:26:50',1,10) LIMIT 5"
>>
>>
>> Stack-Trace
>>
>> SQL: select  * from demotable where cal_date > SUBSTRING('2018-01-01
>> 10:26:50',1,10);
>>
>> User:
>>
>> Success: false
>>
>> Duration: 0.04
>>
>> Project: stage2
>>
>> Realization Names: [CUBE[name=stage2]]
>>
>> Cuboid Ids: [4095]
>>
>> Total scan count: 0
>>
>> Total scan bytes: 0
>>
>> Result row count: 0
>>
>> Accept Partial: true
>>
>> Is Partial Result: false
>>
>> Hit Exception Cache: false
>>
>> Storage cache used: false
>>
>> Is Query Push-Down: false
>>
>> Is Prepare: false
>>
>> Trace URL: null
>>
>> Message: null
>>
>> while executing SQL: "select  * from demotable where cal_date >
>> SUBSTRING('2018-01-01 10:26:50',1,10) LIMIT 5"
>>
>> ==========================[QUERY]===============================
>>
>>
>> 2018-09-05 11:23:49,691 WARN  [Query
>> a14f0f7e-d834-4cd9-bff5-a7179a30fdda-52] service.QueryService:471 : Write
>> metric error.
>>
>> java.lang.NullPointerException
>>
>> at
>>
>> org.apache.kylin.rest.metrics.QueryMetricsFacade.updateMetricsToReservoir(QueryMetricsFacade.java:147)
>>
>> at
>>
>> org.apache.kylin.rest.metrics.QueryMetricsFacade.updateMetrics(QueryMetricsFacade.java:73)
>>
>> at
>>
>> org.apache.kylin.rest.service.QueryService.recordMetric(QueryService.java:560)
>>
>> at
>>
>> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:469)
>>
>> at
>>
>> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:390)
>>
>> at
>>
>> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:86)
>>
>> at sun.reflect.GeneratedMethodAccessor145.invoke(Unknown Source)
>>
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> at java.lang.reflect.Method.invoke(Method.java:498)
>>
>> at
>>
>> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
>>
>> at
>>
>> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
>>
>> at
>>
>> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
>>
>> at
>>
>> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
>>
>> at
>>
>> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
>>
>> at
>>
>> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
>>
>> at
>>
>> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
>>
>> at
>>
>> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
>>
>> at
>>
>> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
>>
>> at
>>
>> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
>>
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
>>
>> at
>>
>> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
>>
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
>>
>> at
>>
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
>>
>> at
>>
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
>>
>> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
>>
>> at
>>
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
>>
>> at
>>
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
>>
>> at
>>
>> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
>>
>> at
>>
>> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
>>
>> at
>>
>> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
>>
>> at
>>
>> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
>>
>> at
>>
>> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
>>
>> at
>>
>> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
>>
>> at
>>
>> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
>>
>> at
>>
>> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
>>
>> at
>>
>> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
>>
>> at
>>
>> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
>>
>> at
>>
>> org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
>>
>> at
>>
>> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
>>
>> at
>>
>> org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
>>
>> at
>>
>> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
>>
>> at
>>
>> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
>>
>> at
>>
>> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
>>
>> at
>>
>> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
>>
>> at
>>
>> org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
>>
>> at
>>
>> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
>>
>> at
>>
>> org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
>>
>> at
>>
>> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
>>
>> at
>>
>> org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:64)
>>
>> at
>>
>> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
>>
>> at
>>
>> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
>>
>> at
>>
>> org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
>>
>> at
>>
>> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
>>
>> at
>>
>> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
>>
>> at
>>
>> org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
>>
>> at
>>
>> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
>>
>> at
>>
>> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
>>
>> at
>>
>> org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
>>
>> at
>>
>> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
>>
>> at
>>
>> org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
>>
>> at
>>
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
>>
>> at
>>
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
>>
>> at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:209)
>>
>> at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:244)
>>
>> at
>>
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
>>
>> at
>>
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
>>
>> at
>>
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
>>
>> at
>>
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
>>
>> at
>>
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
>>
>> at
>>
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
>>
>> at
>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
>>
>> at
>>
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
>>
>> at
>>
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
>>
>> at
>>
>> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1115)
>>
>> at
>>
>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
>>
>> at
>> org.apache.tomcat.util.net
>> .JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
>>
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>
>> at
>>
>> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
>>
>> at java.lang.Thread.run(Thread.java:745)
>>
>> 2018-09-05 11:23:49,692 ERROR [http-bio-7070-exec-3]
>> controller.BasicController:61 :
>>
>> org.apache.kylin.rest.exception.InternalErrorException: null
>> --
>>
>> Thanks & Regards
>>
>> Sachin Aggarwal
>>
>
>

-- 

Thanks & Regards

Sachin Aggarwal
7760502772