You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by huaicui <27...@qq.com> on 2018/09/20 06:30:56 UTC
Kylin concurrent issue for query engine;
We use Kylin to design our project; But we have a problem:
A single query will be responded in 0.07 to 0.2 seconds. Following the
number of query concurrency, response time will be longer. When the number
of query concurrency is about 150, this value will be extended to 4 seconds
or more. Can you help me to fix this issue or give me some suggestion?
<http://apache-kylin.74782.x6.nabble.com/file/t824/10.jpg>
<http://apache-kylin.74782.x6.nabble.com/file/t824/50.jpg>
<http://apache-kylin.74782.x6.nabble.com/file/t824/100.jpg>
<http://apache-kylin.74782.x6.nabble.com/file/t824/150.jpg>
--
Sent from: http://apache-kylin.74782.x6.nabble.com/
Re:Re: Re:Kylin concurrent issue for query engine;
Posted by Ma Gang <mg...@163.com>.
Just send prepare request to Kylin server as before, by default, the server side cache is enable:
POST /kylin/api/query
{"sql":"select minute_start,count(1) from TABLE where minute_start>=? and minute_start<? group by minute_start","offset":0,"limit":50000,"params":[{"className":"java.sql.Timestamp","value":"2012-01-01 02:00:00"},{"className":"java.sql.Timestamp","value":"2012-01-01 05:00:00"}],"acceptPartial":true,"project":"default","backdoorToggles": {}}
在 2018-09-25 17:35:18，"ShaoFeng Shi" <sh...@apache.org> 写道：
>+1 to add this into Kylin documentation.
>
>huaicui <27...@qq.com> 于2018年9月25日周二 下午5:22写道：
>
>> I want to enable this function in V2.5.0, but i don't how to use this
>> PreparedStatement.
>>
>> --
>> Sent from: http://apache-kylin.74782.x6.nabble.com/
>>
>
>
>--
>Best regards,
>
>Shaofeng Shi 史少锋
Re: Re:Kylin concurrent issue for query engine;
Posted by ShaoFeng Shi <sh...@apache.org>.
+1 to add this into Kylin documentation.
huaicui <27...@qq.com> 于2018年9月25日周二 下午5:22写道：
> I want to enable this function in V2.5.0, but i don't how to use this
> PreparedStatement.
>
> --
> Sent from: http://apache-kylin.74782.x6.nabble.com/
>
--
Best regards,
Shaofeng Shi 史少锋
Re: Re:Kylin concurrent issue for query engine;
Posted by huaicui <27...@qq.com>.
I want to enable this function in V2.5.0, but i don't how to use this
PreparedStatement.
--
Sent from: http://apache-kylin.74782.x6.nabble.com/
Re:Kylin concurrent issue for query engine;
Posted by Ma Gang <mg...@163.com>.
Could you run jstack on the Kylin process and send out result? So that we can know what the query threads are blocked on.
I encountered the same issue when doing load test, and found that most of query threads are blocked on class loading, and the class is code gen by calcite, so we may need calcite to fix it(Just a simple fix to change the generated class name). If you use Kylin 2.5 version, and most of your queries have similar pattern, you may try to use PreparedStatement, that may greatly increase the load performance. Kylin already support to cached preparedStatement parse result in Kylin server side since 2.5 (https://issues.apache.org/jira/browse/KYLIN-3434)
At 2018-09-20 14:30:56, "huaicui" <27...@qq.com> wrote:
>We use Kylin to design our project; But we have a problem:
>A single query will be responded in 0.07 to 0.2 seconds. Following the
>number of query concurrency, response time will be longer. When the number
>of query concurrency is about 150, this value will be extended to 4 seconds
>or more. Can you help me to fix this issue or give me some suggestion?
>
><http://apache-kylin.74782.x6.nabble.com/file/t824/10.jpg>
><http://apache-kylin.74782.x6.nabble.com/file/t824/50.jpg>
><http://apache-kylin.74782.x6.nabble.com/file/t824/100.jpg>
><http://apache-kylin.74782.x6.nabble.com/file/t824/150.jpg>
>
>--
>Sent from: http://apache-kylin.74782.x6.nabble.com/