You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by "Liangfei.Su" <su...@gmail.com> on 2016/02/03 04:35:00 UTC
[DISCUSS] Query explanation
Hi, Kylin Community
When troubleshooting kylin query, we could use "Insight" tab to issue
query, then checking the kylin.log. This definitely the right way, but make
it only available to kylin admin who can access the kylin log.
Since most query system provide query explanation(same as calcite), Kylin
might also output query explanation in response. Say when issue query with
explain=true , the query response would return structured explanation, like
how many htable touched, how many records read from each htable and the
time consumption.
Both kylin owner and user should benefit from such feature for
troubleshooting and query tuning.
I filed KYLIN-1398, any comments? Please suggest.
Thanks,
Ralph
Re: [DISCUSS] Query explanation
Posted by Luke Han <lu...@gmail.com>.
I think we should keep same as Hive/MySQL one, then some JDBC/ODBC tool
could leverage it.
Thanks.
Best Regards!
---------------------
Luke Han
On Wed, Feb 3, 2016 at 3:59 PM, Liangfei.Su <su...@gmail.com> wrote:
> It should have execution plan and some of storage engine details as well.
>
> On Wed, Feb 3, 2016 at 2:46 PM, hongbin ma <ma...@apache.org> wrote:
>
> > it also depends on what info you're planning to expose via this function.
> > if only calcite logical/physical execution plans then it could apply to
> > both.
> >
> > if you want to expose storage engine level details then my last email
> might
> > help.
> >
> > On Wed, Feb 3, 2016 at 2:44 PM, hongbin ma <ma...@apache.org> wrote:
> >
> > > query engine changed a lot in 2.x. If it's too much duplicate work I
> > > think it is reasonable to skip the 1.x branch. after all we don't need
> to
> > > port all new features to historical versions.
> > >
> > > On Wed, Feb 3, 2016 at 2:30 PM, Liangfei.Su <su...@gmail.com>
> > wrote:
> > >
> > >> I'm making the fix version as 2.1. Not sure if 1.x should applied or
> > not.
> > >> Technically, it should.
> > >>
> > >> On Wed, Feb 3, 2016 at 2:16 PM, Luke Han <lu...@gmail.com> wrote:
> > >>
> > >> > That's great, please help to double check is this could be applied
> to
> > >> 1.x
> > >> > and 2.x branches both.
> > >> >
> > >> > Thanks.
> > >> >
> > >> >
> > >> > Best Regards!
> > >> > ---------------------
> > >> >
> > >> > Luke Han
> > >> >
> > >> > On Wed, Feb 3, 2016 at 2:03 PM, Liangfei.Su <su...@gmail.com>
> > >> wrote:
> > >> >
> > >> > > Yeah, kylin-1398 means for service side, created kylin-1399
> > >> > >
> > >> > > Ralph
> > >> > >
> > >> > > On Wed, Feb 3, 2016 at 1:18 PM, Luke Han <lu...@gmail.com>
> wrote:
> > >> > >
> > >> > > > Hi Ralph,
> > >> > > > This is great feature we are looking for long time.
> > >> > > > Would you mind to create another JIRA for UI part? I would
> > like
> > >> to
> > >> > > say
> > >> > > > KYLIN-1398 should focus on service side, right?
> > >> > > >
> > >> > > > Thanks.
> > >> > > > Luke
> > >> > > >
> > >> > > >
> > >> > > > Best Regards!
> > >> > > > ---------------------
> > >> > > >
> > >> > > > Luke Han
> > >> > > >
> > >> > > > On Wed, Feb 3, 2016 at 11:35 AM, Liangfei.Su <
> > suliangfei@gmail.com>
> > >> > > wrote:
> > >> > > >
> > >> > > > > Hi, Kylin Community
> > >> > > > >
> > >> > > > > When troubleshooting kylin query, we could use "Insight" tab
> to
> > >> issue
> > >> > > > > query, then checking the kylin.log. This definitely the right
> > way,
> > >> > but
> > >> > > > make
> > >> > > > > it only available to kylin admin who can access the kylin log.
> > >> > > > >
> > >> > > > > Since most query system provide query explanation(same as
> > >> calcite),
> > >> > > Kylin
> > >> > > > > might also output query explanation in response. Say when
> issue
> > >> query
> > >> > > > with
> > >> > > > > explain=true , the query response would return structured
> > >> > explanation,
> > >> > > > like
> > >> > > > > how many htable touched, how many records read from each
> htable
> > >> and
> > >> > the
> > >> > > > > time consumption.
> > >> > > > >
> > >> > > > > Both kylin owner and user should benefit from such feature for
> > >> > > > > troubleshooting and query tuning.
> > >> > > > >
> > >> > > > > I filed KYLIN-1398, any comments? Please suggest.
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > > Ralph
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Regards,
> > >
> > > *Bin Mahone | 马洪宾*
> > > Apache Kylin: http://kylin.io
> > > Github: https://github.com/binmahone
> > >
> >
> >
> >
> > --
> > Regards,
> >
> > *Bin Mahone | 马洪宾*
> > Apache Kylin: http://kylin.io
> > Github: https://github.com/binmahone
> >
>
Re: [DISCUSS] Query explanation
Posted by "Liangfei.Su" <su...@gmail.com>.
It should have execution plan and some of storage engine details as well.
On Wed, Feb 3, 2016 at 2:46 PM, hongbin ma <ma...@apache.org> wrote:
> it also depends on what info you're planning to expose via this function.
> if only calcite logical/physical execution plans then it could apply to
> both.
>
> if you want to expose storage engine level details then my last email might
> help.
>
> On Wed, Feb 3, 2016 at 2:44 PM, hongbin ma <ma...@apache.org> wrote:
>
> > query engine changed a lot in 2.x. If it's too much duplicate work I
> > think it is reasonable to skip the 1.x branch. after all we don't need to
> > port all new features to historical versions.
> >
> > On Wed, Feb 3, 2016 at 2:30 PM, Liangfei.Su <su...@gmail.com>
> wrote:
> >
> >> I'm making the fix version as 2.1. Not sure if 1.x should applied or
> not.
> >> Technically, it should.
> >>
> >> On Wed, Feb 3, 2016 at 2:16 PM, Luke Han <lu...@gmail.com> wrote:
> >>
> >> > That's great, please help to double check is this could be applied to
> >> 1.x
> >> > and 2.x branches both.
> >> >
> >> > Thanks.
> >> >
> >> >
> >> > Best Regards!
> >> > ---------------------
> >> >
> >> > Luke Han
> >> >
> >> > On Wed, Feb 3, 2016 at 2:03 PM, Liangfei.Su <su...@gmail.com>
> >> wrote:
> >> >
> >> > > Yeah, kylin-1398 means for service side, created kylin-1399
> >> > >
> >> > > Ralph
> >> > >
> >> > > On Wed, Feb 3, 2016 at 1:18 PM, Luke Han <lu...@gmail.com> wrote:
> >> > >
> >> > > > Hi Ralph,
> >> > > > This is great feature we are looking for long time.
> >> > > > Would you mind to create another JIRA for UI part? I would
> like
> >> to
> >> > > say
> >> > > > KYLIN-1398 should focus on service side, right?
> >> > > >
> >> > > > Thanks.
> >> > > > Luke
> >> > > >
> >> > > >
> >> > > > Best Regards!
> >> > > > ---------------------
> >> > > >
> >> > > > Luke Han
> >> > > >
> >> > > > On Wed, Feb 3, 2016 at 11:35 AM, Liangfei.Su <
> suliangfei@gmail.com>
> >> > > wrote:
> >> > > >
> >> > > > > Hi, Kylin Community
> >> > > > >
> >> > > > > When troubleshooting kylin query, we could use "Insight" tab to
> >> issue
> >> > > > > query, then checking the kylin.log. This definitely the right
> way,
> >> > but
> >> > > > make
> >> > > > > it only available to kylin admin who can access the kylin log.
> >> > > > >
> >> > > > > Since most query system provide query explanation(same as
> >> calcite),
> >> > > Kylin
> >> > > > > might also output query explanation in response. Say when issue
> >> query
> >> > > > with
> >> > > > > explain=true , the query response would return structured
> >> > explanation,
> >> > > > like
> >> > > > > how many htable touched, how many records read from each htable
> >> and
> >> > the
> >> > > > > time consumption.
> >> > > > >
> >> > > > > Both kylin owner and user should benefit from such feature for
> >> > > > > troubleshooting and query tuning.
> >> > > > >
> >> > > > > I filed KYLIN-1398, any comments? Please suggest.
> >> > > > >
> >> > > > > Thanks,
> >> > > > > Ralph
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
> >
> > --
> > Regards,
> >
> > *Bin Mahone | 马洪宾*
> > Apache Kylin: http://kylin.io
> > Github: https://github.com/binmahone
> >
>
>
>
> --
> Regards,
>
> *Bin Mahone | 马洪宾*
> Apache Kylin: http://kylin.io
> Github: https://github.com/binmahone
>
Re: [DISCUSS] Query explanation
Posted by hongbin ma <ma...@apache.org>.
it also depends on what info you're planning to expose via this function.
if only calcite logical/physical execution plans then it could apply to
both.
if you want to expose storage engine level details then my last email might
help.
On Wed, Feb 3, 2016 at 2:44 PM, hongbin ma <ma...@apache.org> wrote:
> query engine changed a lot in 2.x. If it's too much duplicate work I
> think it is reasonable to skip the 1.x branch. after all we don't need to
> port all new features to historical versions.
>
> On Wed, Feb 3, 2016 at 2:30 PM, Liangfei.Su <su...@gmail.com> wrote:
>
>> I'm making the fix version as 2.1. Not sure if 1.x should applied or not.
>> Technically, it should.
>>
>> On Wed, Feb 3, 2016 at 2:16 PM, Luke Han <lu...@gmail.com> wrote:
>>
>> > That's great, please help to double check is this could be applied to
>> 1.x
>> > and 2.x branches both.
>> >
>> > Thanks.
>> >
>> >
>> > Best Regards!
>> > ---------------------
>> >
>> > Luke Han
>> >
>> > On Wed, Feb 3, 2016 at 2:03 PM, Liangfei.Su <su...@gmail.com>
>> wrote:
>> >
>> > > Yeah, kylin-1398 means for service side, created kylin-1399
>> > >
>> > > Ralph
>> > >
>> > > On Wed, Feb 3, 2016 at 1:18 PM, Luke Han <lu...@gmail.com> wrote:
>> > >
>> > > > Hi Ralph,
>> > > > This is great feature we are looking for long time.
>> > > > Would you mind to create another JIRA for UI part? I would like
>> to
>> > > say
>> > > > KYLIN-1398 should focus on service side, right?
>> > > >
>> > > > Thanks.
>> > > > Luke
>> > > >
>> > > >
>> > > > Best Regards!
>> > > > ---------------------
>> > > >
>> > > > Luke Han
>> > > >
>> > > > On Wed, Feb 3, 2016 at 11:35 AM, Liangfei.Su <su...@gmail.com>
>> > > wrote:
>> > > >
>> > > > > Hi, Kylin Community
>> > > > >
>> > > > > When troubleshooting kylin query, we could use "Insight" tab to
>> issue
>> > > > > query, then checking the kylin.log. This definitely the right way,
>> > but
>> > > > make
>> > > > > it only available to kylin admin who can access the kylin log.
>> > > > >
>> > > > > Since most query system provide query explanation(same as
>> calcite),
>> > > Kylin
>> > > > > might also output query explanation in response. Say when issue
>> query
>> > > > with
>> > > > > explain=true , the query response would return structured
>> > explanation,
>> > > > like
>> > > > > how many htable touched, how many records read from each htable
>> and
>> > the
>> > > > > time consumption.
>> > > > >
>> > > > > Both kylin owner and user should benefit from such feature for
>> > > > > troubleshooting and query tuning.
>> > > > >
>> > > > > I filed KYLIN-1398, any comments? Please suggest.
>> > > > >
>> > > > > Thanks,
>> > > > > Ralph
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
>
> --
> Regards,
>
> *Bin Mahone | 马洪宾*
> Apache Kylin: http://kylin.io
> Github: https://github.com/binmahone
>
--
Regards,
*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone
Re: [DISCUSS] Query explanation
Posted by hongbin ma <ma...@apache.org>.
query engine changed a lot in 2.x. If it's too much duplicate work I think
it is reasonable to skip the 1.x branch. after all we don't need to port
all new features to historical versions.
On Wed, Feb 3, 2016 at 2:30 PM, Liangfei.Su <su...@gmail.com> wrote:
> I'm making the fix version as 2.1. Not sure if 1.x should applied or not.
> Technically, it should.
>
> On Wed, Feb 3, 2016 at 2:16 PM, Luke Han <lu...@gmail.com> wrote:
>
> > That's great, please help to double check is this could be applied to 1.x
> > and 2.x branches both.
> >
> > Thanks.
> >
> >
> > Best Regards!
> > ---------------------
> >
> > Luke Han
> >
> > On Wed, Feb 3, 2016 at 2:03 PM, Liangfei.Su <su...@gmail.com>
> wrote:
> >
> > > Yeah, kylin-1398 means for service side, created kylin-1399
> > >
> > > Ralph
> > >
> > > On Wed, Feb 3, 2016 at 1:18 PM, Luke Han <lu...@gmail.com> wrote:
> > >
> > > > Hi Ralph,
> > > > This is great feature we are looking for long time.
> > > > Would you mind to create another JIRA for UI part? I would like
> to
> > > say
> > > > KYLIN-1398 should focus on service side, right?
> > > >
> > > > Thanks.
> > > > Luke
> > > >
> > > >
> > > > Best Regards!
> > > > ---------------------
> > > >
> > > > Luke Han
> > > >
> > > > On Wed, Feb 3, 2016 at 11:35 AM, Liangfei.Su <su...@gmail.com>
> > > wrote:
> > > >
> > > > > Hi, Kylin Community
> > > > >
> > > > > When troubleshooting kylin query, we could use "Insight" tab to
> issue
> > > > > query, then checking the kylin.log. This definitely the right way,
> > but
> > > > make
> > > > > it only available to kylin admin who can access the kylin log.
> > > > >
> > > > > Since most query system provide query explanation(same as calcite),
> > > Kylin
> > > > > might also output query explanation in response. Say when issue
> query
> > > > with
> > > > > explain=true , the query response would return structured
> > explanation,
> > > > like
> > > > > how many htable touched, how many records read from each htable and
> > the
> > > > > time consumption.
> > > > >
> > > > > Both kylin owner and user should benefit from such feature for
> > > > > troubleshooting and query tuning.
> > > > >
> > > > > I filed KYLIN-1398, any comments? Please suggest.
> > > > >
> > > > > Thanks,
> > > > > Ralph
> > > > >
> > > >
> > >
> >
>
--
Regards,
*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone
Re: [DISCUSS] Query explanation
Posted by "Liangfei.Su" <su...@gmail.com>.
I'm making the fix version as 2.1. Not sure if 1.x should applied or not.
Technically, it should.
On Wed, Feb 3, 2016 at 2:16 PM, Luke Han <lu...@gmail.com> wrote:
> That's great, please help to double check is this could be applied to 1.x
> and 2.x branches both.
>
> Thanks.
>
>
> Best Regards!
> ---------------------
>
> Luke Han
>
> On Wed, Feb 3, 2016 at 2:03 PM, Liangfei.Su <su...@gmail.com> wrote:
>
> > Yeah, kylin-1398 means for service side, created kylin-1399
> >
> > Ralph
> >
> > On Wed, Feb 3, 2016 at 1:18 PM, Luke Han <lu...@gmail.com> wrote:
> >
> > > Hi Ralph,
> > > This is great feature we are looking for long time.
> > > Would you mind to create another JIRA for UI part? I would like to
> > say
> > > KYLIN-1398 should focus on service side, right?
> > >
> > > Thanks.
> > > Luke
> > >
> > >
> > > Best Regards!
> > > ---------------------
> > >
> > > Luke Han
> > >
> > > On Wed, Feb 3, 2016 at 11:35 AM, Liangfei.Su <su...@gmail.com>
> > wrote:
> > >
> > > > Hi, Kylin Community
> > > >
> > > > When troubleshooting kylin query, we could use "Insight" tab to issue
> > > > query, then checking the kylin.log. This definitely the right way,
> but
> > > make
> > > > it only available to kylin admin who can access the kylin log.
> > > >
> > > > Since most query system provide query explanation(same as calcite),
> > Kylin
> > > > might also output query explanation in response. Say when issue query
> > > with
> > > > explain=true , the query response would return structured
> explanation,
> > > like
> > > > how many htable touched, how many records read from each htable and
> the
> > > > time consumption.
> > > >
> > > > Both kylin owner and user should benefit from such feature for
> > > > troubleshooting and query tuning.
> > > >
> > > > I filed KYLIN-1398, any comments? Please suggest.
> > > >
> > > > Thanks,
> > > > Ralph
> > > >
> > >
> >
>
Re: [DISCUSS] Query explanation
Posted by Luke Han <lu...@gmail.com>.
That's great, please help to double check is this could be applied to 1.x
and 2.x branches both.
Thanks.
Best Regards!
---------------------
Luke Han
On Wed, Feb 3, 2016 at 2:03 PM, Liangfei.Su <su...@gmail.com> wrote:
> Yeah, kylin-1398 means for service side, created kylin-1399
>
> Ralph
>
> On Wed, Feb 3, 2016 at 1:18 PM, Luke Han <lu...@gmail.com> wrote:
>
> > Hi Ralph,
> > This is great feature we are looking for long time.
> > Would you mind to create another JIRA for UI part? I would like to
> say
> > KYLIN-1398 should focus on service side, right?
> >
> > Thanks.
> > Luke
> >
> >
> > Best Regards!
> > ---------------------
> >
> > Luke Han
> >
> > On Wed, Feb 3, 2016 at 11:35 AM, Liangfei.Su <su...@gmail.com>
> wrote:
> >
> > > Hi, Kylin Community
> > >
> > > When troubleshooting kylin query, we could use "Insight" tab to issue
> > > query, then checking the kylin.log. This definitely the right way, but
> > make
> > > it only available to kylin admin who can access the kylin log.
> > >
> > > Since most query system provide query explanation(same as calcite),
> Kylin
> > > might also output query explanation in response. Say when issue query
> > with
> > > explain=true , the query response would return structured explanation,
> > like
> > > how many htable touched, how many records read from each htable and the
> > > time consumption.
> > >
> > > Both kylin owner and user should benefit from such feature for
> > > troubleshooting and query tuning.
> > >
> > > I filed KYLIN-1398, any comments? Please suggest.
> > >
> > > Thanks,
> > > Ralph
> > >
> >
>
Re: [DISCUSS] Query explanation
Posted by "Liangfei.Su" <su...@gmail.com>.
Yeah, kylin-1398 means for service side, created kylin-1399
Ralph
On Wed, Feb 3, 2016 at 1:18 PM, Luke Han <lu...@gmail.com> wrote:
> Hi Ralph,
> This is great feature we are looking for long time.
> Would you mind to create another JIRA for UI part? I would like to say
> KYLIN-1398 should focus on service side, right?
>
> Thanks.
> Luke
>
>
> Best Regards!
> ---------------------
>
> Luke Han
>
> On Wed, Feb 3, 2016 at 11:35 AM, Liangfei.Su <su...@gmail.com> wrote:
>
> > Hi, Kylin Community
> >
> > When troubleshooting kylin query, we could use "Insight" tab to issue
> > query, then checking the kylin.log. This definitely the right way, but
> make
> > it only available to kylin admin who can access the kylin log.
> >
> > Since most query system provide query explanation(same as calcite), Kylin
> > might also output query explanation in response. Say when issue query
> with
> > explain=true , the query response would return structured explanation,
> like
> > how many htable touched, how many records read from each htable and the
> > time consumption.
> >
> > Both kylin owner and user should benefit from such feature for
> > troubleshooting and query tuning.
> >
> > I filed KYLIN-1398, any comments? Please suggest.
> >
> > Thanks,
> > Ralph
> >
>
Re: [DISCUSS] Query explanation
Posted by Luke Han <lu...@gmail.com>.
Hi Ralph,
This is great feature we are looking for long time.
Would you mind to create another JIRA for UI part? I would like to say
KYLIN-1398 should focus on service side, right?
Thanks.
Luke
Best Regards!
---------------------
Luke Han
On Wed, Feb 3, 2016 at 11:35 AM, Liangfei.Su <su...@gmail.com> wrote:
> Hi, Kylin Community
>
> When troubleshooting kylin query, we could use "Insight" tab to issue
> query, then checking the kylin.log. This definitely the right way, but make
> it only available to kylin admin who can access the kylin log.
>
> Since most query system provide query explanation(same as calcite), Kylin
> might also output query explanation in response. Say when issue query with
> explain=true , the query response would return structured explanation, like
> how many htable touched, how many records read from each htable and the
> time consumption.
>
> Both kylin owner and user should benefit from such feature for
> troubleshooting and query tuning.
>
> I filed KYLIN-1398, any comments? Please suggest.
>
> Thanks,
> Ralph
>