You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by Edward Zhang <yo...@apache.org> on 2016/01/18 07:44:07 UTC
Kylin service crash easily while building cube in HDP sandbox
Hi,
I was trying to play Kylin 1.2 in HDP sandbox, but unfortunately it always
fails while it builds cube through Kylin service. The symptom is that Kylin
tomcat service failed each time when it is about in the step 12, which
is "Calculate
HTable Region Splits"
I suspected it is out-of-memory issue, so I enabled GC log but did not see
any full gc and no dump also.
What is the possible reason for Kylin service crash? Do you have similar
experience.
Also what I observed is that Kylin service is used for both job client and
ad-hoc query, any consideration of why not decouple the 2 functionalities
with different lifecycles into different processes?
Thanks
Edward
Re: Kylin service crash easily while building cube in HDP sandbox
Posted by Li Yang <li...@apache.org>.
What's the HDP version? Kylin 1.2 only works with HDP 2.2.4. Make sure
you are NOT running on latest HDP 2.3
On Tue, Jan 19, 2016 at 1:34 PM, 宋轶 <yi...@outlook.com> wrote:
> I remember we can config the service to be a job engine or a query engine.
>
> > From: mahongbin@apache.org
> > Date: Mon, 18 Jan 2016 15:50:07 +0800
> > Subject: Re: Kylin service crash easily while building cube in HDP
> sandbox
> > To: dev@kylin.apache.org
> >
> > btw "Calculate HTable Region Splits" should not cause OOM, can you do an
> > analysis on the dump?
> >
> > On Mon, Jan 18, 2016 at 3:48 PM, hongbin ma <ma...@apache.org>
> wrote:
> >
> > > what's the size of the cube you're building? the default configuration
> for
> > > Xmx and Xms is not very big.
> > >
> > > kylin does provided sth similar to what you meant by "different
> > > lifecycles", checkout
> > > http://kylin.apache.org/docs/install/kylin_cluster.html. The reason
> why
> > > we put them into same tomcat instance is to ease the installation for
> > > newbies.
> > >
> > >
> > > On Mon, Jan 18, 2016 at 2:44 PM, Edward Zhang <yonzhang2012@apache.org
> >
> > > wrote:
> > >
> > >> Hi,
> > >>
> > >> I was trying to play Kylin 1.2 in HDP sandbox, but unfortunately it
> always
> > >> fails while it builds cube through Kylin service. The symptom is that
> > >> Kylin
> > >> tomcat service failed each time when it is about in the step 12, which
> > >> is "Calculate
> > >> HTable Region Splits"
> > >>
> > >> I suspected it is out-of-memory issue, so I enabled GC log but did
> not see
> > >> any full gc and no dump also.
> > >>
> > >> What is the possible reason for Kylin service crash? Do you have
> similar
> > >> experience.
> > >>
> > >> Also what I observed is that Kylin service is used for both job
> client and
> > >> ad-hoc query, any consideration of why not decouple the 2
> functionalities
> > >> with different lifecycles into different processes?
> > >>
> > >>
> > >> Thanks
> > >> Edward
> > >>
> > >
> > >
> > >
> > > --
> > > Regards,
> > >
> > > *Bin Mahone | 马洪宾*
> > > Apache Kylin: http://kylin.io
> > > Github: https://github.com/binmahone
> > >
> >
> >
> >
> > --
> > Regards,
> >
> > *Bin Mahone | 马洪宾*
> > Apache Kylin: http://kylin.io
> > Github: https://github.com/binmahone
>
>
RE: Kylin service crash easily while building cube in HDP sandbox
Posted by 宋轶 <yi...@outlook.com>.
I remember we can config the service to be a job engine or a query engine.
> From: mahongbin@apache.org
> Date: Mon, 18 Jan 2016 15:50:07 +0800
> Subject: Re: Kylin service crash easily while building cube in HDP sandbox
> To: dev@kylin.apache.org
>
> btw "Calculate HTable Region Splits" should not cause OOM, can you do an
> analysis on the dump?
>
> On Mon, Jan 18, 2016 at 3:48 PM, hongbin ma <ma...@apache.org> wrote:
>
> > what's the size of the cube you're building? the default configuration for
> > Xmx and Xms is not very big.
> >
> > kylin does provided sth similar to what you meant by "different
> > lifecycles", checkout
> > http://kylin.apache.org/docs/install/kylin_cluster.html. The reason why
> > we put them into same tomcat instance is to ease the installation for
> > newbies.
> >
> >
> > On Mon, Jan 18, 2016 at 2:44 PM, Edward Zhang <yo...@apache.org>
> > wrote:
> >
> >> Hi,
> >>
> >> I was trying to play Kylin 1.2 in HDP sandbox, but unfortunately it always
> >> fails while it builds cube through Kylin service. The symptom is that
> >> Kylin
> >> tomcat service failed each time when it is about in the step 12, which
> >> is "Calculate
> >> HTable Region Splits"
> >>
> >> I suspected it is out-of-memory issue, so I enabled GC log but did not see
> >> any full gc and no dump also.
> >>
> >> What is the possible reason for Kylin service crash? Do you have similar
> >> experience.
> >>
> >> Also what I observed is that Kylin service is used for both job client and
> >> ad-hoc query, any consideration of why not decouple the 2 functionalities
> >> with different lifecycles into different processes?
> >>
> >>
> >> Thanks
> >> Edward
> >>
> >
> >
> >
> > --
> > Regards,
> >
> > *Bin Mahone | 马洪宾*
> > Apache Kylin: http://kylin.io
> > Github: https://github.com/binmahone
> >
>
>
>
> --
> Regards,
>
> *Bin Mahone | 马洪宾*
> Apache Kylin: http://kylin.io
> Github: https://github.com/binmahone
Re: Kylin service crash easily while building cube in HDP sandbox
Posted by hongbin ma <ma...@apache.org>.
btw "Calculate HTable Region Splits" should not cause OOM, can you do an
analysis on the dump?
On Mon, Jan 18, 2016 at 3:48 PM, hongbin ma <ma...@apache.org> wrote:
> what's the size of the cube you're building? the default configuration for
> Xmx and Xms is not very big.
>
> kylin does provided sth similar to what you meant by "different
> lifecycles", checkout
> http://kylin.apache.org/docs/install/kylin_cluster.html. The reason why
> we put them into same tomcat instance is to ease the installation for
> newbies.
>
>
> On Mon, Jan 18, 2016 at 2:44 PM, Edward Zhang <yo...@apache.org>
> wrote:
>
>> Hi,
>>
>> I was trying to play Kylin 1.2 in HDP sandbox, but unfortunately it always
>> fails while it builds cube through Kylin service. The symptom is that
>> Kylin
>> tomcat service failed each time when it is about in the step 12, which
>> is "Calculate
>> HTable Region Splits"
>>
>> I suspected it is out-of-memory issue, so I enabled GC log but did not see
>> any full gc and no dump also.
>>
>> What is the possible reason for Kylin service crash? Do you have similar
>> experience.
>>
>> Also what I observed is that Kylin service is used for both job client and
>> ad-hoc query, any consideration of why not decouple the 2 functionalities
>> with different lifecycles into different processes?
>>
>>
>> Thanks
>> Edward
>>
>
>
>
> --
> Regards,
>
> *Bin Mahone | 马洪宾*
> Apache Kylin: http://kylin.io
> Github: https://github.com/binmahone
>
--
Regards,
*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone
Re: Kylin service crash easily while building cube in HDP sandbox
Posted by hongbin ma <ma...@apache.org>.
what's the size of the cube you're building? the default configuration for
Xmx and Xms is not very big.
kylin does provided sth similar to what you meant by "different
lifecycles", checkout
http://kylin.apache.org/docs/install/kylin_cluster.html. The reason why we
put them into same tomcat instance is to ease the installation for newbies.
On Mon, Jan 18, 2016 at 2:44 PM, Edward Zhang <yo...@apache.org>
wrote:
> Hi,
>
> I was trying to play Kylin 1.2 in HDP sandbox, but unfortunately it always
> fails while it builds cube through Kylin service. The symptom is that Kylin
> tomcat service failed each time when it is about in the step 12, which
> is "Calculate
> HTable Region Splits"
>
> I suspected it is out-of-memory issue, so I enabled GC log but did not see
> any full gc and no dump also.
>
> What is the possible reason for Kylin service crash? Do you have similar
> experience.
>
> Also what I observed is that Kylin service is used for both job client and
> ad-hoc query, any consideration of why not decouple the 2 functionalities
> with different lifecycles into different processes?
>
>
> Thanks
> Edward
>
--
Regards,
*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone