You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by Karthigeyan K <ka...@gmail.com> on 2016/07/22 06:11:51 UTC
Kylin auto refresh is not working
Hi,
I am trying to use incremental cube building .
I have a hive table partitioned by date. I created and built kylin cube
for it.
I attached the image of my cube refresh settings. I am trying to auto
merge new data records every 30 minutes.
I added a new partition(date=2016-07-21) to hive table with 10000 more
records. when I do a count(*) in hive the added rows got reflected in
result.
But Kylin cube in not updated even after many hours.
I don't know what am I missing here, Your Kind help is appreciated.
[image: Inline image 2]
Thanks,
Karthigeyan.
Re: Kylin auto refresh is not working
Posted by hongbin ma <ma...@apache.org>.
hi alberto
1892 and 774 are not same thing. 1892 is same as
https://issues.apache.org/jira/browse/KYLIN-1864
On Thu, Jul 28, 2016 at 3:57 PM, Alberto Ramón <a....@gmail.com>
wrote:
> Kylin 1892 its same that 774 ??
>
> https://issues.apache.org/jira/browse/KYLIN-1892
> https://issues.apache.org/jira/browse/KYLIN-774
>
> Thanks
>
> 2016-07-28 8:53 GMT+02:00 Alberto Ramón <a....@gmail.com>:
>
>> Hi
>>
>> Interesting:
>> https://issues.apache.org/jira/browse/KYLIN-1892
>>
>> Thanks, Alberto
>>
>> 2016-07-28 8:28 GMT+02:00 ShaoFeng Shi <sh...@apache.org>:
>>
>>> yes you need trigger the build from outside; The simpliest way is using
>>> Linux cron + curl comand; As that is quite easy to achieve, Kylin didn't
>>> re-invent the wheel.
>>>
>>> This page has the descriptions about the refresh settings, please have a
>>> check if you haven't :
>>> https://kylin.apache.org/docs15/tutorial/create_cube.html
>>> Please come back if it couldn't answer your question.
>>>
>>> 2016-07-25 13:03 GMT+08:00 Karthigeyan K <ka...@gmail.com>:
>>>
>>>> Thanks for the reply ShaoFeng. So we need to trigger the cube refresh
>>>> externally always? Or Is there a way kylin do it automatically.
>>>> I am confused with this Cube refresh settings . could you enlight me
>>>> about that.
>>>>
>>>> On Fri, Jul 22, 2016 at 4:08 PM, ShaoFeng Shi <sh...@apache.org>
>>>> wrote:
>>>>
>>>>> Kylin exposes RESTful APIs for user to trigger the cube
>>>>> build/merge/refresh; you can easily integrate it with your scheduling tools
>>>>> or workflow system, please check
>>>>> https://kylin.apache.org/docs15/howto/howto_use_restapi.html#build-cube
>>>>>
>>>>>
>>>>>
>>>>> 2016-07-22 14:31 GMT+08:00 Karthigeyan K <ka...@gmail.com>:
>>>>>
>>>>>> Hi,
>>>>>> I am trying to use incremental cube building .
>>>>>> I have a hive table partitioned by date. I created and built kylin
>>>>>> cube for it.
>>>>>> I attached the image of my cube refresh settings. I am trying to
>>>>>> auto merge new data records every 30 minutes.
>>>>>> I added a new partition(date=2016-07-21) to hive table with 10000
>>>>>> more records. when I do a count(*) in hive the added rows got reflected in
>>>>>> result.
>>>>>> But Kylin cube in not updated even after many hours.
>>>>>> When I manually click refresh on cube Its updating. But I like to
>>>>>> happen automatically.
>>>>>> I don't know what am I missing here, Your Kind help is appreciated.
>>>>>>
>>>>>>
>>>>>> [image: Inline image 2]
>>>>>>
>>>>>> Thanks,
>>>>>> Karthigeyan.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>>
>>>>> Shaofeng Shi
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Best regards,
>>>
>>> Shaofeng Shi
>>>
>>>
>>
>
--
Regards,
*Bin Mahone | 马洪宾*
Re: Kylin auto refresh is not working
Posted by Alberto Ramón <a....@gmail.com>.
Kylin 1892 its same that 774 ??
https://issues.apache.org/jira/browse/KYLIN-1892
https://issues.apache.org/jira/browse/KYLIN-774
Thanks
2016-07-28 8:53 GMT+02:00 Alberto Ramón <a....@gmail.com>:
> Hi
>
> Interesting:
> https://issues.apache.org/jira/browse/KYLIN-1892
>
> Thanks, Alberto
>
> 2016-07-28 8:28 GMT+02:00 ShaoFeng Shi <sh...@apache.org>:
>
>> yes you need trigger the build from outside; The simpliest way is using
>> Linux cron + curl comand; As that is quite easy to achieve, Kylin didn't
>> re-invent the wheel.
>>
>> This page has the descriptions about the refresh settings, please have a
>> check if you haven't :
>> https://kylin.apache.org/docs15/tutorial/create_cube.html
>> Please come back if it couldn't answer your question.
>>
>> 2016-07-25 13:03 GMT+08:00 Karthigeyan K <ka...@gmail.com>:
>>
>>> Thanks for the reply ShaoFeng. So we need to trigger the cube refresh
>>> externally always? Or Is there a way kylin do it automatically.
>>> I am confused with this Cube refresh settings . could you enlight me
>>> about that.
>>>
>>> On Fri, Jul 22, 2016 at 4:08 PM, ShaoFeng Shi <sh...@apache.org>
>>> wrote:
>>>
>>>> Kylin exposes RESTful APIs for user to trigger the cube
>>>> build/merge/refresh; you can easily integrate it with your scheduling tools
>>>> or workflow system, please check
>>>> https://kylin.apache.org/docs15/howto/howto_use_restapi.html#build-cube
>>>>
>>>>
>>>>
>>>> 2016-07-22 14:31 GMT+08:00 Karthigeyan K <ka...@gmail.com>:
>>>>
>>>>> Hi,
>>>>> I am trying to use incremental cube building .
>>>>> I have a hive table partitioned by date. I created and built kylin
>>>>> cube for it.
>>>>> I attached the image of my cube refresh settings. I am trying to
>>>>> auto merge new data records every 30 minutes.
>>>>> I added a new partition(date=2016-07-21) to hive table with 10000 more
>>>>> records. when I do a count(*) in hive the added rows got reflected in
>>>>> result.
>>>>> But Kylin cube in not updated even after many hours.
>>>>> When I manually click refresh on cube Its updating. But I like to
>>>>> happen automatically.
>>>>> I don't know what am I missing here, Your Kind help is appreciated.
>>>>>
>>>>>
>>>>> [image: Inline image 2]
>>>>>
>>>>> Thanks,
>>>>> Karthigeyan.
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>>
>>>> Shaofeng Shi
>>>>
>>>>
>>>
>>
>>
>> --
>> Best regards,
>>
>> Shaofeng Shi
>>
>>
>
Re: Kylin auto refresh is not working
Posted by Alberto Ramón <a....@gmail.com>.
Hi
Interesting:
https://issues.apache.org/jira/browse/KYLIN-1892
Thanks, Alberto
2016-07-28 8:28 GMT+02:00 ShaoFeng Shi <sh...@apache.org>:
> yes you need trigger the build from outside; The simpliest way is using
> Linux cron + curl comand; As that is quite easy to achieve, Kylin didn't
> re-invent the wheel.
>
> This page has the descriptions about the refresh settings, please have a
> check if you haven't :
> https://kylin.apache.org/docs15/tutorial/create_cube.html
> Please come back if it couldn't answer your question.
>
> 2016-07-25 13:03 GMT+08:00 Karthigeyan K <ka...@gmail.com>:
>
>> Thanks for the reply ShaoFeng. So we need to trigger the cube refresh
>> externally always? Or Is there a way kylin do it automatically.
>> I am confused with this Cube refresh settings . could you enlight me
>> about that.
>>
>> On Fri, Jul 22, 2016 at 4:08 PM, ShaoFeng Shi <sh...@apache.org>
>> wrote:
>>
>>> Kylin exposes RESTful APIs for user to trigger the cube
>>> build/merge/refresh; you can easily integrate it with your scheduling tools
>>> or workflow system, please check
>>> https://kylin.apache.org/docs15/howto/howto_use_restapi.html#build-cube
>>>
>>>
>>>
>>> 2016-07-22 14:31 GMT+08:00 Karthigeyan K <ka...@gmail.com>:
>>>
>>>> Hi,
>>>> I am trying to use incremental cube building .
>>>> I have a hive table partitioned by date. I created and built kylin
>>>> cube for it.
>>>> I attached the image of my cube refresh settings. I am trying to
>>>> auto merge new data records every 30 minutes.
>>>> I added a new partition(date=2016-07-21) to hive table with 10000 more
>>>> records. when I do a count(*) in hive the added rows got reflected in
>>>> result.
>>>> But Kylin cube in not updated even after many hours.
>>>> When I manually click refresh on cube Its updating. But I like to
>>>> happen automatically.
>>>> I don't know what am I missing here, Your Kind help is appreciated.
>>>>
>>>>
>>>> [image: Inline image 2]
>>>>
>>>> Thanks,
>>>> Karthigeyan.
>>>>
>>>>
>>>
>>>
>>> --
>>> Best regards,
>>>
>>> Shaofeng Shi
>>>
>>>
>>
>
>
> --
> Best regards,
>
> Shaofeng Shi
>
>
Re: Kylin auto refresh is not working
Posted by ShaoFeng Shi <sh...@apache.org>.
yes you need trigger the build from outside; The simpliest way is using
Linux cron + curl comand; As that is quite easy to achieve, Kylin didn't
re-invent the wheel.
This page has the descriptions about the refresh settings, please have a
check if you haven't :
https://kylin.apache.org/docs15/tutorial/create_cube.html
Please come back if it couldn't answer your question.
2016-07-25 13:03 GMT+08:00 Karthigeyan K <ka...@gmail.com>:
> Thanks for the reply ShaoFeng. So we need to trigger the cube refresh
> externally always? Or Is there a way kylin do it automatically.
> I am confused with this Cube refresh settings . could you enlight me about
> that.
>
> On Fri, Jul 22, 2016 at 4:08 PM, ShaoFeng Shi <sh...@apache.org>
> wrote:
>
>> Kylin exposes RESTful APIs for user to trigger the cube
>> build/merge/refresh; you can easily integrate it with your scheduling tools
>> or workflow system, please check
>> https://kylin.apache.org/docs15/howto/howto_use_restapi.html#build-cube
>>
>>
>>
>> 2016-07-22 14:31 GMT+08:00 Karthigeyan K <ka...@gmail.com>:
>>
>>> Hi,
>>> I am trying to use incremental cube building .
>>> I have a hive table partitioned by date. I created and built kylin
>>> cube for it.
>>> I attached the image of my cube refresh settings. I am trying to auto
>>> merge new data records every 30 minutes.
>>> I added a new partition(date=2016-07-21) to hive table with 10000 more
>>> records. when I do a count(*) in hive the added rows got reflected in
>>> result.
>>> But Kylin cube in not updated even after many hours.
>>> When I manually click refresh on cube Its updating. But I like to happen
>>> automatically.
>>> I don't know what am I missing here, Your Kind help is appreciated.
>>>
>>>
>>> [image: Inline image 2]
>>>
>>> Thanks,
>>> Karthigeyan.
>>>
>>>
>>
>>
>> --
>> Best regards,
>>
>> Shaofeng Shi
>>
>>
>
--
Best regards,
Shaofeng Shi
Re: Kylin auto refresh is not working
Posted by Karthigeyan K <ka...@gmail.com>.
Thanks for the reply ShaoFeng. So we need to trigger the cube refresh
externally always? Or Is there a way kylin do it automatically.
I am confused with this Cube refresh settings . could you enlight me about
that.
On Fri, Jul 22, 2016 at 4:08 PM, ShaoFeng Shi <sh...@apache.org>
wrote:
> Kylin exposes RESTful APIs for user to trigger the cube
> build/merge/refresh; you can easily integrate it with your scheduling tools
> or workflow system, please check
> https://kylin.apache.org/docs15/howto/howto_use_restapi.html#build-cube
>
>
>
> 2016-07-22 14:31 GMT+08:00 Karthigeyan K <ka...@gmail.com>:
>
>> Hi,
>> I am trying to use incremental cube building .
>> I have a hive table partitioned by date. I created and built kylin
>> cube for it.
>> I attached the image of my cube refresh settings. I am trying to auto
>> merge new data records every 30 minutes.
>> I added a new partition(date=2016-07-21) to hive table with 10000 more
>> records. when I do a count(*) in hive the added rows got reflected in
>> result.
>> But Kylin cube in not updated even after many hours.
>> When I manually click refresh on cube Its updating. But I like to happen
>> automatically.
>> I don't know what am I missing here, Your Kind help is appreciated.
>>
>>
>> [image: Inline image 2]
>>
>> Thanks,
>> Karthigeyan.
>>
>>
>
>
> --
> Best regards,
>
> Shaofeng Shi
>
>
Re: Kylin auto refresh is not working
Posted by ShaoFeng Shi <sh...@apache.org>.
Kylin exposes RESTful APIs for user to trigger the cube
build/merge/refresh; you can easily integrate it with your scheduling tools
or workflow system, please check
https://kylin.apache.org/docs15/howto/howto_use_restapi.html#build-cube
2016-07-22 14:31 GMT+08:00 Karthigeyan K <ka...@gmail.com>:
> Hi,
> I am trying to use incremental cube building .
> I have a hive table partitioned by date. I created and built kylin cube
> for it.
> I attached the image of my cube refresh settings. I am trying to auto
> merge new data records every 30 minutes.
> I added a new partition(date=2016-07-21) to hive table with 10000 more
> records. when I do a count(*) in hive the added rows got reflected in
> result.
> But Kylin cube in not updated even after many hours.
> When I manually click refresh on cube Its updating. But I like to happen
> automatically.
> I don't know what am I missing here, Your Kind help is appreciated.
>
>
> [image: Inline image 2]
>
> Thanks,
> Karthigeyan.
>
>
--
Best regards,
Shaofeng Shi
Fwd: Kylin auto refresh is not working
Posted by Karthigeyan K <ka...@gmail.com>.
Hi,
I am trying to use incremental cube building .
I have a hive table partitioned by date. I created and built kylin cube
for it.
I attached the image of my cube refresh settings. I am trying to auto
merge new data records every 30 minutes.
I added a new partition(date=2016-07-21) to hive table with 10000 more
records. when I do a count(*) in hive the added rows got reflected in
result.
But Kylin cube in not updated even after many hours.
When I manually click refresh on cube Its updating. But I like to happen
automatically.
I don't know what am I missing here, Your Kind help is appreciated.
[image: Inline image 2]
Thanks,
Karthigeyan.