You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by GuoGuo <gu...@163.com> on 2016/07/24 10:33:38 UTC
Process hang up during buiding cube on kylin?
Hi,
I started to build a cube with Kylin Web UI, and after several hours the process seemed hang up during buiding cube section. For several times I purged and rebuilt the cube, and the Monitor UI for the job like this:
First I thought the data scale may be shrinked to reduce the time, and I truncated all data in the fact table in hive, but still it took a very very long time for me to wait during the "Build Cube" step. Then I discarded the job in the Monitor view, and tried to export the diagnose info, but the page's always loading and didn't export any file or result.
I don't know what root causes for this problem, and how can I fix it?
Thanks in advance~!
Lee
Re: Re: Re: Process hang up during buiding cube on kylin?
Posted by hongbin ma <ma...@apache.org>.
seems yarn cannot allocate enough containers.
what's your yarn config? can you increase resources?
if you're using a hdp or cdh sandbox, try to give it more than 8G RAM
On Mon, Jul 25, 2016 at 10:08 AM, GuoGuo <gu...@163.com> wrote:
> 2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=<memory:2048, vCores:0>2016-07-25 05:46:07,532 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce
>
>
--
Regards,
*Bin Mahone | 马洪宾*
Re:Re: Re: Process hang up during buiding cube on kylin?
Posted by GuoGuo <gu...@163.com>.
I checked the link for yarn job tracking page, and I saw one map task and one reduce task, however they were all in "pending" status. Then I drilled down by click the number, I saw the item shows "scheduled" in the state column. I went back to the yarn's job tracking page and clicked the "logs" link, and the page shows three links:
stderr : Total file length is 2661 bytes.
stdout : Total file length is 0 bytes.
syslog : Total file length is 481344 bytes.
I clicked the stderr link and found nothing wrong, but the syslog link shows the message repeatedly:
2016-07-25 05:46:07,532 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Ramping down all scheduled reduces:0
2016-07-25 05:46:07,532 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Going to preempt 3 due to lack of space for maps
2016-07-25 05:46:07,532 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=<memory:2048, vCores:0>
2016-07-25 05:46:07,532 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1
I am eager to know what should I do to fix it and make the job start to run.
Thanks~!
At 2016-07-25 09:30:09, "hongbin ma" <ma...@apache.org> wrote:
Skipping is normal as it reflects switching from layered cubing to fast cubing.
There's a icon in "Build Cube" step linking to the hadoop job status, you should checkout that
--
Regards,
Bin Mahone | 马洪宾
Re: Re: Process hang up during buiding cube on kylin?
Posted by hongbin ma <ma...@apache.org>.
Skipping is normal as it reflects switching from layered cubing to fast
cubing.
There's a icon in "Build Cube" step linking to the hadoop job status, you
should checkout that
--
Regards,
*Bin Mahone | 马洪宾*
Re:Re: Process hang up during buiding cube on kylin?
Posted by GuoGuo <gu...@163.com>.
When I started to run a job building a cube, I saw the steps worked fine from create intermediate flat hive tables to creating HTable (the output log shows "exit code 1" in the creating HTable step). However, from the step of building base cuboid data, the output log shows "skipped", and so did the next steps of building N-Dimension Cuboid Data. After these "skipped" steps, the step of "Build Cube" seemed hang up since after several hours this step still not finished.
I don't know why this happened, and how can I fix the problem? Thanks in advance~!
At 2016-07-24 20:50:52, "hongbin ma" <ma...@apache.org> wrote:
the picture does not display
if you want to use picture attachments you'd better use http://apache-spark-user-list.1001560.n3.nabble.com/ to post.
BTW can you attach yarn logs for the hanging job/mappers/reducers?
On Sun, Jul 24, 2016 at 6:33 PM, GuoGuo <gu...@163.com> wrote:
ong time for me to wait during the "Build Cube" step. Then I discarded the job in the Monitor view, and tried to export the diagnose info, but the page's always loading and didn't export any file or result.
--
Regards,
Bin Mahone | 马洪宾
Re: Process hang up during buiding cube on kylin?
Posted by hongbin ma <ma...@apache.org>.
the picture does not display
if you want to use picture attachments you'd better use
http://apache-spark-user-list.1001560.n3.nabble.com/ to post.
BTW can you attach yarn logs for the hanging job/mappers/reducers?
On Sun, Jul 24, 2016 at 6:33 PM, GuoGuo <gu...@163.com> wrote:
> ong time for me to wait during the "Build Cube" step. Then I discarded the
> job in the Monitor view, and tried to export the diagnose info, but the
> page's always loading and didn't export any file or result.
>
--
Regards,
*Bin Mahone | 马洪宾*