You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Li Li <fa...@gmail.com> on 2014/09/17 04:25:11 UTC
hadoop cluster crash problem
hi all,
I know it's not a problem related to hadoop but administrator can
not find any clues.
I have a machine with 24 core and 64GB memory with ubuntu 12.04
LTS. we use virtual box to create 4 virtual machine. Each vm has 10GB
memory and 6 core.
I have setup a small hadoop 1.2.1 cluster with one
jobtracker/namenode and 3 tasktracker/datanode. Each tasktrack has 4
mapper slots and 4 reducers slot.
But it always crashs(the host machine crash, not vm crash).
Sometimes it crashes for the first map-reduce job. Sometimes it can
run a few jobs.
is there any clues? I have checked the sys log and can find any
thing useful. Using monitor system, The cpu and io is not abnormal.
The only abnormal phenomenon is context switch is high. about 40k.
Re: hadoop cluster crash problem
Posted by Andre Kelpe <ak...@concurrentinc.com>.
virtualbox is known for causing instabilities in the host-kernel (or at
least, it used to). You might be better off asking for support there:
https://www.virtualbox.org/wiki/Bugtracker
- André
On Wed, Sep 17, 2014 at 4:25 AM, Li Li <fa...@gmail.com> wrote:
> hi all,
> I know it's not a problem related to hadoop but administrator can
> not find any clues.
> I have a machine with 24 core and 64GB memory with ubuntu 12.04
> LTS. we use virtual box to create 4 virtual machine. Each vm has 10GB
> memory and 6 core.
> I have setup a small hadoop 1.2.1 cluster with one
> jobtracker/namenode and 3 tasktracker/datanode. Each tasktrack has 4
> mapper slots and 4 reducers slot.
> But it always crashs(the host machine crash, not vm crash).
> Sometimes it crashes for the first map-reduce job. Sometimes it can
> run a few jobs.
> is there any clues? I have checked the sys log and can find any
> thing useful. Using monitor system, The cpu and io is not abnormal.
> The only abnormal phenomenon is context switch is high. about 40k.
>
--
André Kelpe
andre@concurrentinc.com
http://concurrentinc.com
Re: hadoop cluster crash problem
Posted by Andre Kelpe <ak...@concurrentinc.com>.
virtualbox is known for causing instabilities in the host-kernel (or at
least, it used to). You might be better off asking for support there:
https://www.virtualbox.org/wiki/Bugtracker
- André
On Wed, Sep 17, 2014 at 4:25 AM, Li Li <fa...@gmail.com> wrote:
> hi all,
> I know it's not a problem related to hadoop but administrator can
> not find any clues.
> I have a machine with 24 core and 64GB memory with ubuntu 12.04
> LTS. we use virtual box to create 4 virtual machine. Each vm has 10GB
> memory and 6 core.
> I have setup a small hadoop 1.2.1 cluster with one
> jobtracker/namenode and 3 tasktracker/datanode. Each tasktrack has 4
> mapper slots and 4 reducers slot.
> But it always crashs(the host machine crash, not vm crash).
> Sometimes it crashes for the first map-reduce job. Sometimes it can
> run a few jobs.
> is there any clues? I have checked the sys log and can find any
> thing useful. Using monitor system, The cpu and io is not abnormal.
> The only abnormal phenomenon is context switch is high. about 40k.
>
--
André Kelpe
andre@concurrentinc.com
http://concurrentinc.com
Re: hadoop cluster crash problem
Posted by Andre Kelpe <ak...@concurrentinc.com>.
virtualbox is known for causing instabilities in the host-kernel (or at
least, it used to). You might be better off asking for support there:
https://www.virtualbox.org/wiki/Bugtracker
- André
On Wed, Sep 17, 2014 at 4:25 AM, Li Li <fa...@gmail.com> wrote:
> hi all,
> I know it's not a problem related to hadoop but administrator can
> not find any clues.
> I have a machine with 24 core and 64GB memory with ubuntu 12.04
> LTS. we use virtual box to create 4 virtual machine. Each vm has 10GB
> memory and 6 core.
> I have setup a small hadoop 1.2.1 cluster with one
> jobtracker/namenode and 3 tasktracker/datanode. Each tasktrack has 4
> mapper slots and 4 reducers slot.
> But it always crashs(the host machine crash, not vm crash).
> Sometimes it crashes for the first map-reduce job. Sometimes it can
> run a few jobs.
> is there any clues? I have checked the sys log and can find any
> thing useful. Using monitor system, The cpu and io is not abnormal.
> The only abnormal phenomenon is context switch is high. about 40k.
>
--
André Kelpe
andre@concurrentinc.com
http://concurrentinc.com
Re: hadoop cluster crash problem
Posted by Andre Kelpe <ak...@concurrentinc.com>.
virtualbox is known for causing instabilities in the host-kernel (or at
least, it used to). You might be better off asking for support there:
https://www.virtualbox.org/wiki/Bugtracker
- André
On Wed, Sep 17, 2014 at 4:25 AM, Li Li <fa...@gmail.com> wrote:
> hi all,
> I know it's not a problem related to hadoop but administrator can
> not find any clues.
> I have a machine with 24 core and 64GB memory with ubuntu 12.04
> LTS. we use virtual box to create 4 virtual machine. Each vm has 10GB
> memory and 6 core.
> I have setup a small hadoop 1.2.1 cluster with one
> jobtracker/namenode and 3 tasktracker/datanode. Each tasktrack has 4
> mapper slots and 4 reducers slot.
> But it always crashs(the host machine crash, not vm crash).
> Sometimes it crashes for the first map-reduce job. Sometimes it can
> run a few jobs.
> is there any clues? I have checked the sys log and can find any
> thing useful. Using monitor system, The cpu and io is not abnormal.
> The only abnormal phenomenon is context switch is high. about 40k.
>
--
André Kelpe
andre@concurrentinc.com
http://concurrentinc.com