You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Xuzhan Sun <su...@outlook.com> on 2015/03/24 17:53:39 UTC
Can Pseudo-Distributed Mode take advantage of multi-core structure?
Hello,
I want to do some test on my single node cluster for Speed. I know it is easy to set up the Pseudo-Distributed Mode, and Hadoop will start one Java process for each single map/reduce.
My question is: is it parallel enough on multi-core CPU? I mean if I have 4 mappers at the same time while my CPU have 4 cores, will the 4 mappers be running on different cores at the same time?
I know another way to simulate a Hadoop cluster with one machine is to use virtual machine software such as VMware to simulate multiple machines and set up a cluster upon these virtual machines. What's the difference between the two methods for SPEED and PARALLEL on multi-core structure?
Thanks,
Xuzhan
Re: Can Pseudo-Distributed Mode take advantage of multi-core structure?
Posted by Michael Segel <ms...@hotmail.com>.
Short answer yes.
> On Mar 24, 2015, at 11:53 AM, Xuzhan Sun <su...@outlook.com> wrote:
>
> Hello,
>
> I want to do some test on my single node cluster for Speed. I know it is easy to set up the Pseudo-Distributed Mode, and Hadoop will start one Java process for each single map/reduce.
>
> My question is: is it parallel enough on multi-core CPU? I mean if I have 4 mappers at the same time while my CPU have 4 cores, will the 4 mappers be running on different cores at the same time?
>
> I know another way to simulate a Hadoop cluster with one machine is to use virtual machine software such as VMware to simulate multiple machines and set up a cluster upon these virtual machines. What's the difference between the two methods for SPEED and PARALLEL on multi-core structure?
>
> Thanks,
>
> Xuzhan
Re: Can Pseudo-Distributed Mode take advantage of multi-core structure?
Posted by Michael Segel <ms...@hotmail.com>.
Short answer yes.
> On Mar 24, 2015, at 11:53 AM, Xuzhan Sun <su...@outlook.com> wrote:
>
> Hello,
>
> I want to do some test on my single node cluster for Speed. I know it is easy to set up the Pseudo-Distributed Mode, and Hadoop will start one Java process for each single map/reduce.
>
> My question is: is it parallel enough on multi-core CPU? I mean if I have 4 mappers at the same time while my CPU have 4 cores, will the 4 mappers be running on different cores at the same time?
>
> I know another way to simulate a Hadoop cluster with one machine is to use virtual machine software such as VMware to simulate multiple machines and set up a cluster upon these virtual machines. What's the difference between the two methods for SPEED and PARALLEL on multi-core structure?
>
> Thanks,
>
> Xuzhan
Re: Can Pseudo-Distributed Mode take advantage of multi-core structure?
Posted by Michael Segel <ms...@hotmail.com>.
Short answer yes.
> On Mar 24, 2015, at 11:53 AM, Xuzhan Sun <su...@outlook.com> wrote:
>
> Hello,
>
> I want to do some test on my single node cluster for Speed. I know it is easy to set up the Pseudo-Distributed Mode, and Hadoop will start one Java process for each single map/reduce.
>
> My question is: is it parallel enough on multi-core CPU? I mean if I have 4 mappers at the same time while my CPU have 4 cores, will the 4 mappers be running on different cores at the same time?
>
> I know another way to simulate a Hadoop cluster with one machine is to use virtual machine software such as VMware to simulate multiple machines and set up a cluster upon these virtual machines. What's the difference between the two methods for SPEED and PARALLEL on multi-core structure?
>
> Thanks,
>
> Xuzhan
Re: Can Pseudo-Distributed Mode take advantage of multi-core structure?
Posted by Michael Segel <ms...@hotmail.com>.
Short answer yes.
> On Mar 24, 2015, at 11:53 AM, Xuzhan Sun <su...@outlook.com> wrote:
>
> Hello,
>
> I want to do some test on my single node cluster for Speed. I know it is easy to set up the Pseudo-Distributed Mode, and Hadoop will start one Java process for each single map/reduce.
>
> My question is: is it parallel enough on multi-core CPU? I mean if I have 4 mappers at the same time while my CPU have 4 cores, will the 4 mappers be running on different cores at the same time?
>
> I know another way to simulate a Hadoop cluster with one machine is to use virtual machine software such as VMware to simulate multiple machines and set up a cluster upon these virtual machines. What's the difference between the two methods for SPEED and PARALLEL on multi-core structure?
>
> Thanks,
>
> Xuzhan