回复: Re: where is example of the configuration about multi nodes on one machine?

[beneo_7 <be...@163.com>]

awesome, thx, i will try it later

2010-12-01



beneo_7




发件人: Matthew Foley <ma...@yahoo-inc.com>
发送时间: 2010-12-01 02:01
主 题: Re: where is example of the configuration about multi nodes on one machine?
收件人: "common-user@hadoop.apache.org" <co...@hadoop.apache.org>
抄 送: Matthew Foley <ma...@yahoo-inc.com>





Here is a "recipe" for how to run multiple datanodes on a single server, posted to this list on Sept. 15: 
    http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201009.mbox/%3C8A898C33-DC4E-418C-ADC0-5689D434B05E@yahoo-inc.com%3E 

If you're having trouble getting multiple cores utilized,  this should help, assuming the computation is well expressed in Map/Reduce.   
There is no point in running VMs in your situation, as it just adds overhead without changing the amount of physical processor power  
available. 

--Matt 

On Nov 30, 2010, at 1:21 AM, beneo_7 wrote: 

> If you want to just use one machine, why do you want to use hadoop? Hadoop's 
> power lies in distributed computing. That being said, it is possible to use 
> hadoop on a single machine by using the pseudo-distributed mode (Read 
> http://hadoop.apache.org/common/docs/current/single_node_setup.html and 
> http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/). 
> If you are using just one machine, at least ensure that your machine has 
> lots of cores (8 core/16 cores would be great) to get benefit out of hadoop. 
>  
> I am not sure, but using virtual machines won't be helpful here as a virtual 
> machine is just an abstraction and not real hardware. 


thanks very much, i use the hadoop because the apache Mahout project need it for clustering. 

my machine is only one and powerful, 16cores and 32g mem, but i have only one, so i need configuration about multi nodes on one machine. 

i have used the "pseudo-distributed mode", however, the project always used 1 core,  the cpu freq always 100% ~ 103%, the time the execution is 4 hours, it's too slow. 

i can not change the mahout project source code, the trunk always update sometime, it's difficulty to solute confliction. 


is there any way to conf some slaves in one machine? 

At 2010-11-30 17:07:49，"Hari Sreekumar" <hs...@clickable.com> wrote: 

> Hi beneo, 
>  
> If you want to just use one machine, why do you want to use hadoop? Hadoop's 
> power lies in distributed computing. That being said, it is possible to use 
> hadoop on a single machine by using the pseudo-distributed mode (Read 
> http://hadoop.apache.org/common/docs/current/single_node_setup.html and 
> http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/). 
> If you are using just one machine, at least ensure that your machine has 
> lots of cores (8 core/16 cores would be great) to get benefit out of hadoop. 
>  
> I am not sure, but using virtual machines won't be helpful here as a virtual 
> machine is just an abstraction and not real hardware. 
>  
> Cheers, 
> Hari 
>  
> 2010/11/30 beneo_7 <be...@163.com> 
>  
>> i'm sorry, but, are you sure?? 
>> At 2010-11-30 15:53:58，"rahul patodi" <pa...@gmail.com> wrote: 
>>> you can create virtual machines on your single machine: 
>>> for you have to install sun virtual box(other tools are also available 
>> like 
>>> VMware) 
>>> now you can create as many virtual machine as you want 
>>> then you can create one master and all slaves 
>>>  
>>> -Thanks and Regards, 
>>> Rahul Patodi 
>>> Associate Software Engineer, 
>>> Impetus Infotech (India) Private Limited, 
>>> www.impetus.com 
>>> Mob:09907074413 
>>>  
>>> 2010/11/30 beneo_7 <be...@163.com> 
>>>  
>>>> i have only one machine and it's powerful. 
>>>> so, i want the all the slaves and master on one machine? 
>>>>  
>>>> thx in advanced 
>>>>  
>>>  
>>>  
>>>  
>>> -- 
>>  
>>