You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by ArunKumar <ar...@gmail.com> on 2011/12/10 08:50:10 UTC

Grouping nodes into different racks in Hadoop Cluster

Hi guys !

I am able to set up Hadoop Multinode Clusters as per 
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/
.
I have all my nodes in a LAN. 
How do i group them in different racks ?
I have only a 5 node cluster of say 4GB RAM and i want to create a big
cluster with these using virtualization and group the resulting linux
instances and some real nodes under different racks. How i can do it ?

Any help ?

Thanks,
Arun


--
View this message in context: http://lucene.472066.n3.nabble.com/Grouping-nodes-into-different-racks-in-Hadoop-Cluster-tp3574978p3574978.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.

RE: Grouping nodes into different racks in Hadoop Cluster

Posted by ArunKumar <ar...@gmail.com>.

Hi !

i have three node cluster set up according to 
http://ankitasblogger.blogspot.com/2011/01/hadoop-cluster-setup.html
http://ankitasblogger.blogspot.com/2011/01/hadoop-cluster-setup.html 

I have written a topology script and it doesn't work.
For testing purpose i have also put simple script:

node=$0
echo /cluster$node

I get following error :
2011-12-12 15:33:45,878 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 1 on 9001, call
heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@c77b27, true, true,
true, -1) from 10.0.1.54:35885: error: java.io.IOException:
java.lang.NullPointerException
java.io.IOException: java.lang.NullPointerException
	at
org.apache.hadoop.mapred.JobTracker.resolveAndAddToTopology(JobTracker.java:2385)
	at org.apache.hadoop.mapred.JobTracker.addNewTracker(JobTracker.java:2377)
	at
org.apache.hadoop.mapred.JobTracker.processHeartbeat(JobTracker.java:2756)
	at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2556)
	at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
2011-12-12 15:33:45,879 INFO org.apache.hadoop.mapred.JobTracker: Lost
tracker 'tracker_hbase-master:localhost/127.0.0.1:46028'

Any help ?

Arun





--
View this message in context: http://lucene.472066.n3.nabble.com/Grouping-nodes-into-different-racks-in-Hadoop-Cluster-tp3574978p3579101.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.

RE: Grouping nodes into different racks in Hadoop Cluster

Posted by Devaraj K <de...@huawei.com>.

Hi Arun,

	You can enable rack awareness for your hadoop cluster by configuring
the "topology.script.file.name" property.


Please go through this link for more details about rack awareness.

http://hadoop.apache.org/common/docs/r0.19.2/cluster_setup.html#Hadoop+Rack+
Awareness


Devaraj K 


-----Original Message-----
From: ArunKumar [mailto:arunk786@gmail.com] 
Sent: Saturday, December 10, 2011 1:20 PM
To: hadoop-user@lucene.apache.org
Subject: Grouping nodes into different racks in Hadoop Cluster

Hi guys !

I am able to set up Hadoop Multinode Clusters as per 
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-
node-cluster/
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-
node-cluster/
.
I have all my nodes in a LAN. 
How do i group them in different racks ?
I have only a 5 node cluster of say 4GB RAM and i want to create a big
cluster with these using virtualization and group the resulting linux
instances and some real nodes under different racks. How i can do it ?

Any help ?

Thanks,
Arun


--
View this message in context:
http://lucene.472066.n3.nabble.com/Grouping-nodes-into-different-racks-in-Ha
doop-Cluster-tp3574978p3574978.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.