RE: debug of TaskNode using IsolationRunner is different than real run!!! (fixed)

["Hiller, Dean (Contractor)" <de...@broadridge.com>]

I seem to have tracked it down.

 

It seems the environment variable HBASE_HOME is not set when the masters
start up the data nodes, but when I run locally on a datanode, it is set
just fine.  

 

I had variables set in .bash_profile and moved those exports to the
hadoop-env.sh and everything works now.  It took a lot of looking at the
classpaths with ps -ef | grep Child etc. to figure it all out.

 

Dean

 

From: Hiller, Dean (Contractor) 
Sent: Sunday, January 02, 2011 4:35 PM
To: 'mapreduce-user@hadoop.apache.org'
Subject: debug of TaskNode using IsolationRunner is different than real
run!!! ugh...help please

 

My guess is execution of hadoop script vs. launch of a child task differ
in the *.site.xml files they load????  Ie. My mapred-site.xml which
contains the Zookeeper quorum is read during IsolationRunner's run but
the childTask does not have the same access.  Has anyone setup zookeeper
and pointed datanodes to zookeeper?  Is that done through the
mapred-site.xml or zoo.cfg file?

 

Okay, so I have been running into my TaskNode not connecting to
Zookeeper(map/reduce involving hbase and hbase connects there not me).
I get this in the logs

 

011-01-02 15:28:26,250 INFO org.apache.zookeeper.ZooKeeper: Initiating
client connection, connectString=localhost:2181 sessionTimeout=60000
watcher=org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatche
r@165f738

 

 

Clearly, it should not be connecting to localhost since I have zookeeper
running on a different node than the DataNodes-TaskNodes.  

 

Now, the kicker, when I run IsolationRunner, the log is the following
now...

 

11/01/02 16:28:54 INFO zookeeper.ZooKeeper: Initiating client
connection, connec

tString=DENVER-DEV3.test.jsq.bsg.ad.adp.com:2181 sessionTimeout=60000
watcher=org.

apache.hadoop.hbase.client.HConnectionManager$ClientZKWatcher@3b4f0e

 

which is the correct node.  I don't get it.  

 


This message and any attachments are intended only for the use of the addressee and
may contain information that is privileged and confidential. If the reader of the 
message is not the intended recipient or an authorized representative of the
intended recipient, you are hereby notified that any dissemination of this
communication is strictly prohibited. If you have received this communication in
error, please notify us immediately by e-mail and delete the message and any
attachments from your system.