Re: problem running on a cluster of mixed hardware due to "Incompatible" buildVersion of JobTracker adn TaskTracker

[Brian Bockelman <bb...@cse.unl.edu>]

Hey Bill,

I might be giving you bad advice (I've only verified this for HDFS  
components on the 0.19.x branch, not the JT/TT or the 0.18.x branch),  
but...

In my understanding, Hadoop only compares the base SVN revision  
number, not the build strings.  Make sure that both have the SVN rev.   
Our build machines don't have svn installed, so we actually have to  
generate the correct SVN rev, then submit it to the build clusters.   
We certainly don't have our build time strings match up.

Hope this helps!

Brian

On Apr 6, 2009, at 4:46 PM, Bill Au wrote:

> I am using a cluster of mixed hardware, 32-bit and 64-bit machines,  
> to run
> Hadoop 0.18.3.  I can't use the distribution tar ball since I need  
> to apply
> a couple of patches.  So I build my own Hadoop binaries after  
> applying the
> patches that I need.  I build two copies, one for 32-bit machines  
> and one
> for 64-bit machines.  I am having problem starting the TaskTracker  
> that are
> not the same hardware type as the JobTracker.  I get the Incompatible
> buildVesion error because the compile time is part of the  
> buildVersion:
>
> JobTracker's: 0.18.3 from  by httpd on Mon Apr  6 07:35:15 PDT 2009
> TaskTracker's: 0.18.3 from  by httpd on Mon Apr  6 07:34:56 PDT 2009
>
> Any advice on how I can get arournd this problem?  Is there a way to  
> build a
> single version of Hadoop that will run on both 32-bit and 64-bit  
> machines.
> I notice that there are some native libraries under
> $HADOOP_HOME/lib/native/Linux-amd64-64 and
> $HADOOP_HOME/lib/native/Linux-i386-32.  Do I need to compile my own  
> version
> of those libraries since I am applying pathes to the distribution?
>
> I hope I don't have to hack the code to take the compile time out of
> buildVersion.
>
> Thanks in advance for your help.
>
> Bill

Re: problem running on a cluster of mixed hardware due to "Incompatible" buildVersion of JobTracker adn TaskTracker

[Brian Bockelman <bb...@cse.unl.edu>]

Ah yes, there you go ... so much for extrapolating on a Monday :).   
Sorry Bill!

Brian

On Apr 6, 2009, at 6:03 PM, Todd Lipcon wrote:

> On Mon, Apr 6, 2009 at 4:01 PM, Brian Bockelman  
> <bb...@cse.unl.edu>wrote:
>
>> Hey Bill,
>>
>> I might be giving you bad advice (I've only verified this for HDFS
>> components on the 0.19.x branch, not the JT/TT or the 0.18.x  
>> branch), but...
>>
>> In my understanding, Hadoop only compares the base SVN revision  
>> number, not
>> the build strings.  Make sure that both have the SVN rev.  Our build
>> machines don't have svn installed, so we actually have to generate  
>> the
>> correct SVN rev, then submit it to the build clusters.  We  
>> certainly don't
>> have our build time strings match up.
>>
>
> Nope - the JT/TT definitely do verify the entirety of the build  
> string (at
> least in the case when not built from SVN). I saw the same behavior  
> as Bill
> is reporting last week on 0.18.3.
>
> -Todd

Re: problem running on a cluster of mixed hardware due to "Incompatible" buildVersion of JobTracker adn TaskTracker

[Todd Lipcon <to...@cloudera.com>]

On Mon, Apr 6, 2009 at 4:01 PM, Brian Bockelman <bb...@cse.unl.edu>wrote:

> Hey Bill,
>
> I might be giving you bad advice (I've only verified this for HDFS
> components on the 0.19.x branch, not the JT/TT or the 0.18.x branch), but...
>
> In my understanding, Hadoop only compares the base SVN revision number, not
> the build strings.  Make sure that both have the SVN rev.  Our build
> machines don't have svn installed, so we actually have to generate the
> correct SVN rev, then submit it to the build clusters.  We certainly don't
> have our build time strings match up.
>

Nope - the JT/TT definitely do verify the entirety of the build string (at
least in the case when not built from SVN). I saw the same behavior as Bill
is reporting last week on 0.18.3.

-Todd