You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by bikash sharma <sh...@gmail.com> on 2011/03/25 15:10:26 UTC
observe the effect of changes to Hadoop
Hi,
For my research project, I need to add a couple of functions in
JobTracker.java source file to include additional information about
TaskTrackers resource usage through heartbeat messages. I made those changes
to JobTracker.java file. However, I am not very clear how to see these
effects. I mean what are the next steps in terms of building the entire
Hadoop code base, using the built distribution and installing it again in
the cluster, etc?
Any elaborate updates on these will be very useful since I do not have much
experience in doing modifications to Hadoop like huge code base and
observing the effects of these changes.
Thanks,
Bikash
Re: observe the effect of changes to Hadoop
Posted by bikash sharma <sh...@gmail.com>.
Thanks Steve. It worked.
On Sun, Mar 27, 2011 at 2:08 PM, Steve Loughran <st...@apache.org> wrote:
> On 25/03/2011 14:10, bikash sharma wrote:
>
>> Hi,
>> For my research project, I need to add a couple of functions in
>> JobTracker.java source file to include additional information about
>> TaskTrackers resource usage through heartbeat messages. I made those
>> changes
>> to JobTracker.java file. However, I am not very clear how to see these
>> effects. I mean what are the next steps in terms of building the entire
>> Hadoop code base, using the built distribution and installing it again in
>> the cluster, etc?
>>
>
> If you are working with the Job Tracker you only need to rebuild the
> mapreduce JARs and push the new JAR out to the Job Tracker server, restart
> that process.
>
> For more safety, put the same JAR on all the task trackers and shut down
> HDFS before the updates, but that's potentially overkil
>
>
> Any elaborate updates on these will be very useful since I do not have
>> much
>> experience in doing modifications to Hadoop like huge code base and
>> observing the effects of these changes.
>>
>
> I'd recommend getting everything working on a local machine single VM (the
> MiniMRCluster class helps), then move to multiple VMs and finally, if the
> code looks good, a real cluster with data you don't value.
>
> -stee
>
Re: observe the effect of changes to Hadoop
Posted by Steve Loughran <st...@apache.org>.
On 25/03/2011 14:10, bikash sharma wrote:
> Hi,
> For my research project, I need to add a couple of functions in
> JobTracker.java source file to include additional information about
> TaskTrackers resource usage through heartbeat messages. I made those changes
> to JobTracker.java file. However, I am not very clear how to see these
> effects. I mean what are the next steps in terms of building the entire
> Hadoop code base, using the built distribution and installing it again in
> the cluster, etc?
If you are working with the Job Tracker you only need to rebuild the
mapreduce JARs and push the new JAR out to the Job Tracker server,
restart that process.
For more safety, put the same JAR on all the task trackers and shut down
HDFS before the updates, but that's potentially overkil
> Any elaborate updates on these will be very useful since I do not have much
> experience in doing modifications to Hadoop like huge code base and
> observing the effects of these changes.
I'd recommend getting everything working on a local machine single VM
(the MiniMRCluster class helps), then move to multiple VMs and finally,
if the code looks good, a real cluster with data you don't value.
-stee