Master Node in Hadoop Map/Reduce Implementation.

[Fabrizio detto Mario <jf...@gmail.com>]

Hello Hadoop community,
I read about Hadoop framework (
http://hadoop.apache.org/core/docs/r0.16.3/mapred_tutorial.html) this
phrase:
"The Map-Reduce framework consists of a single master JobTracker and one
slave TaskTracker per cluster-node..."

Is The Job Tracker (Master Node) unique and static for my cluster? Is The
Job Tracker the same for each map/reduce run?

If the response is positive, hadoop is different from map/reduce Google
implementation. In Google Implementation the Master node is "one copies of
program" (See MapReduce: Simplified Data Processing on Large Clusters).

Thanks.

-- 
Fabrizio

Re: Master Node in Hadoop Map/Reduce Implementation.

[Amar Kamat <am...@yahoo-inc.com>]

Amar Kamat wrote:
> Fabrizio detto Mario wrote:
>> Hello Hadoop community,
>> I read about Hadoop framework (
>> http://hadoop.apache.org/core/docs/r0.16.3/mapred_tutorial.html) this
>> phrase:
>> "The Map-Reduce framework consists of a single master JobTracker and one
>> slave TaskTracker per cluster-node..."
>>
>> Is The Job Tracker (Master Node) unique and static for my cluster? Is 
>> The
>> Job Tracker the same for each map/reduce run?
>>
>>   
> The job-tracker process controls certain task-trackers processes. 
> These task-trackers can be on same machine or different machines. One 
> task-tracker is controlled by a single job-tracker. You could have 
> multiple map-reduce clusters (each with different job-trackers) on a 
> same physical cluster. So here are the answers
> 1) The job-tracker is unique for a map-reduce cluster (logical) 
> consisting of a set of trackers. There can be multiple such 
> job-trackers each controlling different set of trackers, possibly in 
> the same physical cluster.
> 2) The job-tracker is same for all jobs that run on the same set of 
> trackers.
> It is possible to have multiple task-trackers running simultaneously 
> on a same machine (differentiated by ports) and hence each tracker can 
> be under different job-trackers and hence one machine
"and hence each tracker can be under different job-trackers .."
should be
"and hence each node can be under different job-trackers .."
Amar
> can be under multiple logical clusters.
> Amar
>> If the response is positive, hadoop is different from map/reduce Google
>> implementation. In Google Implementation the Master node is "one 
>> copies of
>> program" (See MapReduce: Simplified Data Processing on Large Clusters).
>>
>> Thanks.
>>
>>   
>

Re: Master Node in Hadoop Map/Reduce Implementation.

[Amar Kamat <am...@yahoo-inc.com>]

Fabrizio detto Mario wrote:
> Hello Hadoop community,
> I read about Hadoop framework (
> http://hadoop.apache.org/core/docs/r0.16.3/mapred_tutorial.html) this
> phrase:
> "The Map-Reduce framework consists of a single master JobTracker and one
> slave TaskTracker per cluster-node..."
>
> Is The Job Tracker (Master Node) unique and static for my cluster? Is The
> Job Tracker the same for each map/reduce run?
>
>   
The job-tracker process controls certain task-trackers processes. These 
task-trackers can be on same machine or different machines. One 
task-tracker is controlled by a single job-tracker. You could have 
multiple map-reduce clusters (each with different job-trackers) on a 
same physical cluster. So here are the answers
1) The job-tracker is unique for a map-reduce cluster (logical) 
consisting of a set of trackers. There can be multiple such job-trackers 
each controlling different set of trackers, possibly in the same 
physical cluster.
2) The job-tracker is same for all jobs that run on the same set of 
trackers.
It is possible to have multiple task-trackers running simultaneously on 
a same machine (differentiated by ports) and hence each tracker can be 
under different job-trackers and hence one machine can be under multiple 
logical clusters.
Amar
> If the response is positive, hadoop is different from map/reduce Google
> implementation. In Google Implementation the Master node is "one copies of
> program" (See MapReduce: Simplified Data Processing on Large Clusters).
>
> Thanks.
>
>