You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Uddipan Mukherjee <Ud...@infosys.com> on 2012/11/01 08:41:47 UTC
Map Reduce slot
Hi Hadoop Gurus.
What is the good way to know following information in my Hadoop cluster.
# number of Map slots
# number of Reducer slots
JVM Heapsize for Map task
JVM heapsize for reduce task
reuse JVM flag
Thanks and Regards
Uddipan
**************** CAUTION - Disclaimer *****************
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely
for the use of the addressee(s). If you are not the intended recipient, please
notify the sender by e-mail and delete the original message. Further, you are not
to copy, disclose, or distribute this e-mail or its contents to any other person and
any such actions are unlawful. This e-mail may contain viruses. Infosys has taken
every reasonable precaution to minimize this risk, but is not liable for any damage
you may sustain as a result of any virus in this e-mail. You should carry out your
own virus checks before opening the e-mail or attachment. Infosys reserves the
right to monitor and review the content of all messages sent to or from this e-mail
address. Messages sent to or from this e-mail address may be stored on the
Infosys e-mail system.
***INFOSYS******** End of Disclaimer ********INFOSYS***
RE: Map Reduce slot
Posted by "Kartashov, Andy" <An...@mpac.ca>.
I checked my MR admin page running on <host>:50030:jobtracker.jsp this is what I am learning.
I run 2-core processor, so the admin page told me that my max map/reduce slot capacity was 14 or so I assume 7 nodes x 2slots.
I did not touch the property of .map.tasks. It seemed that MR set it nicely to 14 automatically, processing 45 of my maps. However, I increased reduce task from the default 1 to 14 in the mapred-site.xml of the JobTracker.
<property>
<name>mapred.reduce.tasks</name>
<value>14</value>
</property>
The job worked really fast.
Running Map TasksRunning Reduce TasksTotal SubmissionsNodesOccupied Map SlotsOccupied Reduce SlotsReserved Map SlotsReserved Reduce SlotsMap Task CapacityReduce Task CapacityAvg. Tasks/NodeBlacklisted NodesExcluded Nodes14017<http://ec2-54-242-185-163.compute-1.amazonaws.com:50030/machines.jsp?type=active>1400014144.000<http://ec2-54-242-185-163.compute-1.amazonaws.com:50030/machines.jsp?type=blacklisted>1<http://ec2-54-242-185-163.compute-1.amazonaws.com:50030/machines.jsp?type=excluded>
From: kapil bhosale [mailto:kapils.07@gmail.com]
Sent: Thursday, November 01, 2012 3:51 AM
To: user@hadoop.apache.org
Subject: Re: Map Reduce slot
Hi Uddipan,
In my Opinion
Hadoop Jobtracker Interface provides every Details Including
# number of Map slots
# number of Reducer slots
and other Infromation can also be found in Log files generated.
Which can also be explored through the same Interface.
Jobtracker address : Masternode:your_port/jobtracker.jsp<http://>
Other useful details can also be found.
-
kapil
On Thu, Nov 1, 2012 at 1:11 PM, Uddipan Mukherjee <Ud...@infosys.com>> wrote:
Hi Hadoop Gurus.
What is the good way to know following information in my Hadoop cluster.
# number of Map slots
# number of Reducer slots
JVM Heapsize for Map task
JVM heapsize for reduce task
reuse JVM flag
Thanks and Regards
Uddipan
**************** CAUTION - Disclaimer *****************
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely
for the use of the addressee(s). If you are not the intended recipient, please
notify the sender by e-mail and delete the original message. Further, you are not
to copy, disclose, or distribute this e-mail or its contents to any other person and
any such actions are unlawful. This e-mail may contain viruses. Infosys has taken
every reasonable precaution to minimize this risk, but is not liable for any damage
you may sustain as a result of any virus in this e-mail. You should carry out your
own virus checks before opening the e-mail or attachment. Infosys reserves the
right to monitor and review the content of all messages sent to or from this e-mail
address. Messages sent to or from this e-mail address may be stored on the
Infosys e-mail system.
***INFOSYS******** End of Disclaimer ********INFOSYS***
--
kap's
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le pr?sent courriel et toute pi?ce jointe qui l'accompagne sont confidentiels, prot?g?s par le droit d'auteur et peuvent ?tre couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autoris?e est interdite. Si vous n'?tes pas le destinataire pr?vu de ce courriel, supprimez-le et contactez imm?diatement l'exp?diteur. Veuillez penser ? l'environnement avant d'imprimer le pr?sent courriel
Re: Map Reduce slot
Posted by Michael Segel <mi...@hotmail.com>.
"However in production clustes the jvm size is marked final to prevent abuses that may lead to OOMs."
Not necessarily.
On Nov 1, 2012, at 6:43 AM, Bejoy Ks <be...@gmail.com> wrote:
> However in production clustes the jvm size is marked final to prevent abuses that may lead to OOMs.
Re: Map Reduce slot
Posted by Michael Segel <mi...@hotmail.com>.
"However in production clustes the jvm size is marked final to prevent abuses that may lead to OOMs."
Not necessarily.
On Nov 1, 2012, at 6:43 AM, Bejoy Ks <be...@gmail.com> wrote:
> However in production clustes the jvm size is marked final to prevent abuses that may lead to OOMs.
Re: Map Reduce slot
Posted by Michael Segel <mi...@hotmail.com>.
"However in production clustes the jvm size is marked final to prevent abuses that may lead to OOMs."
Not necessarily.
On Nov 1, 2012, at 6:43 AM, Bejoy Ks <be...@gmail.com> wrote:
> However in production clustes the jvm size is marked final to prevent abuses that may lead to OOMs.
Re: Map Reduce slot
Posted by Michael Segel <mi...@hotmail.com>.
"However in production clustes the jvm size is marked final to prevent abuses that may lead to OOMs."
Not necessarily.
On Nov 1, 2012, at 6:43 AM, Bejoy Ks <be...@gmail.com> wrote:
> However in production clustes the jvm size is marked final to prevent abuses that may lead to OOMs.
Re: Map Reduce slot
Posted by Bejoy Ks <be...@gmail.com>.
Hi Udippan
The number of map/reduce tasks slots on your cluster is the sum of all the
slots from your TaskTracker nodes. Based on your node's resource
availability you can even configure that on a per node basis. The slots are
defined on a node level using the following properties
mapred.tasktracker.map.tasks.maximum and
mapred.tasktracker.reduce.tasks.maximum .
As Kapil mentioned the total number of slots across the cluster can be
obtained from JT web UI.
The other 3 properties can be defined on a job level. However in production
clustes the jvm size is marked final to prevent abuses that may lead to
OOMs.
The jvm size of task jvms are defined by 'mapred.child.java.opts' which
defaults to 200 Mb and the jvm reuse is defined by
'mapred.job.reuse.jvm.num.tasks' which defaults to 1 task per jvm.
HTH.
Regards
Bejoy KS
Re: Map Reduce slot
Posted by Bejoy Ks <be...@gmail.com>.
Hi Udippan
The number of map/reduce tasks slots on your cluster is the sum of all the
slots from your TaskTracker nodes. Based on your node's resource
availability you can even configure that on a per node basis. The slots are
defined on a node level using the following properties
mapred.tasktracker.map.tasks.maximum and
mapred.tasktracker.reduce.tasks.maximum .
As Kapil mentioned the total number of slots across the cluster can be
obtained from JT web UI.
The other 3 properties can be defined on a job level. However in production
clustes the jvm size is marked final to prevent abuses that may lead to
OOMs.
The jvm size of task jvms are defined by 'mapred.child.java.opts' which
defaults to 200 Mb and the jvm reuse is defined by
'mapred.job.reuse.jvm.num.tasks' which defaults to 1 task per jvm.
HTH.
Regards
Bejoy KS
RE: Map Reduce slot
Posted by "Kartashov, Andy" <An...@mpac.ca>.
I checked my MR admin page running on <host>:50030:jobtracker.jsp this is what I am learning.
I run 2-core processor, so the admin page told me that my max map/reduce slot capacity was 14 or so I assume 7 nodes x 2slots.
I did not touch the property of .map.tasks. It seemed that MR set it nicely to 14 automatically, processing 45 of my maps. However, I increased reduce task from the default 1 to 14 in the mapred-site.xml of the JobTracker.
<property>
<name>mapred.reduce.tasks</name>
<value>14</value>
</property>
The job worked really fast.
Running Map TasksRunning Reduce TasksTotal SubmissionsNodesOccupied Map SlotsOccupied Reduce SlotsReserved Map SlotsReserved Reduce SlotsMap Task CapacityReduce Task CapacityAvg. Tasks/NodeBlacklisted NodesExcluded Nodes14017<http://ec2-54-242-185-163.compute-1.amazonaws.com:50030/machines.jsp?type=active>1400014144.000<http://ec2-54-242-185-163.compute-1.amazonaws.com:50030/machines.jsp?type=blacklisted>1<http://ec2-54-242-185-163.compute-1.amazonaws.com:50030/machines.jsp?type=excluded>
From: kapil bhosale [mailto:kapils.07@gmail.com]
Sent: Thursday, November 01, 2012 3:51 AM
To: user@hadoop.apache.org
Subject: Re: Map Reduce slot
Hi Uddipan,
In my Opinion
Hadoop Jobtracker Interface provides every Details Including
# number of Map slots
# number of Reducer slots
and other Infromation can also be found in Log files generated.
Which can also be explored through the same Interface.
Jobtracker address : Masternode:your_port/jobtracker.jsp<http://>
Other useful details can also be found.
-
kapil
On Thu, Nov 1, 2012 at 1:11 PM, Uddipan Mukherjee <Ud...@infosys.com>> wrote:
Hi Hadoop Gurus.
What is the good way to know following information in my Hadoop cluster.
# number of Map slots
# number of Reducer slots
JVM Heapsize for Map task
JVM heapsize for reduce task
reuse JVM flag
Thanks and Regards
Uddipan
**************** CAUTION - Disclaimer *****************
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely
for the use of the addressee(s). If you are not the intended recipient, please
notify the sender by e-mail and delete the original message. Further, you are not
to copy, disclose, or distribute this e-mail or its contents to any other person and
any such actions are unlawful. This e-mail may contain viruses. Infosys has taken
every reasonable precaution to minimize this risk, but is not liable for any damage
you may sustain as a result of any virus in this e-mail. You should carry out your
own virus checks before opening the e-mail or attachment. Infosys reserves the
right to monitor and review the content of all messages sent to or from this e-mail
address. Messages sent to or from this e-mail address may be stored on the
Infosys e-mail system.
***INFOSYS******** End of Disclaimer ********INFOSYS***
--
kap's
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le pr?sent courriel et toute pi?ce jointe qui l'accompagne sont confidentiels, prot?g?s par le droit d'auteur et peuvent ?tre couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autoris?e est interdite. Si vous n'?tes pas le destinataire pr?vu de ce courriel, supprimez-le et contactez imm?diatement l'exp?diteur. Veuillez penser ? l'environnement avant d'imprimer le pr?sent courriel
RE: Map Reduce slot
Posted by "Kartashov, Andy" <An...@mpac.ca>.
I checked my MR admin page running on <host>:50030:jobtracker.jsp this is what I am learning.
I run 2-core processor, so the admin page told me that my max map/reduce slot capacity was 14 or so I assume 7 nodes x 2slots.
I did not touch the property of .map.tasks. It seemed that MR set it nicely to 14 automatically, processing 45 of my maps. However, I increased reduce task from the default 1 to 14 in the mapred-site.xml of the JobTracker.
<property>
<name>mapred.reduce.tasks</name>
<value>14</value>
</property>
The job worked really fast.
Running Map TasksRunning Reduce TasksTotal SubmissionsNodesOccupied Map SlotsOccupied Reduce SlotsReserved Map SlotsReserved Reduce SlotsMap Task CapacityReduce Task CapacityAvg. Tasks/NodeBlacklisted NodesExcluded Nodes14017<http://ec2-54-242-185-163.compute-1.amazonaws.com:50030/machines.jsp?type=active>1400014144.000<http://ec2-54-242-185-163.compute-1.amazonaws.com:50030/machines.jsp?type=blacklisted>1<http://ec2-54-242-185-163.compute-1.amazonaws.com:50030/machines.jsp?type=excluded>
From: kapil bhosale [mailto:kapils.07@gmail.com]
Sent: Thursday, November 01, 2012 3:51 AM
To: user@hadoop.apache.org
Subject: Re: Map Reduce slot
Hi Uddipan,
In my Opinion
Hadoop Jobtracker Interface provides every Details Including
# number of Map slots
# number of Reducer slots
and other Infromation can also be found in Log files generated.
Which can also be explored through the same Interface.
Jobtracker address : Masternode:your_port/jobtracker.jsp<http://>
Other useful details can also be found.
-
kapil
On Thu, Nov 1, 2012 at 1:11 PM, Uddipan Mukherjee <Ud...@infosys.com>> wrote:
Hi Hadoop Gurus.
What is the good way to know following information in my Hadoop cluster.
# number of Map slots
# number of Reducer slots
JVM Heapsize for Map task
JVM heapsize for reduce task
reuse JVM flag
Thanks and Regards
Uddipan
**************** CAUTION - Disclaimer *****************
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely
for the use of the addressee(s). If you are not the intended recipient, please
notify the sender by e-mail and delete the original message. Further, you are not
to copy, disclose, or distribute this e-mail or its contents to any other person and
any such actions are unlawful. This e-mail may contain viruses. Infosys has taken
every reasonable precaution to minimize this risk, but is not liable for any damage
you may sustain as a result of any virus in this e-mail. You should carry out your
own virus checks before opening the e-mail or attachment. Infosys reserves the
right to monitor and review the content of all messages sent to or from this e-mail
address. Messages sent to or from this e-mail address may be stored on the
Infosys e-mail system.
***INFOSYS******** End of Disclaimer ********INFOSYS***
--
kap's
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le pr?sent courriel et toute pi?ce jointe qui l'accompagne sont confidentiels, prot?g?s par le droit d'auteur et peuvent ?tre couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autoris?e est interdite. Si vous n'?tes pas le destinataire pr?vu de ce courriel, supprimez-le et contactez imm?diatement l'exp?diteur. Veuillez penser ? l'environnement avant d'imprimer le pr?sent courriel
RE: Map Reduce slot
Posted by "Kartashov, Andy" <An...@mpac.ca>.
I checked my MR admin page running on <host>:50030:jobtracker.jsp this is what I am learning.
I run 2-core processor, so the admin page told me that my max map/reduce slot capacity was 14 or so I assume 7 nodes x 2slots.
I did not touch the property of .map.tasks. It seemed that MR set it nicely to 14 automatically, processing 45 of my maps. However, I increased reduce task from the default 1 to 14 in the mapred-site.xml of the JobTracker.
<property>
<name>mapred.reduce.tasks</name>
<value>14</value>
</property>
The job worked really fast.
Running Map TasksRunning Reduce TasksTotal SubmissionsNodesOccupied Map SlotsOccupied Reduce SlotsReserved Map SlotsReserved Reduce SlotsMap Task CapacityReduce Task CapacityAvg. Tasks/NodeBlacklisted NodesExcluded Nodes14017<http://ec2-54-242-185-163.compute-1.amazonaws.com:50030/machines.jsp?type=active>1400014144.000<http://ec2-54-242-185-163.compute-1.amazonaws.com:50030/machines.jsp?type=blacklisted>1<http://ec2-54-242-185-163.compute-1.amazonaws.com:50030/machines.jsp?type=excluded>
From: kapil bhosale [mailto:kapils.07@gmail.com]
Sent: Thursday, November 01, 2012 3:51 AM
To: user@hadoop.apache.org
Subject: Re: Map Reduce slot
Hi Uddipan,
In my Opinion
Hadoop Jobtracker Interface provides every Details Including
# number of Map slots
# number of Reducer slots
and other Infromation can also be found in Log files generated.
Which can also be explored through the same Interface.
Jobtracker address : Masternode:your_port/jobtracker.jsp<http://>
Other useful details can also be found.
-
kapil
On Thu, Nov 1, 2012 at 1:11 PM, Uddipan Mukherjee <Ud...@infosys.com>> wrote:
Hi Hadoop Gurus.
What is the good way to know following information in my Hadoop cluster.
# number of Map slots
# number of Reducer slots
JVM Heapsize for Map task
JVM heapsize for reduce task
reuse JVM flag
Thanks and Regards
Uddipan
**************** CAUTION - Disclaimer *****************
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely
for the use of the addressee(s). If you are not the intended recipient, please
notify the sender by e-mail and delete the original message. Further, you are not
to copy, disclose, or distribute this e-mail or its contents to any other person and
any such actions are unlawful. This e-mail may contain viruses. Infosys has taken
every reasonable precaution to minimize this risk, but is not liable for any damage
you may sustain as a result of any virus in this e-mail. You should carry out your
own virus checks before opening the e-mail or attachment. Infosys reserves the
right to monitor and review the content of all messages sent to or from this e-mail
address. Messages sent to or from this e-mail address may be stored on the
Infosys e-mail system.
***INFOSYS******** End of Disclaimer ********INFOSYS***
--
kap's
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le pr?sent courriel et toute pi?ce jointe qui l'accompagne sont confidentiels, prot?g?s par le droit d'auteur et peuvent ?tre couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autoris?e est interdite. Si vous n'?tes pas le destinataire pr?vu de ce courriel, supprimez-le et contactez imm?diatement l'exp?diteur. Veuillez penser ? l'environnement avant d'imprimer le pr?sent courriel
Re: Map Reduce slot
Posted by Bejoy Ks <be...@gmail.com>.
Hi Udippan
The number of map/reduce tasks slots on your cluster is the sum of all the
slots from your TaskTracker nodes. Based on your node's resource
availability you can even configure that on a per node basis. The slots are
defined on a node level using the following properties
mapred.tasktracker.map.tasks.maximum and
mapred.tasktracker.reduce.tasks.maximum .
As Kapil mentioned the total number of slots across the cluster can be
obtained from JT web UI.
The other 3 properties can be defined on a job level. However in production
clustes the jvm size is marked final to prevent abuses that may lead to
OOMs.
The jvm size of task jvms are defined by 'mapred.child.java.opts' which
defaults to 200 Mb and the jvm reuse is defined by
'mapred.job.reuse.jvm.num.tasks' which defaults to 1 task per jvm.
HTH.
Regards
Bejoy KS
Re: Map Reduce slot
Posted by Bejoy Ks <be...@gmail.com>.
Hi Udippan
The number of map/reduce tasks slots on your cluster is the sum of all the
slots from your TaskTracker nodes. Based on your node's resource
availability you can even configure that on a per node basis. The slots are
defined on a node level using the following properties
mapred.tasktracker.map.tasks.maximum and
mapred.tasktracker.reduce.tasks.maximum .
As Kapil mentioned the total number of slots across the cluster can be
obtained from JT web UI.
The other 3 properties can be defined on a job level. However in production
clustes the jvm size is marked final to prevent abuses that may lead to
OOMs.
The jvm size of task jvms are defined by 'mapred.child.java.opts' which
defaults to 200 Mb and the jvm reuse is defined by
'mapred.job.reuse.jvm.num.tasks' which defaults to 1 task per jvm.
HTH.
Regards
Bejoy KS
Re: Map Reduce slot
Posted by kapil bhosale <ka...@gmail.com>.
Hi Uddipan,
In my Opinion
Hadoop Jobtracker Interface provides every Details Including
# number of Map slots
# number of Reducer slots
and other Infromation can also be found in Log files generated.
Which can also be explored through the same Interface.
Jobtracker address : Masternode:your_port/jobtracker.jsp <http://#>
Other useful details can also be found.
-
kapil
On Thu, Nov 1, 2012 at 1:11 PM, Uddipan Mukherjee <
Uddipan_Mukherjee@infosys.com> wrote:
> Hi Hadoop Gurus.
>
> What is the good way to know following information in my Hadoop cluster.
>
> # number of Map slots
> # number of Reducer slots
> JVM Heapsize for Map task
> JVM heapsize for reduce task
> reuse JVM flag
>
> Thanks and Regards
> Uddipan
>
> **************** CAUTION - Disclaimer *****************
> This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended
> solely
> for the use of the addressee(s). If you are not the intended recipient,
> please
> notify the sender by e-mail and delete the original message. Further, you
> are not
> to copy, disclose, or distribute this e-mail or its contents to any other
> person and
> any such actions are unlawful. This e-mail may contain viruses. Infosys
> has taken
> every reasonable precaution to minimize this risk, but is not liable for
> any damage
> you may sustain as a result of any virus in this e-mail. You should carry
> out your
> own virus checks before opening the e-mail or attachment. Infosys reserves
> the
> right to monitor and review the content of all messages sent to or from
> this e-mail
> address. Messages sent to or from this e-mail address may be stored on the
> Infosys e-mail system.
> ***INFOSYS******** End of Disclaimer ********INFOSYS***
>
--
kap's
Re: Map Reduce slot
Posted by kapil bhosale <ka...@gmail.com>.
Hi Uddipan,
In my Opinion
Hadoop Jobtracker Interface provides every Details Including
# number of Map slots
# number of Reducer slots
and other Infromation can also be found in Log files generated.
Which can also be explored through the same Interface.
Jobtracker address : Masternode:your_port/jobtracker.jsp <http://#>
Other useful details can also be found.
-
kapil
On Thu, Nov 1, 2012 at 1:11 PM, Uddipan Mukherjee <
Uddipan_Mukherjee@infosys.com> wrote:
> Hi Hadoop Gurus.
>
> What is the good way to know following information in my Hadoop cluster.
>
> # number of Map slots
> # number of Reducer slots
> JVM Heapsize for Map task
> JVM heapsize for reduce task
> reuse JVM flag
>
> Thanks and Regards
> Uddipan
>
> **************** CAUTION - Disclaimer *****************
> This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended
> solely
> for the use of the addressee(s). If you are not the intended recipient,
> please
> notify the sender by e-mail and delete the original message. Further, you
> are not
> to copy, disclose, or distribute this e-mail or its contents to any other
> person and
> any such actions are unlawful. This e-mail may contain viruses. Infosys
> has taken
> every reasonable precaution to minimize this risk, but is not liable for
> any damage
> you may sustain as a result of any virus in this e-mail. You should carry
> out your
> own virus checks before opening the e-mail or attachment. Infosys reserves
> the
> right to monitor and review the content of all messages sent to or from
> this e-mail
> address. Messages sent to or from this e-mail address may be stored on the
> Infosys e-mail system.
> ***INFOSYS******** End of Disclaimer ********INFOSYS***
>
--
kap's
Re: Map Reduce slot
Posted by kapil bhosale <ka...@gmail.com>.
Hi Uddipan,
In my Opinion
Hadoop Jobtracker Interface provides every Details Including
# number of Map slots
# number of Reducer slots
and other Infromation can also be found in Log files generated.
Which can also be explored through the same Interface.
Jobtracker address : Masternode:your_port/jobtracker.jsp <http://#>
Other useful details can also be found.
-
kapil
On Thu, Nov 1, 2012 at 1:11 PM, Uddipan Mukherjee <
Uddipan_Mukherjee@infosys.com> wrote:
> Hi Hadoop Gurus.
>
> What is the good way to know following information in my Hadoop cluster.
>
> # number of Map slots
> # number of Reducer slots
> JVM Heapsize for Map task
> JVM heapsize for reduce task
> reuse JVM flag
>
> Thanks and Regards
> Uddipan
>
> **************** CAUTION - Disclaimer *****************
> This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended
> solely
> for the use of the addressee(s). If you are not the intended recipient,
> please
> notify the sender by e-mail and delete the original message. Further, you
> are not
> to copy, disclose, or distribute this e-mail or its contents to any other
> person and
> any such actions are unlawful. This e-mail may contain viruses. Infosys
> has taken
> every reasonable precaution to minimize this risk, but is not liable for
> any damage
> you may sustain as a result of any virus in this e-mail. You should carry
> out your
> own virus checks before opening the e-mail or attachment. Infosys reserves
> the
> right to monitor and review the content of all messages sent to or from
> this e-mail
> address. Messages sent to or from this e-mail address may be stored on the
> Infosys e-mail system.
> ***INFOSYS******** End of Disclaimer ********INFOSYS***
>
--
kap's
Re: Map Reduce slot
Posted by kapil bhosale <ka...@gmail.com>.
Hi Uddipan,
In my Opinion
Hadoop Jobtracker Interface provides every Details Including
# number of Map slots
# number of Reducer slots
and other Infromation can also be found in Log files generated.
Which can also be explored through the same Interface.
Jobtracker address : Masternode:your_port/jobtracker.jsp <http://#>
Other useful details can also be found.
-
kapil
On Thu, Nov 1, 2012 at 1:11 PM, Uddipan Mukherjee <
Uddipan_Mukherjee@infosys.com> wrote:
> Hi Hadoop Gurus.
>
> What is the good way to know following information in my Hadoop cluster.
>
> # number of Map slots
> # number of Reducer slots
> JVM Heapsize for Map task
> JVM heapsize for reduce task
> reuse JVM flag
>
> Thanks and Regards
> Uddipan
>
> **************** CAUTION - Disclaimer *****************
> This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended
> solely
> for the use of the addressee(s). If you are not the intended recipient,
> please
> notify the sender by e-mail and delete the original message. Further, you
> are not
> to copy, disclose, or distribute this e-mail or its contents to any other
> person and
> any such actions are unlawful. This e-mail may contain viruses. Infosys
> has taken
> every reasonable precaution to minimize this risk, but is not liable for
> any damage
> you may sustain as a result of any virus in this e-mail. You should carry
> out your
> own virus checks before opening the e-mail or attachment. Infosys reserves
> the
> right to monitor and review the content of all messages sent to or from
> this e-mail
> address. Messages sent to or from this e-mail address may be stored on the
> Infosys e-mail system.
> ***INFOSYS******** End of Disclaimer ********INFOSYS***
>
--
kap's