You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Rajesh Jain <rj...@gmail.com> on 2013/08/20 05:02:29 UTC
YARN Container JVMs
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html
In this architecture -- The NodeManager is the per-machine framework agent
who is responsible for containers.
Questions:
Are these container which execute the Map and Reduce Jobs, separate JVM's?
What are the name of the this container JVM's and how can I control the
Java properties for this container JVM's?
Does mapred.reduce.java.opts/mapred.map.java.opts control the JVM
properties?
I am not seeing seperate JVM's being started by the NodeManager when I
execute the WordCount program.
Thanks,
Rajesh
Re: YARN Container JVMs
Posted by Harsh J <ha...@cloudera.com>.
If you use MR, then the options you mention are valid for controlling
the MRv2 app spawned task JVMs.
If you're not seeing any JVMs being spawned across the cluster when
you run an MRv2 job, chances are you're running jobs via the
LocalJobRunner accidentally instead.
On Tue, Aug 20, 2013 at 8:32 AM, Rajesh Jain <rj...@gmail.com> wrote:
> http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html
>
> In this architecture -- The NodeManager is the per-machine framework agent
> who is responsible for containers.
>
> Questions:
>
> Are these container which execute the Map and Reduce Jobs, separate JVM's?
>
> What are the name of the this container JVM's and how can I control the Java
> properties for this container JVM's?
>
> Does mapred.reduce.java.opts/mapred.map.java.opts control the JVM
> properties?
>
> I am not seeing seperate JVM's being started by the NodeManager when I
> execute the WordCount program.
>
> Thanks,
> Rajesh
>
>
>
--
Harsh J
Re: YARN Container JVMs
Posted by Harsh J <ha...@cloudera.com>.
If you use MR, then the options you mention are valid for controlling
the MRv2 app spawned task JVMs.
If you're not seeing any JVMs being spawned across the cluster when
you run an MRv2 job, chances are you're running jobs via the
LocalJobRunner accidentally instead.
On Tue, Aug 20, 2013 at 8:32 AM, Rajesh Jain <rj...@gmail.com> wrote:
> http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html
>
> In this architecture -- The NodeManager is the per-machine framework agent
> who is responsible for containers.
>
> Questions:
>
> Are these container which execute the Map and Reduce Jobs, separate JVM's?
>
> What are the name of the this container JVM's and how can I control the Java
> properties for this container JVM's?
>
> Does mapred.reduce.java.opts/mapred.map.java.opts control the JVM
> properties?
>
> I am not seeing seperate JVM's being started by the NodeManager when I
> execute the WordCount program.
>
> Thanks,
> Rajesh
>
>
>
--
Harsh J
Re: YARN Container JVMs
Posted by Harsh J <ha...@cloudera.com>.
If you use MR, then the options you mention are valid for controlling
the MRv2 app spawned task JVMs.
If you're not seeing any JVMs being spawned across the cluster when
you run an MRv2 job, chances are you're running jobs via the
LocalJobRunner accidentally instead.
On Tue, Aug 20, 2013 at 8:32 AM, Rajesh Jain <rj...@gmail.com> wrote:
> http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html
>
> In this architecture -- The NodeManager is the per-machine framework agent
> who is responsible for containers.
>
> Questions:
>
> Are these container which execute the Map and Reduce Jobs, separate JVM's?
>
> What are the name of the this container JVM's and how can I control the Java
> properties for this container JVM's?
>
> Does mapred.reduce.java.opts/mapred.map.java.opts control the JVM
> properties?
>
> I am not seeing seperate JVM's being started by the NodeManager when I
> execute the WordCount program.
>
> Thanks,
> Rajesh
>
>
>
--
Harsh J
Re: YARN Container JVMs
Posted by Harsh J <ha...@cloudera.com>.
If you use MR, then the options you mention are valid for controlling
the MRv2 app spawned task JVMs.
If you're not seeing any JVMs being spawned across the cluster when
you run an MRv2 job, chances are you're running jobs via the
LocalJobRunner accidentally instead.
On Tue, Aug 20, 2013 at 8:32 AM, Rajesh Jain <rj...@gmail.com> wrote:
> http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html
>
> In this architecture -- The NodeManager is the per-machine framework agent
> who is responsible for containers.
>
> Questions:
>
> Are these container which execute the Map and Reduce Jobs, separate JVM's?
>
> What are the name of the this container JVM's and how can I control the Java
> properties for this container JVM's?
>
> Does mapred.reduce.java.opts/mapred.map.java.opts control the JVM
> properties?
>
> I am not seeing seperate JVM's being started by the NodeManager when I
> execute the WordCount program.
>
> Thanks,
> Rajesh
>
>
>
--
Harsh J