You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Andy Srine <an...@gmail.com> on 2014/04/18 03:35:52 UTC
JVM option
Quick question. How would I pass the following JVM option to the Hadoop
command line?
"-Xmx4G"
hadoop jar <jar file> <class_name> <args>
Thanks,
Andy
Re: JVM option
Posted by Bjorn Jonsson <bj...@gmail.com>.
Hi Andy,
This is a client side property that you can set in hadoop-env.sh
(/etc/hadoop/conf). The hadoop script (/bin/hadoop)
itself, which is a bash shell script, contains code like the following.
if [ "$HADOOP_HEAPSIZE" != "" ]; then
#echo "run with heapsize $HADOOP_HEAPSIZE"
JAVA_HEAP_MAX="-Xmx""$HADOOP_HEAPSIZE""m"
#echo $JAVA_HEAP_MAX
fi
Use hadoop-env.sh.
Best,
Bjorn
On Fri, Apr 18, 2014 at 7:20 PM, Andy Srine <an...@gmail.com> wrote:
> I guess then the question is, how do I set the (system?) property and what
> property exactly is this?
>
> "export HADOOP_HEAPSIZE=4096" - Works from the environment.
>
> And none of the following seem to work from the command line.
> -Xmx4G
>
> -Dmapred.child.java.opts=-Xmx4G
>
> -DHADOOP_HEAPSIZE=4096
>
> -DHADOOP_OPTS=-Xmx4G
>
>
> I tried this from Java code as well and no luck.
>
> System.setProperty("HADOOP_HEAPSIZE", "4096");
>
>
> What am I missing?
>
>
> Thanks,
>
> Andy
>
>
>
>
> On Fri, Apr 18, 2014 at 11:28 AM, Shahab Yunus <sh...@gmail.com>wrote:
>
>> You can pass hadoop conf properties through the -D option. Have you seen
>> this?
>>
>> http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
>>
>> This is not for system properties. The assumption is that you want to
>> specify hadoop conf property through the command line.
>>
>> Regards,
>> Shahab
>>
>>
>> On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <an...@gmail.com> wrote:
>>
>>> Thanks for the tip. May be it's a different setting I need. Let me
>>> explain the problem.
>>>
>>> I get an heap error on the command line - "Exception in thread "main"
>>> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
>>> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
>>> command line (hadoop jar ...)? Next choice would be to do it
>>> programmatically in Java using hadoop conf. I can't really change the XML
>>> files across the cluster.
>>>
>>> Regards,
>>> Andy
>>>
>>>
>>> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
>>>
>>>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>>>> "mapred.child.java.opts" in the mapred-site.xml
>>>>
>>>>
>>>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com>wrote:
>>>>
>>>>> Quick question. How would I pass the following JVM option to the
>>>>> Hadoop command line?
>>>>>
>>>>> "-Xmx4G"
>>>>>
>>>>> hadoop jar <jar file> <class_name> <args>
>>>>>
>>>>> Thanks,
>>>>> Andy
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Andy
>>>
>>>
>>
>
>
> --
> Thanks,
> Andy
>
>
Re: JVM option
Posted by Shahab Yunus <sh...@gmail.com>.
HADOOP_HEAPSIZE and HADOOP_OPTS are not hadoop properties. These seem to
be environment variables of the shell script.
Try this with a space between -D and the property name (
-Dmapred.child.java.opts)
See the properties available (for mrv1) here:
http://hadoop.apache.org/docs/r1.0.4/mapred-default.html
or this:
http://hadoop.apache.org/docs/r2.3.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
As for passing system properties, the option that I know of (others can
chime in if they know something better) is to pass the hadoop property and
then in the java code, set the system property form the configuration
object.
Regards,
Shahab
On Fri, Apr 18, 2014 at 3:20 PM, Andy Srine <an...@gmail.com> wrote:
> I guess then the question is, how do I set the (system?) property and what
> property exactly is this?
>
> "export HADOOP_HEAPSIZE=4096" - Works from the environment.
>
> And none of the following seem to work from the command line.
> -Xmx4G
>
> -Dmapred.child.java.opts=-Xmx4G
>
> -DHADOOP_HEAPSIZE=4096
>
> -DHADOOP_OPTS=-Xmx4G
>
>
> I tried this from Java code as well and no luck.
>
> System.setProperty("HADOOP_HEAPSIZE", "4096");
>
>
> What am I missing?
>
>
> Thanks,
>
> Andy
>
>
>
>
> On Fri, Apr 18, 2014 at 11:28 AM, Shahab Yunus <sh...@gmail.com>wrote:
>
>> You can pass hadoop conf properties through the -D option. Have you seen
>> this?
>>
>> http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
>>
>> This is not for system properties. The assumption is that you want to
>> specify hadoop conf property through the command line.
>>
>> Regards,
>> Shahab
>>
>>
>> On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <an...@gmail.com> wrote:
>>
>>> Thanks for the tip. May be it's a different setting I need. Let me
>>> explain the problem.
>>>
>>> I get an heap error on the command line - "Exception in thread "main"
>>> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
>>> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
>>> command line (hadoop jar ...)? Next choice would be to do it
>>> programmatically in Java using hadoop conf. I can't really change the XML
>>> files across the cluster.
>>>
>>> Regards,
>>> Andy
>>>
>>>
>>> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
>>>
>>>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>>>> "mapred.child.java.opts" in the mapred-site.xml
>>>>
>>>>
>>>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com>wrote:
>>>>
>>>>> Quick question. How would I pass the following JVM option to the
>>>>> Hadoop command line?
>>>>>
>>>>> "-Xmx4G"
>>>>>
>>>>> hadoop jar <jar file> <class_name> <args>
>>>>>
>>>>> Thanks,
>>>>> Andy
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Andy
>>>
>>>
>>
>
>
> --
> Thanks,
> Andy
>
>
Re: JVM option
Posted by Bjorn Jonsson <bj...@gmail.com>.
Hi Andy,
This is a client side property that you can set in hadoop-env.sh
(/etc/hadoop/conf). The hadoop script (/bin/hadoop)
itself, which is a bash shell script, contains code like the following.
if [ "$HADOOP_HEAPSIZE" != "" ]; then
#echo "run with heapsize $HADOOP_HEAPSIZE"
JAVA_HEAP_MAX="-Xmx""$HADOOP_HEAPSIZE""m"
#echo $JAVA_HEAP_MAX
fi
Use hadoop-env.sh.
Best,
Bjorn
On Fri, Apr 18, 2014 at 7:20 PM, Andy Srine <an...@gmail.com> wrote:
> I guess then the question is, how do I set the (system?) property and what
> property exactly is this?
>
> "export HADOOP_HEAPSIZE=4096" - Works from the environment.
>
> And none of the following seem to work from the command line.
> -Xmx4G
>
> -Dmapred.child.java.opts=-Xmx4G
>
> -DHADOOP_HEAPSIZE=4096
>
> -DHADOOP_OPTS=-Xmx4G
>
>
> I tried this from Java code as well and no luck.
>
> System.setProperty("HADOOP_HEAPSIZE", "4096");
>
>
> What am I missing?
>
>
> Thanks,
>
> Andy
>
>
>
>
> On Fri, Apr 18, 2014 at 11:28 AM, Shahab Yunus <sh...@gmail.com>wrote:
>
>> You can pass hadoop conf properties through the -D option. Have you seen
>> this?
>>
>> http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
>>
>> This is not for system properties. The assumption is that you want to
>> specify hadoop conf property through the command line.
>>
>> Regards,
>> Shahab
>>
>>
>> On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <an...@gmail.com> wrote:
>>
>>> Thanks for the tip. May be it's a different setting I need. Let me
>>> explain the problem.
>>>
>>> I get an heap error on the command line - "Exception in thread "main"
>>> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
>>> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
>>> command line (hadoop jar ...)? Next choice would be to do it
>>> programmatically in Java using hadoop conf. I can't really change the XML
>>> files across the cluster.
>>>
>>> Regards,
>>> Andy
>>>
>>>
>>> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
>>>
>>>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>>>> "mapred.child.java.opts" in the mapred-site.xml
>>>>
>>>>
>>>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com>wrote:
>>>>
>>>>> Quick question. How would I pass the following JVM option to the
>>>>> Hadoop command line?
>>>>>
>>>>> "-Xmx4G"
>>>>>
>>>>> hadoop jar <jar file> <class_name> <args>
>>>>>
>>>>> Thanks,
>>>>> Andy
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Andy
>>>
>>>
>>
>
>
> --
> Thanks,
> Andy
>
>
Re: JVM option
Posted by Shahab Yunus <sh...@gmail.com>.
HADOOP_HEAPSIZE and HADOOP_OPTS are not hadoop properties. These seem to
be environment variables of the shell script.
Try this with a space between -D and the property name (
-Dmapred.child.java.opts)
See the properties available (for mrv1) here:
http://hadoop.apache.org/docs/r1.0.4/mapred-default.html
or this:
http://hadoop.apache.org/docs/r2.3.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
As for passing system properties, the option that I know of (others can
chime in if they know something better) is to pass the hadoop property and
then in the java code, set the system property form the configuration
object.
Regards,
Shahab
On Fri, Apr 18, 2014 at 3:20 PM, Andy Srine <an...@gmail.com> wrote:
> I guess then the question is, how do I set the (system?) property and what
> property exactly is this?
>
> "export HADOOP_HEAPSIZE=4096" - Works from the environment.
>
> And none of the following seem to work from the command line.
> -Xmx4G
>
> -Dmapred.child.java.opts=-Xmx4G
>
> -DHADOOP_HEAPSIZE=4096
>
> -DHADOOP_OPTS=-Xmx4G
>
>
> I tried this from Java code as well and no luck.
>
> System.setProperty("HADOOP_HEAPSIZE", "4096");
>
>
> What am I missing?
>
>
> Thanks,
>
> Andy
>
>
>
>
> On Fri, Apr 18, 2014 at 11:28 AM, Shahab Yunus <sh...@gmail.com>wrote:
>
>> You can pass hadoop conf properties through the -D option. Have you seen
>> this?
>>
>> http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
>>
>> This is not for system properties. The assumption is that you want to
>> specify hadoop conf property through the command line.
>>
>> Regards,
>> Shahab
>>
>>
>> On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <an...@gmail.com> wrote:
>>
>>> Thanks for the tip. May be it's a different setting I need. Let me
>>> explain the problem.
>>>
>>> I get an heap error on the command line - "Exception in thread "main"
>>> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
>>> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
>>> command line (hadoop jar ...)? Next choice would be to do it
>>> programmatically in Java using hadoop conf. I can't really change the XML
>>> files across the cluster.
>>>
>>> Regards,
>>> Andy
>>>
>>>
>>> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
>>>
>>>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>>>> "mapred.child.java.opts" in the mapred-site.xml
>>>>
>>>>
>>>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com>wrote:
>>>>
>>>>> Quick question. How would I pass the following JVM option to the
>>>>> Hadoop command line?
>>>>>
>>>>> "-Xmx4G"
>>>>>
>>>>> hadoop jar <jar file> <class_name> <args>
>>>>>
>>>>> Thanks,
>>>>> Andy
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Andy
>>>
>>>
>>
>
>
> --
> Thanks,
> Andy
>
>
Re: JVM option
Posted by Shahab Yunus <sh...@gmail.com>.
HADOOP_HEAPSIZE and HADOOP_OPTS are not hadoop properties. These seem to
be environment variables of the shell script.
Try this with a space between -D and the property name (
-Dmapred.child.java.opts)
See the properties available (for mrv1) here:
http://hadoop.apache.org/docs/r1.0.4/mapred-default.html
or this:
http://hadoop.apache.org/docs/r2.3.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
As for passing system properties, the option that I know of (others can
chime in if they know something better) is to pass the hadoop property and
then in the java code, set the system property form the configuration
object.
Regards,
Shahab
On Fri, Apr 18, 2014 at 3:20 PM, Andy Srine <an...@gmail.com> wrote:
> I guess then the question is, how do I set the (system?) property and what
> property exactly is this?
>
> "export HADOOP_HEAPSIZE=4096" - Works from the environment.
>
> And none of the following seem to work from the command line.
> -Xmx4G
>
> -Dmapred.child.java.opts=-Xmx4G
>
> -DHADOOP_HEAPSIZE=4096
>
> -DHADOOP_OPTS=-Xmx4G
>
>
> I tried this from Java code as well and no luck.
>
> System.setProperty("HADOOP_HEAPSIZE", "4096");
>
>
> What am I missing?
>
>
> Thanks,
>
> Andy
>
>
>
>
> On Fri, Apr 18, 2014 at 11:28 AM, Shahab Yunus <sh...@gmail.com>wrote:
>
>> You can pass hadoop conf properties through the -D option. Have you seen
>> this?
>>
>> http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
>>
>> This is not for system properties. The assumption is that you want to
>> specify hadoop conf property through the command line.
>>
>> Regards,
>> Shahab
>>
>>
>> On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <an...@gmail.com> wrote:
>>
>>> Thanks for the tip. May be it's a different setting I need. Let me
>>> explain the problem.
>>>
>>> I get an heap error on the command line - "Exception in thread "main"
>>> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
>>> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
>>> command line (hadoop jar ...)? Next choice would be to do it
>>> programmatically in Java using hadoop conf. I can't really change the XML
>>> files across the cluster.
>>>
>>> Regards,
>>> Andy
>>>
>>>
>>> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
>>>
>>>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>>>> "mapred.child.java.opts" in the mapred-site.xml
>>>>
>>>>
>>>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com>wrote:
>>>>
>>>>> Quick question. How would I pass the following JVM option to the
>>>>> Hadoop command line?
>>>>>
>>>>> "-Xmx4G"
>>>>>
>>>>> hadoop jar <jar file> <class_name> <args>
>>>>>
>>>>> Thanks,
>>>>> Andy
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Andy
>>>
>>>
>>
>
>
> --
> Thanks,
> Andy
>
>
Re: JVM option
Posted by Bjorn Jonsson <bj...@gmail.com>.
Hi Andy,
This is a client side property that you can set in hadoop-env.sh
(/etc/hadoop/conf). The hadoop script (/bin/hadoop)
itself, which is a bash shell script, contains code like the following.
if [ "$HADOOP_HEAPSIZE" != "" ]; then
#echo "run with heapsize $HADOOP_HEAPSIZE"
JAVA_HEAP_MAX="-Xmx""$HADOOP_HEAPSIZE""m"
#echo $JAVA_HEAP_MAX
fi
Use hadoop-env.sh.
Best,
Bjorn
On Fri, Apr 18, 2014 at 7:20 PM, Andy Srine <an...@gmail.com> wrote:
> I guess then the question is, how do I set the (system?) property and what
> property exactly is this?
>
> "export HADOOP_HEAPSIZE=4096" - Works from the environment.
>
> And none of the following seem to work from the command line.
> -Xmx4G
>
> -Dmapred.child.java.opts=-Xmx4G
>
> -DHADOOP_HEAPSIZE=4096
>
> -DHADOOP_OPTS=-Xmx4G
>
>
> I tried this from Java code as well and no luck.
>
> System.setProperty("HADOOP_HEAPSIZE", "4096");
>
>
> What am I missing?
>
>
> Thanks,
>
> Andy
>
>
>
>
> On Fri, Apr 18, 2014 at 11:28 AM, Shahab Yunus <sh...@gmail.com>wrote:
>
>> You can pass hadoop conf properties through the -D option. Have you seen
>> this?
>>
>> http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
>>
>> This is not for system properties. The assumption is that you want to
>> specify hadoop conf property through the command line.
>>
>> Regards,
>> Shahab
>>
>>
>> On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <an...@gmail.com> wrote:
>>
>>> Thanks for the tip. May be it's a different setting I need. Let me
>>> explain the problem.
>>>
>>> I get an heap error on the command line - "Exception in thread "main"
>>> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
>>> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
>>> command line (hadoop jar ...)? Next choice would be to do it
>>> programmatically in Java using hadoop conf. I can't really change the XML
>>> files across the cluster.
>>>
>>> Regards,
>>> Andy
>>>
>>>
>>> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
>>>
>>>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>>>> "mapred.child.java.opts" in the mapred-site.xml
>>>>
>>>>
>>>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com>wrote:
>>>>
>>>>> Quick question. How would I pass the following JVM option to the
>>>>> Hadoop command line?
>>>>>
>>>>> "-Xmx4G"
>>>>>
>>>>> hadoop jar <jar file> <class_name> <args>
>>>>>
>>>>> Thanks,
>>>>> Andy
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Andy
>>>
>>>
>>
>
>
> --
> Thanks,
> Andy
>
>
Re: JVM option
Posted by Shahab Yunus <sh...@gmail.com>.
HADOOP_HEAPSIZE and HADOOP_OPTS are not hadoop properties. These seem to
be environment variables of the shell script.
Try this with a space between -D and the property name (
-Dmapred.child.java.opts)
See the properties available (for mrv1) here:
http://hadoop.apache.org/docs/r1.0.4/mapred-default.html
or this:
http://hadoop.apache.org/docs/r2.3.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
As for passing system properties, the option that I know of (others can
chime in if they know something better) is to pass the hadoop property and
then in the java code, set the system property form the configuration
object.
Regards,
Shahab
On Fri, Apr 18, 2014 at 3:20 PM, Andy Srine <an...@gmail.com> wrote:
> I guess then the question is, how do I set the (system?) property and what
> property exactly is this?
>
> "export HADOOP_HEAPSIZE=4096" - Works from the environment.
>
> And none of the following seem to work from the command line.
> -Xmx4G
>
> -Dmapred.child.java.opts=-Xmx4G
>
> -DHADOOP_HEAPSIZE=4096
>
> -DHADOOP_OPTS=-Xmx4G
>
>
> I tried this from Java code as well and no luck.
>
> System.setProperty("HADOOP_HEAPSIZE", "4096");
>
>
> What am I missing?
>
>
> Thanks,
>
> Andy
>
>
>
>
> On Fri, Apr 18, 2014 at 11:28 AM, Shahab Yunus <sh...@gmail.com>wrote:
>
>> You can pass hadoop conf properties through the -D option. Have you seen
>> this?
>>
>> http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
>>
>> This is not for system properties. The assumption is that you want to
>> specify hadoop conf property through the command line.
>>
>> Regards,
>> Shahab
>>
>>
>> On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <an...@gmail.com> wrote:
>>
>>> Thanks for the tip. May be it's a different setting I need. Let me
>>> explain the problem.
>>>
>>> I get an heap error on the command line - "Exception in thread "main"
>>> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
>>> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
>>> command line (hadoop jar ...)? Next choice would be to do it
>>> programmatically in Java using hadoop conf. I can't really change the XML
>>> files across the cluster.
>>>
>>> Regards,
>>> Andy
>>>
>>>
>>> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
>>>
>>>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>>>> "mapred.child.java.opts" in the mapred-site.xml
>>>>
>>>>
>>>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com>wrote:
>>>>
>>>>> Quick question. How would I pass the following JVM option to the
>>>>> Hadoop command line?
>>>>>
>>>>> "-Xmx4G"
>>>>>
>>>>> hadoop jar <jar file> <class_name> <args>
>>>>>
>>>>> Thanks,
>>>>> Andy
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Andy
>>>
>>>
>>
>
>
> --
> Thanks,
> Andy
>
>
Re: JVM option
Posted by Bjorn Jonsson <bj...@gmail.com>.
Hi Andy,
This is a client side property that you can set in hadoop-env.sh
(/etc/hadoop/conf). The hadoop script (/bin/hadoop)
itself, which is a bash shell script, contains code like the following.
if [ "$HADOOP_HEAPSIZE" != "" ]; then
#echo "run with heapsize $HADOOP_HEAPSIZE"
JAVA_HEAP_MAX="-Xmx""$HADOOP_HEAPSIZE""m"
#echo $JAVA_HEAP_MAX
fi
Use hadoop-env.sh.
Best,
Bjorn
On Fri, Apr 18, 2014 at 7:20 PM, Andy Srine <an...@gmail.com> wrote:
> I guess then the question is, how do I set the (system?) property and what
> property exactly is this?
>
> "export HADOOP_HEAPSIZE=4096" - Works from the environment.
>
> And none of the following seem to work from the command line.
> -Xmx4G
>
> -Dmapred.child.java.opts=-Xmx4G
>
> -DHADOOP_HEAPSIZE=4096
>
> -DHADOOP_OPTS=-Xmx4G
>
>
> I tried this from Java code as well and no luck.
>
> System.setProperty("HADOOP_HEAPSIZE", "4096");
>
>
> What am I missing?
>
>
> Thanks,
>
> Andy
>
>
>
>
> On Fri, Apr 18, 2014 at 11:28 AM, Shahab Yunus <sh...@gmail.com>wrote:
>
>> You can pass hadoop conf properties through the -D option. Have you seen
>> this?
>>
>> http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
>>
>> This is not for system properties. The assumption is that you want to
>> specify hadoop conf property through the command line.
>>
>> Regards,
>> Shahab
>>
>>
>> On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <an...@gmail.com> wrote:
>>
>>> Thanks for the tip. May be it's a different setting I need. Let me
>>> explain the problem.
>>>
>>> I get an heap error on the command line - "Exception in thread "main"
>>> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
>>> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
>>> command line (hadoop jar ...)? Next choice would be to do it
>>> programmatically in Java using hadoop conf. I can't really change the XML
>>> files across the cluster.
>>>
>>> Regards,
>>> Andy
>>>
>>>
>>> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
>>>
>>>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>>>> "mapred.child.java.opts" in the mapred-site.xml
>>>>
>>>>
>>>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com>wrote:
>>>>
>>>>> Quick question. How would I pass the following JVM option to the
>>>>> Hadoop command line?
>>>>>
>>>>> "-Xmx4G"
>>>>>
>>>>> hadoop jar <jar file> <class_name> <args>
>>>>>
>>>>> Thanks,
>>>>> Andy
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Andy
>>>
>>>
>>
>
>
> --
> Thanks,
> Andy
>
>
Re: JVM option
Posted by Andy Srine <an...@gmail.com>.
I guess then the question is, how do I set the (system?) property and what
property exactly is this?
"export HADOOP_HEAPSIZE=4096" - Works from the environment.
And none of the following seem to work from the command line.
-Xmx4G
-Dmapred.child.java.opts=-Xmx4G
-DHADOOP_HEAPSIZE=4096
-DHADOOP_OPTS=-Xmx4G
I tried this from Java code as well and no luck.
System.setProperty("HADOOP_HEAPSIZE", "4096");
What am I missing?
Thanks,
Andy
On Fri, Apr 18, 2014 at 11:28 AM, Shahab Yunus <sh...@gmail.com>wrote:
> You can pass hadoop conf properties through the -D option. Have you seen
> this?
>
> http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
>
> This is not for system properties. The assumption is that you want to
> specify hadoop conf property through the command line.
>
> Regards,
> Shahab
>
>
> On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <an...@gmail.com> wrote:
>
>> Thanks for the tip. May be it's a different setting I need. Let me
>> explain the problem.
>>
>> I get an heap error on the command line - "Exception in thread "main"
>> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
>> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
>> command line (hadoop jar ...)? Next choice would be to do it
>> programmatically in Java using hadoop conf. I can't really change the XML
>> files across the cluster.
>>
>> Regards,
>> Andy
>>
>>
>> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
>>
>>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>>> "mapred.child.java.opts" in the mapred-site.xml
>>>
>>>
>>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com>wrote:
>>>
>>>> Quick question. How would I pass the following JVM option to the Hadoop
>>>> command line?
>>>>
>>>> "-Xmx4G"
>>>>
>>>> hadoop jar <jar file> <class_name> <args>
>>>>
>>>> Thanks,
>>>> Andy
>>>>
>>>>
>>>
>>
>>
>> --
>> Thanks,
>> Andy
>>
>>
>
--
Thanks,
Andy
Re: JVM option
Posted by Andy Srine <an...@gmail.com>.
I guess then the question is, how do I set the (system?) property and what
property exactly is this?
"export HADOOP_HEAPSIZE=4096" - Works from the environment.
And none of the following seem to work from the command line.
-Xmx4G
-Dmapred.child.java.opts=-Xmx4G
-DHADOOP_HEAPSIZE=4096
-DHADOOP_OPTS=-Xmx4G
I tried this from Java code as well and no luck.
System.setProperty("HADOOP_HEAPSIZE", "4096");
What am I missing?
Thanks,
Andy
On Fri, Apr 18, 2014 at 11:28 AM, Shahab Yunus <sh...@gmail.com>wrote:
> You can pass hadoop conf properties through the -D option. Have you seen
> this?
>
> http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
>
> This is not for system properties. The assumption is that you want to
> specify hadoop conf property through the command line.
>
> Regards,
> Shahab
>
>
> On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <an...@gmail.com> wrote:
>
>> Thanks for the tip. May be it's a different setting I need. Let me
>> explain the problem.
>>
>> I get an heap error on the command line - "Exception in thread "main"
>> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
>> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
>> command line (hadoop jar ...)? Next choice would be to do it
>> programmatically in Java using hadoop conf. I can't really change the XML
>> files across the cluster.
>>
>> Regards,
>> Andy
>>
>>
>> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
>>
>>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>>> "mapred.child.java.opts" in the mapred-site.xml
>>>
>>>
>>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com>wrote:
>>>
>>>> Quick question. How would I pass the following JVM option to the Hadoop
>>>> command line?
>>>>
>>>> "-Xmx4G"
>>>>
>>>> hadoop jar <jar file> <class_name> <args>
>>>>
>>>> Thanks,
>>>> Andy
>>>>
>>>>
>>>
>>
>>
>> --
>> Thanks,
>> Andy
>>
>>
>
--
Thanks,
Andy
Re: JVM option
Posted by Andy Srine <an...@gmail.com>.
I guess then the question is, how do I set the (system?) property and what
property exactly is this?
"export HADOOP_HEAPSIZE=4096" - Works from the environment.
And none of the following seem to work from the command line.
-Xmx4G
-Dmapred.child.java.opts=-Xmx4G
-DHADOOP_HEAPSIZE=4096
-DHADOOP_OPTS=-Xmx4G
I tried this from Java code as well and no luck.
System.setProperty("HADOOP_HEAPSIZE", "4096");
What am I missing?
Thanks,
Andy
On Fri, Apr 18, 2014 at 11:28 AM, Shahab Yunus <sh...@gmail.com>wrote:
> You can pass hadoop conf properties through the -D option. Have you seen
> this?
>
> http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
>
> This is not for system properties. The assumption is that you want to
> specify hadoop conf property through the command line.
>
> Regards,
> Shahab
>
>
> On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <an...@gmail.com> wrote:
>
>> Thanks for the tip. May be it's a different setting I need. Let me
>> explain the problem.
>>
>> I get an heap error on the command line - "Exception in thread "main"
>> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
>> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
>> command line (hadoop jar ...)? Next choice would be to do it
>> programmatically in Java using hadoop conf. I can't really change the XML
>> files across the cluster.
>>
>> Regards,
>> Andy
>>
>>
>> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
>>
>>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>>> "mapred.child.java.opts" in the mapred-site.xml
>>>
>>>
>>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com>wrote:
>>>
>>>> Quick question. How would I pass the following JVM option to the Hadoop
>>>> command line?
>>>>
>>>> "-Xmx4G"
>>>>
>>>> hadoop jar <jar file> <class_name> <args>
>>>>
>>>> Thanks,
>>>> Andy
>>>>
>>>>
>>>
>>
>>
>> --
>> Thanks,
>> Andy
>>
>>
>
--
Thanks,
Andy
Re: JVM option
Posted by Andy Srine <an...@gmail.com>.
I guess then the question is, how do I set the (system?) property and what
property exactly is this?
"export HADOOP_HEAPSIZE=4096" - Works from the environment.
And none of the following seem to work from the command line.
-Xmx4G
-Dmapred.child.java.opts=-Xmx4G
-DHADOOP_HEAPSIZE=4096
-DHADOOP_OPTS=-Xmx4G
I tried this from Java code as well and no luck.
System.setProperty("HADOOP_HEAPSIZE", "4096");
What am I missing?
Thanks,
Andy
On Fri, Apr 18, 2014 at 11:28 AM, Shahab Yunus <sh...@gmail.com>wrote:
> You can pass hadoop conf properties through the -D option. Have you seen
> this?
>
> http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
>
> This is not for system properties. The assumption is that you want to
> specify hadoop conf property through the command line.
>
> Regards,
> Shahab
>
>
> On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <an...@gmail.com> wrote:
>
>> Thanks for the tip. May be it's a different setting I need. Let me
>> explain the problem.
>>
>> I get an heap error on the command line - "Exception in thread "main"
>> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
>> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
>> command line (hadoop jar ...)? Next choice would be to do it
>> programmatically in Java using hadoop conf. I can't really change the XML
>> files across the cluster.
>>
>> Regards,
>> Andy
>>
>>
>> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
>>
>>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>>> "mapred.child.java.opts" in the mapred-site.xml
>>>
>>>
>>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com>wrote:
>>>
>>>> Quick question. How would I pass the following JVM option to the Hadoop
>>>> command line?
>>>>
>>>> "-Xmx4G"
>>>>
>>>> hadoop jar <jar file> <class_name> <args>
>>>>
>>>> Thanks,
>>>> Andy
>>>>
>>>>
>>>
>>
>>
>> --
>> Thanks,
>> Andy
>>
>>
>
--
Thanks,
Andy
Re: JVM option
Posted by Shahab Yunus <sh...@gmail.com>.
You can pass hadoop conf properties through the -D option. Have you seen
this?
http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
This is not for system properties. The assumption is that you want to
specify hadoop conf property through the command line.
Regards,
Shahab
On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <an...@gmail.com> wrote:
> Thanks for the tip. May be it's a different setting I need. Let me explain
> the problem.
>
> I get an heap error on the command line - "Exception in thread "main"
> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
> command line (hadoop jar ...)? Next choice would be to do it
> programmatically in Java using hadoop conf. I can't really change the XML
> files across the cluster.
>
> Regards,
> Andy
>
>
> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
>
>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>> "mapred.child.java.opts" in the mapred-site.xml
>>
>>
>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com> wrote:
>>
>>> Quick question. How would I pass the following JVM option to the Hadoop
>>> command line?
>>>
>>> "-Xmx4G"
>>>
>>> hadoop jar <jar file> <class_name> <args>
>>>
>>> Thanks,
>>> Andy
>>>
>>>
>>
>
>
> --
> Thanks,
> Andy
>
>
Re: JVM option
Posted by Shahab Yunus <sh...@gmail.com>.
You can pass hadoop conf properties through the -D option. Have you seen
this?
http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
This is not for system properties. The assumption is that you want to
specify hadoop conf property through the command line.
Regards,
Shahab
On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <an...@gmail.com> wrote:
> Thanks for the tip. May be it's a different setting I need. Let me explain
> the problem.
>
> I get an heap error on the command line - "Exception in thread "main"
> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
> command line (hadoop jar ...)? Next choice would be to do it
> programmatically in Java using hadoop conf. I can't really change the XML
> files across the cluster.
>
> Regards,
> Andy
>
>
> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
>
>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>> "mapred.child.java.opts" in the mapred-site.xml
>>
>>
>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com> wrote:
>>
>>> Quick question. How would I pass the following JVM option to the Hadoop
>>> command line?
>>>
>>> "-Xmx4G"
>>>
>>> hadoop jar <jar file> <class_name> <args>
>>>
>>> Thanks,
>>> Andy
>>>
>>>
>>
>
>
> --
> Thanks,
> Andy
>
>
Re: JVM option
Posted by Shahab Yunus <sh...@gmail.com>.
You can pass hadoop conf properties through the -D option. Have you seen
this?
http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
This is not for system properties. The assumption is that you want to
specify hadoop conf property through the command line.
Regards,
Shahab
On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <an...@gmail.com> wrote:
> Thanks for the tip. May be it's a different setting I need. Let me explain
> the problem.
>
> I get an heap error on the command line - "Exception in thread "main"
> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
> command line (hadoop jar ...)? Next choice would be to do it
> programmatically in Java using hadoop conf. I can't really change the XML
> files across the cluster.
>
> Regards,
> Andy
>
>
> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
>
>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>> "mapred.child.java.opts" in the mapred-site.xml
>>
>>
>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com> wrote:
>>
>>> Quick question. How would I pass the following JVM option to the Hadoop
>>> command line?
>>>
>>> "-Xmx4G"
>>>
>>> hadoop jar <jar file> <class_name> <args>
>>>
>>> Thanks,
>>> Andy
>>>
>>>
>>
>
>
> --
> Thanks,
> Andy
>
>
Re: JVM option
Posted by Shahab Yunus <sh...@gmail.com>.
You can pass hadoop conf properties through the -D option. Have you seen
this?
http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
This is not for system properties. The assumption is that you want to
specify hadoop conf property through the command line.
Regards,
Shahab
On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <an...@gmail.com> wrote:
> Thanks for the tip. May be it's a different setting I need. Let me explain
> the problem.
>
> I get an heap error on the command line - "Exception in thread "main"
> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
> command line (hadoop jar ...)? Next choice would be to do it
> programmatically in Java using hadoop conf. I can't really change the XML
> files across the cluster.
>
> Regards,
> Andy
>
>
> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
>
>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>> "mapred.child.java.opts" in the mapred-site.xml
>>
>>
>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com> wrote:
>>
>>> Quick question. How would I pass the following JVM option to the Hadoop
>>> command line?
>>>
>>> "-Xmx4G"
>>>
>>> hadoop jar <jar file> <class_name> <args>
>>>
>>> Thanks,
>>> Andy
>>>
>>>
>>
>
>
> --
> Thanks,
> Andy
>
>
Re: JVM option
Posted by Andy Srine <an...@gmail.com>.
Thanks for the tip. May be it's a different setting I need. Let me explain
the problem.
I get an heap error on the command line - "Exception in thread "main"
java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
command line (hadoop jar ...)? Next choice would be to do it
programmatically in Java using hadoop conf. I can't really change the XML
files across the cluster.
Regards,
Andy
On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
> "mapred.child.java.opts" in the mapred-site.xml
>
>
> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com> wrote:
>
>> Quick question. How would I pass the following JVM option to the Hadoop
>> command line?
>>
>> "-Xmx4G"
>>
>> hadoop jar <jar file> <class_name> <args>
>>
>> Thanks,
>> Andy
>>
>>
>
--
Thanks,
Andy
Re: JVM option
Posted by Andy Srine <an...@gmail.com>.
Thanks for the tip. May be it's a different setting I need. Let me explain
the problem.
I get an heap error on the command line - "Exception in thread "main"
java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
command line (hadoop jar ...)? Next choice would be to do it
programmatically in Java using hadoop conf. I can't really change the XML
files across the cluster.
Regards,
Andy
On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
> "mapred.child.java.opts" in the mapred-site.xml
>
>
> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com> wrote:
>
>> Quick question. How would I pass the following JVM option to the Hadoop
>> command line?
>>
>> "-Xmx4G"
>>
>> hadoop jar <jar file> <class_name> <args>
>>
>> Thanks,
>> Andy
>>
>>
>
--
Thanks,
Andy
Re: JVM option
Posted by Andy Srine <an...@gmail.com>.
Thanks for the tip. May be it's a different setting I need. Let me explain
the problem.
I get an heap error on the command line - "Exception in thread "main"
java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
command line (hadoop jar ...)? Next choice would be to do it
programmatically in Java using hadoop conf. I can't really change the XML
files across the cluster.
Regards,
Andy
On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
> "mapred.child.java.opts" in the mapred-site.xml
>
>
> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com> wrote:
>
>> Quick question. How would I pass the following JVM option to the Hadoop
>> command line?
>>
>> "-Xmx4G"
>>
>> hadoop jar <jar file> <class_name> <args>
>>
>> Thanks,
>> Andy
>>
>>
>
--
Thanks,
Andy
Re: JVM option
Posted by Andy Srine <an...@gmail.com>.
Thanks for the tip. May be it's a different setting I need. Let me explain
the problem.
I get an heap error on the command line - "Exception in thread "main"
java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
command line (hadoop jar ...)? Next choice would be to do it
programmatically in Java using hadoop conf. I can't really change the XML
files across the cluster.
Regards,
Andy
On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <az...@gmail.com> wrote:
> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
> "mapred.child.java.opts" in the mapred-site.xml
>
>
> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com> wrote:
>
>> Quick question. How would I pass the following JVM option to the Hadoop
>> command line?
>>
>> "-Xmx4G"
>>
>> hadoop jar <jar file> <class_name> <args>
>>
>> Thanks,
>> Andy
>>
>>
>
--
Thanks,
Andy
Re: JVM option
Posted by Azuryy Yu <az...@gmail.com>.
Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
"mapred.child.java.opts" in the mapred-site.xml
On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com> wrote:
> Quick question. How would I pass the following JVM option to the Hadoop
> command line?
>
> "-Xmx4G"
>
> hadoop jar <jar file> <class_name> <args>
>
> Thanks,
> Andy
>
>
Re: JVM option
Posted by Azuryy Yu <az...@gmail.com>.
Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
"mapred.child.java.opts" in the mapred-site.xml
On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com> wrote:
> Quick question. How would I pass the following JVM option to the Hadoop
> command line?
>
> "-Xmx4G"
>
> hadoop jar <jar file> <class_name> <args>
>
> Thanks,
> Andy
>
>
Re: JVM option
Posted by Azuryy Yu <az...@gmail.com>.
Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
"mapred.child.java.opts" in the mapred-site.xml
On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com> wrote:
> Quick question. How would I pass the following JVM option to the Hadoop
> command line?
>
> "-Xmx4G"
>
> hadoop jar <jar file> <class_name> <args>
>
> Thanks,
> Andy
>
>
Re: JVM option
Posted by Azuryy Yu <az...@gmail.com>.
Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
"mapred.child.java.opts" in the mapred-site.xml
On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <an...@gmail.com> wrote:
> Quick question. How would I pass the following JVM option to the Hadoop
> command line?
>
> "-Xmx4G"
>
> hadoop jar <jar file> <class_name> <args>
>
> Thanks,
> Andy
>
>