You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Kun Ling <lk...@gmail.com> on 2013/08/15 11:26:53 UTC
Is there any possible way to use hostname variable in mapred-site.xml file
Hi all,
I have a Hadoop MapReduce Cluster. In which I want to adjust the
mapred.local.dir, so that each TaskTracker can write to a mapred.local.dir
with different name. And also make the conf file looks the same to make
deployment easier.
Currently, my plan is that each TaskTracker have hostname in its
mapred.local.dir configuration. So the configuration in mapred-site.xml is
just like this:
<property>
<name>mapred.local.dir</name>
<value>/var/mapred_local/*HOSTNAME*/</value>
</property>
The problem is how to make TaskTracker automatically get the HOSTNAME？ I
have look through all the .xml files in conf/ and jar files, but only get
an variable ${user.name} which could be used to indicate the current hadoop
username.
Thanks very much .
yours,
Kun Ling
--
http://www.lingcc.com
Re: Is there any possible way to use hostname variable in
mapred-site.xml file
Posted by Kun Ling <lk...@gmail.com>.
Hi Binglin,
Thanks for your kindly help.
And your advice works well for me.
yours,
Kun Ling
On Thu, Aug 15, 2013 at 5:51 PM, Binglin Chang <de...@gmail.com> wrote:
> How about add -Dhost.name=`hostname` in HADOOP_OPTS
> and get this variable in config file ${host.name} ?
> I have not tried this, you can try this.
>
>
> On Thu, Aug 15, 2013 at 5:26 PM, Kun Ling <lk...@gmail.com> wrote:
>
>> Hi all,
>> I have a Hadoop MapReduce Cluster. In which I want to adjust the
>> mapred.local.dir, so that each TaskTracker can write to a mapred.local.dir
>> with different name. And also make the conf file looks the same to make
>> deployment easier.
>>
>> Currently, my plan is that each TaskTracker have hostname in its
>> mapred.local.dir configuration. So the configuration in mapred-site.xml is
>> just like this:
>>
>> <property>
>> <name>mapred.local.dir</name>
>> <value>/var/mapred_local/*HOSTNAME*/</value>
>> </property>
>>
>> The problem is how to make TaskTracker automatically get the HOSTNAME？
>> I have look through all the .xml files in conf/ and jar files, but only
>> get an variable ${user.name} which could be used to indicate the current
>> hadoop username.
>>
>>
>> Thanks very much .
>>
>>
>> yours,
>> Kun Ling
>>
>>
>> --
>> http://www.lingcc.com
>>
>
>
--
http://www.lingcc.com
Re: Is there any possible way to use hostname variable in
mapred-site.xml file
Posted by Kun Ling <lk...@gmail.com>.
Hi Binglin,
Thanks for your kindly help.
And your advice works well for me.
yours,
Kun Ling
On Thu, Aug 15, 2013 at 5:51 PM, Binglin Chang <de...@gmail.com> wrote:
> How about add -Dhost.name=`hostname` in HADOOP_OPTS
> and get this variable in config file ${host.name} ?
> I have not tried this, you can try this.
>
>
> On Thu, Aug 15, 2013 at 5:26 PM, Kun Ling <lk...@gmail.com> wrote:
>
>> Hi all,
>> I have a Hadoop MapReduce Cluster. In which I want to adjust the
>> mapred.local.dir, so that each TaskTracker can write to a mapred.local.dir
>> with different name. And also make the conf file looks the same to make
>> deployment easier.
>>
>> Currently, my plan is that each TaskTracker have hostname in its
>> mapred.local.dir configuration. So the configuration in mapred-site.xml is
>> just like this:
>>
>> <property>
>> <name>mapred.local.dir</name>
>> <value>/var/mapred_local/*HOSTNAME*/</value>
>> </property>
>>
>> The problem is how to make TaskTracker automatically get the HOSTNAME？
>> I have look through all the .xml files in conf/ and jar files, but only
>> get an variable ${user.name} which could be used to indicate the current
>> hadoop username.
>>
>>
>> Thanks very much .
>>
>>
>> yours,
>> Kun Ling
>>
>>
>> --
>> http://www.lingcc.com
>>
>
>
--
http://www.lingcc.com
Re: Is there any possible way to use hostname variable in
mapred-site.xml file
Posted by Kun Ling <lk...@gmail.com>.
Hi Binglin,
Thanks for your kindly help.
And your advice works well for me.
yours,
Kun Ling
On Thu, Aug 15, 2013 at 5:51 PM, Binglin Chang <de...@gmail.com> wrote:
> How about add -Dhost.name=`hostname` in HADOOP_OPTS
> and get this variable in config file ${host.name} ?
> I have not tried this, you can try this.
>
>
> On Thu, Aug 15, 2013 at 5:26 PM, Kun Ling <lk...@gmail.com> wrote:
>
>> Hi all,
>> I have a Hadoop MapReduce Cluster. In which I want to adjust the
>> mapred.local.dir, so that each TaskTracker can write to a mapred.local.dir
>> with different name. And also make the conf file looks the same to make
>> deployment easier.
>>
>> Currently, my plan is that each TaskTracker have hostname in its
>> mapred.local.dir configuration. So the configuration in mapred-site.xml is
>> just like this:
>>
>> <property>
>> <name>mapred.local.dir</name>
>> <value>/var/mapred_local/*HOSTNAME*/</value>
>> </property>
>>
>> The problem is how to make TaskTracker automatically get the HOSTNAME？
>> I have look through all the .xml files in conf/ and jar files, but only
>> get an variable ${user.name} which could be used to indicate the current
>> hadoop username.
>>
>>
>> Thanks very much .
>>
>>
>> yours,
>> Kun Ling
>>
>>
>> --
>> http://www.lingcc.com
>>
>
>
--
http://www.lingcc.com
Re: Is there any possible way to use hostname variable in
mapred-site.xml file
Posted by Kun Ling <lk...@gmail.com>.
Hi Binglin,
Thanks for your kindly help.
And your advice works well for me.
yours,
Kun Ling
On Thu, Aug 15, 2013 at 5:51 PM, Binglin Chang <de...@gmail.com> wrote:
> How about add -Dhost.name=`hostname` in HADOOP_OPTS
> and get this variable in config file ${host.name} ?
> I have not tried this, you can try this.
>
>
> On Thu, Aug 15, 2013 at 5:26 PM, Kun Ling <lk...@gmail.com> wrote:
>
>> Hi all,
>> I have a Hadoop MapReduce Cluster. In which I want to adjust the
>> mapred.local.dir, so that each TaskTracker can write to a mapred.local.dir
>> with different name. And also make the conf file looks the same to make
>> deployment easier.
>>
>> Currently, my plan is that each TaskTracker have hostname in its
>> mapred.local.dir configuration. So the configuration in mapred-site.xml is
>> just like this:
>>
>> <property>
>> <name>mapred.local.dir</name>
>> <value>/var/mapred_local/*HOSTNAME*/</value>
>> </property>
>>
>> The problem is how to make TaskTracker automatically get the HOSTNAME？
>> I have look through all the .xml files in conf/ and jar files, but only
>> get an variable ${user.name} which could be used to indicate the current
>> hadoop username.
>>
>>
>> Thanks very much .
>>
>>
>> yours,
>> Kun Ling
>>
>>
>> --
>> http://www.lingcc.com
>>
>
>
--
http://www.lingcc.com
Re: Is there any possible way to use hostname variable in
mapred-site.xml file
Posted by Binglin Chang <de...@gmail.com>.
How about add -Dhost.name=`hostname` in HADOOP_OPTS
and get this variable in config file ${host.name} ?
I have not tried this, you can try this.
On Thu, Aug 15, 2013 at 5:26 PM, Kun Ling <lk...@gmail.com> wrote:
> Hi all,
> I have a Hadoop MapReduce Cluster. In which I want to adjust the
> mapred.local.dir, so that each TaskTracker can write to a mapred.local.dir
> with different name. And also make the conf file looks the same to make
> deployment easier.
>
> Currently, my plan is that each TaskTracker have hostname in its
> mapred.local.dir configuration. So the configuration in mapred-site.xml is
> just like this:
>
> <property>
> <name>mapred.local.dir</name>
> <value>/var/mapred_local/*HOSTNAME*/</value>
> </property>
>
> The problem is how to make TaskTracker automatically get the HOSTNAME？ I
> have look through all the .xml files in conf/ and jar files, but only get
> an variable ${user.name} which could be used to indicate the current
> hadoop username.
>
>
> Thanks very much .
>
>
> yours,
> Kun Ling
>
>
> --
> http://www.lingcc.com
>
Re: Is there any possible way to use hostname variable in
mapred-site.xml file
Posted by Binglin Chang <de...@gmail.com>.
How about add -Dhost.name=`hostname` in HADOOP_OPTS
and get this variable in config file ${host.name} ?
I have not tried this, you can try this.
On Thu, Aug 15, 2013 at 5:26 PM, Kun Ling <lk...@gmail.com> wrote:
> Hi all,
> I have a Hadoop MapReduce Cluster. In which I want to adjust the
> mapred.local.dir, so that each TaskTracker can write to a mapred.local.dir
> with different name. And also make the conf file looks the same to make
> deployment easier.
>
> Currently, my plan is that each TaskTracker have hostname in its
> mapred.local.dir configuration. So the configuration in mapred-site.xml is
> just like this:
>
> <property>
> <name>mapred.local.dir</name>
> <value>/var/mapred_local/*HOSTNAME*/</value>
> </property>
>
> The problem is how to make TaskTracker automatically get the HOSTNAME？ I
> have look through all the .xml files in conf/ and jar files, but only get
> an variable ${user.name} which could be used to indicate the current
> hadoop username.
>
>
> Thanks very much .
>
>
> yours,
> Kun Ling
>
>
> --
> http://www.lingcc.com
>
Re: Is there any possible way to use hostname variable in
mapred-site.xml file
Posted by Binglin Chang <de...@gmail.com>.
How about add -Dhost.name=`hostname` in HADOOP_OPTS
and get this variable in config file ${host.name} ?
I have not tried this, you can try this.
On Thu, Aug 15, 2013 at 5:26 PM, Kun Ling <lk...@gmail.com> wrote:
> Hi all,
> I have a Hadoop MapReduce Cluster. In which I want to adjust the
> mapred.local.dir, so that each TaskTracker can write to a mapred.local.dir
> with different name. And also make the conf file looks the same to make
> deployment easier.
>
> Currently, my plan is that each TaskTracker have hostname in its
> mapred.local.dir configuration. So the configuration in mapred-site.xml is
> just like this:
>
> <property>
> <name>mapred.local.dir</name>
> <value>/var/mapred_local/*HOSTNAME*/</value>
> </property>
>
> The problem is how to make TaskTracker automatically get the HOSTNAME？ I
> have look through all the .xml files in conf/ and jar files, but only get
> an variable ${user.name} which could be used to indicate the current
> hadoop username.
>
>
> Thanks very much .
>
>
> yours,
> Kun Ling
>
>
> --
> http://www.lingcc.com
>
Re: Is there any possible way to use hostname variable in
mapred-site.xml file
Posted by Binglin Chang <de...@gmail.com>.
How about add -Dhost.name=`hostname` in HADOOP_OPTS
and get this variable in config file ${host.name} ?
I have not tried this, you can try this.
On Thu, Aug 15, 2013 at 5:26 PM, Kun Ling <lk...@gmail.com> wrote:
> Hi all,
> I have a Hadoop MapReduce Cluster. In which I want to adjust the
> mapred.local.dir, so that each TaskTracker can write to a mapred.local.dir
> with different name. And also make the conf file looks the same to make
> deployment easier.
>
> Currently, my plan is that each TaskTracker have hostname in its
> mapred.local.dir configuration. So the configuration in mapred-site.xml is
> just like this:
>
> <property>
> <name>mapred.local.dir</name>
> <value>/var/mapred_local/*HOSTNAME*/</value>
> </property>
>
> The problem is how to make TaskTracker automatically get the HOSTNAME？ I
> have look through all the .xml files in conf/ and jar files, but only get
> an variable ${user.name} which could be used to indicate the current
> hadoop username.
>
>
> Thanks very much .
>
>
> yours,
> Kun Ling
>
>
> --
> http://www.lingcc.com
>