You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Gang Luo <lg...@yahoo.com.cn> on 2010/02/03 22:48:52 UTC

configuration file

Hi,
I am writing script to run whole bunch of jobs automatically. But the configuration file doesn't seems working. I think there is something wrong in my command. 

The command is my script is like:
bin/hadoop jar myJarFile myClass -conf myConfigurationFilr.xml  arg1  agr2 ....

I use conf.get() so show the value of some parameters. But the values are not what I define in that xml file.  Is there something wrong? 

Thanks.
-Gang



      ___________________________________________________________ 
  好玩贺卡等你发，邮箱贺卡全新上线！ 
http://card.mail.cn.yahoo.com/

Re: configuration file

Posted by Eric Arenas <ea...@rocketmail.com>.

Hi Gang,

You have to load the XML config file in your M/R code.

Something like this:
FSDataInputStream inS = fs.open(in);
conf.addResource(inS); 

 
Where "conf" is your Configuration.

This will in effect read all the parameters from that XML and override anything that you have previously set with:
conf.set("parameter",parameterValue);

regards,
Eric Arenas



----- Original Message ----
From: Gang Luo <lg...@yahoo.com.cn>
To: common-user@hadoop.apache.org
Sent: Thu, February 4, 2010 6:14:54 AM
Subject: Re: configuration file

I give the path to that xml file in that command. Do I need to add that path to classpath? I try to give a wrong path, there is no error reported.

Aren't those parameters all configurable? like io.sort.mb, mapred.reduce.tasks, io.sort.factor, etc. 

Thanks.
-Gang




----- 原始邮件 ----
发件人： Amogh Vasekar <am...@yahoo-inc.com>
收件人： "common-user@hadoop.apache.org" <co...@hadoop.apache.org>
发送日期： 2010/2/4 (周四) 6:09:04 上午
主   题： Re: configuration file

Hi,
A shot in the dark, is the conf file in your classpath? If yes, are the parameters you are trying to override marked final?

Amogh


On 2/4/10 3:18 AM, "Gang Luo" <lg...@yahoo.com.cn> wrote:

Hi,
I am writing script to run whole bunch of jobs automatically. But the configuration file doesn't seems working. I think there is something wrong in my command.

The command is my script is like:
bin/hadoop jar myJarFile myClass -conf myConfigurationFilr.xml  arg1  agr2 ....

I use conf.get() so show the value of some parameters. But the values are not what I define in that xml file.  Is there something wrong?

Thanks.
-Gang


      ___________________________________________________________ 
  好玩贺卡等你发，邮箱贺卡全新上线！ 
http://card.mail.cn.yahoo.com/

Re: configuration file

Posted by Gang Luo <lg...@yahoo.com.cn>.

I give the path to that xml file in that command. Do I need to add that path to classpath? I try to give a wrong path, there is no error reported.

Aren't those parameters all configurable? like io.sort.mb, mapred.reduce.tasks, io.sort.factor, etc. 

Thanks.
-Gang




----- 原始邮件 ----
发件人： Amogh Vasekar <am...@yahoo-inc.com>
收件人： "common-user@hadoop.apache.org" <co...@hadoop.apache.org>
发送日期： 2010/2/4 (周四) 6:09:04 上午
主   题： Re: configuration file

Hi,
A shot in the dark, is the conf file in your classpath? If yes, are the parameters you are trying to override marked final?

Amogh


On 2/4/10 3:18 AM, "Gang Luo" <lg...@yahoo.com.cn> wrote:

Hi,
I am writing script to run whole bunch of jobs automatically. But the configuration file doesn't seems working. I think there is something wrong in my command.

The command is my script is like:
bin/hadoop jar myJarFile myClass -conf myConfigurationFilr.xml  arg1  agr2 ....

I use conf.get() so show the value of some parameters. But the values are not what I define in that xml file.  Is there something wrong?

Thanks.
-Gang


      ___________________________________________________________ 
  好玩贺卡等你发，邮箱贺卡全新上线！ 
http://card.mail.cn.yahoo.com/

Re: configuration file

Posted by Amogh Vasekar <am...@yahoo-inc.com>.

Hi,
A shot in the dark, is the conf file in your classpath? If yes, are the parameters you are trying to override marked final?

Amogh


On 2/4/10 3:18 AM, "Gang Luo" <lg...@yahoo.com.cn> wrote:

Hi,
I am writing script to run whole bunch of jobs automatically. But the configuration file doesn't seems working. I think there is something wrong in my command.

The command is my script is like:
bin/hadoop jar myJarFile myClass -conf myConfigurationFilr.xml  arg1  agr2 ....

I use conf.get() so show the value of some parameters. But the values are not what I define in that xml file.  Is there something wrong?

Thanks.
-Gang