You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-user@hadoop.apache.org by Bejoy Ks <be...@gmail.com> on 2011/12/09 07:44:16 UTC

Are the values available in job.xml the actual values used for job

Hi experts
             I have a query with the job.xml file in map reduce.I set some
value in mapred-site.xml and *marked as final*, say
mapred.num.reduce.tasks=15. When I submit my job I explicitly specified the
number of reducers as -D mapred.num.reduce.tasks=4. Now as expected my my
job should run with 15 reducers as I marked this value as final in my
config file. Now what would be the value that would be available for
mapred.num.reduce.tasks *in job.xml*,  15 or 4 ?

Thank you

Regards
Bejoy.K.S

Re: Are the values available in job.xml the actual values used for job

Posted by Bejoy Ks <be...@gmail.com>.

Correction not mapred.num.reduce.tasks but mapred.reduce.tasks . :)

On Fri, Dec 9, 2011 at 12:14 PM, Bejoy Ks <be...@gmail.com> wrote:

> Hi experts
>              I have a query with the job.xml file in map reduce.I set some
> value in mapred-site.xml and *marked as final*, say
> mapred.num.reduce.tasks=15. When I submit my job I explicitly specified the
> number of reducers as -D mapred.num.reduce.tasks=4. Now as expected my my
> job should run with 15 reducers as I marked this value as final in my
> config file. Now what would be the value that would be available for
> mapred.num.reduce.tasks *in job.xml*,  15 or 4 ?
>
> Thank you
>
> Regards
> Bejoy.K.S
>

Re: Are the values available in job.xml the actual values used for job

Posted by Bejoy Ks <be...@gmail.com>.

Thanks Arun.
                To make my understanding a little more crisp. All the
values for config parameters available in job.xml are the actual values
used by a job for its execution. There won't be any further changes in
these values at any point further down the lane of execution. (ie no
changes in confing param values after the job.xml is generated).  Is my
understanding correct?
                I waned to make sure of this as in some clusters I'm
working on I don't have acess to the config files *-site.xml . So I just
want to make sure that job.xml is fully dependable and the right reference
for the config values used by my job.

Thanks a lot!....

Regards
Bejoy.KS

On Sat, Dec 10, 2011 at 4:44 AM, Arun C Murthy <ac...@hortonworks.com> wrote:

> 'final' is meant for admins to ensure certain values aren't overridable.
>
> However, in the example you gave, you'll see 15 (since it's 'final').
>
> Arun
>
> On Dec 8, 2011, at 10:44 PM, Bejoy Ks wrote:
>
> Hi experts
>              I have a query with the job.xml file in map reduce.I set some
> value in mapred-site.xml and *marked as final*, say
> mapred.num.reduce.tasks=15. When I submit my job I explicitly specified the
> number of reducers as -D mapred.num.reduce.tasks=4. Now as expected my my
> job should run with 15 reducers as I marked this value as final in my
> config file. Now what would be the value that would be available for
> mapred.num.reduce.tasks *in job.xml*,  15 or 4 ?
>
> Thank you
>
> Regards
> Bejoy.K.S
>
>
>

Re: Are the values available in job.xml the actual values used for job

Posted by Arun C Murthy <ac...@hortonworks.com>.

'final' is meant for admins to ensure certain values aren't overridable.

However, in the example you gave, you'll see 15 (since it's 'final').

Arun

On Dec 8, 2011, at 10:44 PM, Bejoy Ks wrote:

> Hi experts
>              I have a query with the job.xml file in map reduce.I set some value in mapred-site.xml and marked as final, say mapred.num.reduce.tasks=15. When I submit my job I explicitly specified the number of reducers as -D mapred.num.reduce.tasks=4. Now as expected my my job should run with 15 reducers as I marked this value as final in my config file. Now what would be the value that would be available for mapred.num.reduce.tasks in job.xml,  15 or 4 ?
> 
> Thank you
> 
> Regards
> Bejoy.K.S