Pig default properties/configuration

[Jeff Yuan <qu...@gmail.com>]

I'm calling Pig (with entry point from Main.java) from another Java
program. When it runs, I get a ton of warnings like these:

13/03/03 15:38:55 WARN conf.Configuration: dfs.max.objects is
deprecated. Instead, use dfs.namenode.max.objects
13/03/03 15:38:55 WARN conf.Configuration: mapred.task.id is
deprecated. Instead, use mapreduce.task.attempt.id

My question is, am I missing a step to load the default
configuration/properties? Or does Pig always show a lot of these
similar warnings? Any way to hide these during execution or to
otherwise?

Thanks.

Re: Pig default properties/configuration

[Cheolsoo Park <ch...@cloudera.com>]

Hi Jeff,

That's a side effect of that Pig has to support both Hadoop1.x and
Hadoop2.x, and Hadoop2.x depreciates a lot of old properties. One
workaround is upping the logging level of Hadoop Configuration as follows:

1. Edit conf/log4j.properties as follows:

log4j.logger.org.apache.hadoop.conf.Configuration=ERROR, A

2. Edit conf/pig.properties as follows:

# log4jconf log4j configuration file
log4jconf=<path_to_log4j.properties>/log4j.properties

Thanks,
Cheolsoo




On Mon, Mar 4, 2013 at 8:28 AM, Jeff Yuan <qu...@gmail.com> wrote:

> I'm calling Pig (with entry point from Main.java) from another Java
> program. When it runs, I get a ton of warnings like these:
>
> 13/03/03 15:38:55 WARN conf.Configuration: dfs.max.objects is
> deprecated. Instead, use dfs.namenode.max.objects
> 13/03/03 15:38:55 WARN conf.Configuration: mapred.task.id is
> deprecated. Instead, use mapreduce.task.attempt.id
>
> My question is, am I missing a step to load the default
> configuration/properties? Or does Pig always show a lot of these
> similar warnings? Any way to hide these during execution or to
> otherwise?
>
> Thanks.
>