You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Renato Marroquín Mogrovejo <re...@gmail.com> on 2010/10/01 01:24:03 UTC
Re: how to use option mapred.max.map.failures.percent ?
Hi all, I'm also having a similar problem, does anyone have a
suggestion/opinion/thought?
The thing is that my MR jobs are dying because of java heap space error (I
am planning to get some more). I want a way to control this, I mean this
parameter would also help me handling this memory.
Renato M.
2010/9/28 Wojciech Langiewicz <wl...@gmail.com>
> Hello,
> I'm running very big MR job with Pig, and sometimes some maps fail, but I
> would like this job to finish anyway. I know that option
> "mapred.max.map.failures.percent" is what I need, but how to pass this
> option when running Pig? (from command line or interpreter)
>
> Thanks for help.
>
> --
> Wojciech Langiewicz
>
Re: how to use option mapred.max.map.failures.percent ?
Posted by Wojciech Langiewicz <wl...@gmail.com>.
Thanks, this option solves my problem:)
2010/10/1 Thejas M Nair <te...@yahoo-inc.com>
>
> To set heap size you can set the java property when you invoke pig-
> -Dmapred.child.java.opts=-Xmx2048m
>
> This should work for mapred.max.map.failures.percent as well.
>
> -Thejas
>
>
> On 9/30/10 4:24 PM, "Renato Marroquín Mogrovejo" <
> renatoj.marroquin@gmail.com> wrote:
>
> Hi all, I'm also having a similar problem, does anyone have a
> suggestion/opinion/thought?
> The thing is that my MR jobs are dying because of java heap space error (I
> am planning to get some more). I want a way to control this, I mean this
> parameter would also help me handling this memory.
>
>
> Renato M.
>
>
> 2010/9/28 Wojciech Langiewicz <wl...@gmail.com>
>
> > Hello,
> > I'm running very big MR job with Pig, and sometimes some maps fail, but I
> > would like this job to finish anyway. I know that option
> > "mapred.max.map.failures.percent" is what I need, but how to pass this
> > option when running Pig? (from command line or interpreter)
> >
> > Thanks for help.
> >
> > --
> > Wojciech Langiewicz
> >
>
>
>
--
Wojciech Langiewicz
Re: how to use option mapred.max.map.failures.percent ?
Posted by Thejas M Nair <te...@yahoo-inc.com>.
To set heap size you can set the java property when you invoke pig-
-Dmapred.child.java.opts=-Xmx2048m
This should work for mapred.max.map.failures.percent as well.
-Thejas
On 9/30/10 4:24 PM, "Renato Marroquín Mogrovejo" <re...@gmail.com> wrote:
Hi all, I'm also having a similar problem, does anyone have a
suggestion/opinion/thought?
The thing is that my MR jobs are dying because of java heap space error (I
am planning to get some more). I want a way to control this, I mean this
parameter would also help me handling this memory.
Renato M.
2010/9/28 Wojciech Langiewicz <wl...@gmail.com>
> Hello,
> I'm running very big MR job with Pig, and sometimes some maps fail, but I
> would like this job to finish anyway. I know that option
> "mapred.max.map.failures.percent" is what I need, but how to pass this
> option when running Pig? (from command line or interpreter)
>
> Thanks for help.
>
> --
> Wojciech Langiewicz
>