You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Bertrand Dechoux <de...@gmail.com> on 2013/05/13 11:25:12 UTC
Disable ColumnMapKeyPrune rule with PigTest
Hi,
I am using PigTest in order to verify a script reading and storing data in
avro format.
However, at the moment, the script fails due to the optimisation rule
ColumnMapKeyPrune.
I known I can disable it using the -optimizer_off flag. But is there a way
to do that using PigTest?
It seems to me that the option is only for the command line, that there is
no way to provide it with PigTest and that there is no way to turn off the
rule from the script itself.
Could someone confirm that?
Regards
Bertrand
PS : I am using a Cloudera 0.10 pig version but the question is not
specific to Cloudera.
Re: Disable ColumnMapKeyPrune rule with PigTest
Posted by Bertrand Dechoux <de...@gmail.com>.
Thanks for the information!
Bertrand
On Mon, May 13, 2013 at 6:27 PM, Cheolsoo Park <pi...@gmail.com> wrote:
> Hi Bertand,
>
> I don't think there is a way to do it now.
>
> But PIG-3317 will allow you to disable optimization rules via Pig
> properties:
> https://issues.apache.org/jira/browse/PIG-3317
>
> So you will be able to set the properties in PigContext and pass it to
> PigServer.
>
> The patch is not committed yet, but it's likely to be in next release.
>
> Thanks,
> Cheolsoo
>
>
>
> On Mon, May 13, 2013 at 2:25 AM, Bertrand Dechoux <dechouxb@gmail.com
> >wrote:
>
> > Hi,
> >
> > I am using PigTest in order to verify a script reading and storing data
> in
> > avro format.
> > However, at the moment, the script fails due to the optimisation rule
> > ColumnMapKeyPrune.
> > I known I can disable it using the -optimizer_off flag. But is there a
> way
> > to do that using PigTest?
> >
> > It seems to me that the option is only for the command line, that there
> is
> > no way to provide it with PigTest and that there is no way to turn off
> the
> > rule from the script itself.
> >
> > Could someone confirm that?
> >
> > Regards
> >
> > Bertrand
> >
> > PS : I am using a Cloudera 0.10 pig version but the question is not
> > specific to Cloudera.
> >
>
Re: Disable ColumnMapKeyPrune rule with PigTest
Posted by Cheolsoo Park <pi...@gmail.com>.
Hi Bertand,
I don't think there is a way to do it now.
But PIG-3317 will allow you to disable optimization rules via Pig
properties:
https://issues.apache.org/jira/browse/PIG-3317
So you will be able to set the properties in PigContext and pass it to
PigServer.
The patch is not committed yet, but it's likely to be in next release.
Thanks,
Cheolsoo
On Mon, May 13, 2013 at 2:25 AM, Bertrand Dechoux <de...@gmail.com>wrote:
> Hi,
>
> I am using PigTest in order to verify a script reading and storing data in
> avro format.
> However, at the moment, the script fails due to the optimisation rule
> ColumnMapKeyPrune.
> I known I can disable it using the -optimizer_off flag. But is there a way
> to do that using PigTest?
>
> It seems to me that the option is only for the command line, that there is
> no way to provide it with PigTest and that there is no way to turn off the
> rule from the script itself.
>
> Could someone confirm that?
>
> Regards
>
> Bertrand
>
> PS : I am using a Cloudera 0.10 pig version but the question is not
> specific to Cloudera.
>