You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by James Newhaven <ja...@gmail.com> on 2012/04/10 22:33:01 UTC
LIMIT operator doesn't work with variables
Hi,
I am trying to a limit the output size using LIMIT. I want to the limit
size to be 5 percent of the total output size like this:
-- Put all the inids in a bag so we can count them.
G = GROUP F ALL;
-- Count everything in the bag
H = FOREACH G GENERATE COUNT_STAR(F) AS total;
-- Limit out to 5 percent of the total.
I = LIMIT F H.total * 0.05;
However, when I define the last line with the LIMIT operator, I get the
following error. Does anyone know a way around this?
ERROR 1200: <line 362, column 12> mismatched input 'H' expecting set null
Failed to parse: <line 362, column 12> mismatched input 'H' expecting set
null
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:222)
Thanks,
James
Re: LIMIT operator doesn't work with variables
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Yeah, that would work.
10 should be released within weeks.
D
On Tue, Apr 10, 2012 at 2:42 PM, James Newhaven
<ja...@gmail.com> wrote:
> Thanks for the responses. Until 0.10 is released what alternatives do
> I have if limit can only take constants?
>
> I suspect I could use TOP if that supports variables?
>
>
>
> On 10 Apr 2012, at 10:20 PM, Dmitriy Ryaboy <dv...@gmail.com> wrote:
>
>> Fixed in 0.10 actually
>>
>> https://issues.apache.org/jira/browse/PIG-1926
>>
>> But if you are using the scalar feature, you should cast explicitly.
>>
>> D
>>
>> On Tue, Apr 10, 2012 at 2:11 PM, Stan Rosenberg
>> <st...@gmail.com> wrote:
>>> I believe the syntax of LIMIT does not admit an arbitrary expression;
>>> it only admits constants. At least this is what the documentation
>>> says.
>>>
>>> stan
>>>
>>>
>>> On Tue, Apr 10, 2012 at 4:33 PM, James Newhaven
>>> <ja...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> I am trying to a limit the output size using LIMIT. I want to the limit
>>>> size to be 5 percent of the total output size like this:
>>>>
>>>> -- Put all the inids in a bag so we can count them.
>>>> G = GROUP F ALL;
>>>>
>>>> -- Count everything in the bag
>>>> H = FOREACH G GENERATE COUNT_STAR(F) AS total;
>>>>
>>>> -- Limit out to 5 percent of the total.
>>>> I = LIMIT F H.total * 0.05;
>>>>
>>>> However, when I define the last line with the LIMIT operator, I get the
>>>> following error. Does anyone know a way around this?
>>>>
>>>> ERROR 1200: <line 362, column 12> mismatched input 'H' expecting set null
>>>>
>>>> Failed to parse: <line 362, column 12> mismatched input 'H' expecting set
>>>> null
>>>> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:222)
>>>>
>>>>
>>>> Thanks,
>>>> James
Re: LIMIT operator doesn't work with variables
Posted by James Newhaven <ja...@gmail.com>.
Thanks for the responses. Until 0.10 is released what alternatives do
I have if limit can only take constants?
I suspect I could use TOP if that supports variables?
On 10 Apr 2012, at 10:20 PM, Dmitriy Ryaboy <dv...@gmail.com> wrote:
> Fixed in 0.10 actually
>
> https://issues.apache.org/jira/browse/PIG-1926
>
> But if you are using the scalar feature, you should cast explicitly.
>
> D
>
> On Tue, Apr 10, 2012 at 2:11 PM, Stan Rosenberg
> <st...@gmail.com> wrote:
>> I believe the syntax of LIMIT does not admit an arbitrary expression;
>> it only admits constants. At least this is what the documentation
>> says.
>>
>> stan
>>
>>
>> On Tue, Apr 10, 2012 at 4:33 PM, James Newhaven
>> <ja...@gmail.com> wrote:
>>> Hi,
>>>
>>> I am trying to a limit the output size using LIMIT. I want to the limit
>>> size to be 5 percent of the total output size like this:
>>>
>>> -- Put all the inids in a bag so we can count them.
>>> G = GROUP F ALL;
>>>
>>> -- Count everything in the bag
>>> H = FOREACH G GENERATE COUNT_STAR(F) AS total;
>>>
>>> -- Limit out to 5 percent of the total.
>>> I = LIMIT F H.total * 0.05;
>>>
>>> However, when I define the last line with the LIMIT operator, I get the
>>> following error. Does anyone know a way around this?
>>>
>>> ERROR 1200: <line 362, column 12> mismatched input 'H' expecting set null
>>>
>>> Failed to parse: <line 362, column 12> mismatched input 'H' expecting set
>>> null
>>> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:222)
>>>
>>>
>>> Thanks,
>>> James
Re: LIMIT operator doesn't work with variables
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Fixed in 0.10 actually
https://issues.apache.org/jira/browse/PIG-1926
But if you are using the scalar feature, you should cast explicitly.
D
On Tue, Apr 10, 2012 at 2:11 PM, Stan Rosenberg
<st...@gmail.com> wrote:
> I believe the syntax of LIMIT does not admit an arbitrary expression;
> it only admits constants. At least this is what the documentation
> says.
>
> stan
>
>
> On Tue, Apr 10, 2012 at 4:33 PM, James Newhaven
> <ja...@gmail.com> wrote:
>> Hi,
>>
>> I am trying to a limit the output size using LIMIT. I want to the limit
>> size to be 5 percent of the total output size like this:
>>
>> -- Put all the inids in a bag so we can count them.
>> G = GROUP F ALL;
>>
>> -- Count everything in the bag
>> H = FOREACH G GENERATE COUNT_STAR(F) AS total;
>>
>> -- Limit out to 5 percent of the total.
>> I = LIMIT F H.total * 0.05;
>>
>> However, when I define the last line with the LIMIT operator, I get the
>> following error. Does anyone know a way around this?
>>
>> ERROR 1200: <line 362, column 12> mismatched input 'H' expecting set null
>>
>> Failed to parse: <line 362, column 12> mismatched input 'H' expecting set
>> null
>> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:222)
>>
>>
>> Thanks,
>> James
Re: LIMIT operator doesn't work with variables
Posted by Stan Rosenberg <st...@gmail.com>.
I believe the syntax of LIMIT does not admit an arbitrary expression;
it only admits constants. At least this is what the documentation
says.
stan
On Tue, Apr 10, 2012 at 4:33 PM, James Newhaven
<ja...@gmail.com> wrote:
> Hi,
>
> I am trying to a limit the output size using LIMIT. I want to the limit
> size to be 5 percent of the total output size like this:
>
> -- Put all the inids in a bag so we can count them.
> G = GROUP F ALL;
>
> -- Count everything in the bag
> H = FOREACH G GENERATE COUNT_STAR(F) AS total;
>
> -- Limit out to 5 percent of the total.
> I = LIMIT F H.total * 0.05;
>
> However, when I define the last line with the LIMIT operator, I get the
> following error. Does anyone know a way around this?
>
> ERROR 1200: <line 362, column 12> mismatched input 'H' expecting set null
>
> Failed to parse: <line 362, column 12> mismatched input 'H' expecting set
> null
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:222)
>
>
> Thanks,
> James