You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Rusty Klophaus <rk...@gmail.com> on 2010/02/07 18:02:25 UTC
Error 1026 when FILTER then ORDER inner bag?
Hello,
I am trying to FILTER and then ORDER an inner bag, for example:
A = LOAD ...blah... AS (first, last, age, kids:bag{kid:tuple(name, age)});
B = FOREACH A {
filteredkids = FILTER kids BY age != 1;
sortedkids = ORDER filteredkids BY age;
GENERATE sortedkids;
}
DUMP B;
When I run this, I get a stacktrace with a root error of:
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1026:
Attempt to fetch field 1 from schema of size 1
at org.apache.pig.impl.logicalLayer.schema.Schema.getField(Schema.java:749)
at
org.apache.pig.impl.logicalLayer.LOProject.getFieldSchema(LOProject.java:289)
$ pig --version
Apache Pig version 0.5.0 (r829623)
compiled Oct 25 2009, 18:58:38
If I run just the FILTER or ORDER operation in the inner bag, it works. But
when I put them both in there, it fails.
Has anybody else experienced this problem? Am I doing something wrong?
Thanks,
Rusty Klophaus
Re: Error 1026 when FILTER then ORDER inner bag?
Posted by Alan Gates <ga...@yahoo-inc.com>.
I wonder if the parser is getting confused between the inner and outer
'age' fields. What happens if you switch the name of one of them?
Alan.
On Feb 7, 2010, at 9:02 AM, Rusty Klophaus wrote:
> Hello,
>
> I am trying to FILTER and then ORDER an inner bag, for example:
>
> A = LOAD ...blah... AS (first, last, age, kids:bag{kid:tuple(name,
> age)});
> B = FOREACH A {
> filteredkids = FILTER kids BY age != 1;
> sortedkids = ORDER filteredkids BY age;
> GENERATE sortedkids;
> }
> DUMP B;
>
> When I run this, I get a stacktrace with a root error of:
>
> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR
> 1026:
> Attempt to fetch field 1 from schema of size 1
> at
> org.apache.pig.impl.logicalLayer.schema.Schema.getField(Schema.java:
> 749)
> at
> org
> .apache
> .pig.impl.logicalLayer.LOProject.getFieldSchema(LOProject.java:289)
>
> $ pig --version
> Apache Pig version 0.5.0 (r829623)
> compiled Oct 25 2009, 18:58:38
>
> If I run just the FILTER or ORDER operation in the inner bag, it
> works. But
> when I put them both in there, it fails.
>
> Has anybody else experienced this problem? Am I doing something wrong?
>
> Thanks,
> Rusty Klophaus
Re: Error 1026 when FILTER then ORDER inner bag?
Posted by Rusty Klophaus <rk...@gmail.com>.
Hi,
Thanks for your help and suggestions.
Unfortunately, the problem happens during the parsing stage, before any
steps are run. Something is going wrong when Pig tries to turn the query
into a LogicalPlan.
To see for yourself, try running the query below in Grunt. Doesn't matter if
you are running in "-x local" or "-x mapreduce" mode:
A = LOAD 'ignored' AS (ignored, kids:bag{kid:tuple(name, age)});
B = FOREACH A {
filteredkids = FILTER kids BY age != 1;
sortedkids = ORDER filteredkids BY age;
GENERATE sortedkids;
}
DUMP B;
You should see:
2010-02-09 11:54:40,065 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1026: Attempt to fetch field 1 from schema of size 1
When I split up the foreach block it seems to work correctly, leading me to
believe that this is a bug, just want someone to verify as I'm relatively
new to Pig:
A = LOAD 'ignored' AS (ignored, kids:bag{kid:tuple(name, age)});
B = FOREACH A {
filteredkids = FILTER kids BY age != 1;
GENERATE $0, filteredkids;
}
C = FOREACH B {
sortedkids = ORDER $1 BY age;
GENERATE $0, sortedkids;
}
DUMP C;
Best,
Rusty
On Sun, Feb 7, 2010 at 11:00 PM, Rekha Joshi <re...@yahoo-inc.com> wrote:
> What might possibly be happening is you get empty bags after filtering.Try
> filtering them out before trying the order.Try checking the describe/dump of
> filtered data.
> And at what stage are you meaning which 'age' element is not very clear
> here.
>
> Cheers,
> /R
>
> On 2/7/10 10:32 PM, "Rusty Klophaus" <rk...@gmail.com> wrote:
>
> Hello,
>
> I am trying to FILTER and then ORDER an inner bag, for example:
>
> A = LOAD ...blah... AS (first, last, age, kids:bag{kid:tuple(name, age)});
> B = FOREACH A {
> filteredkids = FILTER kids BY age != 1;
> sortedkids = ORDER filteredkids BY age;
> GENERATE sortedkids;
> }
> DUMP B;
>
> When I run this, I get a stacktrace with a root error of:
>
> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1026:
> Attempt to fetch field 1 from schema of size 1
> at org.apache.pig.impl.logicalLayer.schema.Schema.getField(Schema.java:749)
> at
>
> org.apache.pig.impl.logicalLayer.LOProject.getFieldSchema(LOProject.java:289)
>
> $ pig --version
> Apache Pig version 0.5.0 (r829623)
> compiled Oct 25 2009, 18:58:38
>
> If I run just the FILTER or ORDER operation in the inner bag, it works. But
> when I put them both in there, it fails.
>
> Has anybody else experienced this problem? Am I doing something wrong?
>
> Thanks,
> Rusty Klophaus
>
>
Re: Error 1026 when FILTER then ORDER inner bag?
Posted by Rekha Joshi <re...@yahoo-inc.com>.
What might possibly be happening is you get empty bags after filtering.Try filtering them out before trying the order.Try checking the describe/dump of filtered data.
And at what stage are you meaning which 'age' element is not very clear here.
Cheers,
/R
On 2/7/10 10:32 PM, "Rusty Klophaus" <rk...@gmail.com> wrote:
Hello,
I am trying to FILTER and then ORDER an inner bag, for example:
A = LOAD ...blah... AS (first, last, age, kids:bag{kid:tuple(name, age)});
B = FOREACH A {
filteredkids = FILTER kids BY age != 1;
sortedkids = ORDER filteredkids BY age;
GENERATE sortedkids;
}
DUMP B;
When I run this, I get a stacktrace with a root error of:
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1026:
Attempt to fetch field 1 from schema of size 1
at org.apache.pig.impl.logicalLayer.schema.Schema.getField(Schema.java:749)
at
org.apache.pig.impl.logicalLayer.LOProject.getFieldSchema(LOProject.java:289)
$ pig --version
Apache Pig version 0.5.0 (r829623)
compiled Oct 25 2009, 18:58:38
If I run just the FILTER or ORDER operation in the inner bag, it works. But
when I put them both in there, it fails.
Has anybody else experienced this problem? Am I doing something wrong?
Thanks,
Rusty Klophaus