You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Vivek Padmanabhan (JIRA)" <ji...@apache.org> on 2011/04/08 09:25:05 UTC
[jira] [Created] (PIG-1979) New logical plan failing with ERROR
2229: Couldn't find matching uid -1
New logical plan failing with ERROR 2229: Couldn't find matching uid -1
------------------------------------------------------------------------
Key: PIG-1979
URL: https://issues.apache.org/jira/browse/PIG-1979
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.8.0, 0.9.0
Reporter: Vivek Padmanabhan
The below is my script
{code}
register myudf.jar;
c01 = LOAD 'input' USING org.test.MyTableLoader('');
c02 = FILTER c01 BY result == 'OK' AND formatted IS NOT NULL AND formatted != '' ;
c03 = FOREACH c02 GENERATE url, formatted, FLATTEN(usage);
c04 = FOREACH c03 GENERATE usage::domain AS domain, url, formatted;
doc_001 = FOREACH c04 GENERATE domain,url, FLATTEN(MyExtractor(formatted)) AS category;
doc_004_1 = GROUP doc_001 BY (domain,url);
doc_005 = FOREACH doc_004_1 GENERATE group.domain as domain, group.url as url, doc_001.category as category;
STORE doc_005 INTO 'out_final' USING PigStorage();
review1 = FOREACH c04 GENERATE domain,url, MyExtractor(formatted) AS rev;
review2 = FILTER review1 BY SIZE(rev)>0;
joinresult = JOIN review2 by (domain,url), doc_005 by (domain,url);
finalresult = FOREACH joinresult GENERATE doc_005::category;
STORE finalresult INTO 'out_final' using PigStorage();
{code}
The script is failing in building the plan, while applying for logical optimization rule for AddForEach.
ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2229: Couldn't find matching uid -1 for project (Name: Project Type: bytearray Uid: 106 Input: 0 Column: 5)
The problem is happening when I try to include doc_005::category in the projection for relation finalresult. This is field is orginated from the udf org.vivek.udfs.MyExtractor (source given below).
{code}
import java.io.IOException;
import org.apache.pig.EvalFunc;
import org.apache.pig.data.*;
import org.apache.pig.impl.logicalLayer.FrontendException;
import org.apache.pig.impl.logicalLayer.schema.Schema;
import org.apache.pig.impl.logicalLayer.schema.Schema.FieldSchema;
public class MyExtractor extends EvalFunc<DataBag>
{
@Override
public Schema outputSchema(Schema arg0) {
try {
return Schema.generateNestedSchema(DataType.BAG, DataType.CHARARRAY);
} catch (FrontendException e) {
System.err.println("Error while generating schema. "+e);
return new Schema(new FieldSchema(null, DataType.BAG));
}
}
@Override
public DataBag exec(Tuple inputTuple)
throws IOException
{
try {
Tuple tp2 = TupleFactory.getInstance().newTuple(1);
tp2.set(0, (inputTuple.get(0).toString()+inputTuple.hashCode()));
DataBag retBag = BagFactory.getInstance().newDefaultBag();
retBag.add(tp2);
return retBag;
}
catch (Exception e) {
throw new IOException(" Caught exception", e);
}
}
}
{code}
The script goes through fine if I disable AddForEach rule by -t AddForEach
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-1979) New logical plan failing with ERROR
2229: Couldn't find matching uid -1
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1979:
----------------------------
Attachment: PIG-1979-1-trunk.patch
PIG-1979-1-trunk.patch is for trunk.
> New logical plan failing with ERROR 2229: Couldn't find matching uid -1
> ------------------------------------------------------------------------
>
> Key: PIG-1979
> URL: https://issues.apache.org/jira/browse/PIG-1979
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.8.0, 0.9.0
> Reporter: Vivek Padmanabhan
> Assignee: Daniel Dai
> Fix For: 0.8.0
>
> Attachments: PIG-1979-1-trunk.patch, PIG-1979-1.patch
>
>
> The below is my script
> {code}
> register myudf.jar;
> c01 = LOAD 'input' USING org.test.MyTableLoader('');
> c02 = FILTER c01 BY result == 'OK' AND formatted IS NOT NULL AND formatted != '' ;
> c03 = FOREACH c02 GENERATE url, formatted, FLATTEN(usage);
> c04 = FOREACH c03 GENERATE usage::domain AS domain, url, formatted;
> doc_001 = FOREACH c04 GENERATE domain,url, FLATTEN(MyExtractor(formatted)) AS category;
> doc_004_1 = GROUP doc_001 BY (domain,url);
> doc_005 = FOREACH doc_004_1 GENERATE group.domain as domain, group.url as url, doc_001.category as category;
> STORE doc_005 INTO 'out_final' USING PigStorage();
> review1 = FOREACH c04 GENERATE domain,url, MyExtractor(formatted) AS rev;
> review2 = FILTER review1 BY SIZE(rev)>0;
> joinresult = JOIN review2 by (domain,url), doc_005 by (domain,url);
> finalresult = FOREACH joinresult GENERATE doc_005::category;
> STORE finalresult INTO 'out_final' using PigStorage();
> {code}
> The script is failing in building the plan, while applying for logical optimization rule for AddForEach.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2229: Couldn't find matching uid -1 for project (Name: Project Type: bytearray Uid: 106 Input: 0 Column: 5)
> The problem is happening when I try to include doc_005::category in the projection for relation finalresult. This is field is orginated from the udf org.vivek.udfs.MyExtractor (source given below).
> {code}
> import java.io.IOException;
> import org.apache.pig.EvalFunc;
> import org.apache.pig.data.*;
> import org.apache.pig.impl.logicalLayer.FrontendException;
> import org.apache.pig.impl.logicalLayer.schema.Schema;
> import org.apache.pig.impl.logicalLayer.schema.Schema.FieldSchema;
> public class MyExtractor extends EvalFunc<DataBag>
> {
> @Override
> public Schema outputSchema(Schema arg0) {
> try {
> return Schema.generateNestedSchema(DataType.BAG, DataType.CHARARRAY);
> } catch (FrontendException e) {
> System.err.println("Error while generating schema. "+e);
> return new Schema(new FieldSchema(null, DataType.BAG));
> }
> }
> @Override
> public DataBag exec(Tuple inputTuple)
> throws IOException
> {
> try {
> Tuple tp2 = TupleFactory.getInstance().newTuple(1);
> tp2.set(0, (inputTuple.get(0).toString()+inputTuple.hashCode()));
> DataBag retBag = BagFactory.getInstance().newDefaultBag();
> retBag.add(tp2);
> return retBag;
> }
> catch (Exception e) {
> throw new IOException(" Caught exception", e);
> }
> }
> }
> {code}
> The script goes through fine if I disable AddForEach rule by -t AddForEach
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-1979) New logical plan failing with ERROR
2229: Couldn't find matching uid -1
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13019129#comment-13019129 ]
jiraposter@reviews.apache.org commented on PIG-1979:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/587/
-----------------------------------------------------------
Review request for pig and thejas.
Summary
-------
See PIG-1979
This addresses bug PIG-1979.
https://issues.apache.org/jira/browse/PIG-1979
Diffs
-----
http://svn.apache.org/repos/asf/pig/branches/branch-0.8/src/org/apache/pig/newplan/logical/expression/DereferenceExpression.java 1091122
http://svn.apache.org/repos/asf/pig/branches/branch-0.8/test/org/apache/pig/test/TestEvalPipeline2.java 1091122
Diff: https://reviews.apache.org/r/587/diff
Testing
-------
Test-patch:
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
Unit-test:
all pass
Thanks,
Daniel
> New logical plan failing with ERROR 2229: Couldn't find matching uid -1
> ------------------------------------------------------------------------
>
> Key: PIG-1979
> URL: https://issues.apache.org/jira/browse/PIG-1979
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.8.0, 0.9.0
> Reporter: Vivek Padmanabhan
> Assignee: Daniel Dai
> Fix For: 0.9.0
>
> Attachments: PIG-1979-1.patch
>
>
> The below is my script
> {code}
> register myudf.jar;
> c01 = LOAD 'input' USING org.test.MyTableLoader('');
> c02 = FILTER c01 BY result == 'OK' AND formatted IS NOT NULL AND formatted != '' ;
> c03 = FOREACH c02 GENERATE url, formatted, FLATTEN(usage);
> c04 = FOREACH c03 GENERATE usage::domain AS domain, url, formatted;
> doc_001 = FOREACH c04 GENERATE domain,url, FLATTEN(MyExtractor(formatted)) AS category;
> doc_004_1 = GROUP doc_001 BY (domain,url);
> doc_005 = FOREACH doc_004_1 GENERATE group.domain as domain, group.url as url, doc_001.category as category;
> STORE doc_005 INTO 'out_final' USING PigStorage();
> review1 = FOREACH c04 GENERATE domain,url, MyExtractor(formatted) AS rev;
> review2 = FILTER review1 BY SIZE(rev)>0;
> joinresult = JOIN review2 by (domain,url), doc_005 by (domain,url);
> finalresult = FOREACH joinresult GENERATE doc_005::category;
> STORE finalresult INTO 'out_final' using PigStorage();
> {code}
> The script is failing in building the plan, while applying for logical optimization rule for AddForEach.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2229: Couldn't find matching uid -1 for project (Name: Project Type: bytearray Uid: 106 Input: 0 Column: 5)
> The problem is happening when I try to include doc_005::category in the projection for relation finalresult. This is field is orginated from the udf org.vivek.udfs.MyExtractor (source given below).
> {code}
> import java.io.IOException;
> import org.apache.pig.EvalFunc;
> import org.apache.pig.data.*;
> import org.apache.pig.impl.logicalLayer.FrontendException;
> import org.apache.pig.impl.logicalLayer.schema.Schema;
> import org.apache.pig.impl.logicalLayer.schema.Schema.FieldSchema;
> public class MyExtractor extends EvalFunc<DataBag>
> {
> @Override
> public Schema outputSchema(Schema arg0) {
> try {
> return Schema.generateNestedSchema(DataType.BAG, DataType.CHARARRAY);
> } catch (FrontendException e) {
> System.err.println("Error while generating schema. "+e);
> return new Schema(new FieldSchema(null, DataType.BAG));
> }
> }
> @Override
> public DataBag exec(Tuple inputTuple)
> throws IOException
> {
> try {
> Tuple tp2 = TupleFactory.getInstance().newTuple(1);
> tp2.set(0, (inputTuple.get(0).toString()+inputTuple.hashCode()));
> DataBag retBag = BagFactory.getInstance().newDefaultBag();
> retBag.add(tp2);
> return retBag;
> }
> catch (Exception e) {
> throw new IOException(" Caught exception", e);
> }
> }
> }
> {code}
> The script goes through fine if I disable AddForEach rule by -t AddForEach
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-1979) New logical plan failing with ERROR
2229: Couldn't find matching uid -1
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-1979:
--------------------------------
Fix Version/s: 0.9.0
Assignee: Daniel Dai
> New logical plan failing with ERROR 2229: Couldn't find matching uid -1
> ------------------------------------------------------------------------
>
> Key: PIG-1979
> URL: https://issues.apache.org/jira/browse/PIG-1979
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.8.0, 0.9.0
> Reporter: Vivek Padmanabhan
> Assignee: Daniel Dai
> Fix For: 0.9.0
>
>
> The below is my script
> {code}
> register myudf.jar;
> c01 = LOAD 'input' USING org.test.MyTableLoader('');
> c02 = FILTER c01 BY result == 'OK' AND formatted IS NOT NULL AND formatted != '' ;
> c03 = FOREACH c02 GENERATE url, formatted, FLATTEN(usage);
> c04 = FOREACH c03 GENERATE usage::domain AS domain, url, formatted;
> doc_001 = FOREACH c04 GENERATE domain,url, FLATTEN(MyExtractor(formatted)) AS category;
> doc_004_1 = GROUP doc_001 BY (domain,url);
> doc_005 = FOREACH doc_004_1 GENERATE group.domain as domain, group.url as url, doc_001.category as category;
> STORE doc_005 INTO 'out_final' USING PigStorage();
> review1 = FOREACH c04 GENERATE domain,url, MyExtractor(formatted) AS rev;
> review2 = FILTER review1 BY SIZE(rev)>0;
> joinresult = JOIN review2 by (domain,url), doc_005 by (domain,url);
> finalresult = FOREACH joinresult GENERATE doc_005::category;
> STORE finalresult INTO 'out_final' using PigStorage();
> {code}
> The script is failing in building the plan, while applying for logical optimization rule for AddForEach.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2229: Couldn't find matching uid -1 for project (Name: Project Type: bytearray Uid: 106 Input: 0 Column: 5)
> The problem is happening when I try to include doc_005::category in the projection for relation finalresult. This is field is orginated from the udf org.vivek.udfs.MyExtractor (source given below).
> {code}
> import java.io.IOException;
> import org.apache.pig.EvalFunc;
> import org.apache.pig.data.*;
> import org.apache.pig.impl.logicalLayer.FrontendException;
> import org.apache.pig.impl.logicalLayer.schema.Schema;
> import org.apache.pig.impl.logicalLayer.schema.Schema.FieldSchema;
> public class MyExtractor extends EvalFunc<DataBag>
> {
> @Override
> public Schema outputSchema(Schema arg0) {
> try {
> return Schema.generateNestedSchema(DataType.BAG, DataType.CHARARRAY);
> } catch (FrontendException e) {
> System.err.println("Error while generating schema. "+e);
> return new Schema(new FieldSchema(null, DataType.BAG));
> }
> }
> @Override
> public DataBag exec(Tuple inputTuple)
> throws IOException
> {
> try {
> Tuple tp2 = TupleFactory.getInstance().newTuple(1);
> tp2.set(0, (inputTuple.get(0).toString()+inputTuple.hashCode()));
> DataBag retBag = BagFactory.getInstance().newDefaultBag();
> retBag.add(tp2);
> return retBag;
> }
> catch (Exception e) {
> throw new IOException(" Caught exception", e);
> }
> }
> }
> {code}
> The script goes through fine if I disable AddForEach rule by -t AddForEach
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-1979) New logical plan failing with ERROR
2229: Couldn't find matching uid -1
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020039#comment-13020039 ]
jiraposter@reviews.apache.org commented on PIG-1979:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/587/#review461
-----------------------------------------------------------
Ship it!
+1
- thejas
On 2011-04-13 00:11:37, Daniel Dai wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/587/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-04-13 00:11:37)
bq.
bq.
bq. Review request for pig and thejas.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. See PIG-1979
bq.
bq.
bq. This addresses bug PIG-1979.
bq. https://issues.apache.org/jira/browse/PIG-1979
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. http://svn.apache.org/repos/asf/pig/branches/branch-0.8/src/org/apache/pig/newplan/logical/expression/DereferenceExpression.java 1091122
bq. http://svn.apache.org/repos/asf/pig/branches/branch-0.8/test/org/apache/pig/test/TestEvalPipeline2.java 1091122
bq.
bq. Diff: https://reviews.apache.org/r/587/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Test-patch:
bq. [exec] +1 overall.
bq. [exec]
bq. [exec] +1 @author. The patch does not contain any @author tags.
bq. [exec]
bq. [exec] +1 tests included. The patch appears to include 3 new or modified tests.
bq. [exec]
bq. [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
bq. [exec]
bq. [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
bq. [exec]
bq. [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
bq. [exec]
bq. [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
bq.
bq. Unit-test:
bq. all pass
bq.
bq.
bq. Thanks,
bq.
bq. Daniel
bq.
bq.
> New logical plan failing with ERROR 2229: Couldn't find matching uid -1
> ------------------------------------------------------------------------
>
> Key: PIG-1979
> URL: https://issues.apache.org/jira/browse/PIG-1979
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.8.0, 0.9.0
> Reporter: Vivek Padmanabhan
> Assignee: Daniel Dai
> Fix For: 0.8.0
>
> Attachments: PIG-1979-1-trunk.patch, PIG-1979-1.patch
>
>
> The below is my script
> {code}
> register myudf.jar;
> c01 = LOAD 'input' USING org.test.MyTableLoader('');
> c02 = FILTER c01 BY result == 'OK' AND formatted IS NOT NULL AND formatted != '' ;
> c03 = FOREACH c02 GENERATE url, formatted, FLATTEN(usage);
> c04 = FOREACH c03 GENERATE usage::domain AS domain, url, formatted;
> doc_001 = FOREACH c04 GENERATE domain,url, FLATTEN(MyExtractor(formatted)) AS category;
> doc_004_1 = GROUP doc_001 BY (domain,url);
> doc_005 = FOREACH doc_004_1 GENERATE group.domain as domain, group.url as url, doc_001.category as category;
> STORE doc_005 INTO 'out_final' USING PigStorage();
> review1 = FOREACH c04 GENERATE domain,url, MyExtractor(formatted) AS rev;
> review2 = FILTER review1 BY SIZE(rev)>0;
> joinresult = JOIN review2 by (domain,url), doc_005 by (domain,url);
> finalresult = FOREACH joinresult GENERATE doc_005::category;
> STORE finalresult INTO 'out_final' using PigStorage();
> {code}
> The script is failing in building the plan, while applying for logical optimization rule for AddForEach.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2229: Couldn't find matching uid -1 for project (Name: Project Type: bytearray Uid: 106 Input: 0 Column: 5)
> The problem is happening when I try to include doc_005::category in the projection for relation finalresult. This is field is orginated from the udf org.vivek.udfs.MyExtractor (source given below).
> {code}
> import java.io.IOException;
> import org.apache.pig.EvalFunc;
> import org.apache.pig.data.*;
> import org.apache.pig.impl.logicalLayer.FrontendException;
> import org.apache.pig.impl.logicalLayer.schema.Schema;
> import org.apache.pig.impl.logicalLayer.schema.Schema.FieldSchema;
> public class MyExtractor extends EvalFunc<DataBag>
> {
> @Override
> public Schema outputSchema(Schema arg0) {
> try {
> return Schema.generateNestedSchema(DataType.BAG, DataType.CHARARRAY);
> } catch (FrontendException e) {
> System.err.println("Error while generating schema. "+e);
> return new Schema(new FieldSchema(null, DataType.BAG));
> }
> }
> @Override
> public DataBag exec(Tuple inputTuple)
> throws IOException
> {
> try {
> Tuple tp2 = TupleFactory.getInstance().newTuple(1);
> tp2.set(0, (inputTuple.get(0).toString()+inputTuple.hashCode()));
> DataBag retBag = BagFactory.getInstance().newDefaultBag();
> retBag.add(tp2);
> return retBag;
> }
> catch (Exception e) {
> throw new IOException(" Caught exception", e);
> }
> }
> }
> {code}
> The script goes through fine if I disable AddForEach rule by -t AddForEach
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-1979) New logical plan failing with ERROR
2229: Couldn't find matching uid -1
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1979:
----------------------------
Attachment: PIG-1979-1.patch
> New logical plan failing with ERROR 2229: Couldn't find matching uid -1
> ------------------------------------------------------------------------
>
> Key: PIG-1979
> URL: https://issues.apache.org/jira/browse/PIG-1979
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.8.0, 0.9.0
> Reporter: Vivek Padmanabhan
> Assignee: Daniel Dai
> Fix For: 0.9.0
>
> Attachments: PIG-1979-1.patch
>
>
> The below is my script
> {code}
> register myudf.jar;
> c01 = LOAD 'input' USING org.test.MyTableLoader('');
> c02 = FILTER c01 BY result == 'OK' AND formatted IS NOT NULL AND formatted != '' ;
> c03 = FOREACH c02 GENERATE url, formatted, FLATTEN(usage);
> c04 = FOREACH c03 GENERATE usage::domain AS domain, url, formatted;
> doc_001 = FOREACH c04 GENERATE domain,url, FLATTEN(MyExtractor(formatted)) AS category;
> doc_004_1 = GROUP doc_001 BY (domain,url);
> doc_005 = FOREACH doc_004_1 GENERATE group.domain as domain, group.url as url, doc_001.category as category;
> STORE doc_005 INTO 'out_final' USING PigStorage();
> review1 = FOREACH c04 GENERATE domain,url, MyExtractor(formatted) AS rev;
> review2 = FILTER review1 BY SIZE(rev)>0;
> joinresult = JOIN review2 by (domain,url), doc_005 by (domain,url);
> finalresult = FOREACH joinresult GENERATE doc_005::category;
> STORE finalresult INTO 'out_final' using PigStorage();
> {code}
> The script is failing in building the plan, while applying for logical optimization rule for AddForEach.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2229: Couldn't find matching uid -1 for project (Name: Project Type: bytearray Uid: 106 Input: 0 Column: 5)
> The problem is happening when I try to include doc_005::category in the projection for relation finalresult. This is field is orginated from the udf org.vivek.udfs.MyExtractor (source given below).
> {code}
> import java.io.IOException;
> import org.apache.pig.EvalFunc;
> import org.apache.pig.data.*;
> import org.apache.pig.impl.logicalLayer.FrontendException;
> import org.apache.pig.impl.logicalLayer.schema.Schema;
> import org.apache.pig.impl.logicalLayer.schema.Schema.FieldSchema;
> public class MyExtractor extends EvalFunc<DataBag>
> {
> @Override
> public Schema outputSchema(Schema arg0) {
> try {
> return Schema.generateNestedSchema(DataType.BAG, DataType.CHARARRAY);
> } catch (FrontendException e) {
> System.err.println("Error while generating schema. "+e);
> return new Schema(new FieldSchema(null, DataType.BAG));
> }
> }
> @Override
> public DataBag exec(Tuple inputTuple)
> throws IOException
> {
> try {
> Tuple tp2 = TupleFactory.getInstance().newTuple(1);
> tp2.set(0, (inputTuple.get(0).toString()+inputTuple.hashCode()));
> DataBag retBag = BagFactory.getInstance().newDefaultBag();
> retBag.add(tp2);
> return retBag;
> }
> catch (Exception e) {
> throw new IOException(" Caught exception", e);
> }
> }
> }
> {code}
> The script goes through fine if I disable AddForEach rule by -t AddForEach
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-1979) New logical plan failing with ERROR
2229: Couldn't find matching uid -1
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-1979:
--------------------------------
Fix Version/s: (was: 0.9.0)
0.8.0
> New logical plan failing with ERROR 2229: Couldn't find matching uid -1
> ------------------------------------------------------------------------
>
> Key: PIG-1979
> URL: https://issues.apache.org/jira/browse/PIG-1979
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.8.0, 0.9.0
> Reporter: Vivek Padmanabhan
> Assignee: Daniel Dai
> Fix For: 0.8.0
>
> Attachments: PIG-1979-1.patch
>
>
> The below is my script
> {code}
> register myudf.jar;
> c01 = LOAD 'input' USING org.test.MyTableLoader('');
> c02 = FILTER c01 BY result == 'OK' AND formatted IS NOT NULL AND formatted != '' ;
> c03 = FOREACH c02 GENERATE url, formatted, FLATTEN(usage);
> c04 = FOREACH c03 GENERATE usage::domain AS domain, url, formatted;
> doc_001 = FOREACH c04 GENERATE domain,url, FLATTEN(MyExtractor(formatted)) AS category;
> doc_004_1 = GROUP doc_001 BY (domain,url);
> doc_005 = FOREACH doc_004_1 GENERATE group.domain as domain, group.url as url, doc_001.category as category;
> STORE doc_005 INTO 'out_final' USING PigStorage();
> review1 = FOREACH c04 GENERATE domain,url, MyExtractor(formatted) AS rev;
> review2 = FILTER review1 BY SIZE(rev)>0;
> joinresult = JOIN review2 by (domain,url), doc_005 by (domain,url);
> finalresult = FOREACH joinresult GENERATE doc_005::category;
> STORE finalresult INTO 'out_final' using PigStorage();
> {code}
> The script is failing in building the plan, while applying for logical optimization rule for AddForEach.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2229: Couldn't find matching uid -1 for project (Name: Project Type: bytearray Uid: 106 Input: 0 Column: 5)
> The problem is happening when I try to include doc_005::category in the projection for relation finalresult. This is field is orginated from the udf org.vivek.udfs.MyExtractor (source given below).
> {code}
> import java.io.IOException;
> import org.apache.pig.EvalFunc;
> import org.apache.pig.data.*;
> import org.apache.pig.impl.logicalLayer.FrontendException;
> import org.apache.pig.impl.logicalLayer.schema.Schema;
> import org.apache.pig.impl.logicalLayer.schema.Schema.FieldSchema;
> public class MyExtractor extends EvalFunc<DataBag>
> {
> @Override
> public Schema outputSchema(Schema arg0) {
> try {
> return Schema.generateNestedSchema(DataType.BAG, DataType.CHARARRAY);
> } catch (FrontendException e) {
> System.err.println("Error while generating schema. "+e);
> return new Schema(new FieldSchema(null, DataType.BAG));
> }
> }
> @Override
> public DataBag exec(Tuple inputTuple)
> throws IOException
> {
> try {
> Tuple tp2 = TupleFactory.getInstance().newTuple(1);
> tp2.set(0, (inputTuple.get(0).toString()+inputTuple.hashCode()));
> DataBag retBag = BagFactory.getInstance().newDefaultBag();
> retBag.add(tp2);
> return retBag;
> }
> catch (Exception e) {
> throw new IOException(" Caught exception", e);
> }
> }
> }
> {code}
> The script goes through fine if I disable AddForEach rule by -t AddForEach
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (PIG-1979) New logical plan failing with ERROR
2229: Couldn't find matching uid -1
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai resolved PIG-1979.
-----------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Patch committed to both trunk and 0.8 branch.
> New logical plan failing with ERROR 2229: Couldn't find matching uid -1
> ------------------------------------------------------------------------
>
> Key: PIG-1979
> URL: https://issues.apache.org/jira/browse/PIG-1979
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.8.0, 0.9.0
> Reporter: Vivek Padmanabhan
> Assignee: Daniel Dai
> Fix For: 0.8.0
>
> Attachments: PIG-1979-1-trunk.patch, PIG-1979-1.patch
>
>
> The below is my script
> {code}
> register myudf.jar;
> c01 = LOAD 'input' USING org.test.MyTableLoader('');
> c02 = FILTER c01 BY result == 'OK' AND formatted IS NOT NULL AND formatted != '' ;
> c03 = FOREACH c02 GENERATE url, formatted, FLATTEN(usage);
> c04 = FOREACH c03 GENERATE usage::domain AS domain, url, formatted;
> doc_001 = FOREACH c04 GENERATE domain,url, FLATTEN(MyExtractor(formatted)) AS category;
> doc_004_1 = GROUP doc_001 BY (domain,url);
> doc_005 = FOREACH doc_004_1 GENERATE group.domain as domain, group.url as url, doc_001.category as category;
> STORE doc_005 INTO 'out_final' USING PigStorage();
> review1 = FOREACH c04 GENERATE domain,url, MyExtractor(formatted) AS rev;
> review2 = FILTER review1 BY SIZE(rev)>0;
> joinresult = JOIN review2 by (domain,url), doc_005 by (domain,url);
> finalresult = FOREACH joinresult GENERATE doc_005::category;
> STORE finalresult INTO 'out_final' using PigStorage();
> {code}
> The script is failing in building the plan, while applying for logical optimization rule for AddForEach.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2229: Couldn't find matching uid -1 for project (Name: Project Type: bytearray Uid: 106 Input: 0 Column: 5)
> The problem is happening when I try to include doc_005::category in the projection for relation finalresult. This is field is orginated from the udf org.vivek.udfs.MyExtractor (source given below).
> {code}
> import java.io.IOException;
> import org.apache.pig.EvalFunc;
> import org.apache.pig.data.*;
> import org.apache.pig.impl.logicalLayer.FrontendException;
> import org.apache.pig.impl.logicalLayer.schema.Schema;
> import org.apache.pig.impl.logicalLayer.schema.Schema.FieldSchema;
> public class MyExtractor extends EvalFunc<DataBag>
> {
> @Override
> public Schema outputSchema(Schema arg0) {
> try {
> return Schema.generateNestedSchema(DataType.BAG, DataType.CHARARRAY);
> } catch (FrontendException e) {
> System.err.println("Error while generating schema. "+e);
> return new Schema(new FieldSchema(null, DataType.BAG));
> }
> }
> @Override
> public DataBag exec(Tuple inputTuple)
> throws IOException
> {
> try {
> Tuple tp2 = TupleFactory.getInstance().newTuple(1);
> tp2.set(0, (inputTuple.get(0).toString()+inputTuple.hashCode()));
> DataBag retBag = BagFactory.getInstance().newDefaultBag();
> retBag.add(tp2);
> return retBag;
> }
> catch (Exception e) {
> throw new IOException(" Caught exception", e);
> }
> }
> }
> {code}
> The script goes through fine if I disable AddForEach rule by -t AddForEach
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira