You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Daniel Dai (Created) (JIRA)" <ji...@apache.org> on 2012/03/15 21:14:37 UTC
[jira] [Created] (PIG-2593) Filter by a boolean value does not work
Filter by a boolean value does not work
---------------------------------------
Key: PIG-2593
URL: https://issues.apache.org/jira/browse/PIG-2593
Project: Pig
Issue Type: Bug
Components: build
Reporter: Daniel Dai
The following script does not work:
{code}
a = load 'allscalar10k' as (name, age, gpa, instate);
b = filter a by instate;
explain b;
{code}
Exception:
ERROR 1200: <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1598)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1541)
at org.apache.pig.PigServer.registerQuery(PigServer.java:541)
at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:945)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:392)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:190)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
at org.apache.pig.Main.run(Main.java:599)
at org.apache.pig.Main.main(Main.java:153)
Caused by: Failed to parse: <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:222)
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:164)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1590)
... 9 more
It works if we change the script into:
{code}
a = load 'allscalar10k' as (name, age, gpa, instate);
b = filter a by instate==TRUE;
explain b;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2593) Filter by a boolean value does not work
Posted by "Jie Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jie Li updated PIG-2593:
------------------------
Attachment: PIG-2593.patch
Attached the patch of case 5. The patch for case 4 is in PIG-2736.
> Filter by a boolean value does not work
> ---------------------------------------
>
> Key: PIG-2593
> URL: https://issues.apache.org/jira/browse/PIG-2593
> Project: Pig
> Issue Type: Bug
> Components: build
> Reporter: Daniel Dai
> Attachments: PIG-2593.patch
>
>
> The following script does not work:
> {code}
> a = load 'allscalar10k' as (name, age, gpa, instate);
> b = filter a by instate;
> explain b;
> {code}
> Exception:
> ERROR 1200: <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1598)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1541)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:541)
> at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:945)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:392)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:190)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
> at org.apache.pig.Main.run(Main.java:599)
> at org.apache.pig.Main.main(Main.java:153)
> Caused by: Failed to parse: <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:222)
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:164)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1590)
> ... 9 more
> It works if we change the script into:
> {code}
> a = load 'allscalar10k' as (name, age, gpa, instate);
> b = filter a by instate==TRUE;
> explain b;
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2593) Filter by a boolean value does not
work
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13287649#comment-13287649 ]
Daniel Dai commented on PIG-2593:
---------------------------------
Yes, you are right. We need to fix both 4 & 5.
There exists some e2e test cases in FilterBoolean, which is disabled due to the issue. You can make use of them.
> Filter by a boolean value does not work
> ---------------------------------------
>
> Key: PIG-2593
> URL: https://issues.apache.org/jira/browse/PIG-2593
> Project: Pig
> Issue Type: Bug
> Components: build
> Reporter: Daniel Dai
>
> The following script does not work:
> {code}
> a = load 'allscalar10k' as (name, age, gpa, instate);
> b = filter a by instate;
> explain b;
> {code}
> Exception:
> ERROR 1200: <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1598)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1541)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:541)
> at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:945)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:392)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:190)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
> at org.apache.pig.Main.run(Main.java:599)
> at org.apache.pig.Main.main(Main.java:153)
> Caused by: Failed to parse: <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:222)
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:164)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1590)
> ... 9 more
> It works if we change the script into:
> {code}
> a = load 'allscalar10k' as (name, age, gpa, instate);
> b = filter a by instate==TRUE;
> explain b;
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2593) Filter by a boolean value does not
work
Posted by "Jie Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13289845#comment-13289845 ]
Jie Li commented on PIG-2593:
-----------------------------
Fixed case 5 in a light way: added a virtual node "BOOL_COND" in QueryParser and implemented it with the original expression in LogicalPlanGenerator. Will attach the patch soon.
Actually there is a Case 6 in the e2e test:
|6|no schema and filter|a = load 'allscalar10k' as (name, age, gpa, instate);
b = filter a by instate;
dump b;| Error: <file 2.pig, line 9, column 4> Filter's condition must evaluate to boolean. Found: bytearray
After discussing with Daniel, we decided to leave this case for now, as it may require some heavy-weight changes.
> Filter by a boolean value does not work
> ---------------------------------------
>
> Key: PIG-2593
> URL: https://issues.apache.org/jira/browse/PIG-2593
> Project: Pig
> Issue Type: Bug
> Components: build
> Reporter: Daniel Dai
>
> The following script does not work:
> {code}
> a = load 'allscalar10k' as (name, age, gpa, instate);
> b = filter a by instate;
> explain b;
> {code}
> Exception:
> ERROR 1200: <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1598)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1541)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:541)
> at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:945)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:392)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:190)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
> at org.apache.pig.Main.run(Main.java:599)
> at org.apache.pig.Main.main(Main.java:153)
> Caused by: Failed to parse: <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:222)
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:164)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1590)
> ... 9 more
> It works if we change the script into:
> {code}
> a = load 'allscalar10k' as (name, age, gpa, instate);
> b = filter a by instate==TRUE;
> explain b;
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2593) Filter by a boolean value does not work
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2593:
----------------------------
Resolution: Fixed
Fix Version/s: 0.11
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
All unit tests pass.
Patch committed to trunk. Thanks Jie!
> Filter by a boolean value does not work
> ---------------------------------------
>
> Key: PIG-2593
> URL: https://issues.apache.org/jira/browse/PIG-2593
> Project: Pig
> Issue Type: Bug
> Components: build
> Reporter: Daniel Dai
> Assignee: Jie Li
> Fix For: 0.11
>
> Attachments: PIG-2593.patch
>
>
> The following script does not work:
> {code}
> a = load 'allscalar10k' as (name, age, gpa, instate);
> b = filter a by instate;
> explain b;
> {code}
> Exception:
> ERROR 1200: <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1598)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1541)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:541)
> at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:945)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:392)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:190)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
> at org.apache.pig.Main.run(Main.java:599)
> at org.apache.pig.Main.main(Main.java:153)
> Caused by: Failed to parse: <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:222)
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:164)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1590)
> ... 9 more
> It works if we change the script into:
> {code}
> a = load 'allscalar10k' as (name, age, gpa, instate);
> b = filter a by instate==TRUE;
> explain b;
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2593) Filter by a boolean value does not
work
Posted by "Jie Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13289662#comment-13289662 ]
Jie Li commented on PIG-2593:
-----------------------------
Fixed case 4 in the sub-task PIG-2736.
For case 5, one alternative is to rewrite the expression "by instate" to "by instate == true;" in QueryParser, but the downside is if instate is not a boolean type, then the error messages would be confusing: "the left side is a xxx type and the right side is a boolean type", as there is actually no right side. Any means to customize the error message?
Another alternative is to create a BooleanExpression to handle this case, which would involve more changes all the way from the parser to the physical operator.
> Filter by a boolean value does not work
> ---------------------------------------
>
> Key: PIG-2593
> URL: https://issues.apache.org/jira/browse/PIG-2593
> Project: Pig
> Issue Type: Bug
> Components: build
> Reporter: Daniel Dai
>
> The following script does not work:
> {code}
> a = load 'allscalar10k' as (name, age, gpa, instate);
> b = filter a by instate;
> explain b;
> {code}
> Exception:
> ERROR 1200: <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1598)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1541)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:541)
> at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:945)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:392)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:190)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
> at org.apache.pig.Main.run(Main.java:599)
> at org.apache.pig.Main.main(Main.java:153)
> Caused by: Failed to parse: <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:222)
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:164)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1590)
> ... 9 more
> It works if we change the script into:
> {code}
> a = load 'allscalar10k' as (name, age, gpa, instate);
> b = filter a by instate==TRUE;
> explain b;
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2593) Filter by a boolean value does not
work
Posted by "Jie Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13287596#comment-13287596 ]
Jie Li commented on PIG-2593:
-----------------------------
I got slightly different results:
||Case||Description||Query||Result||
|1|declare as boolean and filter as boolean|a = load 'allscalar10k' as (name, age, gpa, instate:boolean);
b = filter a by instate==TRUE;
dump b;|successful|
|2|explicitly cast to boolean|a = load 'allscalar10k' as (name, age, gpa, instate);
b = filter a by (boolean) instate == TRUE;
dump b;|successful|
|3|implicitly conversion to string|a = load 'allscalar10k' as (name, age, gpa, instate);
b = filter a by instate == 'true';
dump b;|successful|
|4|implicitly conversion to boolean|a = load 'allscalar10k' as (name, age, gpa, instate);
b = filter a by instate==TRUE;
dump b;|Error:In alias b, incompatible types in Equal Operator left hand side:bytearray right hand side:boolean|
|5|declare as boolean and filter| a = load 'allscalar10k' as (name, age, gpa, instate:boolean);
b = filter a by instate;
dump b;|Error: <file 2.pig, line 2, column 23> mismatched input ';' expecting IS|
So we have two cases not working here. We want to make Case 4 work by supporting implicit conversion of boolean, as we've already supported implicit conversion of chararray in Case 3. Also it makes sense that Case 5 should work as boolean is a valid conditional expression.
> Filter by a boolean value does not work
> ---------------------------------------
>
> Key: PIG-2593
> URL: https://issues.apache.org/jira/browse/PIG-2593
> Project: Pig
> Issue Type: Bug
> Components: build
> Reporter: Daniel Dai
>
> The following script does not work:
> {code}
> a = load 'allscalar10k' as (name, age, gpa, instate);
> b = filter a by instate;
> explain b;
> {code}
> Exception:
> ERROR 1200: <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1598)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1541)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:541)
> at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:945)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:392)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:190)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
> at org.apache.pig.Main.run(Main.java:599)
> at org.apache.pig.Main.main(Main.java:153)
> Caused by: Failed to parse: <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:222)
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:164)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1590)
> ... 9 more
> It works if we change the script into:
> {code}
> a = load 'allscalar10k' as (name, age, gpa, instate);
> b = filter a by instate==TRUE;
> explain b;
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2593) Filter by a boolean value does not work
Posted by "Jie Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jie Li updated PIG-2593:
------------------------
Assignee: Jie Li
Status: Patch Available (was: Open)
> Filter by a boolean value does not work
> ---------------------------------------
>
> Key: PIG-2593
> URL: https://issues.apache.org/jira/browse/PIG-2593
> Project: Pig
> Issue Type: Bug
> Components: build
> Reporter: Daniel Dai
> Assignee: Jie Li
> Attachments: PIG-2593.patch
>
>
> The following script does not work:
> {code}
> a = load 'allscalar10k' as (name, age, gpa, instate);
> b = filter a by instate;
> explain b;
> {code}
> Exception:
> ERROR 1200: <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1598)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1541)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:541)
> at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:945)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:392)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:190)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
> at org.apache.pig.Main.run(Main.java:599)
> at org.apache.pig.Main.main(Main.java:153)
> Caused by: Failed to parse: <file 18.pig, line 2, column 23> mismatched input ';' expecting IS
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:222)
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:164)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1590)
> ... 9 more
> It works if we change the script into:
> {code}
> a = load 'allscalar10k' as (name, age, gpa, instate);
> b = filter a by instate==TRUE;
> explain b;
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira