You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Rafael Turk <ra...@gmail.com> on 2008/08/02 04:04:03 UTC

Regular Expression escape

Hi All,

 I cant run any script when using regular expressions with escape codes

If I try to run the following line of code:

 raw_filtered = FILTER raw BY ngram matches '^[a-zA-Z0-9\s]$';
OR
 raw_filtered = FILTER raw BY ngram matches "^[a-zA-Z0-9\s]$";

I get the following error:

org.apache.pig.impl.logicalLayer.parser.TokenMgrError: Lexical error at line
1, column 57.  Encountered: "s" (115), after : "\'^[a-zA-Z0-9\\"
        at
org.apache.pig.impl.logicalLayer.parser.QueryParserTokenManager.getNextToken(QueryParserTokenManager.java:1623)
        at
org.apache.pig.impl.logicalLayer.parser.QueryParser.jj_consume_token(QueryParser.java:4744)
        at
org.apache.pig.impl.logicalLayer.parser.QueryParser.PUnaryCond(QueryParser.java:1117)
        at
org.apache.pig.impl.logicalLayer.parser.QueryParser.PAndCond(QueryParser.java:1055)
        at
org.apache.pig.impl.logicalLayer.parser.QueryParser.POrCond(QueryParser.java:1005)
        at
org.apache.pig.impl.logicalLayer.parser.QueryParser.PCond(QueryParser.java:973)
        at
org.apache.pig.impl.logicalLayer.parser.QueryParser.FilterClause(QueryParser.java:941)
        at
org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:686)
        at
org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:512)
        at
org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:362)
        at
org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:47)
        at org.apache.pig.PigServer.registerQuery(PigServer.java:275)
        at
org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:475)
        at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:233)
        at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:81)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:60)
        at org.apache.pig.Main.main(Main.java:294)

I am using Pig Trunk from 01-Aug-2008

Thanks,
--Rafael

Re: Regular Expression escape

Posted by Alan Gates <ga...@yahoo-inc.com>.
You're hitting a bug in the parser where it is trying to read those 
escapes itself.  Please file a JIRA 
(http://issues.apache.org/jira/browse/PIG) so we can track and resolve 
this issue.

Alan.

Rafael Turk wrote:
> Hi All,
>
>  I cant run any script when using regular expressions with escape codes
>
> If I try to run the following line of code:
>
>  raw_filtered = FILTER raw BY ngram matches '^[a-zA-Z0-9\s]$';
> OR
>  raw_filtered = FILTER raw BY ngram matches "^[a-zA-Z0-9\s]$";
>
> I get the following error:
>
> org.apache.pig.impl.logicalLayer.parser.TokenMgrError: Lexical error at line
> 1, column 57.  Encountered: "s" (115), after : "\'^[a-zA-Z0-9\\"
>         at
> org.apache.pig.impl.logicalLayer.parser.QueryParserTokenManager.getNextToken(QueryParserTokenManager.java:1623)
>         at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.jj_consume_token(QueryParser.java:4744)
>         at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.PUnaryCond(QueryParser.java:1117)
>         at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.PAndCond(QueryParser.java:1055)
>         at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.POrCond(QueryParser.java:1005)
>         at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.PCond(QueryParser.java:973)
>         at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.FilterClause(QueryParser.java:941)
>         at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:686)
>         at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:512)
>         at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:362)
>         at
> org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:47)
>         at org.apache.pig.PigServer.registerQuery(PigServer.java:275)
>         at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:475)
>         at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:233)
>         at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:81)
>         at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:60)
>         at org.apache.pig.Main.main(Main.java:294)
>
> I am using Pig Trunk from 01-Aug-2008
>
> Thanks,
> --Rafael
>
>