[jira] Commented: (PIG-1475) Large number of FILTER matching clauses causes internal error 2998

["Jeff Zhang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-1475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12883768#action_12883768 ] 

Jeff Zhang commented on PIG-1475:
---------------------------------

Mike, Could you paste your pig script and the full exception call stack ? Attach the sample data would be better

> Large number of FILTER matching clauses causes internal error 2998
> ------------------------------------------------------------------
>
>                 Key: PIG-1475
>                 URL: https://issues.apache.org/jira/browse/PIG-1475
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt
>    Affects Versions: 0.6.0
>         Environment: Amazon Elastic MapReduce with Pig 0.6 and Hadoop 0.20.2
>            Reporter: Mike Subelsky
>            Priority: Minor
>
> I'm generating reports using Pig where I only want to report on rows matching a set of regular expressions, but those regular expressions are pretty numerous.
> Pig fails with this internal error when I run FILTER with 500 terms:
> 2010-06-29 15:46:04,579 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal error. null
> This only occurred when I ran my Pig script against Hadoop with the full dataset. When I ran Pig in local mode, with a smaller sample file, Pig handled the FILTER command just fine.
> The workaround has been to split my list into two separate lists of 250 then UNION the results, but I assume this is something that could be addressed in the code.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.