[jira] [Updated] (PIG-3002) Pig client should handle CountersExceededException

["Bill Graham (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bill Graham updated PIG-3002:
-----------------------------

    Attachment: PIG-3002.2.patch

I was able to run some tests using both the patched and unpatched version and both behave the same w.r.t. what's output to the console.

The behavior of {{MapReduceLauncher.computeWarningAggregate()}} is  already to catch {{IOException}} and log a warning, so it would be aceptable to catch {{Exception}} around the iterator on the {{PigWarning}} enum and just {{log.warn}} there as well. No need to modify the shim code. Just log the error and move on. This will cause the console output to show the success state of the jobs, along with the exception with the counters.
                
> Pig client should handle CountersExceededException
> --------------------------------------------------
>
>                 Key: PIG-3002
>                 URL: https://issues.apache.org/jira/browse/PIG-3002
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Bill Graham
>            Assignee: Jarek Jarcec Cecho
>              Labels: newbie, simple
>         Attachments: PIG-3002.2.patch, PIG-3002.patch
>
>
> Running a pig job that uses more than 120 counters will succeed, but a grunt exception will occur when trying to output counter info to the console. This exception should be caught and handled with friendly messaging:
> {noformat}
> org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during execution.
>         at org.apache.pig.PigServer.launchPlan(PigServer.java:1275)
>         at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249)
>         at org.apache.pig.PigServer.execute(PigServer.java:1239)
>         at org.apache.pig.PigServer.executeBatch(PigServer.java:333)
>         at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:136)
>         at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:197)
>         at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169)
>         at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
>         at org.apache.pig.Main.run(Main.java:604)
>         at org.apache.pig.Main.main(Main.java:154)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> Caused by: org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters - Counters=120 Limit=120
>         at org.apache.hadoop.mapred.Counters$Group.getCounterForName(Counters.java:312)
>         at org.apache.hadoop.mapred.Counters.findCounter(Counters.java:431)
>         at org.apache.hadoop.mapred.Counters.getCounter(Counters.java:495)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.computeWarningAggregate(MapReduceLauncher.java:707)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:442)
>         at org.apache.pig.PigServer.launchPlan(PigServer.java:1264)
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira