You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Thejas M Nair (JIRA)" <ji...@apache.org> on 2010/08/26 17:50:57 UTC
[jira] Created: (PIG-1570) native mapreduce operator MR job does
not follow same failure handling logic as other pig MR jobs
native mapreduce operator MR job does not follow same failure handling logic as other pig MR jobs
-------------------------------------------------------------------------------------------------
Key: PIG-1570
URL: https://issues.apache.org/jira/browse/PIG-1570
Project: Pig
Issue Type: Bug
Reporter: Thejas M Nair
Fix For: 0.8.0
The code path for handling failure in MR job corresponding to native MR is different and does not have the same behavior.
For example, even if the MR job for mapreduce operator fails, the number of jobs that failed is being reported as 0 in PigStats log.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1570) native mapreduce operator MR job does
not follow same failure handling logic as other pig MR jobs
Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904263#action_12904263 ]
Thejas M Nair commented on PIG-1570:
------------------------------------
The code path that is followed in case of the native MR job is still different because the jar is a black box, and pig just calls the main function, pig doesn't even know if it is a MR job that is actually being run.
This fixes the pig stats reporting (log messages) for failed native MR job and also the feature list in the native MR job.
> native mapreduce operator MR job does not follow same failure handling logic as other pig MR jobs
> -------------------------------------------------------------------------------------------------
>
> Key: PIG-1570
> URL: https://issues.apache.org/jira/browse/PIG-1570
> Project: Pig
> Issue Type: Bug
> Reporter: Thejas M Nair
> Assignee: Thejas M Nair
> Fix For: 0.8.0
>
> Attachments: PIG-1570.1.patch
>
>
> The code path for handling failure in MR job corresponding to native MR is different and does not have the same behavior.
> For example, even if the MR job for mapreduce operator fails, the number of jobs that failed is being reported as 0 in PigStats log.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1570) native mapreduce operator MR job does
not follow same failure handling logic as other pig MR jobs
Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904258#action_12904258 ]
Thejas M Nair commented on PIG-1570:
------------------------------------
Regarding
bq. Another thing to investigate (somewhat related) - there seems to be a problem when PigServer is used to execute query having native mr operator - i was unable to run the tests in local mode . But i am able to run query in local mode from commandline.
The problem was that in test setup, the MiniCluster hadoop-site.xml (~/pigtest/conf/hadoop-site.xml) is in classpath. The WordCount.jar would end up trying to run the MR job using minicluster and fail, if rest of the test is using local mode.
> native mapreduce operator MR job does not follow same failure handling logic as other pig MR jobs
> -------------------------------------------------------------------------------------------------
>
> Key: PIG-1570
> URL: https://issues.apache.org/jira/browse/PIG-1570
> Project: Pig
> Issue Type: Bug
> Reporter: Thejas M Nair
> Assignee: Thejas M Nair
> Fix For: 0.8.0
>
>
> The code path for handling failure in MR job corresponding to native MR is different and does not have the same behavior.
> For example, even if the MR job for mapreduce operator fails, the number of jobs that failed is being reported as 0 in PigStats log.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1570) native mapreduce operator MR job does
not follow same failure handling logic as other pig MR jobs
Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902919#action_12902919 ]
Thejas M Nair commented on PIG-1570:
------------------------------------
Another thing to investigate (somewhat related) - there seems to be a problem when PigServer is used to execute query having native mr operator - i was unable to run the tests in local mode . But i am able to run query in local mode from commandline.
> native mapreduce operator MR job does not follow same failure handling logic as other pig MR jobs
> -------------------------------------------------------------------------------------------------
>
> Key: PIG-1570
> URL: https://issues.apache.org/jira/browse/PIG-1570
> Project: Pig
> Issue Type: Bug
> Reporter: Thejas M Nair
> Fix For: 0.8.0
>
>
> The code path for handling failure in MR job corresponding to native MR is different and does not have the same behavior.
> For example, even if the MR job for mapreduce operator fails, the number of jobs that failed is being reported as 0 in PigStats log.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1570) native mapreduce operator MR job does
not follow same failure handling logic as other pig MR jobs
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904321#action_12904321 ]
Richard Ding commented on PIG-1570:
-----------------------------------
+1.
> native mapreduce operator MR job does not follow same failure handling logic as other pig MR jobs
> -------------------------------------------------------------------------------------------------
>
> Key: PIG-1570
> URL: https://issues.apache.org/jira/browse/PIG-1570
> Project: Pig
> Issue Type: Bug
> Reporter: Thejas M Nair
> Assignee: Thejas M Nair
> Fix For: 0.8.0
>
> Attachments: PIG-1570.1.patch
>
>
> The code path for handling failure in MR job corresponding to native MR is different and does not have the same behavior.
> For example, even if the MR job for mapreduce operator fails, the number of jobs that failed is being reported as 0 in PigStats log.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-1570) native mapreduce operator MR job does
not follow same failure handling logic as other pig MR jobs
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich reassigned PIG-1570:
-----------------------------------
Assignee: Thejas M Nair
> native mapreduce operator MR job does not follow same failure handling logic as other pig MR jobs
> -------------------------------------------------------------------------------------------------
>
> Key: PIG-1570
> URL: https://issues.apache.org/jira/browse/PIG-1570
> Project: Pig
> Issue Type: Bug
> Reporter: Thejas M Nair
> Assignee: Thejas M Nair
> Fix For: 0.8.0
>
>
> The code path for handling failure in MR job corresponding to native MR is different and does not have the same behavior.
> For example, even if the MR job for mapreduce operator fails, the number of jobs that failed is being reported as 0 in PigStats log.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1570) native mapreduce operator MR job does
not follow same failure handling logic as other pig MR jobs
Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thejas M Nair updated PIG-1570:
-------------------------------
Attachment: PIG-1570.1.patch
Patch passed test-patch and core tests. Patch is ready for review.
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 5 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec]
> native mapreduce operator MR job does not follow same failure handling logic as other pig MR jobs
> -------------------------------------------------------------------------------------------------
>
> Key: PIG-1570
> URL: https://issues.apache.org/jira/browse/PIG-1570
> Project: Pig
> Issue Type: Bug
> Reporter: Thejas M Nair
> Assignee: Thejas M Nair
> Fix For: 0.8.0
>
> Attachments: PIG-1570.1.patch
>
>
> The code path for handling failure in MR job corresponding to native MR is different and does not have the same behavior.
> For example, even if the MR job for mapreduce operator fails, the number of jobs that failed is being reported as 0 in PigStats log.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (PIG-1570) native mapreduce operator MR job does
not follow same failure handling logic as other pig MR jobs
Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thejas M Nair resolved PIG-1570.
--------------------------------
Hadoop Flags: [Reviewed]
Resolution: Fixed
Patch committed to trunk.
> native mapreduce operator MR job does not follow same failure handling logic as other pig MR jobs
> -------------------------------------------------------------------------------------------------
>
> Key: PIG-1570
> URL: https://issues.apache.org/jira/browse/PIG-1570
> Project: Pig
> Issue Type: Bug
> Reporter: Thejas M Nair
> Assignee: Thejas M Nair
> Fix For: 0.8.0
>
> Attachments: PIG-1570.1.patch
>
>
> The code path for handling failure in MR job corresponding to native MR is different and does not have the same behavior.
> For example, even if the MR job for mapreduce operator fails, the number of jobs that failed is being reported as 0 in PigStats log.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.