You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "niraj rai (JIRA)" <ji...@apache.org> on 2010/09/01 22:43:55 UTC
[jira] Created: (PIG-1591) pig does not create a log file, if tje
MR job succeeds but front end fails.
pig does not create a log file, if tje MR job succeeds but front end fails.
---------------------------------------------------------------------------
Key: PIG-1591
URL: https://issues.apache.org/jira/browse/PIG-1591
Project: Pig
Issue Type: Bug
Reporter: niraj rai
Assignee: niraj rai
When I run this script:
A = load 'limit_empty.input_a' as (a1:int);
B = load 'limit_empty.input_b' as (b1:int);
C =COGROUP A by a1, B by b1;
C1 = foreach C { Alim = limit A 1; Blim = limit B 1; generate Alim, Blim; }
D1 = FOREACH C1 generate Alim,Blim, (IsEmpty(Alim)? 0:1), (IsEmpty(Blim)? 0:1), COUNT(Alim), COUNT(Blim);
dump D1;
The MR job succeeds but the pig job fails with the fillowing error:
2010-08-31 13:33:09,960 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2010-08-31 13:33:09,962 [main] INFO org.apache.pig.impl.io.InterStorage - Pig Internal storage in use
2010-08-31 13:33:09,963 [main] INFO org.apache.pig.impl.io.InterStorage - Pig Internal storage in use
2010-08-31 13:33:09,963 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!
2010-08-31 13:33:09,964 [main] INFO org.apache.pig.impl.io.InterStorage - Pig Internal storage in use
2010-08-31 13:33:09,965 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2010-08-31 13:33:09,969 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2010-08-31 13:33:09,969 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
2010-08-31 13:33:09,973 [main] ERROR org.apache.pig.backend.hadoop.executionengine.HJob - java.lang.ClassCastException: java.lang.Integer cannot be cast to org.apache.pig.data.Tuple
since MR job is succeeded, so the pig does not create any log file, but it should still create a log file, giving the cause of failure in the pig.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1591) pig does not create a log file, if tje
MR job succeeds but front end fails.
Posted by "niraj rai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
niraj rai updated PIG-1591:
---------------------------
Attachment: pig_1591.patch
> pig does not create a log file, if tje MR job succeeds but front end fails.
> ---------------------------------------------------------------------------
>
> Key: PIG-1591
> URL: https://issues.apache.org/jira/browse/PIG-1591
> Project: Pig
> Issue Type: Bug
> Reporter: niraj rai
> Assignee: niraj rai
> Attachments: pig_1591.patch
>
>
> When I run this script:
> A = load 'limit_empty.input_a' as (a1:int);
> B = load 'limit_empty.input_b' as (b1:int);
> C =COGROUP A by a1, B by b1;
> C1 = foreach C { Alim = limit A 1; Blim = limit B 1; generate Alim, Blim; }
> D1 = FOREACH C1 generate Alim,Blim, (IsEmpty(Alim)? 0:1), (IsEmpty(Blim)? 0:1), COUNT(Alim), COUNT(Blim);
> dump D1;
> The MR job succeeds but the pig job fails with the fillowing error:
> 2010-08-31 13:33:09,960 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
> 2010-08-31 13:33:09,962 [main] INFO org.apache.pig.impl.io.InterStorage - Pig Internal storage in use
> 2010-08-31 13:33:09,963 [main] INFO org.apache.pig.impl.io.InterStorage - Pig Internal storage in use
> 2010-08-31 13:33:09,963 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!
> 2010-08-31 13:33:09,964 [main] INFO org.apache.pig.impl.io.InterStorage - Pig Internal storage in use
> 2010-08-31 13:33:09,965 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
> 2010-08-31 13:33:09,969 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
> 2010-08-31 13:33:09,969 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
> 2010-08-31 13:33:09,973 [main] ERROR org.apache.pig.backend.hadoop.executionengine.HJob - java.lang.ClassCastException: java.lang.Integer cannot be cast to org.apache.pig.data.Tuple
> since MR job is succeeded, so the pig does not create any log file, but it should still create a log file, giving the cause of failure in the pig.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1591) pig does not create a log file, if tje
MR job succeeds but front end fails.
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12905966#action_12905966 ]
Daniel Dai commented on PIG-1591:
---------------------------------
+1. No unit test needed since it is about error message. Manually tested and it works. Will commit it shortly.
> pig does not create a log file, if tje MR job succeeds but front end fails.
> ---------------------------------------------------------------------------
>
> Key: PIG-1591
> URL: https://issues.apache.org/jira/browse/PIG-1591
> Project: Pig
> Issue Type: Bug
> Reporter: niraj rai
> Assignee: niraj rai
> Attachments: pig_1591.patch
>
>
> When I run this script:
> A = load 'limit_empty.input_a' as (a1:int);
> B = load 'limit_empty.input_b' as (b1:int);
> C =COGROUP A by a1, B by b1;
> C1 = foreach C { Alim = limit A 1; Blim = limit B 1; generate Alim, Blim; }
> D1 = FOREACH C1 generate Alim,Blim, (IsEmpty(Alim)? 0:1), (IsEmpty(Blim)? 0:1), COUNT(Alim), COUNT(Blim);
> dump D1;
> The MR job succeeds but the pig job fails with the fillowing error:
> 2010-08-31 13:33:09,960 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
> 2010-08-31 13:33:09,962 [main] INFO org.apache.pig.impl.io.InterStorage - Pig Internal storage in use
> 2010-08-31 13:33:09,963 [main] INFO org.apache.pig.impl.io.InterStorage - Pig Internal storage in use
> 2010-08-31 13:33:09,963 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!
> 2010-08-31 13:33:09,964 [main] INFO org.apache.pig.impl.io.InterStorage - Pig Internal storage in use
> 2010-08-31 13:33:09,965 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
> 2010-08-31 13:33:09,969 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
> 2010-08-31 13:33:09,969 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
> 2010-08-31 13:33:09,973 [main] ERROR org.apache.pig.backend.hadoop.executionengine.HJob - java.lang.ClassCastException: java.lang.Integer cannot be cast to org.apache.pig.data.Tuple
> since MR job is succeeded, so the pig does not create any log file, but it should still create a log file, giving the cause of failure in the pig.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (PIG-1591) pig does not create a log file, if tje
MR job succeeds but front end fails.
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai resolved PIG-1591.
-----------------------------
Hadoop Flags: [Reviewed]
Fix Version/s: 0.8.0
Resolution: Fixed
Patch committed to both trunk and 0.8 branch.
> pig does not create a log file, if tje MR job succeeds but front end fails.
> ---------------------------------------------------------------------------
>
> Key: PIG-1591
> URL: https://issues.apache.org/jira/browse/PIG-1591
> Project: Pig
> Issue Type: Bug
> Reporter: niraj rai
> Assignee: niraj rai
> Fix For: 0.8.0
>
> Attachments: pig_1591.patch
>
>
> When I run this script:
> A = load 'limit_empty.input_a' as (a1:int);
> B = load 'limit_empty.input_b' as (b1:int);
> C =COGROUP A by a1, B by b1;
> C1 = foreach C { Alim = limit A 1; Blim = limit B 1; generate Alim, Blim; }
> D1 = FOREACH C1 generate Alim,Blim, (IsEmpty(Alim)? 0:1), (IsEmpty(Blim)? 0:1), COUNT(Alim), COUNT(Blim);
> dump D1;
> The MR job succeeds but the pig job fails with the fillowing error:
> 2010-08-31 13:33:09,960 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
> 2010-08-31 13:33:09,962 [main] INFO org.apache.pig.impl.io.InterStorage - Pig Internal storage in use
> 2010-08-31 13:33:09,963 [main] INFO org.apache.pig.impl.io.InterStorage - Pig Internal storage in use
> 2010-08-31 13:33:09,963 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!
> 2010-08-31 13:33:09,964 [main] INFO org.apache.pig.impl.io.InterStorage - Pig Internal storage in use
> 2010-08-31 13:33:09,965 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
> 2010-08-31 13:33:09,969 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
> 2010-08-31 13:33:09,969 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
> 2010-08-31 13:33:09,973 [main] ERROR org.apache.pig.backend.hadoop.executionengine.HJob - java.lang.ClassCastException: java.lang.Integer cannot be cast to org.apache.pig.data.Tuple
> since MR job is succeeded, so the pig does not create any log file, but it should still create a log file, giving the cause of failure in the pig.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.