You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Rohini Palaniswamy (JIRA)" <ji...@apache.org> on 2013/02/11 18:35:13 UTC
[jira] [Commented] (PIG-3179) Task Information Header only prints
out the first split for each task
[ https://issues.apache.org/jira/browse/PIG-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13575923#comment-13575923 ]
Rohini Palaniswamy commented on PIG-3179:
-----------------------------------------
Input-split file: hdfs://gridx.yahoo.com:8020/tmp/bz-6086044/msh_grouped.bz2/part-r-00032.bz2
Input-split start-offset: 0
Input-split length: 11814548
Input-split file: hdfs://gridx.ygrid.yahoo.com:8020/tmp/bz-6086044/msh_grouped.bz2/part-r-00033.bz2
Input-split start-offset: 0
Input-split length: 11953088
Since we are printing all splits can we avoid the duplication of Input-split in each line and combine them into one statement to save log space?
> Task Information Header only prints out the first split for each task
> ---------------------------------------------------------------------
>
> Key: PIG-3179
> URL: https://issues.apache.org/jira/browse/PIG-3179
> Project: Pig
> Issue Type: Improvement
> Reporter: Koji Noguchi
> Assignee: Koji Noguchi
> Priority: Trivial
> Attachments: pig-3179-v01.patch
>
>
> When a task's PigSplit is containing more than wrappedSplit, it only logs the first fileinfo.
> When debugging, I saw
> {noformat}
> ===== Task Information Header =====
> Command: bash ....
> Start time: Mon Feb 11 16:41:21 UTC 2013
> Input-split file: hdfs://abc.bcd.efg:8020/tmp/hij/part-r-00000.bz2
> Input-split start-offset: 0Input-split length: 11854247
> {noformat}
> but the actual error was happing while reading part-r-00007.bz2. It would have been nice if the log showed all the info that task was going to read.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira