You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Adam Warrington (JIRA)" <ji...@apache.org> on 2011/04/04 21:43:05 UTC
[jira] [Commented] (PIG-1702) Streaming debug output outputs null
input-split information
[ https://issues.apache.org/jira/browse/PIG-1702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015571#comment-13015571 ]
Adam Warrington commented on PIG-1702:
--------------------------------------
Review request can be found here:
https://reviews.apache.org/r/547/
> Streaming debug output outputs null input-split information
> -----------------------------------------------------------
>
> Key: PIG-1702
> URL: https://issues.apache.org/jira/browse/PIG-1702
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.7.0
> Reporter: Adam Warrington
> Priority: Minor
> Fix For: 0.9.0
>
> Attachments: PIG-1702-0.patch
>
>
> Within the Pig streaming command execution, debug information is printed out to stderr which specified the input file, as well as split information. The function is org.apache.pig.backend.hadoop.streaming.HadoopExecutableManager.writeDebugHeader(). Pig 0.7 outputs null for the split file, and -1 for the split start-offset and split length. Example output:
> ===== Task Information Header =====
> Command: test.pl (stdin-org.apache.pig.builtin.PigStreaming/stdout-org.apache.pig.builtin.PigStreaming)
> Start time: Mon Oct 25 21:24:45 EDT 2010
> Input-split file: null
> Input-split start-offset: -1
> Input-split length: -1
> Within the writeDebugHeader() function, the input file information is obtained by querying for the "map.input.file" configuration variable. This configuration variable was set by the old hadoop m/r api, but not by the 0.20 api, which Pig 0.7 now uses. The new way to get this information is with something like: ((FileSplit) context.getInputSplit).getPath(). See HADOOP-5973.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira