[jira] [Commented] (PIG-4788) the value BytesRead metric info always returns 0 even the length of input file is not 0 in spark engine

["liyunzhang_intel (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-4788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15120680#comment-15120680 ] 

liyunzhang_intel commented on PIG-4788:
---------------------------------------

[~xuefuz], [~mohitsabharwal],[~pallavi.rao] and [~kexianda]:
PIG-4788.patch changes original code of PigSplit to fix the problem that taskMetrics.inputMetrics().get().bytesRead() always returns 0.
I explain why do this change in the above comment. If anyone knows how to fix this problem and not change original code, please tell me.

> the value BytesRead metric info always returns 0 even the length of input file is not 0 in spark engine
> -------------------------------------------------------------------------------------------------------
>
>                 Key: PIG-4788
>                 URL: https://issues.apache.org/jira/browse/PIG-4788
>             Project: Pig
>          Issue Type: Sub-task
>          Components: spark
>            Reporter: liyunzhang_intel
>            Assignee: liyunzhang_intel
>             Fix For: spark-branch
>
>         Attachments: PIG-4788.patch
>
>
> In [JobMetricsLinstener#onTaskEnd|https://github.com/apache/pig/blob/spark/src/org/apache/pig/tools/pigstats/spark/SparkJobStats.java#L140], taskMetrics.inputMetrics().get().bytesRead() always returns 0 even the length of input file is not zero.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)