You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Aniket Mokashi (JIRA)" <ji...@apache.org> on 2014/03/06 22:23:44 UTC
[jira] [Updated] (PIG-3754)
InputSizeReducerEstimator.getTotalInputFileSize reports incorrect size
[ https://issues.apache.org/jira/browse/PIG-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aniket Mokashi updated PIG-3754:
--------------------------------
Description:
If you have more than one input, InputSizeReducerEstimator.getTotalInputFileSize can return incorrect value if one of the loader returns \-1 and is not file based (eg- hbase). This causes incorrect reducer estimation and problems in auto.local mode.
If size of input is not found in for any of the inputs, we should bail out with return value of -1.
was:
If you have more than one input, InputSizeReducerEstimator.getTotalInputFileSize can return incorrect value if one of the loader returns -1 and is not file based (eg- hbase). This causes incorrect reducer estimation and problems in auto.local mode.
If size of input is not found in for any of the inputs, we should bail out with return value of -1.
> InputSizeReducerEstimator.getTotalInputFileSize reports incorrect size
> ----------------------------------------------------------------------
>
> Key: PIG-3754
> URL: https://issues.apache.org/jira/browse/PIG-3754
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.13.0
> Reporter: Aniket Mokashi
> Assignee: Aniket Mokashi
> Priority: Trivial
> Fix For: 0.13.0
>
>
> If you have more than one input, InputSizeReducerEstimator.getTotalInputFileSize can return incorrect value if one of the loader returns \-1 and is not file based (eg- hbase). This causes incorrect reducer estimation and problems in auto.local mode.
> If size of input is not found in for any of the inputs, we should bail out with return value of -1.
--
This message was sent by Atlassian JIRA
(v6.2#6252)