You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Thiruvel Thirumoolan (JIRA)" <ji...@apache.org> on 2013/02/12 18:39:12 UTC
[jira] [Commented] (HIVE-3911) udaf_percentile_approx.q fails with
Hadoop 0.23.5 when map-side aggr is disabled.
[ https://issues.apache.org/jira/browse/HIVE-3911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13576811#comment-13576811 ]
Thiruvel Thirumoolan commented on HIVE-3911:
--------------------------------------------
This also happens with all usages of NumericHistogram (udaf histogram_numeric() too). This algorithm deals with double and the order in which inputs goes to the algorithm matter. If the order is different (as in this case), the results will be different. In Hadoop 20.x, the inputs goto the UDAF as it is in the table. But in Hadoop-23, the input order is reversed and the final output also is different. I have uploaded a patch which works fine for histogram_numeric() but fails with a small difference for udaf_percentile_approx. If there is way to tune this in Hadoop-23 that should help.
> udaf_percentile_approx.q fails with Hadoop 0.23.5 when map-side aggr is disabled.
> ---------------------------------------------------------------------------------
>
> Key: HIVE-3911
> URL: https://issues.apache.org/jira/browse/HIVE-3911
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Thiruvel Thirumoolan
> Fix For: 0.11.0
>
> Attachments: HIVE-3911.patch
>
>
> I am running Hive10 unit tests against Hadoop 0.23.5 and udaf_percentile_approx.q fails with a different value when map-side aggr is disabled and only when 3rd argument to this UDAF is 100. Matches expected output when map-side aggr is enabled for the same arguments.
> This test passes when hadoop.version is 1.1.1 and fails when its 0.23.x or 2.0.0-alpha or 2.0.2-alpha.
> [junit] 20c20
> [junit] < 254.08333333333331
> [junit] ---
> [junit] > 252.77777777777777
> [junit] 47c47
> [junit] < 254.08333333333331
> [junit] ---
> [junit] > 252.77777777777777
> [junit] 74c74
> [junit] < [23.355555555555558,254.08333333333331,477.0625,489.5466666666667]
> [junit] ---
> [junit] > [24.07,252.77777777777777,476.9444444444444,487.82]
> [junit] 101c101
> [junit] < [23.355555555555558,254.08333333333331,477.0625,489.5466666666667]
> [junit] ---
> [junit] > [24.07,252.77777777777777,476.9444444444444,487.82]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira