[jira] [Commented] (HIVE-5990) round() function with large scale chokes

["Xuefu Zhang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-5990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843709#comment-13843709 ] 

Xuefu Zhang commented on HIVE-5990:
-----------------------------------

Trunk has a new implementation. However, OOM is observed with the same query, seemingly due to the same issue as above.
{code}
hive> select ROUND(4.42330604244772E-305, 2000000000) from ts;     
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at java.math.BigDecimal.bigTenToThe(BigDecimal.java:3376)
	at java.math.BigDecimal.bigMultiplyPowerTen(BigDecimal.java:3508)
	at java.math.BigDecimal.setScale(BigDecimal.java:2394)
	at java.math.BigDecimal.setScale(BigDecimal.java:2335)
	at org.apache.hadoop.hive.ql.udf.generic.RoundUtils.round(RoundUtils.java:49)
	at org.apache.hadoop.hive.ql.udf.generic.GenericUDFRound.round(GenericUDFRound.java:248)
	at org.apache.hadoop.hive.ql.udf.generic.GenericUDFRound.evaluate(GenericUDFRound.java:229)
	at org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:145)
	at org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:232)
{code}

[~mihirk] Do you have a fix for this, or do you plan to work on this?

> round() function with large scale chokes
> ----------------------------------------
>
>                 Key: HIVE-5990
>                 URL: https://issues.apache.org/jira/browse/HIVE-5990
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor, UDF
>    Affects Versions: 0.9.0
>         Environment: CDH 4.2 running Hive 0.9
>            Reporter: Mihir Kulkarni
>
> ROUND(4.42330604244772E-305, 2000000000) chokes with following error:
> FAILED: SemanticException [Error 10014]: Line 1:7 Wrong arguments '2000000000': org.apache.hadoop.hive.ql.metadata.HiveException: Unable to execute method public org.apache.hadoop.hive.serde2.io.DoubleWritable org.apache.hadoop.hive.ql.udf.UDFRound.evaluate(org.apache.hadoop.hive.serde2.io.DoubleWritable,org.apache.hadoop.io.IntWritable)  on object org.apache.hadoop.hive.ql.udf.UDFRound@51809ab7 of class org.apache.hadoop.hive.ql.udf.UDFRound with arguments {1.001:org.apache.hadoop.hive.serde2.io.DoubleWritable, 2000000000:org.apache.hadoop.io.IntWritable} of size 2
> I think the problem might be in how Hive evaluates 2 arg ROUND(). The code uses
> BigDecimal.valueOf(d).setScale(i, RoundingMode.HALF_UP)
> which chokes by itself. I saw some changes in implementation of ROUND in hive trunk but this is still being used in RoundUtils.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)