[jira] [Updated] (HIVE-4736) Float aggregate of single value loses precission

["Eric Hanson (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-4736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eric Hanson updated HIVE-4736:
------------------------------

    Summary: Float aggregate of single value loses precission  (was: Float aggregate of single value looses precission)

> Float aggregate of single value loses precission
> ------------------------------------------------
>
>                 Key: HIVE-4736
>                 URL: https://issues.apache.org/jira/browse/HIVE-4736
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Query Processor
>    Affects Versions: vectorization-branch
>            Reporter: Remus Rusanu
>            Assignee: Remus Rusanu
>            Priority: Minor
>
> I am seeing differences that are small but greater than expected in the output of TPC-H query 1. Thanks to Jitendra’s patch, the negative result error is gone. However, I am seeing that the results for 
>  
> Sum(l_extendedprice * ( 1 - l_discount ))
> Sum(l_extendedprice * ( 1 - l_discount ) * ( 1 + l_tax ))
>  
> Differ in the low-order 8 or so digits compared with text.  I know the input is float so it should have all zeros in the low digits when cast to double. I would not expect the answers to match exactly but I would expect errors in the last few digits, not the last 8. I could be wrong, but I think this may be worth investigating. Any ideas?
>  
> I also noticed this, where I ran the query against a single row worth of input (order 1, line 1):
>  
> SELECT l_returnflag,
>        l_linestatus,
>        Sum(l_quantity)                                           AS sum_qty,
>        Sum(l_extendedprice)                                      AS sum_base_price,
>        Sum(l_extendedprice * ( 1 - l_discount ))                 AS sum_disc_price,
>        Sum(l_extendedprice * ( 1 - l_discount ) * ( 1 + l_tax )) AS sum_charge,
>        Avg(l_quantity)                                           AS avg_qty,
>        Avg(l_extendedprice)                                      AS avg_price,
>        Avg(l_discount)                                           AS avg_disc,
>        Count(*)                                                  AS count_order
> FROM   lineitem
> WHERE  l_shipdate <= '1998-09-19'
>        and l_orderkey = 1 and l_linenumber = 1 
> GROUP  BY l_returnflag,
>           l_linestatus
> ORDER  BY l_returnflag,
>           l_linestatus; 
>  
> input row
> 1       155190  7706    1       17.0    21168.23        0.04    0.02    N       O       1996-03-13  1996-02-12  1996-03-22      DELIVER IN PERSON   TRUCK   egular courts above the
>  
>  
> Result
>  
> V (orc)
> N       O       17.0    21168.23046875  20321.501268925873      20727.931285219973      17.0    21168.23046875  0.03999999910593033 1
>  
> NV (text)
> N       O       17.0    21168.23046875  20321.5                 20727.9296875           17.0    21168.23046875  0.03999999910593033     1



--
This message was sent by Atlassian JIRA
(v6.1#6144)