You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Amogh Margoor (Code Review)" <ge...@cloudera.org> on 2021/04/13 19:37:44 UTC

[Impala-ASF-CR] IMPALA-10654: Fix precision loss in DecimalValue to double conversion.

Amogh Margoor has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/17303 )

Change subject: IMPALA-10654: Fix precision loss in DecimalValue to double conversion.
......................................................................

IMPALA-10654: Fix precision loss in DecimalValue to double conversion.

Original approach to convert DecimalValue(internal representation
of decimals) to double was not accurate.
It was:
           static_cast<double>(value_) / pow(10.0, scale).
However only integers from −2^53 to 2^53 can be represented
accurately by double precision without any loss.
Hence, it would not work for numbers like -0.43149576573887316.
For DecimalValue representing -0.43149576573887316, value_ would be
-43149576573887316 and scale would be 17. As value_ < -2^53,
 result would not be accurate. In newer approach we are using third
party library https://github.com/lemire/fast_double_parser, which
handles above scenario in a performant manner.

Testing:
1. Added End to End Tests covering following scenarios:
    a. Test to show precision limitation of 16 in the write path
    b. DecimalValue's value_ between -2^53 and 2^53.
    b. value_ outside above range but abs(value_) < UINT64_MAX
    c. abs(value_) > UINT64_MAX -covers DecimalValue<__int128_t>
2. Ran existing  backend and end-to-end tests completely

Change-Id: I56f0652cb8f81a491b87d9b108a94c00ae6c99a1
---
M be/src/runtime/decimal-value.inline.h
A be/src/thirdparty/fast_double_parser/LICENSE
A be/src/thirdparty/fast_double_parser/LICENSE.BSL
A be/src/thirdparty/fast_double_parser/README.md
A be/src/thirdparty/fast_double_parser/fast_double_parser.h
M bin/rat_exclude_files.txt
M testdata/workloads/functional-query/queries/QueryTest/values.test
7 files changed, 1,568 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/03/17303/5
-- 
To view, visit http://gerrit.cloudera.org:8080/17303
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I56f0652cb8f81a491b87d9b108a94c00ae6c99a1
Gerrit-Change-Number: 17303
Gerrit-PatchSet: 5
Gerrit-Owner: Amogh Margoor <am...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>