You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org> on 2020/02/25 18:42:06 UTC

[Impala-ASF-CR] IMPALA-8759: Use double precision for HLL finalize function 

Wenzhe Zhou has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15167


Change subject: IMPALA-8759: Use double precision for HLL finalize function
......................................................................

IMPALA-8759: Use double precision for HLL finalize function

Current HLL finalize function use single precision of data type
float32 to calculate estimate. It's not accurate for the larger
cardinalities beyond 1,000,000 since float32 only has 6~7 decimal
digit precision.
This patch change single precision data type to double precision
type for HLL finalize function.

Testing:
 - Passed all exhaustive tests.
 - Did benchmark for queries with NDV functions. The performance
   impact is negligible.

Change-Id: I0c5a5229b682070b0bc14da287db5231159dbb3d
---
M be/src/exprs/aggregate-functions-ir.cc
1 file changed, 9 insertions(+), 9 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/67/15167/3
-- 
To view, visit http://gerrit.cloudera.org:8080/15167
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0c5a5229b682070b0bc14da287db5231159dbb3d
Gerrit-Change-Number: 15167
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-8759: Use double precision for HLL finalize function

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15167 )

Change subject: IMPALA-8759: Use double precision for HLL finalize function
......................................................................


Patch Set 4: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/15167
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0c5a5229b682070b0bc14da287db5231159dbb3d
Gerrit-Change-Number: 15167
Gerrit-PatchSet: 4
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 26 Feb 2020 06:37:09 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8759: Use double precision for HLL finalize function

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/15167 )

Change subject: IMPALA-8759: Use double precision for HLL finalize function
......................................................................

IMPALA-8759: Use double precision for HLL finalize function

Current HLL finalize function use single precision of data type
float32 to calculate estimate. It's not accurate for the larger
cardinalities beyond 1,000,000 since float32 only has 6~7 decimal
digit precision.
This patch change single precision data type to double precision
type for HLL finalize function.

Testing:
 - Passed all exhaustive tests.
 - Did benchmark for queries with NDV functions. The performance
   impact is negligible.
   See following spreadsheet for the menchmark:
   https://docs.google.com/spreadsheets/d/1DIVOEs5C4MJL1b7O4MA_jkaM3Y-JSMFREjXCUHJ3eHc/edit#gid=0

Change-Id: I0c5a5229b682070b0bc14da287db5231159dbb3d
---
M be/src/exprs/aggregate-functions-ir.cc
1 file changed, 9 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/67/15167/4
-- 
To view, visit http://gerrit.cloudera.org:8080/15167
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0c5a5229b682070b0bc14da287db5231159dbb3d
Gerrit-Change-Number: 15167
Gerrit-PatchSet: 4
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-8759: Use double precision for HLL finalize function

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15167 )

Change subject: IMPALA-8759: Use double precision for HLL finalize function
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/5330/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/15167
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0c5a5229b682070b0bc14da287db5231159dbb3d
Gerrit-Change-Number: 15167
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 25 Feb 2020 19:26:36 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8759: Use double precision for HLL finalize function

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15167 )

Change subject: IMPALA-8759: Use double precision for HLL finalize function
......................................................................

IMPALA-8759: Use double precision for HLL finalize function

Current HLL finalize function use single precision of data type
float32 to calculate estimate. It's not accurate for the larger
cardinalities beyond 1,000,000 since float32 only has 6~7 decimal
digit precision.
This patch change single precision data type to double precision
type for HLL finalize function.

Testing:
 - Passed all exhaustive tests.
 - Did benchmark for queries with NDV functions. The performance
   impact is negligible.
   See following spreadsheet for the menchmark:
   https://docs.google.com/spreadsheets/d/1DIVOEs5C4MJL1b7O4MA_jkaM3Y-JSMFREjXCUHJ3eHc/edit#gid=0

Change-Id: I0c5a5229b682070b0bc14da287db5231159dbb3d
Reviewed-on: http://gerrit.cloudera.org:8080/15167
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/exprs/aggregate-functions-ir.cc
1 file changed, 9 insertions(+), 9 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/15167
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I0c5a5229b682070b0bc14da287db5231159dbb3d
Gerrit-Change-Number: 15167
Gerrit-PatchSet: 6
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-8759: Use double precision for HLL finalize function

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15167 )

Change subject: IMPALA-8759: Use double precision for HLL finalize function
......................................................................


Patch Set 5: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/15167
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0c5a5229b682070b0bc14da287db5231159dbb3d
Gerrit-Change-Number: 15167
Gerrit-PatchSet: 5
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 26 Feb 2020 06:37:22 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8759: Use double precision for HLL finalize function

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15167 )

Change subject: IMPALA-8759: Use double precision for HLL finalize function
......................................................................


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5404/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/15167
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0c5a5229b682070b0bc14da287db5231159dbb3d
Gerrit-Change-Number: 15167
Gerrit-PatchSet: 5
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 26 Feb 2020 06:37:23 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8759: Use double precision for HLL finalize function

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15167 )

Change subject: IMPALA-8759: Use double precision for HLL finalize function
......................................................................


Patch Set 5: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/15167
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0c5a5229b682070b0bc14da287db5231159dbb3d
Gerrit-Change-Number: 15167
Gerrit-PatchSet: 5
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 26 Feb 2020 11:06:10 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8759: Use double precision for HLL finalize function

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15167 )

Change subject: IMPALA-8759: Use double precision for HLL finalize function
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/5341/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/15167
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0c5a5229b682070b0bc14da287db5231159dbb3d
Gerrit-Change-Number: 15167
Gerrit-PatchSet: 4
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 26 Feb 2020 01:45:48 +0000
Gerrit-HasComments: No