You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Xiang Yang (Code Review)" <ge...@cloudera.org> on 2022/09/25 12:09:23 UTC
[Impala-ASF-CR] IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier().
Xiang Yang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18861
Change subject: IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
Currently decimal-util.h didn't specialize DecimalUtil
::GetScaleMultiplier<int256_t>(), causing more performance loss when
calculate Decimal16Value division.
Testing:
- Ran existing jobs.
- Add decimal-util-benchmark.
Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/decimal-util-benchmark.cc
M be/src/util/decimal-util.h
3 files changed, 164 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/18861/3
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 3
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Xiang Yang (Code Review)" <ge...@cloudera.org>.
Xiang Yang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 9:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/18861/8/be/src/benchmarks/decimal-util-benchmark.cc
File be/src/benchmarks/decimal-util-benchmark.cc:
http://gerrit.cloudera.org:8080/#/c/18861/8/be/src/benchmarks/decimal-util-benchmark.cc@25
PS8, Line 25: (ra
> Nit: ran, no need for capital letter.
Done
http://gerrit.cloudera.org:8080/#/c/18861/8/be/src/benchmarks/decimal-util-benchmark.cc@25
PS8, Line 25: // Summary result (ran within a Docker container):
> Nit: add a space before the parenthesis.
Done
http://gerrit.cloudera.org:8080/#/c/18861/8/be/src/benchmarks/decimal-util-benchmark.cc@54
PS8, Line 54: // A volatile variable to hold the return value of the benchmarked functions.
> I think this would be better/more understantable:
Done
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 9
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Sat, 08 Oct 2022 02:46:27 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc
File be/src/benchmarks/decimal-util-benchmark.cc:
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc@51
PS3, Line 51: }
> I guess the difference is a total time comparison, incuding the target func
I meant why it makes a huge difference if the volatile variable is static or not.
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/runtime/decimal-test.cc
File be/src/runtime/decimal-test.cc:
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/runtime/decimal-test.cc@1022
PS4, Line 1022: TestGetScaleMultiplier<int32_t>(DecimalUtil::INT32_SCALE_UPPER_BOUND);
> Since we didn't specialize the double edition, I think we needn't to test i
I still think we should test it, it's good to add tests to old code that hasn't been tested so far.
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 4
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Wed, 28 Sep 2022 15:09:54 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 9: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 9
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Mon, 10 Oct 2022 11:39:25 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
lipenglin@sensorsdata.cn has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/18861/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18861/4//COMMIT_MSG@7
PS4, Line 7: S
nit: Lack of space
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/benchmarks/decimal-util-benchmark.cc
File be/src/benchmarks/decimal-util-benchmark.cc:
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/benchmarks/decimal-util-benchmark.cc@26
PS4, Line 26: Machine Info: AMD Ryzen 9 5950X 16-Core Processor
Whether to indicate that this is in Docker?
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 4
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Tue, 27 Sep 2022 12:02:06 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 6:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8645/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 6
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Fri, 30 Sep 2022 10:41:03 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Xiang Yang (Code Review)" <ge...@cloudera.org>.
Hello Daniel Becker, lipenglin@sensorsdata.cn, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18861
to look at the new patch set (#6).
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
Currently decimal-util.h didn't specialize DecimalUtil
::GetScaleMultiplier<int256_t>(), causing more performance loss when
calculate Decimal16Value division.
Testing:
- Ran existing jobs.
- Add decimal-util-benchmark.
Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/decimal-util-benchmark.cc
M be/src/runtime/decimal-test.cc
M be/src/util/decimal-util.h
4 files changed, 232 insertions(+), 30 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/18861/6
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 6
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Xiang Yang (Code Review)" <ge...@cloudera.org>.
Hello Daniel Becker, lipenglin@sensorsdata.cn, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18861
to look at the new patch set (#7).
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
Currently decimal-util.h didn't specialize DecimalUtil
::GetScaleMultiplier<int256_t>(), causing more performance loss when
calculate Decimal16Value division.
This template specialisation results in 5600X speedup approximately.
Testing:
- Ran existing jobs.
- Add decimal-util-benchmark.
Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/decimal-util-benchmark.cc
M be/src/runtime/decimal-test.cc
M be/src/util/decimal-util.h
4 files changed, 235 insertions(+), 30 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/18861/7
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 7
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
[Impala-ASF-CR] IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 4:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/18861/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18861/3//COMMIT_MSG@15
PS3, Line 15: - Add decimal-util-benchmark.
> done, but ran in a docker container, will that be a problem?
It's ok if it is in a Docker container.
I would mention in the commit msg how big the speedup is.
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc
File be/src/benchmarks/decimal-util-benchmark.cc:
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc@51
PS3, Line 51: }
> done.
I experimented a bit with it and a simple global static volatile variable may be enough. I couldn't get it to work with int256_t but using an int64_t I got an even greater difference, around 6000x.
This time I simply assigned the result of the function to the global volatile int64_t, didn't add anything.
Interestingly if the global volatile variable is not static, the difference is only 1400x...
I don't know the cause of the difference but I think we should try the global, static volatile int64_t variable that we assign the result to.
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/benchmarks/decimal-util-benchmark.cc
File be/src/benchmarks/decimal-util-benchmark.cc:
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/benchmarks/decimal-util-benchmark.cc@56
PS4, Line 56: int256_t
If we stay with summing the results, I'm not sure int256_t is OK, it may have undefined behaviour on overflow.
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/runtime/decimal-test.cc
File be/src/runtime/decimal-test.cc:
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/runtime/decimal-test.cc@1012
PS4, Line 1012: DCHECK(expect * 10 >= expect);
This is not really a good check because if a signed type (such as int32_t etc.) overflows it is undefined behaviour so the value could be anything.
In this test I think we can tolerate not checking overflow.
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/runtime/decimal-test.cc@1013
PS4, Line 1013: expect *= 10;
We don't need to prepare a fresh 'expected' in each iteration, we can start with 1 outside the loop and multiply it with 10 at the end of each iteration.
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/runtime/decimal-test.cc@1022
PS4, Line 1022: TestGetScaleMultiplier<int32_t>(DecimalUtil::INT32_SCALE_UPPER_BOUND);
We could also test with double, I saw this function used with double in the code base.
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/util/decimal-util.h
File be/src/util/decimal-util.h:
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/util/decimal-util.h@37
PS4, Line 37: // The scale upper bound for GetScaleMultiplier<int32_t>()
: static constexpr int INT32_SCALE_UPPER_BOUND = ColumnType::MAX_DECIMAL4_PRECISION + 1;
: // The scale upper bound for GetScaleMultiplier<int64_t>()
: static constexpr int INT64_SCALE_UPPER_BOUND = ColumnType::MAX_DECIMAL8_PRECISION + 1;
: // The scale upper bound for GetScaleMultiplier<int128_t>()
: static constexpr int INT128_SCALE_UPPER_BOUND = ColumnType::MAX_PRECISION + 1;
: // The scale upper bound for GetScaleMultiplier<int256_t>()
: static constexpr int INT256_SCALE_UPPER_BOUND = 77;
I don't think these should be public class-level constants, they should be in the respective functions.
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h
File be/src/util/decimal-util.h:
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@342
PS3, Line 342: * i10e18 * i10e18 * 1000000000ll
> use 'num_values' instead of "sizeof(values) / sizeof(int256_t)"? it seems t
I agree now this assert statement is not needed, we can remove it.
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 4
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Wed, 28 Sep 2022 10:53:42 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 7:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11534/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 7
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Wed, 05 Oct 2022 10:01:21 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
Currently decimal-util.h didn't specialize DecimalUtil
::GetScaleMultiplier<int256_t>(), causing more performance loss when
calculate Decimal16Value division.
This template specialisation results in 5600X speedup approximately.
Testing:
- Ran existing jobs.
- Add decimal-util-benchmark.
Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Reviewed-on: http://gerrit.cloudera.org:8080/18861
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/decimal-util-benchmark.cc
M be/src/runtime/decimal-test.cc
M be/src/util/decimal-util.h
4 files changed, 235 insertions(+), 30 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 11
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
[Impala-ASF-CR] IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Xiang Yang (Code Review)" <ge...@cloudera.org>.
Hello Daniel Becker, lipenglin@sensorsdata.cn, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18861
to look at the new patch set (#5).
Change subject: IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
Currently decimal-util.h didn't specialize DecimalUtil
::GetScaleMultiplier<int256_t>(), causing more performance loss when
calculate Decimal16Value division.
Testing:
- Ran existing jobs.
- Add decimal-util-benchmark.
Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/decimal-util-benchmark.cc
M be/src/runtime/decimal-test.cc
M be/src/util/decimal-util.h
4 files changed, 232 insertions(+), 30 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/18861/5
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 5
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 7: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 7
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Wed, 05 Oct 2022 14:47:28 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 9:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11562/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 9
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Sat, 08 Oct 2022 03:05:41 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
lipenglin@sensorsdata.cn has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 9: Code-Review+1
Thanks for contributing this, the patch can improve the performance of division of Decimal Type.
LGTM!
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 9
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Mon, 10 Oct 2022 08:46:53 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 3:
(31 comments)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h
File be/src/util/decimal-util.h:
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@311
PS3, Line 311: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 10000000000ll,
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@312
PS3, Line 312: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 100000000000ll,
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@313
PS3, Line 313: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000ll,
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@314
PS3, Line 314: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 10000000000000ll,
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@315
PS3, Line 315: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 100000000000000ll,
line too long (95 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@316
PS3, Line 316: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000ll,
line too long (96 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@317
PS3, Line 317: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 10000000000000000ll,
line too long (97 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@318
PS3, Line 318: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 100000000000000000ll,
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@319
PS3, Line 319: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll,
line too long (99 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@320
PS3, Line 320: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 10ll,
line too long (106 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@321
PS3, Line 321: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 100ll,
line too long (107 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@322
PS3, Line 322: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 1000ll,
line too long (108 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@323
PS3, Line 323: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 10000ll,
line too long (109 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@324
PS3, Line 324: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 100000ll,
line too long (110 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@325
PS3, Line 325: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 1000000ll,
line too long (111 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@326
PS3, Line 326: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 10000000ll,
line too long (112 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@327
PS3, Line 327: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 100000000ll,
line too long (113 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@328
PS3, Line 328: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 1000000000ll,
line too long (114 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@329
PS3, Line 329: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 10000000000ll,
line too long (115 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@330
PS3, Line 330: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 100000000000ll,
line too long (116 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@331
PS3, Line 331: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 1000000000000ll,
line too long (117 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@332
PS3, Line 332: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 10000000000000ll,
line too long (118 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@333
PS3, Line 333: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 100000000000000ll,
line too long (119 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@334
PS3, Line 334: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 1000000000000000ll,
line too long (120 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@335
PS3, Line 335: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 10000000000000000ll,
line too long (121 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@336
PS3, Line 336: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 100000000000000000ll,
line too long (122 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@337
PS3, Line 337: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 1000000000000000000ll,
line too long (123 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@338
PS3, Line 338: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 1000000000000000000ll * 10ll,
line too long (130 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@339
PS3, Line 339: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 1000000000000000000ll * 100ll,
line too long (131 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@340
PS3, Line 340: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 1000000000000000000ll * 1000ll,
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@341
PS3, Line 341: static_cast<int256_t>(1000000000000000000ll) * 1000000000000000000ll * 1000000000000000000ll * 1000000000000000000ll * 10000ll};
line too long (134 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 3
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Sun, 25 Sep 2022 12:10:12 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 9: Code-Review+1
Thanks. Let's wait a bit and see if Lipenglin has other remarks before merging this.
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 9
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Mon, 10 Oct 2022 08:21:20 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 10: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 10
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Mon, 10 Oct 2022 16:46:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Xiang Yang (Code Review)" <ge...@cloudera.org>.
Xiang Yang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 4:
(14 comments)
Hi Daniel, thanks for your suggestions! I still have 2 remain questions, waiting for your reply.
http://gerrit.cloudera.org:8080/#/c/18861/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18861/3//COMMIT_MSG@15
PS3, Line 15: - Add decimal-util-benchmark.
> You could include the result of the benchmark in the commit message. Be car
done, but ran in a docker container, will that be a problem?
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc
File be/src/benchmarks/decimal-util-benchmark.cc:
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc@30
PS3, Line 30: // raw 0.704 0.704 0.717 1X 1X 1X
> "for (int i = 0; i < n; ++n)" would be more readable in my opinion.
Done
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc@31
PS3, Line 31:
> Nit: scale.
Done
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc@32
PS3, Line 32:
> constexpr int?
Done
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc@33
PS3, Line 33: m
> We shadow the input parameter 'n' here, it would be better to choose a diff
Done
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc@51
PS3, Line 51: }
> If we don't use the result, the compiler will probably optimise out the cal
done.
But I found another approach : https://stackoverflow.com/questions/7083482#answer-58758133.
that is add '__asm__ volatile("" : "+g" (m) : :);' after GetScaleMultiplier().
the result is 1400X, which has big difference with current.
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc@69
PS3, Line 69: <int
> If you get errors complaining about context switches (you may get them afte
I didn't meet this situation.
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h
File be/src/util/decimal-util.h:
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@262
PS3, Line 262: 00ll,
> We could add unit tests in decimal-test.cc for GetScaleMultiplier().
Done
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@262
PS3, Line 262: i10e18 * 1000000000000000ll,
> I've got a few suggestions that also apply to the other specialisations of
Done
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@264
PS3, Line 264: e18 *
> Could be constexpr.
Done
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@264
PS3, Line 264: i10e18 * 100000000000000000ll,
> The size of the array could be specified explicitly, for example:
Done
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@342
PS3, Line 342: i10e1
> Could be a static_assert.
Done
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@342
PS3, Line 342: * i10e18 * i10e18 * 1000000000ll
> Could use 'num_values' instead.
use 'num_values' instead of "sizeof(values) / sizeof(int256_t)"? it seems that the whole assert statement is meaningless.
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@343
PS3, Line 343: *
> Could use 'num_values' instead.
Done
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 4
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Tue, 27 Sep 2022 07:57:30 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 3:
(14 comments)
http://gerrit.cloudera.org:8080/#/c/18861/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18861/3//COMMIT_MSG@15
PS3, Line 15: - Add decimal-util-benchmark.
You could include the result of the benchmark in the commit message. Be careful to run the benchmark using a release build.
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc
File be/src/benchmarks/decimal-util-benchmark.cc:
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc@30
PS3, Line 30: while (n-- > 0) {
"for (int i = 0; i < n; ++n)" would be more readable in my opinion.
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc@31
PS3, Line 31: sacle
Nit: scale.
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc@32
PS3, Line 32: int
constexpr int?
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc@33
PS3, Line 33: n
We shadow the input parameter 'n' here, it would be better to choose a different variable name.
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc@51
PS3, Line 51: GetScaleMultiplier(j);
If we don't use the result, the compiler will probably optimise out the call entirely.
We should use the result in some way that is also cheap so it doesn't change the running time much.
For example we can try summing the results and at the end of the function write it to some volatile (global) variable. But we need to make sure that overflow is defined, i.e. don't use a signed type to store the sum.
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc@69
PS3, Line 69: 1000
If you get errors complaining about context switches (you may get them after doing what my comment on L51 says), you can lower this number.
The idea is that one call to the benchmarking function should complete without context switches but if it is too long it will not be possible.
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h
File be/src/util/decimal-util.h:
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@262
PS3, Line 262: GetScaleMultiplier
We could add unit tests in decimal-test.cc for GetScaleMultiplier().
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@262
PS3, Line 262: inline int256_t DecimalUtil::GetScaleMultiplier<int256_t>(int scale) {
I've got a few suggestions that also apply to the other specialisations of GetScaleMultiplier(). Now that we touch this file they could also be updated.
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@264
PS3, Line 264: const
Could be constexpr.
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@264
PS3, Line 264: static const int256_t values[] = {
The size of the array could be specified explicitly, for example:
static constexpr int num_values = 77;
static constexpr int256_t values[num_values] = ...
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@342
PS3, Line 342: DCHECK_GE
Could be a static_assert.
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@342
PS3, Line 342: sizeof(values) / sizeof(int256_t)
Could use 'num_values' instead.
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/util/decimal-util.h@343
PS3, Line 343: 77
Could use 'num_values' instead.
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 3
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 26 Sep 2022 10:38:38 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11430/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 3
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Sun, 25 Sep 2022 12:30:24 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11499/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 5
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Fri, 30 Sep 2022 10:39:01 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 8:
(3 comments)
Thanks, only a few minor remarks.
http://gerrit.cloudera.org:8080/#/c/18861/8/be/src/benchmarks/decimal-util-benchmark.cc
File be/src/benchmarks/decimal-util-benchmark.cc:
http://gerrit.cloudera.org:8080/#/c/18861/8/be/src/benchmarks/decimal-util-benchmark.cc@25
PS8, Line 25: // Summary result(Ran within a Docker container):
Nit: add a space before the parenthesis.
http://gerrit.cloudera.org:8080/#/c/18861/8/be/src/benchmarks/decimal-util-benchmark.cc@25
PS8, Line 25: Ran
Nit: ran, no need for capital letter.
http://gerrit.cloudera.org:8080/#/c/18861/8/be/src/benchmarks/decimal-util-benchmark.cc@54
PS8, Line 54: // A variable to accept the result of the target function,
I think this would be better/more understantable:
"A volatile variable to hold the return value of the benchmarked functions. Used to prevent the compiler from optimising out the calls."
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 8
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Thu, 06 Oct 2022 13:23:13 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 7:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8655/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 7
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Wed, 05 Oct 2022 09:41:42 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Xiang Yang (Code Review)" <ge...@cloudera.org>.
Xiang Yang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 4:
(2 comments)
Hi Daniel, I still have a question to discuss, Looking forward to your reply, thanks.
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc
File be/src/benchmarks/decimal-util-benchmark.cc:
http://gerrit.cloudera.org:8080/#/c/18861/3/be/src/benchmarks/decimal-util-benchmark.cc@51
PS3, Line 51: }
> I experimented a bit with it and a simple global static volatile variable m
I guess the difference is a total time comparison, incuding the target function's cost, and the addition's cost, and from int256 to int64 conversion's cost. Because different test method's distribution ratio are different, the result is different.
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/runtime/decimal-test.cc
File be/src/runtime/decimal-test.cc:
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/runtime/decimal-test.cc@1022
PS4, Line 1022: TestGetScaleMultiplier<int32_t>(DecimalUtil::INT32_SCALE_UPPER_BOUND);
> We could also test with double, I saw this function used with double in the
Since we didn't specialize the double edition, I think we needn't to test it.
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 4
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Wed, 28 Sep 2022 12:05:28 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Xiang Yang (Code Review)" <ge...@cloudera.org>.
Xiang Yang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 6:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18861/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18861/4//COMMIT_MSG@7
PS4, Line 7:
> nit: Lack of space
Done
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 6
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Fri, 30 Sep 2022 10:40:23 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 8:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11547/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 8
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Thu, 06 Oct 2022 02:30:54 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 6:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/18861/6//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18861/6//COMMIT_MSG@12
PS6, Line 12:
Indicate also here how much the speedup is resulting from this template specialisation.
http://gerrit.cloudera.org:8080/#/c/18861/6/be/src/benchmarks/decimal-util-benchmark.cc
File be/src/benchmarks/decimal-util-benchmark.cc:
http://gerrit.cloudera.org:8080/#/c/18861/6/be/src/benchmarks/decimal-util-benchmark.cc@26
PS6, Line 26: // Machine Info: AMD Ryzen 9 5950X 16-Core Processor
I agree with Lipenglin, we should mention that it was done in a Docker container.
http://gerrit.cloudera.org:8080/#/c/18861/6/be/src/benchmarks/decimal-util-benchmark.cc@53
PS6, Line 53: static volatile int64_t black_hole = 0;
I think in this case 'volatile_var' or something similar would be more descriptive - when assigning to this variable, the reader could have an idea that we have this variable to block the compiler from removing the code.
Also, a comment before this variable should describe why we have it.
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/util/decimal-util.h
File be/src/util/decimal-util.h:
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/util/decimal-util.h@37
PS4, Line 37: // The scale upper bound for GetScaleMultiplier<int32_t>()
: static constexpr int INT32_SCALE_UPPER_BOUND = ColumnType::MAX_DECIMAL4_PRECISION + 1;
: // The scale upper bound for GetScaleMultiplier<int64_t>()
: static constexpr int INT64_SCALE_UPPER_BOUND = ColumnType::MAX_DECIMAL8_PRECISION + 1;
: // The scale upper bound for GetScaleMultiplier<int128_t>()
: static constexpr int INT128_SCALE_UPPER_BOUND = ColumnType::MAX_PRECISION + 1;
: // The scale upper bound for GetScaleMultiplier<int256_t>()
: static constexpr int INT256_SCALE_UPPER_BOUND = 77;
> These constants are referenced in decimal-test.cc and decimal-util-benchmar
Ok, we can let them stay.
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 6
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Mon, 03 Oct 2022 07:42:41 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 10:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8678/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 10
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Mon, 10 Oct 2022 11:40:06 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 10: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 10
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Mon, 10 Oct 2022 11:40:05 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Xiang Yang (Code Review)" <ge...@cloudera.org>.
Hello Daniel Becker, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18861
to look at the new patch set (#4).
Change subject: IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
Currently decimal-util.h didn't specialize DecimalUtil
::GetScaleMultiplier<int256_t>(), causing more performance loss when
calculate Decimal16Value division.
Testing:
- Ran existing jobs.
- Add decimal-util-benchmark.
Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/decimal-util-benchmark.cc
M be/src/runtime/decimal-test.cc
M be/src/util/decimal-util.h
4 files changed, 246 insertions(+), 30 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/18861/4
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 4
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Xiang Yang (Code Review)" <ge...@cloudera.org>.
Xiang Yang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/util/decimal-util.h
File be/src/util/decimal-util.h:
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/util/decimal-util.h@37
PS4, Line 37: // The scale upper bound for GetScaleMultiplier<int32_t>()
: static constexpr int INT32_SCALE_UPPER_BOUND = ColumnType::MAX_DECIMAL4_PRECISION + 1;
: // The scale upper bound for GetScaleMultiplier<int64_t>()
: static constexpr int INT64_SCALE_UPPER_BOUND = ColumnType::MAX_DECIMAL8_PRECISION + 1;
: // The scale upper bound for GetScaleMultiplier<int128_t>()
: static constexpr int INT128_SCALE_UPPER_BOUND = ColumnType::MAX_PRECISION + 1;
: // The scale upper bound for GetScaleMultiplier<int256_t>()
: static constexpr int INT256_SCALE_UPPER_BOUND = 77;
> I don't think these should be public class-level constants, they should be
These constants are referenced in decimal-test.cc and decimal-util-benchmark.cc, if we don't make them public, we should redefine other constants with same value in decimal-test.cc and decimal-util-benchmark.cc. Do you have any better solution?
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 4
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Fri, 30 Sep 2022 09:50:49 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504:Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11452/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 4
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 27 Sep 2022 07:34:44 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Xiang Yang (Code Review)" <ge...@cloudera.org>.
Xiang Yang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 8:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/18861/6//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18861/6//COMMIT_MSG@12
PS6, Line 12:
> Indicate also here how much the speedup is resulting from this template spe
Done
http://gerrit.cloudera.org:8080/#/c/18861/6/be/src/benchmarks/decimal-util-benchmark.cc
File be/src/benchmarks/decimal-util-benchmark.cc:
http://gerrit.cloudera.org:8080/#/c/18861/6/be/src/benchmarks/decimal-util-benchmark.cc@26
PS6, Line 26: //
> I agree with Lipenglin, we should mention that it was done in a Docker cont
Done
http://gerrit.cloudera.org:8080/#/c/18861/6/be/src/benchmarks/decimal-util-benchmark.cc@53
PS6, Line 53:
> I think in this case 'volatile_var' or something similar would be more desc
Done
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/util/decimal-util.h
File be/src/util/decimal-util.h:
http://gerrit.cloudera.org:8080/#/c/18861/4/be/src/util/decimal-util.h@37
PS4, Line 37: // The scale's exclusive upper bound for GetScaleMultiplier<int32_t>()
: static constexpr int INT32_SCALE_UPPER_BOUND = ColumnType::MAX_DECIMAL4_PRECISION + 1;
: // The scale's exclusive upper bound for GetScaleMultiplier<int64_t>()
: static constexpr int INT64_SCALE_UPPER_BOUND = ColumnType::MAX_DECIMAL8_PRECISION + 1;
: // The scale's exclusive upper bound for GetScaleMultiplier<int128_t>()
: static constexpr int INT128_SCALE_UPPER_BOUND = ColumnType::MAX_PRECISION + 1;
: // The scale's exclusive upper bound for GetScaleMultiplier<int256_t>()
: static constexpr int INT256_SCALE_UPPER_BOUND = 77;
> Ok, we can let them stay.
Done
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 8
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Thu, 06 Oct 2022 02:13:55 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Xiang Yang (Code Review)" <ge...@cloudera.org>.
Hello Daniel Becker, lipenglin@sensorsdata.cn, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18861
to look at the new patch set (#8).
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
Currently decimal-util.h didn't specialize DecimalUtil
::GetScaleMultiplier<int256_t>(), causing more performance loss when
calculate Decimal16Value division.
This template specialisation results in 5600X speedup approximately.
Testing:
- Ran existing jobs.
- Add decimal-util-benchmark.
Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/decimal-util-benchmark.cc
M be/src/runtime/decimal-test.cc
M be/src/util/decimal-util.h
4 files changed, 235 insertions(+), 30 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/18861/8
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 8
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Xiang Yang (Code Review)" <ge...@cloudera.org>.
Hello Daniel Becker, lipenglin@sensorsdata.cn, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18861
to look at the new patch set (#9).
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
Currently decimal-util.h didn't specialize DecimalUtil
::GetScaleMultiplier<int256_t>(), causing more performance loss when
calculate Decimal16Value division.
This template specialisation results in 5600X speedup approximately.
Testing:
- Ran existing jobs.
- Add decimal-util-benchmark.
Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
---
M be/src/benchmarks/CMakeLists.txt
A be/src/benchmarks/decimal-util-benchmark.cc
M be/src/runtime/decimal-test.cc
M be/src/util/decimal-util.h
4 files changed, 235 insertions(+), 30 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/18861/9
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 9
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 6: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8645/
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 6
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Fri, 30 Sep 2022 15:48:58 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier().
Posted by "Xiang Yang (Code Review)" <ge...@cloudera.org>.
Xiang Yang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18861 )
Change subject: IMPALA-11504: Specializing DecimalUtil::GetScaleMultiplier<int256_t>().
......................................................................
Patch Set 9:
Thank you for your patient guidance!
--
To view, visit http://gerrit.cloudera.org:8080/18861
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I969e2977d51313e738f72c8246db003ae43a3782
Gerrit-Change-Number: 18861
Gerrit-PatchSet: 9
Gerrit-Owner: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Mon, 10 Oct 2022 10:14:34 +0000
Gerrit-HasComments: No