You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/09/03 03:25:59 UTC
[GitHub] [arrow] jianxind opened a new pull request #8103: ARROW-9904: [C++] Unroll the loop of CountSetBits.
jianxind opened a new pull request #8103:
URL: https://github.com/apache/arrow/pull/8103
1. Unroll the loop of CountSetBits to generate parallel instructions.
2. Add count benchmark
Signed-off-by: Frank Du <fr...@intel.com>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #8103: ARROW-9904: [C++] Unroll the loop of CountSetBits.
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #8103:
URL: https://github.com/apache/arrow/pull/8103#discussion_r482910530
##########
File path: cpp/src/arrow/compute/kernels/aggregate_benchmark.cc
##########
@@ -398,5 +398,17 @@ MINMAX_KERNEL_BENCHMARK(MinMaxKernelInt16, Int16Type);
MINMAX_KERNEL_BENCHMARK(MinMaxKernelInt32, Int32Type);
MINMAX_KERNEL_BENCHMARK(MinMaxKernelInt64, Int64Type);
+static void CountKernelBenchInt64(benchmark::State& state) {
+ RegressionArgs args(state);
+ const int64_t array_size = args.size / sizeof(int64_t);
+ auto rand = random::RandomArrayGenerator(1923);
+ auto array = rand.Numeric<Int64Type>(array_size, -100, 100, args.null_proportion);
+
+ for (auto _ : state) {
+ ABORT_NOT_OK(Count(array).status());
Review comment:
The null count is only computed once and then cached on the array, so this benchmark isn't correct.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jianxind commented on pull request #8103: ARROW-9904: [C++] Unroll the loop of CountSetBits.
Posted by GitBox <gi...@apache.org>.
jianxind commented on pull request #8103:
URL: https://github.com/apache/arrow/pull/8103#issuecomment-686230245
Below is the results on the new count benchmark for two devices on gcc and clang.
CLX 8280L | gcc default | gcc unrolled | gcc speed up | clang default | clang unrolled | clang speed up
-- | -- | -- | -- | -- | -- | --
Count benchmark(G/s) | 662.234 | 813.84 | 1.23 | 576.81 | 815.193 | 1.413278203
i7-8700 | gcc default | gcc unrolled | gcc speed up | clang default | clang unrolled | clang speed up
-- | -- | -- | -- | -- | -- | --
Count benchmark(G/s) | 622.174 | 736.495 | 1.18 | 559.015 | 743.022 | 1.329162903
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jianxind commented on a change in pull request #8103: ARROW-9904: [C++] Unroll the loop of CountSetBits.
Posted by GitBox <gi...@apache.org>.
jianxind commented on a change in pull request #8103:
URL: https://github.com/apache/arrow/pull/8103#discussion_r483344463
##########
File path: cpp/src/arrow/compute/kernels/aggregate_benchmark.cc
##########
@@ -398,5 +398,17 @@ MINMAX_KERNEL_BENCHMARK(MinMaxKernelInt16, Int16Type);
MINMAX_KERNEL_BENCHMARK(MinMaxKernelInt32, Int32Type);
MINMAX_KERNEL_BENCHMARK(MinMaxKernelInt64, Int64Type);
+static void CountKernelBenchInt64(benchmark::State& state) {
+ RegressionArgs args(state);
+ const int64_t array_size = args.size / sizeof(int64_t);
+ auto rand = random::RandomArrayGenerator(1923);
+ auto array = rand.Numeric<Int64Type>(array_size, -100, 100, args.null_proportion);
+
+ for (auto _ : state) {
+ ABORT_NOT_OK(Count(array).status());
Review comment:
Indeed it's computed again every time. The exec engine runs on batch iteration though for this case only one batch actually occurs. Each iteration use Slice to fetch the part of input array.
See https://github.com/apache/arrow/blob/master/cpp/src/arrow/compute/exec.cc#L174
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jianxind commented on a change in pull request #8103: ARROW-9904: [C++] Unroll the loop of CountSetBits.
Posted by GitBox <gi...@apache.org>.
jianxind commented on a change in pull request #8103:
URL: https://github.com/apache/arrow/pull/8103#discussion_r483359355
##########
File path: cpp/src/arrow/compute/kernels/aggregate_benchmark.cc
##########
@@ -398,5 +398,17 @@ MINMAX_KERNEL_BENCHMARK(MinMaxKernelInt16, Int16Type);
MINMAX_KERNEL_BENCHMARK(MinMaxKernelInt32, Int32Type);
MINMAX_KERNEL_BENCHMARK(MinMaxKernelInt64, Int64Type);
+static void CountKernelBenchInt64(benchmark::State& state) {
+ RegressionArgs args(state);
+ const int64_t array_size = args.size / sizeof(int64_t);
+ auto rand = random::RandomArrayGenerator(1923);
+ auto array = rand.Numeric<Int64Type>(array_size, -100, 100, args.null_proportion);
+
+ for (auto _ : state) {
+ ABORT_NOT_OK(Count(array).status());
Review comment:
Just add a path to Slice if it's a exactly copy of original. Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou closed pull request #8103: ARROW-9904: [C++] Unroll the loop of CountSetBits.
Posted by GitBox <gi...@apache.org>.
pitrou closed pull request #8103:
URL: https://github.com/apache/arrow/pull/8103
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jianxind edited a comment on pull request #8103: ARROW-9904: [C++] Unroll the loop of CountSetBits.
Posted by GitBox <gi...@apache.org>.
jianxind edited a comment on pull request #8103:
URL: https://github.com/apache/arrow/pull/8103#issuecomment-686230245
Below is the results on the new count benchmark for two devices on gcc and clang.
CLX 8280L | gcc default | gcc unrolled | gcc speed up | clang default | clang unrolled | clang speed up
-- | -- | -- | -- | -- | -- | --
Count benchmark(G/s) | 499.845 | 680.873 | 1.36 | 523.869 | 667.37 | 1.27
i7-8700 | gcc default | gcc unrolled | gcc speed up | clang default | clang unrolled | clang speed up
-- | -- | -- | -- | -- | -- | --
Count benchmark(G/s) | 492.842 | 631.361 | 1.28 | 499.446 | 651.659 | 1.30
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jianxind commented on a change in pull request #8103: ARROW-9904: [C++] Unroll the loop of CountSetBits.
Posted by GitBox <gi...@apache.org>.
jianxind commented on a change in pull request #8103:
URL: https://github.com/apache/arrow/pull/8103#discussion_r483345094
##########
File path: cpp/src/arrow/compute/kernels/aggregate_benchmark.cc
##########
@@ -398,5 +398,17 @@ MINMAX_KERNEL_BENCHMARK(MinMaxKernelInt16, Int16Type);
MINMAX_KERNEL_BENCHMARK(MinMaxKernelInt32, Int32Type);
MINMAX_KERNEL_BENCHMARK(MinMaxKernelInt64, Int64Type);
+static void CountKernelBenchInt64(benchmark::State& state) {
+ RegressionArgs args(state);
+ const int64_t array_size = args.size / sizeof(int64_t);
+ auto rand = random::RandomArrayGenerator(1923);
+ auto array = rand.Numeric<Int64Type>(array_size, -100, 100, args.null_proportion);
+
+ for (auto _ : state) {
+ ABORT_NOT_OK(Count(array).status());
Review comment:
Should I change this benchmark to use Slice in case the exec engine optimize the iteration?
ABORT_NOT_OK(Count(array->Slice(0, array_size)).status());
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8103: ARROW-9904: [C++] Unroll the loop of CountSetBits.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8103:
URL: https://github.com/apache/arrow/pull/8103#issuecomment-686230855
https://issues.apache.org/jira/browse/ARROW-9904
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org