You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/05/29 05:46:38 UTC
[GitHub] [arrow] cyb70289 opened a new pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
cyb70289 opened a new pull request #7300:
URL: https://github.com/apache/arrow/pull/7300
Copy/Invert bitmap in words improves performance significantly.
Partial aligned case jumps from 914M/s to 6.6G/s, and non aligned
case from 82M/s to 3.8G/s on my test machine.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
wesm commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-636401125
Is there a JIRA about fixing the legibility of the benchmark diff?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] fsaintjacques commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
fsaintjacques commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-636849068
@ursabot --help
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm closed pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
wesm closed pull request #7300:
URL: https://github.com/apache/arrow/pull/7300
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-636864585
[AMD64 Ubuntu 18.04 C++ Benchmark (#108741)](https://ci.ursalabs.org/#builders/73/builds/67) builder has been succeeded.
Revision: be1ecf709e862a84284ed354239859a15a32d702
```diff
========================= ========== =========== ========
benchmark baseline contender change
========================= ========== =========== ========
CopyBitmapWithOffset/8192 5.9798e+08 4.4428e+09 6.42968
========================= ========== =========== ========
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] fsaintjacques commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
fsaintjacques commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-636844431
@ursabot benchmark --suite-filter=arrow-bit-util-benchmark --benchmark-filter=CopyBitmapWithOffset origin/master
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] cyb70289 commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
cyb70289 commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-635776890
There are similar codes in my three recent patches optimizing unaligned bitmap operations (logical, compare, copy). I opened a new jira to refine. https://issues.apache.org/jira/browse/ARROW-8979
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-637030201
[AMD64 Ubuntu 18.04 C++ Benchmark (#108807)](https://ci.ursalabs.org/#builders/73/builds/69) builder has been succeeded.
Revision: be1ecf709e862a84284ed354239859a15a32d702
```diff
========================= =============== ============= ========
benchmark baseline contender change
========================= =============== ============= ========
CopyBitmapWithOffset/8192 570.298 MiB/sec 4.261 GiB/sec 665.006%
========================= =============== ============= ========
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-636863219
```
Usage: @ursabot benchmark [OPTIONS] [<baseline>]
Run the benchmark suite in comparison mode.
This command will run the benchmark suite for tip of the branch commit
against `<baseline>` (or master if not provided).
Examples:
# Run the all the benchmarks
@ursabot benchmark
# Compare only benchmarks where the name matches the /^Sum/ regex
@ursabot benchmark --benchmark-filter=^Sum
# Compare only benchmarks where the suite matches the /compute-/ regex.
# A suite is the C++ binary.
@ursabot benchmark --suite-filter=compute-
# Sometimes a new optimization requires the addition of new benchmarks to
# quantify the performance increase. When doing this be sure to add the
# benchmark in a separate commit before introducing the optimization.
#
# Note that specifying the baseline is the only way to compare using a new
# benchmark, since master does not contain the new benchmark and no
# comparison is possible.
#
# The following command compares the results of matching benchmarks,
# compiling against HEAD and the provided baseline commit, e.g. eaf8302.
# You can use this to quantify the performance improvement of new
# optimizations or to check for regressions.
@ursabot benchmark --benchmark-filter=MyBenchmark eaf8302
Options:
--suite-filter <regex> Regex filtering benchmark suites.
--benchmark-filter <regex> Regex filtering benchmarks.
--help Show this message and exit.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot removed a comment on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
ursabot removed a comment on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-636858419
[AMD64 Ubuntu 18.04 C++ Benchmark (#108740)](https://ci.ursalabs.org/#builders/73/builds/66) builder was cancelled.
Revision: be1ecf709e862a84284ed354239859a15a32d702
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] fsaintjacques commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
fsaintjacques commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-637024748
@ursabot benchmark --suite-filter=arrow-bit-util-benchmark --benchmark-filter=CopyBitmapWithOffset origin/master
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs removed a comment on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
kszucs removed a comment on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-636857436
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-636860479
@ursabot build
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
wesm commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-637070516
Needs rebase
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] fsaintjacques removed a comment on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
fsaintjacques removed a comment on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-636849068
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] cyb70289 commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
cyb70289 commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-635773456
Tested on AMD Zen1 with gcc-7.5
before (CopyBitmapWithOffsetBoth is added by this patch)
```
CopyBitmapWithoutOffset/8192 190 ns 190 ns 3678223 bytes_per_second=40.0804G/s
CopyBitmapWithOffset/8192 8548 ns 8548 ns 81846 bytes_per_second=914.007M/s
CopyBitmapWithOffsetBoth/8192 95447 ns 95445 ns 7334 bytes_per_second=81.8531M/s
```
after
```
CopyBitmapWithoutOffset/8192 191 ns 191 ns 3670699 bytes_per_second=39.9939G/s
CopyBitmapWithOffset/8192 1151 ns 1151 ns 600632 bytes_per_second=6.62768G/s
CopyBitmapWithOffsetBoth/8192 2018 ns 2018 ns 346761 bytes_per_second=3.78089G/s
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-636857436
@ursabot build
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-636857905
@ursabot benchmark
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] fsaintjacques commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
fsaintjacques commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-637030938
once https://github.com/ursa-labs/ursabot/pull/197 is merged, we shouldn't need to specify `origin/master` as a baseline.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] fsaintjacques commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
fsaintjacques commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-636863207
@ursabot benchmark --help
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs removed a comment on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
kszucs removed a comment on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-636860479
@ursabot build
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-636858419
[AMD64 Ubuntu 18.04 C++ Benchmark (#108740)](https://ci.ursalabs.org/#builders/73/builds/66) builder was cancelled.
Revision: be1ecf709e862a84284ed354239859a15a32d702
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] cyb70289 commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
cyb70289 commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-635798096
@ursabot benchmark --suite-filter=arrow-bit-util-benchmark --benchmark-filter=CopyBitmapWithOffset
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
wesm commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-637166114
The RTools 4.0 build was hanging so I triggered a new build
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-635802797
[AMD64 Ubuntu 18.04 C++ Benchmark (#108408)](https://ci.ursalabs.org/#builders/73/builds/65) builder has been succeeded.
Revision: ef4d77ceabd23b51e052e4da58310ebacdc0fe9d
```diff
============================= =========== =========== ============
benchmark baseline contender change
============================= =========== =========== ============
CopyBitmapWithOffsetBoth/8192 2.48133e+09 2.48132e+09 -3.12173e-06
CopyBitmapWithOffset/8192 4.14268e+09 4.14273e+09 1.20158e-05
============================= =========== =========== ============
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-636858234
@ursabot benchmark --suite-filter=arrow-bit-util-benchmark --benchmark-filter=CopyBitmapWithOffset origin/master
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] fsaintjacques commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
fsaintjacques commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-635967095
> [AMD64 Ubuntu 18.04 C++ Benchmark (#108408)](https://ci.ursalabs.org/#builders/73/builds/65) builder has been succeeded.
>
> Revision: [ef4d77c](https://github.com/apache/arrow/commit/ef4d77ceabd23b51e052e4da58310ebacdc0fe9d)
>
> ```diff
> ============================= =========== =========== ============
> benchmark baseline contender change
> ============================= =========== =========== ============
> CopyBitmapWithOffsetBoth/8192 2.48133e+09 2.48132e+09 -3.12173e-06
> CopyBitmapWithOffset/8192 4.14268e+09 4.14273e+09 1.20158e-05
> ```
I'll need to fix this, tried to address this last time, weirdly it was fixed locally...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #7300: ARROW-8844: [C++] Transfer bitmap in words
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #7300:
URL: https://github.com/apache/arrow/pull/7300#issuecomment-635772970
https://issues.apache.org/jira/browse/ARROW-8844
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org