You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/09/19 08:03:10 UTC
[GitHub] [arrow-datafusion] poonai opened a new pull request, #3529: Use direct indexing for small sized keys to improve group by aggregation performance
poonai opened a new pull request, #3529:
URL: https://github.com/apache/arrow-datafusion/pull/3529
## Note
will update the description after clean up and verifying the performance gain 😅
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] andygrove commented on a diff in pull request #3529: Use direct indexing for small sized keys to improve group by aggregation performance
Posted by GitBox <gi...@apache.org>.
andygrove commented on code in PR #3529:
URL: https://github.com/apache/arrow-datafusion/pull/3529#discussion_r974566908
##########
datafusion/core/src/physical_plan/aggregates/groupby_map.rs:
##########
@@ -0,0 +1,18 @@
+use arrow::datatypes::Schema;
+// Copyright 2022 Balaji (rbalajis25@gmail.com)
Review Comment:
Please remove the copyright statement
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] codecov-commenter commented on pull request #3529: Use direct indexing for small sized keys to improve group by aggregation performance
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #3529:
URL: https://github.com/apache/arrow-datafusion/pull/3529#issuecomment-1252726272
# [Codecov](https://codecov.io/gh/apache/arrow-datafusion/pull/3529?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3529](https://codecov.io/gh/apache/arrow-datafusion/pull/3529?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (75ec7f5) into [master](https://codecov.io/gh/apache/arrow-datafusion/commit/331922036a23265084ab091fa2ab5c57da42c841?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3319220) will **increase** coverage by `0.03%`.
> The diff coverage is `100.00%`.
```diff
@@ Coverage Diff @@
## master #3529 +/- ##
==========================================
+ Coverage 85.75% 85.78% +0.03%
==========================================
Files 299 299
Lines 55311 55391 +80
==========================================
+ Hits 47431 47519 +88
+ Misses 7880 7872 -8
```
| [Impacted Files](https://codecov.io/gh/apache/arrow-datafusion/pull/3529?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...atafusion/core/src/physical\_plan/aggregates/mod.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/3529/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb3JlL3NyYy9waHlzaWNhbF9wbGFuL2FnZ3JlZ2F0ZXMvbW9kLnJz) | `91.41% <ø> (ø)` | |
| [...sion/core/src/physical\_plan/aggregates/row\_hash.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/3529/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb3JlL3NyYy9waHlzaWNhbF9wbGFuL2FnZ3JlZ2F0ZXMvcm93X2hhc2gucnM=) | `96.41% <100.00%> (+0.62%)` | :arrow_up: |
| [datafusion/core/src/physical\_plan/hash\_utils.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/3529/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb3JlL3NyYy9waHlzaWNhbF9wbGFuL2hhc2hfdXRpbHMucnM=) | `41.36% <100.00%> (+0.32%)` | :arrow_up: |
| [datafusion/row/src/accessor.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/3529/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9yb3cvc3JjL2FjY2Vzc29yLnJz) | `61.53% <100.00%> (+5.12%)` | :arrow_up: |
| [datafusion/optimizer/src/simplify\_expressions.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/3529/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9vcHRpbWl6ZXIvc3JjL3NpbXBsaWZ5X2V4cHJlc3Npb25zLnJz) | `82.80% <0.00%> (-0.04%)` | :arrow_down: |
| [datafusion/core/src/execution/context.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/3529/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb3JlL3NyYy9leGVjdXRpb24vY29udGV4dC5ycw==) | `79.29% <0.00%> (-0.03%)` | :arrow_down: |
| [datafusion/common/src/lib.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/3529/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb21tb24vc3JjL2xpYi5ycw==) | `0.00% <0.00%> (ø)` | |
| [datafusion/physical-expr/src/aggregate/count.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/3529/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9waHlzaWNhbC1leHByL3NyYy9hZ2dyZWdhdGUvY291bnQucnM=) | `98.05% <0.00%> (ø)` | |
| [datafusion/physical-expr/src/aggregate/average.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/3529/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9waHlzaWNhbC1leHByL3NyYy9hZ2dyZWdhdGUvYXZlcmFnZS5ycw==) | `94.96% <0.00%> (ø)` | |
| [...tafusion/core/src/physical\_plan/file\_format/mod.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/3529/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb3JlL3NyYy9waHlzaWNhbF9wbGFuL2ZpbGVfZm9ybWF0L21vZC5ycw==) | `96.95% <0.00%> (ø)` | |
| ... and [13 more](https://codecov.io/gh/apache/arrow-datafusion/pull/3529/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] poonai commented on pull request #3529: Use direct indexing for small sized keys to improve group by aggregation performance
Posted by GitBox <gi...@apache.org>.
poonai commented on PR #3529:
URL: https://github.com/apache/arrow-datafusion/pull/3529#issuecomment-1253289532
will reopen this, after working it properly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] poonai closed pull request #3529: Use direct indexing for small sized keys to improve group by aggregation performance
Posted by GitBox <gi...@apache.org>.
poonai closed pull request #3529: Use direct indexing for small sized keys to improve group by aggregation performance
URL: https://github.com/apache/arrow-datafusion/pull/3529
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org