You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by GitBox <gi...@apache.org> on 2022/07/01 05:51:15 UTC
[GitHub] [incubator-kyuubi] pan3793 opened a new pull request, #2981: Improve TPC-DS scan performance
pan3793 opened a new pull request, #2981:
URL: https://github.com/apache/incubator-kyuubi/pull/2981
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Before
```
OpenJDK 64-Bit Server VM 1.8.0_332-b09 on Mac OS X 12.4
Apple M1 Pro
TPCDS table generates 1000000 rows benchmark: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
----------------------------------------------------------------------------------------------------------------------------
catalog_returns benchmark 13956 13975 21 0.1 13955.7 1.0X
catalog_sales benchmark 10229 10277 42 0.1 10229.2 1.4X
customer benchmark 9305 9464 249 0.1 9305.0 1.5X
customer_address benchmark 5612 5737 162 0.2 5611.7 2.5X
customer_demographics benchmark 1108 1182 66 0.9 1107.5 12.6X
inventory benchmark 665 695 27 1.5 664.7 21.0X
store_returns benchmark 11260 11409 132 0.1 11260.1 1.2X
store_sales benchmark 7894 7909 15 0.1 7894.1 1.8X
web_returns benchmark 13042 13082 38 0.1 13042.1 1.1X
web_sales benchmark 11182 11201 23 0.1 11182.4 1.2X
```
After
```
OpenJDK 64-Bit Server VM 1.8.0_332-b09 on Mac OS X 12.4
Apple M1 Pro
TPCDS table generates 1000000 rows benchmark: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
----------------------------------------------------------------------------------------------------------------------------
catalog_returns benchmark 13496 13836 398 0.1 13496.3 1.0X
catalog_sales benchmark 10084 10119 56 0.1 10084.0 1.3X
customer benchmark 8900 8917 18 0.1 8900.0 1.5X
customer_address benchmark 5287 5295 10 0.2 5287.2 2.6X
customer_demographics benchmark 880 888 6 1.1 880.3 15.3X
inventory benchmark 523 550 34 1.9 522.6 25.8X
store_returns benchmark 10853 10876 40 0.1 10852.9 1.2X
store_sales benchmark 7764 7818 50 0.1 7764.0 1.7X
web_returns benchmark 12840 12863 23 0.1 12839.7 1.1X
web_sales benchmark 10810 10955 193 0.1 10809.6 1.2X
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #2981: Improve TPC-DS scan performance
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #2981:
URL: https://github.com/apache/incubator-kyuubi/pull/2981#issuecomment-1172084270
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2981?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#2981](https://codecov.io/gh/apache/incubator-kyuubi/pull/2981?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0ca494e) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/3bc299d01e1399d6e82cad00419b1c647f9c1915?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3bc299d) will **increase** coverage by `0.02%`.
> The diff coverage is `85.71%`.
```diff
@@ Coverage Diff @@
## master #2981 +/- ##
============================================
+ Coverage 62.79% 62.82% +0.02%
Complexity 6 6
============================================
Files 418 418
Lines 20336 20339 +3
Branches 2570 2570
============================================
+ Hits 12771 12778 +7
+ Misses 6338 6337 -1
+ Partials 1227 1224 -3
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/2981?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../kyuubi/spark/connector/tpcds/TPCDSBatchScan.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZW5zaW9ucy9zcGFyay9reXV1Ymktc3BhcmstY29ubmVjdG9yLXRwY2RzL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL3NwYXJrL2Nvbm5lY3Rvci90cGNkcy9UUENEU0JhdGNoU2Nhbi5zY2FsYQ==) | `85.48% <85.71%> (+2.43%)` | :arrow_up: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `97.19% <0.00%> (+0.09%)` | :arrow_up: |
| [.../apache/kyuubi/client/KyuubiSyncThriftClient.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jbGllbnQvS3l1dWJpU3luY1RocmlmdENsaWVudC5zY2FsYQ==) | `88.28% <0.00%> (+0.41%)` | :arrow_up: |
| [.../org/apache/kyuubi/operation/KyuubiOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uLnNjYWxh) | `69.01% <0.00%> (+1.40%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2981?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2981?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [3bc299d...0ca494e](https://codecov.io/gh/apache/incubator-kyuubi/pull/2981?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] pan3793 closed pull request #2981: Improve TPC-DS scan performance
Posted by GitBox <gi...@apache.org>.
pan3793 closed pull request #2981: Improve TPC-DS scan performance
URL: https://github.com/apache/incubator-kyuubi/pull/2981
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] pan3793 commented on pull request #2981: Improve TPC-DS scan performance
Posted by GitBox <gi...@apache.org>.
pan3793 commented on PR #2981:
URL: https://github.com/apache/incubator-kyuubi/pull/2981#issuecomment-1172145929
Thanks, merging to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] pan3793 commented on pull request #2981: Improve TPC-DS scan performance
Posted by GitBox <gi...@apache.org>.
pan3793 commented on PR #2981:
URL: https://github.com/apache/incubator-kyuubi/pull/2981#issuecomment-1171973376
And I'd like to say, AMD YES 💯
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org