You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "walterddr (via GitHub)" <gi...@apache.org> on 2023/03/28 16:35:47 UTC
[GitHub] [pinot] walterddr opened a new pull request, #10493: [multistage][wip] worker assignment dispatch partition
walterddr opened a new pull request, #10493:
URL: https://github.com/apache/pinot/pull/10493
previously with https://github.com/apache/pinot/pull/10157 we dispatch 1 request per virtual server. this causes lots of issues when parallelism is high --> as broker becomes the bottleneck creating dispatch payloads
This PR makes VirutalServer containing
- physical `ServerInstance`
- list of `partitionIds`
This way the dispatch is generating 1 request per single physical server
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #10493: [multistage][wip] worker assignment dispatch partition
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #10493:
URL: https://github.com/apache/pinot/pull/10493#issuecomment-1487319202
## [Codecov](https://codecov.io/gh/apache/pinot/pull/10493?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#10493](https://codecov.io/gh/apache/pinot/pull/10493?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2688821) into [master](https://codecov.io/gh/apache/pinot/commit/c0d0896ac30ca04229f641b960c2722d98398de3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c0d0896) will **decrease** coverage by `10.49%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #10493 +/- ##
=============================================
- Coverage 24.44% 13.96% -10.49%
- Complexity 49 237 +188
=============================================
Files 2042 2034 -8
Lines 111146 109706 -1440
Branches 16933 16773 -160
=============================================
- Hits 27174 15320 -11854
- Misses 81132 93135 +12003
+ Partials 2840 1251 -1589
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| unittests2 | `13.96% <0.00%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/10493?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../org/apache/pinot/query/routing/VirtualServer.java](https://codecov.io/gh/apache/pinot/pull/10493?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcm91dGluZy9WaXJ0dWFsU2VydmVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...ache/pinot/query/routing/VirtualServerAddress.java](https://codecov.io/gh/apache/pinot/pull/10493?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcm91dGluZy9WaXJ0dWFsU2VydmVyQWRkcmVzcy5qYXZh) | `0.00% <ø> (ø)` | |
| [.../org/apache/pinot/query/routing/WorkerManager.java](https://codecov.io/gh/apache/pinot/pull/10493?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcm91dGluZy9Xb3JrZXJNYW5hZ2VyLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...va/org/apache/pinot/query/runtime/QueryRunner.java](https://codecov.io/gh/apache/pinot/pull/10493?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcnVudGltZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcnVudGltZS9RdWVyeVJ1bm5lci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...query/runtime/operator/MailboxReceiveOperator.java](https://codecov.io/gh/apache/pinot/pull/10493?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcnVudGltZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcnVudGltZS9vcGVyYXRvci9NYWlsYm94UmVjZWl2ZU9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...ot/query/runtime/operator/MailboxSendOperator.java](https://codecov.io/gh/apache/pinot/pull/10493?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcnVudGltZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcnVudGltZS9vcGVyYXRvci9NYWlsYm94U2VuZE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...t/query/runtime/plan/ServerRequestPlanVisitor.java](https://codecov.io/gh/apache/pinot/pull/10493?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcnVudGltZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcnVudGltZS9wbGFuL1NlcnZlclJlcXVlc3RQbGFuVmlzaXRvci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [.../query/runtime/plan/serde/QueryPlanSerDeUtils.java](https://codecov.io/gh/apache/pinot/pull/10493?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcnVudGltZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcnVudGltZS9wbGFuL3NlcmRlL1F1ZXJ5UGxhblNlckRlVXRpbHMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [963 files with indirect coverage changes](https://codecov.io/gh/apache/pinot/pull/10493/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr closed pull request #10493: [multistage][wip] worker assignment dispatch partition
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr closed pull request #10493: [multistage][wip] worker assignment dispatch partition
URL: https://github.com/apache/pinot/pull/10493
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on pull request #10493: [multistage][wip] worker assignment dispatch partition
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr commented on PR #10493:
URL: https://github.com/apache/pinot/pull/10493#issuecomment-1516781203
closing as i discussed with @Jackie-Jiang we are mixing up serialized info and non-serialized context and causing some unrealistic refactor if we decided to go down this path. will redesign the changes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org