You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "xiangfu0 (via GitHub)" <gi...@apache.org> on 2023/05/28 06:04:50 UTC
[GitHub] [pinot] xiangfu0 opened a new pull request, #10813: [multi-stage]Adding ExchangeNode.Type for SubPlan/PlanFragment/PipelineBreaker
xiangfu0 opened a new pull request, #10813:
URL: https://github.com/apache/pinot/pull/10813
Adding ExchangeNode.Type for `SubPlan`/`PlanFragment`/`PipelineBreaker`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #10813: [multi-stage]Adding ExchangeNode.Type for SubPlan/PlanFragment/PipelineBreaker
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr commented on code in PR #10813:
URL: https://github.com/apache/pinot/pull/10813#discussion_r1210611522
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/planner/plannode/ExchangeNode.java:
##########
@@ -102,4 +110,17 @@ public List<RelFieldCollation> getCollations() {
public Set<String> getTableNames() {
return _tableNames;
}
+
+ /** Type of exchange. */
+ public enum Type {
Review Comment:
suggest name it as ExchangeType. later we want to also get rid of RelDistribution.Type dependency from Calcite and we might add DistributionType Enum as well here
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/planner/logical/RelToPlanNodeConverter.java:
##########
@@ -120,8 +120,8 @@ private static PlanNode convertLogicalExchange(Exchange node, int currentStageId
// Compute all the tables involved under this exchange node
Set<String> tableNames = getTableNamesFromRelRoot(node);
- return new ExchangeNode(currentStageId, toDataSchema(node.getRowType()), tableNames, node.getDistribution(),
- fieldCollations, isSortOnSender, isSortOnReceiver);
+ return new ExchangeNode(currentStageId, toDataSchema(node.getRowType()), ExchangeNode.Type.PLAN_FRAGMENT,
+ tableNames, node.getDistribution(), fieldCollations, isSortOnSender, isSortOnReceiver);
Review Comment:
this is the only place to add exchange type which is hard-coded. what's the plan to make a decision on whether to use SUBPLAN or FRAGMENT?
need to address the following questions
- does it use RelHint?
- if so which RelHint to use and how does it apply to Exchange
this PR should provide an example for utilizing it to create something other than PLAN_FRAGMENT exchange
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr merged pull request #10813: [multi-stage]Adding ExchangeNode.Type for SubPlan/PlanFragment/PipelineBreaker
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr merged PR #10813:
URL: https://github.com/apache/pinot/pull/10813
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #10813: [multi-stage]Adding ExchangeNode.Type for SubPlan/PlanFragment/PipelineBreaker
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #10813:
URL: https://github.com/apache/pinot/pull/10813#issuecomment-1565937592
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/10813?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
> Merging [#10813](https://app.codecov.io/gh/apache/pinot/pull/10813?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (d1eccf2) into [master](https://app.codecov.io/gh/apache/pinot/commit/51bf75efa65cbe8bd8b497eb20e34869205d74e8?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (51bf75e) will **decrease** coverage by `56.59%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #10813 +/- ##
=============================================
- Coverage 70.22% 13.63% -56.59%
+ Complexity 6525 439 -6086
=============================================
Files 2164 2110 -54
Lines 116369 113884 -2485
Branches 17599 17301 -298
=============================================
- Hits 81719 15533 -66186
- Misses 28952 97082 +68130
+ Partials 5698 1269 -4429
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `?` | |
| unittests2 | `13.63% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://app.codecov.io/gh/apache/pinot/pull/10813?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...he/pinot/query/planner/logical/PlanFragmenter.java](https://app.codecov.io/gh/apache/pinot/pull/10813?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcGxhbm5lci9sb2dpY2FsL1BsYW5GcmFnbWVudGVyLmphdmE=) | `0.00% <0.00%> (-91.12%)` | :arrow_down: |
| [.../query/planner/logical/RelToPlanNodeConverter.java](https://app.codecov.io/gh/apache/pinot/pull/10813?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcGxhbm5lci9sb2dpY2FsL1JlbFRvUGxhbk5vZGVDb252ZXJ0ZXIuamF2YQ==) | `0.00% <0.00%> (-94.18%)` | :arrow_down: |
| [...pinot/query/planner/logical/SubPlanFragmenter.java](https://app.codecov.io/gh/apache/pinot/pull/10813?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcGxhbm5lci9sb2dpY2FsL1N1YlBsYW5GcmFnbWVudGVyLmphdmE=) | `0.00% <0.00%> (-64.11%)` | :arrow_down: |
| [...che/pinot/query/planner/plannode/ExchangeNode.java](https://app.codecov.io/gh/apache/pinot/pull/10813?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcGxhbm5lci9wbGFubm9kZS9FeGNoYW5nZU5vZGUuamF2YQ==) | `0.00% <0.00%> (-80.00%)` | :arrow_down: |
... and [1704 files with indirect coverage changes](https://app.codecov.io/gh/apache/pinot/pull/10813/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on pull request #10813: [multi-stage]Adding ExchangeNode.Type for SubPlan/PlanFragment/PipelineBreaker
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr commented on PR #10813:
URL: https://github.com/apache/pinot/pull/10813#issuecomment-1590192112
updating this PR with the suggestions above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org