You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/09/29 21:30:03 UTC
[GitHub] [pinot] agavra opened a new pull request, #9497: add debug string to query plan [MINOR]
agavra opened a new pull request, #9497:
URL: https://github.com/apache/pinot/pull/9497
Adds functionality to be able to call `QueryPlan#debugString()` to see an `EXPLAIN`-like string. Here are some examples:
`SELECT * FROM b ORDER BY col1, col2 DESC LIMIT 3`:
```
MAIL_RECEIVE
MAIL_SEND
SORT (LIMIT 3)
MAIL_RECEIVE
MAIL_SEND
TABLE SCAN (b)
```
`SELECT * FROM a JOIN b ON a.col1 = b.col1 JOIN c ON a.col3 = c.col3`:
```
MAIL_RECEIVE
MAIL_SEND
JOIN
MAIL_RECEIVE
MAIL_SEND
JOIN
MAIL_RECEIVE
MAIL_SEND
TABLE SCAN (a)
MAIL_RECEIVE
MAIL_SEND
TABLE SCAN (b)
MAIL_RECEIVE
MAIL_SEND
TABLE SCAN (c)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on pull request #9497: [multi stage] add debug string to query plan
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on PR #9497:
URL: https://github.com/apache/pinot/pull/9497#issuecomment-1263957519
We should consider doing a follow-up to wire this into EXPLAIN - https://github.com/apache/pinot/pull/9280
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] agavra commented on a diff in pull request #9497: add debug string to query plan [MINOR]
Posted by GitBox <gi...@apache.org>.
agavra commented on code in PR #9497:
URL: https://github.com/apache/pinot/pull/9497#discussion_r984779048
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/planner/stage/AggregateNode.java:
##########
@@ -54,4 +54,9 @@ public List<RexExpression> getAggCalls() {
public List<RexExpression> getGroupSet() {
return _groupSet;
}
+
+ @Override
+ public String debugString() {
Review Comment:
@siddharthteotia you read my mind 😉 that's _just_ what I was working on (I'll update PR description with examples once I've worked out the bugs).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #9497: add debug string to query plan [MINOR]
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #9497:
URL: https://github.com/apache/pinot/pull/9497#issuecomment-1262927951
# [Codecov](https://codecov.io/gh/apache/pinot/pull/9497?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#9497](https://codecov.io/gh/apache/pinot/pull/9497?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (01e3af3) into [master](https://codecov.io/gh/apache/pinot/commit/3057712fdc468951d72be9d676c17b5a8f0af850?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3057712) will **increase** coverage by `1.18%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #9497 +/- ##
============================================
+ Coverage 68.66% 69.85% +1.18%
+ Complexity 5136 4869 -267
============================================
Files 1916 1917 +1
Lines 102069 102233 +164
Branches 15481 15511 +30
============================================
+ Hits 70083 71412 +1329
+ Misses 26980 25781 -1199
- Partials 5006 5040 +34
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `25.96% <0.00%> (-0.17%)` | :arrow_down: |
| integration2 | `24.73% <0.00%> (?)` | |
| unittests1 | `67.14% <0.00%> (+0.01%)` | :arrow_up: |
| unittests2 | `15.50% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/9497?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...java/org/apache/pinot/query/planner/QueryPlan.java](https://codecov.io/gh/apache/pinot/pull/9497/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcGxhbm5lci9RdWVyeVBsYW4uamF2YQ==) | `28.57% <0.00%> (-71.43%)` | :arrow_down: |
| [...pache/pinot/query/planner/stage/AggregateNode.java](https://codecov.io/gh/apache/pinot/pull/9497/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcGxhbm5lci9zdGFnZS9BZ2dyZWdhdGVOb2RlLmphdmE=) | `91.66% <0.00%> (-8.34%)` | :arrow_down: |
| [...g/apache/pinot/query/planner/stage/FilterNode.java](https://codecov.io/gh/apache/pinot/pull/9497/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcGxhbm5lci9zdGFnZS9GaWx0ZXJOb2RlLmphdmE=) | `85.71% <0.00%> (-14.29%)` | :arrow_down: |
| [...org/apache/pinot/query/planner/stage/JoinNode.java](https://codecov.io/gh/apache/pinot/pull/9497/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcGxhbm5lci9zdGFnZS9Kb2luTm9kZS5qYXZh) | `94.73% <0.00%> (-5.27%)` | :arrow_down: |
| [.../pinot/query/planner/stage/MailboxReceiveNode.java](https://codecov.io/gh/apache/pinot/pull/9497/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcGxhbm5lci9zdGFnZS9NYWlsYm94UmVjZWl2ZU5vZGUuamF2YQ==) | `90.90% <0.00%> (-9.10%)` | :arrow_down: |
| [...che/pinot/query/planner/stage/MailboxSendNode.java](https://codecov.io/gh/apache/pinot/pull/9497/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcGxhbm5lci9zdGFnZS9NYWlsYm94U2VuZE5vZGUuamF2YQ==) | `90.90% <0.00%> (-9.10%)` | :arrow_down: |
| [.../apache/pinot/query/planner/stage/ProjectNode.java](https://codecov.io/gh/apache/pinot/pull/9497/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcGxhbm5lci9zdGFnZS9Qcm9qZWN0Tm9kZS5qYXZh) | `85.71% <0.00%> (-14.29%)` | :arrow_down: |
| [...org/apache/pinot/query/planner/stage/SortNode.java](https://codecov.io/gh/apache/pinot/pull/9497/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcGxhbm5lci9zdGFnZS9Tb3J0Tm9kZS5qYXZh) | `94.11% <0.00%> (-5.89%)` | :arrow_down: |
| [...pache/pinot/query/planner/stage/TableScanNode.java](https://codecov.io/gh/apache/pinot/pull/9497/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcGxhbm5lci9zdGFnZS9UYWJsZVNjYW5Ob2RlLmphdmE=) | `88.88% <0.00%> (-11.12%)` | :arrow_down: |
| [...rg/apache/pinot/query/planner/stage/ValueNode.java](https://codecov.io/gh/apache/pinot/pull/9497/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcGxhbm5lci9zdGFnZS9WYWx1ZU5vZGUuamF2YQ==) | `28.57% <0.00%> (-2.20%)` | :arrow_down: |
| ... and [163 more](https://codecov.io/gh/apache/pinot/pull/9497/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9497: add debug string to query plan [MINOR]
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on code in PR #9497:
URL: https://github.com/apache/pinot/pull/9497#discussion_r984094036
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/planner/stage/AggregateNode.java:
##########
@@ -54,4 +54,9 @@ public List<RexExpression> getAggCalls() {
public List<RexExpression> getGroupSet() {
return _groupSet;
}
+
+ @Override
+ public String debugString() {
Review Comment:
In the existing engine, we implement `toExplainString` at the physical operator level and each operator also has a EXPLAIN_NAME.
Even in this case, I think the API could be called `explainString` or something similar even though we are doing at the logical node level and not at the operator level.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia merged pull request #9497: add debug string to query plan [MINOR]
Posted by GitBox <gi...@apache.org>.
siddharthteotia merged PR #9497:
URL: https://github.com/apache/pinot/pull/9497
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9497: add debug string to query plan [MINOR]
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on code in PR #9497:
URL: https://github.com/apache/pinot/pull/9497#discussion_r984095507
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/planner/stage/AggregateNode.java:
##########
@@ -54,4 +54,9 @@ public List<RexExpression> getAggCalls() {
public List<RexExpression> getGroupSet() {
return _groupSet;
}
+
+ @Override
+ public String debugString() {
Review Comment:
Ohh I just noticed this is not wired into the EXPLAIN PLAN. We do support EXPLAIN for multi-stage btw. We can probably also do this by wiring this up with EXPLAIN PLAN for multi-stage (can do as a follow up as well).
The output in the description looks reasonably good
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9497: add debug string to query plan [MINOR]
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on code in PR #9497:
URL: https://github.com/apache/pinot/pull/9497#discussion_r984774728
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/planner/stage/AggregateNode.java:
##########
@@ -54,4 +54,9 @@ public List<RexExpression> getAggCalls() {
public List<RexExpression> getGroupSet() {
return _groupSet;
}
+
+ @Override
+ public String debugString() {
Review Comment:
can we also add instances / worker assignment into the output for the overall plan ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9497: add debug string to query plan [MINOR]
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on code in PR #9497:
URL: https://github.com/apache/pinot/pull/9497#discussion_r984095507
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/planner/stage/AggregateNode.java:
##########
@@ -54,4 +54,9 @@ public List<RexExpression> getAggCalls() {
public List<RexExpression> getGroupSet() {
return _groupSet;
}
+
+ @Override
+ public String debugString() {
Review Comment:
Ohh I just noticed this is not wired into the EXPLAIN PLAN. We do support EXPLAIN (just the logical at this point) for multi-stage btw. We can probably also do this by wiring this up with EXPLAIN PLAN for multi-stage (can do as a follow up as well).
The output in the description looks reasonably good
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/planner/stage/AggregateNode.java:
##########
@@ -54,4 +54,9 @@ public List<RexExpression> getAggCalls() {
public List<RexExpression> getGroupSet() {
return _groupSet;
}
+
+ @Override
+ public String debugString() {
Review Comment:
Ohh I just noticed this is not wired into the EXPLAIN PLAN. We do support EXPLAIN (just the logical at this point iirc) for multi-stage btw. We can probably also do this by wiring this up with EXPLAIN PLAN for multi-stage (can do as a follow up as well).
The output in the description looks reasonably good
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9497: add debug string to query plan [MINOR]
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on code in PR #9497:
URL: https://github.com/apache/pinot/pull/9497#discussion_r984774728
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/planner/stage/AggregateNode.java:
##########
@@ -54,4 +54,9 @@ public List<RexExpression> getAggCalls() {
public List<RexExpression> getGroupSet() {
return _groupSet;
}
+
+ @Override
+ public String debugString() {
Review Comment:
can we also add instances / worker assignment into the output ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org