You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "dang-stripe (via GitHub)" <gi...@apache.org> on 2024/10/31 17:53:28 UTC
[I] `stageStats` exceptions when querying empty tables with multistage [pinot]
dang-stripe opened a new issue, #14350:
URL: https://github.com/apache/pinot/issues/14350
We've noticed some exceptions in `stageStats` when queries contain empty tables for multistage.
## IndexOutOfBounds
Simple query to an empty table:
```
select * from table_v1_OFFLINE limit 10
```
Query response contains IndexOutOfBoundsException:
```
{
"resultTable": {
"dataSchema": {
"columnNames": [
"column1",
"column2",
"column3",
],
"columnDataTypes": [
"STRING",
"STRING",
"STRING",
]
},
"rows": []
},
"numRowsResultSet": 0,
"partialResult": false,
"exceptions": [],
"numGroupsLimitReached": false,
"maxRowsInJoinReached": false,
"maxRowsInWindowReached": false,
"timeUsedMs": 10,
"stageStats": {
"error": "Error encountered while collecting multi-stage stats - java.lang.IndexOutOfBoundsException: Index 2 out of bounds for length 2"
},
"maxRowsInOperator": 0,
"requestId": "126145356000000188",
"brokerId": "Broker_broker-1_8099",
"numDocsScanned": 0,
"totalDocs": 0,
...
}
```
Exception from logs
```
[2024-10-31 17:38:09.029739] WARN [MultiStageBrokerRequestHandler] [jersey-server-managed-async-executor-2:16] Error encountered while collecting multi-stage stats
[2024-10-31 17:38:09.029787] java.lang.IndexOutOfBoundsException: Index 2 out of bounds for length 2
[2024-10-31 17:38:09.029798] at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64) ~[?:?]
[2024-10-31 17:38:09.029811] at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70) ~[?:?]
[2024-10-31 17:38:09.029824] at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248) ~[?:?]
[2024-10-31 17:38:09.029833] at java.base/java.util.Objects.checkIndex(Objects.java:374) ~[?:?]
[2024-10-31 17:38:09.029840] at java.base/java.util.ArrayList.get(ArrayList.java:459) ~[?:?]
[2024-10-31 17:38:09.029856] at org.apache.pinot.query.runtime.MultiStageStatsTreeBuilder.jsonStatsByStage(MultiStageStatsTreeBuilder.java:38) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 17:38:09.029872] at org.apache.pinot.query.runtime.InStageStatsTreeBuilder.visitMailboxReceive(InStageStatsTreeBuilder.java:149) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 17:38:09.029890] at org.apache.pinot.query.runtime.InStageStatsTreeBuilder.visitMailboxReceive(InStageStatsTreeBuilder.java:50) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 17:38:09.029905] at org.apache.pinot.query.planner.plannode.MailboxReceiveNode.visit(MailboxReceiveNode.java:102) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 17:38:09.029930] at org.apache.pinot.query.runtime.InStageStatsTreeBuilder.recursiveCase(InStageStatsTreeBuilder.java:120) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 17:38:09.029946] at org.apache.pinot.query.runtime.InStageStatsTreeBuilder.visitSort(InStageStatsTreeBuilder.java:166) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 17:38:09.029962] at org.apache.pinot.query.runtime.InStageStatsTreeBuilder.visitSort(InStageStatsTreeBuilder.java:50) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 17:38:09.029975] at org.apache.pinot.query.planner.plannode.SortNode.visit(SortNode.java:59) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 17:38:09.029988] at org.apache.pinot.query.runtime.InStageStatsTreeBuilder.recursiveCase(InStageStatsTreeBuilder.java:120) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 17:38:09.030004] at org.apache.pinot.query.runtime.InStageStatsTreeBuilder.visitMailboxSend(InStageStatsTreeBuilder.java:156) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 17:38:09.030018] at org.apache.pinot.query.runtime.InStageStatsTreeBuilder.visitMailboxSend(InStageStatsTreeBuilder.java:50) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 17:38:09.030034] at org.apache.pinot.query.planner.plannode.MailboxSendNode.visit(MailboxSendNode.java:102) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 17:38:09.030048] at org.apache.pinot.query.runtime.MultiStageStatsTreeBuilder.jsonStatsByStage(MultiStageStatsTreeBuilder.java:41) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 17:38:09.030062] at org.apache.pinot.query.runtime.InStageStatsTreeBuilder.visitMailboxReceive(InStageStatsTreeBuilder.java:149) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 17:38:09.030076] at org.apache.pinot.query.runtime.InStageStatsTreeBuilder.visitMailboxReceive(InStageStatsTreeBuilder.java:50) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 17:38:09.030089] at org.apache.pinot.query.planner.plannode.MailboxReceiveNode.visit(MailboxReceiveNode.java:102) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 17:38:09.030121] at org.apache.pinot.query.runtime.MultiStageStatsTreeBuilder.jsonStatsByStage(MultiStageStatsTreeBuilder.java:41) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
```
## NullPointerException
We've also seen NullPointerException in `stageStats` for more complex join queries that contain an empty table.
```
[2024-10-31 04:21:35.325052] WARN [MultiStageBrokerRequestHandler] [jersey-server-managed-async-executor-0:16] Error encountered while collecting multi-stage stats
[2024-10-31 04:21:35.325093] java.lang.NullPointerException: null
[2024-10-31 04:21:35.325109] at org.apache.pinot.query.runtime.InStageStatsTreeBuilder.<init>(InStageStatsTreeBuilder.java:60) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 04:21:35.325125] at org.apache.pinot.query.runtime.MultiStageStatsTreeBuilder.jsonStatsByStage(MultiStageStatsTreeBuilder.java:40) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 04:21:35.325139] at org.apache.pinot.query.runtime.InStageStatsTreeBuilder.visitMailboxReceive(InStageStatsTreeBuilder.java:149) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
[2024-10-31 04:21:35.325153] at org.apache.pinot.query.runtime.InStageStatsTreeBuilder.visitMailboxReceive(InStageStatsTreeBuilder.java:50) ~[pinot-all-1.2.0-2024-10-21-2ec5437773-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2024-10-21-2ec5437773-SNAPSHOT-2ec543777306becb1332ba65b50497d3778e7aeb]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] `stageStats` exceptions when querying empty tables with multistage [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang closed issue #14350: `stageStats` exceptions when querying empty tables with multistage
URL: https://github.com/apache/pinot/issues/14350
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] `stageStats` exceptions when querying empty tables with multistage [pinot]
Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on issue #14350:
URL: https://github.com/apache/pinot/issues/14350#issuecomment-2451399972
We added better error handling for this a few months ago here - https://github.com/apache/pinot/pull/13496.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] `stageStats` exceptions when querying empty tables with multistage [pinot]
Posted by "dang-stripe (via GitHub)" <gi...@apache.org>.
dang-stripe commented on issue #14350:
URL: https://github.com/apache/pinot/issues/14350#issuecomment-2455837094
@Jackie-Jiang we are running on 1.2 but it looks like @gortiz has a new fix in https://github.com/apache/pinot/pull/14374
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] `stageStats` exceptions when querying empty tables with multistage [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #14350:
URL: https://github.com/apache/pinot/issues/14350#issuecomment-2450612817
@gortiz Can you help take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] `stageStats` exceptions when querying empty tables with multistage [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #14350:
URL: https://github.com/apache/pinot/issues/14350#issuecomment-2451405359
@dang-stripe Can you help verify that? It is available in `1.2`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org