You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "edg956 (via GitHub)" <gi...@apache.org> on 2026/02/13 20:35:02 UTC
[I] Certain queries produce `QueryExecutionErrors` [pinot]
edg956 opened a new issue, #17701:
URL: https://github.com/apache/pinot/issues/17701
Hi.
While fixing our integration with PinotDb I have found what seem to be some bugs in the query compilation or execution process.
For context, we (OpenMetadata) have a stack of metrics we usually compute for our users' tables as part of our data profiling feature. I reckon non-OpenMetadata users might find themselves writing similar queries for their data observability needs.
I'll detail them separately in the order I have discovered them:
<details>
<summary>
Comparing with floats in some nested functions throws a <code>NumberFormatException</code>
</summary>
One of those metrics is actually a histogram, which for all other DBs we compute with a query roughly like this:
```sql
select
sum(case when (age < 35.0) then 1 else 0 end) as age_group_1,
sum(case when (age >= 35.0 and age < 45.0) then 1 else 0 end) age_group_2,
sum(case when (age >= 45.0) then 1 else 0 end) age_group_3
from data_source;
```
Which returns the following error:
```text
Error Code: 200
QueryExecutionError:
Received error query execution result block: {200=QueryExecutionError:
org.apache.pinot.spi.exception.BadQueryRequestException: java.lang.NumberFormatException: For input string: "35.0"
at org.apache.pinot.core.operator.filter.predicate.PredicateEvaluatorProvider.getPredicateEvaluator(PredicateEvaluatorProvider.java:94)
at org.apache.pinot.core.operator.filter.predicate.PredicateEvaluatorProvider.getPredicateEvaluator(PredicateEvaluatorProvider.java:100)
at org.apache.pinot.core.plan.FilterPlanNode.constructPhysicalOperator(FilterPlanNode.java:310)
at org.apache.pinot.core.plan.FilterPlanNode.run(FilterPlanNode.java:93)
...
Caused by: java.lang.NumberFormatException: For input string: "35.0"
at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
at java.base/java.lang.Integer.parseInt(Integer.java:668)
at java.base/java.lang.Integer.parseInt(Integer.java:786)
at org.apache.pinot.segment.local.segment.index.readers.IntDictionary.insertionIndexOf(IntDictionary.java:44)}
org.apache.pinot.query.service.dispatch.QueryDispatcher.runReducer(QueryDispatcher.java:306)
org.apache.pinot.query.service.dispatch.QueryDispatcher.submitAndReduce(QueryDispatcher.java:96)
org.apache.pinot.broker.requesthandler.MultiStageBrokerRequestHandler.handleRequest(MultiStageBrokerRequestHandler.java:219)
org.apache.pinot.broker.requesthandler.BaseBrokerRequestHandler.handleRequest(BaseBrokerRequestHandler.java:133)
```
This is solved when changing the boundaries to integers.
I initially thought it was an issue with comparing different types but, on top of discovering that it also happens on float<->float comparisons, I discovered another issue:
</details>
<details>
<summary>Nested casting can also break query execution</summary>
I attempted to cast the column to float as a fix (before realizing float<->float comparisons also fail), and got the following:
```sql
select
sum(case when (cast(age as float) < 35.0) then 1 else 0 end) as age_group_1,
sum(case when (cast(age as float) >= 35.0 and cast(age as float) < 45.0) then 1 else 0 end) age_group_2,
sum(case when (cast(age as float) >= 45.0) then 1 else 0 end) age_group_3
from data_source;
```
```text
ProcessingException(errorCode:150, message:SQLParsingError:
java.lang.Exception: Unable to find table for this query
at org.apache.pinot.controller.api.resources.PinotQueryResource.getMultiStageQueryResponse(PinotQueryResource.java:222)
at org.apache.pinot.controller.api.resources.PinotQueryResource.executeSqlQuery(PinotQueryResource.java:179)
at org.apache.pinot.controller.api.resources.PinotQueryResource.handlePostSql(PinotQueryResource.java:127)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
...
Caused by: java.lang.RuntimeException: Error composing query plan for: select
sum(case when (cast(age as float) < 35.0) then 1 else 0 end) as age_group_1,
sum(case when (cast(age as float) >= 35.0 and cast(age as float) < 45.0) then 1 else 0 end) age_group_2,
sum(case when (cast(age as float) >= 45.0) then 1 else 0 end) age_group_3
from data_source;
...
Caused by: java.lang.IllegalArgumentException: Table does not exist: 'data_source'
at org.apache.pinot.shaded.com.google.common.base.Preconditions.checkArgument(Preconditions.java:145)
at org.apache.pinot.query.catalog.PinotCatalog.getTable(PinotCatalog.java:78)
at org.apache.calcite.jdbc.SimpleCalciteSchema.getImplicitTable(SimpleCalciteSchema.java:128)
at org.apache.calcite.jdbc.CalciteSchema.getTable(CalciteSchema.java:283))
```
</details>
<details>
<summary>Finally, I discovered the source of these errors: <code>SUM</code> breaks the whole thing</summary>
If I changed the query to:
```sql
select
case when (cast(age as float) < 35.0) then 1 else 0 end as age_group_1,
case when (cast(age as float) >= 35.0 and cast(age as float) < 45.0) then 1 else 0 end age_group_2,
case when (cast(age as float) >= 45.0) then 1 else 0 end age_group_3
from data_source;
```
And got 1's and 0's where I expected. So there's must be something fishy going about `SUM`.
</details>
For the record I am using PinotDB 1.2.0 in a simple docker compose deployment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Certain queries produce `QueryExecutionErrors` [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #17701:
URL: https://github.com/apache/pinot/issues/17701#issuecomment-5403183243
This issue has been closed because it has been inactive for 180 days. If you believe this issue is still relevant, please feel free to reopen it or create a new issue. Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Certain queries produce `QueryExecutionErrors` [pinot]
Posted by "edg956 (via GitHub)" <gi...@apache.org>.
edg956 commented on issue #17701:
URL: https://github.com/apache/pinot/issues/17701#issuecomment-3923077762
@Jackie-Jiang @gortiz @yashmayya here's the exception for the second query
```
Error Code: 720
QueryPlanningError:
Error composing query plan for: select
sum(case when (cast(age as float) < 35.0) then 1 else 0 end) as age_group_1,
sum(case when (cast(age as float) >= 35.0 and cast(age as float) < 45.0) then 1 else 0 end) age_group_2,
sum(case when (cast(age as float) >= 45.0) then 1 else 0 end) age_group_3
from profiler_test_table;
org.apache.pinot.query.QueryEnvironment.planQuery(QueryEnvironment.java:138)
org.apache.pinot.broker.requesthandler.MultiStageBrokerRequestHandler.handleRequest(MultiStageBrokerRequestHandler.java:158)
org.apache.pinot.broker.requesthandler.BaseBrokerRequestHandler.handleRequest(BaseBrokerRequestHandler.java:133)
org.apache.pinot.broker.requesthandler.BrokerRequestHandlerDelegate.handleRequest(BrokerRequestHandlerDelegate.java:86)
class org.apache.calcite.rex.RexCall cannot be cast to class org.apache.calcite.rex.RexInputRef (org.apache.calcite.rex.RexCall and org.apache.calcite.rex.RexInputRef are in unnamed module of loader 'app')
org.apache.pinot.query.planner.logical.RexExpressionUtils.handleSearch(RexExpressionUtils.java:173)
org.apache.pinot.query.planner.logical.RexExpressionUtils.fromRexCall(RexExpressionUtils.java:133)
org.apache.pinot.query.planner.logical.RexExpressionUtils.fromRexNode(RexExpressionUtils.java:58)
org.apache.pinot.query.planner.logical.RexExpressionUtils.fromRexNodes(RexExpressionUtils.java:67)
```
I should say that I just tested all queries with multi-stage engine off and they worked, but since we use queries that are only supported by the multi-stage engine I discarded that option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Certain queries produce `QueryExecutionErrors` [pinot]
Posted by "edg956 (via GitHub)" <gi...@apache.org>.
edg956 commented on issue #17701:
URL: https://github.com/apache/pinot/issues/17701#issuecomment-3923044803
@Jackie-Jiang apologies, I must've ran this query after changing the name of the table to post it here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Certain queries produce `QueryExecutionErrors` [pinot]
Posted by "edg956 (via GitHub)" <gi...@apache.org>.
edg956 commented on issue #17701:
URL: https://github.com/apache/pinot/issues/17701#issuecomment-3923381717
@yashmayya understood. Thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Certain queries produce `QueryExecutionErrors` [pinot]
Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on issue #17701:
URL: https://github.com/apache/pinot/issues/17701#issuecomment-3923360909
@edg956 Pinot 1.2.0 is a pretty old version. I know for a fact that the `class org.apache.calcite.rex.RexCall cannot be cast to class org.apache.calcite.rex.RexInputRef` issue has been fixed a while ago. Similarly for the `NumberFormatException` issue, I'm unable to reproduce that on the latest code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Certain queries produce `QueryExecutionErrors` [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #17701:
URL: https://github.com/apache/pinot/issues/17701#issuecomment-3922985735
cc @gortiz @yashmayya
The second exception points to `Table does not exist: 'data_source'`, so probably a different issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Certain queries produce `QueryExecutionErrors` [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #17701:
URL: https://github.com/apache/pinot/issues/17701#issuecomment-5064877025
This issue has been automatically marked as stale because it has not had any activity in the last 150 days. It will be closed in 30 days if no further activity occurs. If this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Certain queries produce `QueryExecutionErrors` [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #17701: Certain queries produce `QueryExecutionErrors`
URL: https://github.com/apache/pinot/issues/17701
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org