You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "SebastianPuscas (via GitHub)" <gi...@apache.org> on 2026/03/16 09:33:55 UTC
[I] Calcite execution planner misses timewindow filtering (druid)
SebastianPuscas opened a new issue, #19164:
URL: https://github.com/apache/druid/issues/19164
Please provide a detailed title (e.g. "Broker crashes when using TopN query with Bound filter" instead of just "Broker crashes").
### Affected Version
34
The Druid version where the problem was encountered.
34
### Description
We have a production query using `IN LIST`, the query is extracting data for a given time window when being executed, the many issue is that the timefilter is being lost when execution planner parses the sql query and transfers it to other nodes.
First we have a hot tier with latest 3 months and a cold tier with 3 months to 2 years segments.
It happens that we have many queries inside the cold tier for a datasource which barely has any time-window query specified for more than 3 months.
So we are using on the service side a spring java project which transfers sql statements to druid.
I inspected historical nodes and seen native queries, asking claude to share similar sql query i got the following
Query 1 (88% of traffic — 47,363 hits)
SELECT pbx,
APPROX_COUNT_DISTINCT_DS_HLL(j.callId) AS a0
FROM source_v1
INNERJOIN inline_data j ON c.callId = j.d0
WHERE c.addrType = 'xxxxa'
AND c.addr = 'xxxx'
GROUPBY pbx
on the service side we are building following which return correct data as broker still does the filtering but on cold tier side, I don't see the __time filtering.
SELECT COUNT(DISTINCT callId), pbx
* FROM source_v1
* WHERE ( pbx = :pbx AND addrType = 'xxx' AND addr IN (:LIST)
* AND callId NOT IN ('id1', 'id2', ...) AND __time >= ? AND __time <= ? )
* GROUP BY pbx
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Calcite execution planner misses timewindow filtering (druid)
Posted by "gianm (via GitHub)" <gi...@apache.org>.
gianm commented on issue #19164:
URL: https://github.com/apache/druid/issues/19164#issuecomment-4072133061
This looks like the transformation that happens when the size of an `IN` or `NOT IN` is above `inSubQueryThreshold`. By default it's `Integer.MAX_VALUE` which means we don't do this transformation. Are you setting `inSubQueryThreshold` in your setup?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org