You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/09/28 14:10:36 UTC
[GitHub] [druid] hosswald opened a new issue, #13152: Unsatisfyable WHERE clause leads to exception instead of empty result
hosswald opened a new issue, #13152:
URL: https://github.com/apache/druid/issues/13152
### Affected Version
0.23.0
### Description
The following query
```sql
SELECT *
FROM "druid"."mydata"
WHERE false
ORDER BY "__time" DESC
```
leads to the following error:
`druid error: Unsupported operation (org.apache.druid.java.util.common.UOE): Time-ordering on scan queries is only supported for queries with segment specs of type MultipleSpecificSegmentSpec or SpecificSegmentSpec...a [MultipleIntervalSegmentSpec] was received instead.`
false can be an arbitrary unsatisfiable boolean expression. I came across it when trying to filter for a specific exact timestamp in Superset, which is translated into the unsatisfiable expression
```sql
WHERE MILLIS_TO_TIMESTAMP(recordTimeParsed) >= TIME_PARSE('2022-09-25T00:00:00')
AND MILLIS_TO_TIMESTAMP(recordTimeParsed) < TIME_PARSE('2022-09-25T00:00:00')
```
Since it seems to be valid vor any unsatisfiable WHERE clause, I'm omitting details about my specific dataset.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] abhishekagarwal87 commented on issue #13152: Unsatisfiable WHERE clause leads to exception instead of empty result
Posted by GitBox <gi...@apache.org>.
abhishekagarwal87 commented on issue #13152:
URL: https://github.com/apache/druid/issues/13152#issuecomment-1263359278
@paul-rogers - since you were reviewing that PR, can you take that PR to completion?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] gianm commented on issue #13152: Unsatisfyable WHERE clause leads to exception instead of empty result
Posted by GitBox <gi...@apache.org>.
gianm commented on issue #13152:
URL: https://github.com/apache/druid/issues/13152#issuecomment-1261288990
I tried this locally and what happens here is:
1. The unsatisfiable filter leads to the table being swapped out with an inline empty recordset. This is good, it means the planner realizes no data can possibly match the filter.
2. The query runs as normal on the inline empty recordset. Here it fails because `scan` query does not support ordering on inline data.
We can fix it by allowing the `scan` query to order inline data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] abhishekagarwal87 commented on issue #13152: Unsatisfiable WHERE clause leads to exception instead of empty result
Posted by GitBox <gi...@apache.org>.
abhishekagarwal87 commented on issue #13152:
URL: https://github.com/apache/druid/issues/13152#issuecomment-1271460821
I can't spot the `sorting` operation in the runner. The data from physical segments are already sorted by time. That ordering is used to decide if the data needs to be reversed. That's why we are only passing the `type of ordering (Asc/desc)` down the `makeCursors` call and not the columns on which to order by. Are you suggesting passing the columns to `makeCursors` and then doing the sorting in `RowBasedStorageAdapter`?
Or you are suggesting doing the sorting in the `createRunner` method itself? The former seems a simpler approach to implement. We will just materialize the rows into a list and sort the list.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] 599166320 commented on issue #13152: Unsatisfiable WHERE clause leads to exception instead of empty result
Posted by GitBox <gi...@apache.org>.
599166320 commented on issue #13152:
URL: https://github.com/apache/druid/issues/13152#issuecomment-1263222889
@abhishekagarwal87 I recently improved the sorting function of common fields, which should solve the sorting problem of inline data together. However, I have recently closed this PR for some reasons. If you are interested, I will reopen this pr and let you review it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] abhishekagarwal87 commented on issue #13152: Unsatisfyable WHERE clause leads to exception instead of empty result
Posted by GitBox <gi...@apache.org>.
abhishekagarwal87 commented on issue #13152:
URL: https://github.com/apache/druid/issues/13152#issuecomment-1261841965
Yeah. I was thinking if we can just make a special case of empty inline data. Since we would be returning an empty sequence anyway. @599166320 was also making some changes in this area recently. that could be relevant.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] gianm commented on issue #13152: Unsatisfiable WHERE clause leads to exception instead of empty result
Posted by GitBox <gi...@apache.org>.
gianm commented on issue #13152:
URL: https://github.com/apache/druid/issues/13152#issuecomment-1270569834
Good question. That's another reason that little sketch isn't ready to be a PR 🙂
I think the answer is, if there's a limit then it's done by stableLimitingSort. If there's no limit then I don't see that it actually does get sorted. That'd need to change.
Your idea sounds good. What would we do if there's no limit? Ideally, since it's inline data and it's not likely to be very big, we sort it anyway. It could be done either in the mergeRunners call, or in the runner created by createRunner (since there's only one runner for inline data). Wondering what you think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] gianm commented on issue #13152: Unsatisfiable WHERE clause leads to exception instead of empty result
Posted by GitBox <gi...@apache.org>.
gianm commented on issue #13152:
URL: https://github.com/apache/druid/issues/13152#issuecomment-1270575521
Thinking about it a bit more. I think it makes the most sense to do the sorting in the runner from createRunner. I actually think that is already happening: check out the makeCursors call in ScanQueryEngine. It is either ascending or descending based on the time order of the query.
So mergeRunners just needs to merge runners that are already sorted. In the inline case, there is only one runner, so there's nothing to do. We can return it as-is.
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org