You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2020/10/21 06:53:43 UTC
[GitHub] [druid] shashanksinghal opened a new issue #10525: Incorrect results (including nulls) when querying string column with col <> '' and col is not null
shashanksinghal opened a new issue #10525:
URL: https://github.com/apache/druid/issues/10525
### Affected Version
0.18.0 and 0.20.0
### Description
For druid 0.18.0 and config useDefaultValueForNull to false, querying a table with condition on string column (lets say col) with condition like col <> '' and col is not null returns rows with that column null. Both of these conditions behave correctly if passed separately but when together, nulls are not filtered at all.
- Cluster size
Local docker setup
- Configurations in use
useDefaultValueForNull is False
- Steps to reproduce the problem
1. Setup local druid 0.18.0 using docker setup
2. Load example data viz. wikipedia
3. Query: select * from wikipedia where cityName is not null and cityName <> '' limit 100
- The error message or stack traces encountered.
In the results you can see rows with cityName null as well
- Any debugging that you have already done
I tested it with version 0.18.0 and 0.20.0 and both have these issue
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] Gahen commented on issue #10525: Incorrect results (including nulls) when querying string column with col <> '' and col is not null
Posted by GitBox <gi...@apache.org>.
Gahen commented on issue #10525:
URL: https://github.com/apache/druid/issues/10525#issuecomment-719056162
We'd also notice this issue to when filtering doing an SQL query like `SELECT * FROM some_table WHERE some_string_field IS NOT NULL AND (NOT some_string_field = 'some value') GROUP BY some_string_field`.
The `EXPLAIN PLAN FOR` seems to show that druid discards the "not null" clause when parsing the SQL as it's the exact same when removing the `some_string_field IS NOT NULL AND` part.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] chenyuzhi459 commented on issue #10525: Incorrect results (including nulls) when querying string column with col <> '' and col is not null
Posted by GitBox <gi...@apache.org>.
chenyuzhi459 commented on issue #10525:
URL: https://github.com/apache/druid/issues/10525#issuecomment-784974922
Hey, i met the same problem with you. In my case, `is not null` is invalid in my sql-query for string column(which is string-array type strictly), and I had fix the problem with pr#10921. Hope it can help you
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] Gahen edited a comment on issue #10525: Incorrect results (including nulls) when querying string column with col <> '' and col is not null
Posted by GitBox <gi...@apache.org>.
Gahen edited a comment on issue #10525:
URL: https://github.com/apache/druid/issues/10525#issuecomment-719056162
We'd also notice this issue to when filtering doing an SQL query like `SELECT * FROM some_table WHERE some_string_field IS NOT NULL AND (NOT some_string_field = 'some value') GROUP BY some_string_field`.
The `EXPLAIN PLAN FOR` seems to show that druid discards the "not null" clause when parsing the SQL as it's the exact same when removing the `some_string_field IS NOT NULL AND` part.
One workaround for us was to use `LIKE` instead of `IS`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] chenyuzhi459 edited a comment on issue #10525: Incorrect results (including nulls) when querying string column with col <> '' and col is not null
Posted by GitBox <gi...@apache.org>.
chenyuzhi459 edited a comment on issue #10525:
URL: https://github.com/apache/druid/issues/10525#issuecomment-784974922
Hey, i met the same problem with you. In my case, `is not null` is invalid in my sql-query for string column(which is string-array type strictly), and I had fix the problem with pr #10921. Hope it can help you
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] shashanksinghal commented on issue #10525: Incorrect results (including nulls) when querying string column with col <> '' and col is not null
Posted by GitBox <gi...@apache.org>.
shashanksinghal commented on issue #10525:
URL: https://github.com/apache/druid/issues/10525#issuecomment-719157152
Thanks @Gahen, another possible solution I found is:
instead of
`select * from receiver where some_string_field is not null and some_string_field != ''`
send
`select * from receiver where NVL(some_string_field, 'nullVal') != 'nullVal' and some_string_field != ''`
NOTE that NVL solves the issue but COALESCE does not.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org