You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/08/17 04:05:57 UTC
[GitHub] [druid] zoov-w opened a new issue, #12911: Why druid-sql not support Filtered DimensionSpecs
zoov-w opened a new issue, #12911:
URL: https://github.com/apache/druid/issues/12911
### Motivation
we use multi-value column in project, when use sql `group by` that column, we get more value exceeded our expectation. the document:
https://druid.apache.org/docs/latest/querying/multi-value-dimensions.html
so we have to filter value in application, it not a good solution because we have a lot of multi-value column in a multi-dimition table and lots of `group by` query.
so we try to resolve it. add three UDF (prefix_filter, regex_filter, list_filter) translate DefaultDimensionSpec to FilteredDimensionSpec, udf use as follows:
```
select list_filter(singer_ids, '1', '2'), sum(index)
from table
where singer_ids in ('1', '2')
GROUP BY singer_ids
```
singer_ids is multi-value column, the sql query sum index of singer_id 1, 2. `list_filter(singer_ids, '1', '2')` translate to :
```
"dimension" : {
"type": "listFiltered",
"delegate": {
"type": "default",
"dimension": "d0",
"outputName": "d0",
"outputType": "STRING"
},
"values": ["1", "2"]
}
```
we want to push a path but not sure this is a good solution. hope some advice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Why druid-sql not support Filtered DimensionSpecs (druid)
Posted by "jtuglu1 (via GitHub)" <gi...@apache.org>.
jtuglu1 closed issue #12911: Why druid-sql not support Filtered DimensionSpecs
URL: https://github.com/apache/druid/issues/12911
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] gianm commented on issue #12911: Why druid-sql not support Filtered DimensionSpecs
Posted by GitBox <gi...@apache.org>.
gianm commented on issue #12911:
URL: https://github.com/apache/druid/issues/12911#issuecomment-1227926428
@zoov-w we'll close this issue since I think @clintropolis has pointed to a solution, but if you still have issues, please reopen this issue or join us on Slack: https://druid.apache.org/community/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] clintropolis commented on issue #12911: Why druid-sql not support Filtered DimensionSpecs
Posted by GitBox <gi...@apache.org>.
clintropolis commented on issue #12911:
URL: https://github.com/apache/druid/issues/12911#issuecomment-1217466981
Hi @zoov-w, I have added partial support this functionality in #11650, which added `MV_FILTER_ONLY` and `MV_FILTER_NONE` which covers the list filtered functionality. Instead of using the `DimensionSpec` however, I decided to port the functionality to a `VirtualColumn` implementation (which is the way I would prefer we do column transformations). We have been naming all SQL functions that explicitly deal with multi-value strings with the `MV_` prefix, so by that naming the missing functions are `MV_FILTER_REGEX` and `MV_FILTER_PREFIX`. That PR would probably be a good reference if you would like to try to add them as `VirtualColumn`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] gianm closed issue #12911: Why druid-sql not support Filtered DimensionSpecs
Posted by GitBox <gi...@apache.org>.
gianm closed issue #12911: Why druid-sql not support Filtered DimensionSpecs
URL: https://github.com/apache/druid/issues/12911
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] zoov-w commented on issue #12911: Why druid-sql not support Filtered DimensionSpecs
Posted by GitBox <gi...@apache.org>.
zoov-w commented on issue #12911:
URL: https://github.com/apache/druid/issues/12911#issuecomment-1217641524
thank you for your reply @clintropolis. I will try to do it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Why druid-sql not support Filtered DimensionSpecs (druid)
Posted by "maytasm (via GitHub)" <gi...@apache.org>.
maytasm commented on issue #12911:
URL: https://github.com/apache/druid/issues/12911#issuecomment-2541924688
Before re-opening this issue, do we actually have `MV_FILTER_REGEX` and `MV_FILTER_PREFIX` (or any equivalent to those)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org