You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/04/27 19:32:00 UTC
[GitHub] [druid] gianm opened a new pull request #11172: Add a way to retrieve UTF-8 bytes directly via DimensionDictionarySelector.
gianm opened a new pull request #11172:
URL: https://github.com/apache/druid/pull/11172
The idea is that certain operations (like count distinct on strings) will
be faster if they are able to run directly on UTF-8 bytes instead of on
Java Strings decoded by "lookupName".
I'm looking into modifying the cardinality and datasketch hll build
aggregators to use lookupNameUtf8 instead of lookupName in a
follow-on patch, and it is able to speed them up quite a bit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] gianm edited a comment on pull request #11172: Add a way to retrieve UTF-8 bytes directly via DimensionDictionarySelector.
Posted by GitBox <gi...@apache.org>.
gianm edited a comment on pull request #11172:
URL: https://github.com/apache/druid/pull/11172#issuecomment-830208867
I was also thinking of using this to enable moving the groupBy dictionaries off-heap (we could copy strings from segments to merge buffers as UTF-8 and avoid string transcoding).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] gianm edited a comment on pull request #11172: Add a way to retrieve UTF-8 bytes directly via DimensionDictionarySelector.
Posted by GitBox <gi...@apache.org>.
gianm edited a comment on pull request #11172:
URL: https://github.com/apache/druid/pull/11172#issuecomment-830208867
I was also thinking of using this to enable moving the groupBy dictionaries off-heap (we could store them in merge buffers as UTF-8 and avoid string decoding).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] gianm merged pull request #11172: Add a way to retrieve UTF-8 bytes directly via DimensionDictionarySelector.
Posted by GitBox <gi...@apache.org>.
gianm merged pull request #11172:
URL: https://github.com/apache/druid/pull/11172
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] gianm commented on pull request #11172: Add a way to retrieve UTF-8 bytes directly via DimensionDictionarySelector.
Posted by GitBox <gi...@apache.org>.
gianm commented on pull request #11172:
URL: https://github.com/apache/druid/pull/11172#issuecomment-830208867
I was also thinking of using this to enable moving the groupBy dictionaries off-heap (we could store them in merge buffers as UTF-8 and avoid string decoding)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org