You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "ankitsultana (via GitHub)" <gi...@apache.org> on 2023/10/08 05:44:00 UTC
[I] Support for Dictionary Based Group-By [pinot]
ankitsultana opened a new issue, #11759:
URL: https://github.com/apache/pinot/issues/11759
In a recent discussion with @itschrispeck on some related issue, we wondered if it would make sense to have support for executing Group-By by bypassing DocIdSetOperator, and instead using a Dictionary + Inverted Index.
It could help optimize many use-cases, particularly those where a user wants to run some transform function on a group-by/count query e.g.
```
select
case
when AirTime < 100 then
'ok'
else 'not-ok'
end as airtime_category,
count(*)
from airlineStats
where AirlineID = 19805
group by airtime_category
limit 10
```
We can also optimize json_extract_scalar queries (assuming user has json index on the column):
```
explain plan for select
count(*),
json_extract_scalar(group_json, '$.group_city', 'STRING')
from meetupRsvpJson
group by json_extract_scalar(group_json, '$.group_city', 'STRING')
limit 1000
```
Creating this issue to test the waters and see how other folks feel about this. If there's support we can follow-up with a design doc with more details. Here's a PoC PR I am using to estimate the perf gains: https://github.com/apache/pinot/pull/11758
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Support for Dictionary Based Group-By [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #11759:
URL: https://github.com/apache/pinot/issues/11759#issuecomment-5706514788
This issue has been automatically marked as stale because it has not had any activity in the last 150 days. It will be closed in 30 days if no further activity occurs. If this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Support for Dictionary Based Group-By [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #11759:
URL: https://github.com/apache/pinot/issues/11759#issuecomment-1751931701
This would be a great optimization when we group by a single column, or an expression on a single column
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Support for Dictionary Based Group-By [pinot]
Posted by "wirybeaver (via GitHub)" <gi...@apache.org>.
wirybeaver commented on issue #11759:
URL: https://github.com/apache/pinot/issues/11759#issuecomment-1837355589
> This would be a great optimization when we group by a single column, or an expression on a single column
Low cardinality column is likely to enjoy the benefit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Support for Dictionary Based Group-By [pinot]
Posted by "ankitsultana (via GitHub)" <gi...@apache.org>.
ankitsultana commented on issue #11759:
URL: https://github.com/apache/pinot/issues/11759#issuecomment-1789494825
To keep folks updated, we are starting work on this and hope to close this in November. There's a couple of use-cases where this will help internally and we are trying to find the scope of changes we want to do. cc: @Jackie-Jiang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Support for Dictionary Based Group-By [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #11759:
URL: https://github.com/apache/pinot/issues/11759#issuecomment-6280339026
This issue has been closed because it has been inactive for 180 days. If you believe this issue is still relevant, please feel free to reopen it or create a new issue. Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Support for Dictionary Based Group-By [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #11759: Support for Dictionary Based Group-By
URL: https://github.com/apache/pinot/issues/11759
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org