You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/03/29 04:23:10 UTC
[GitHub] [pinot] siddharthteotia edited a comment on pull request #8398: Allow disabling dict generation for High cardinality columns
siddharthteotia edited a comment on pull request #8398:
URL: https://github.com/apache/pinot/pull/8398#issuecomment-1081386541
> I'd like to request reaching consensus here:
>
> * Do we agree that using raw values for high cardinality metrics makes sense? If so, can we enable no-dict for those either by default or under a flag?
> * Do we agree that [Combine Metadata and Dictionary based plans into single operator #8408](https://github.com/apache/pinot/pull/8408) takes care of issue 1 in @siddharthteotia 's comment? And issues 2 and 3 from the same comment don't apply to this PR.
> * We can independently make the heuristic between recommendation engine / auto-tuner and this PR use the same algorithm, but that should be outside the scope of this PR.
>
> If there are more open items, lets reach consensus on this, and move froward with this PR.
>
> cc: @siddharthteotia @richardstartin @Jackie-Jiang
@mayankshriv my comment was more of a FYI to share our learnings from recommender rule engine. I don't mean to increase scope of this PR.
I agree that for high cardinality metrics, no dictionary is a better option.
I think outright if we want to create metric columns with no dictionary, regardless of cardinality, then we should probably do that under a flag to keep the default behavior same as today ?
I will review @richardstartin PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org