You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "gortiz (via GitHub)" <gi...@apache.org> on 2023/04/05 14:42:49 UTC
[GitHub] [pinot] gortiz opened a new issue, #10550: Unify index validation
gortiz opened a new issue, #10550:
URL: https://github.com/apache/pinot/issues/10550
Each Pinot index have their own constraints. For example, a column of type integer should not have a TEXT index and one with raw encoding should not have a FST index. These constraints are enforced in different ways around the code, which violates RAW and makes it more difficult to change these constraints (as the developer needs to know all the places where the constraint is enforced).
There are also some situations where even when the customer explicitly ask to disable some index or use some specific encoding, the presence of some specific index implies that what the customer explicitly ask is ignored.
There have been some efforts to centralize how these constraints are enforced. For example, `TableConfigUtils.validateIndexConfigs`. But now that we can add indexes, that utility method cannot cover all cases. Ideally we should extend IndexType in order to support some kind of validation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] navina commented on issue #10550: Unify index validation
Posted by "navina (via GitHub)" <gi...@apache.org>.
navina commented on issue #10550:
URL: https://github.com/apache/pinot/issues/10550#issuecomment-1502424292
@gortiz are you going to work on this one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] gortiz commented on issue #10550: Unify index validation
Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on issue #10550:
URL: https://github.com/apache/pinot/issues/10550#issuecomment-1504753772
Not right now. If someone is interested on it, feel free to work on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] gortiz commented on issue #10550: Unify index validation
Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on issue #10550:
URL: https://github.com/apache/pinot/issues/10550#issuecomment-1511683058
Related to what @swaminathanmanish said, I've been working on some changes related to the index documentation. You can find the changes [here](https://github.com/pinot-contrib/pinot-docs/pull/162).
We could move that logic into the dictionary index. But right now the logic applies to all columns. The index configuration is column specific. I think an option like `optimizeDictionaryForMetrics` should be configured per column (I think it makes sense to do not use the same value for all columns), so we could move it there.
But right now there is no way to use the new index configuration objects to configure a property on all columns
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] swaminathanmanish commented on issue #10550: Unify index validation
Posted by "swaminathanmanish (via GitHub)" <gi...@apache.org>.
swaminathanmanish commented on issue #10550:
URL: https://github.com/apache/pinot/issues/10550#issuecomment-1511645897
On a related note - I was recently looking into index auto tuning (eg: dictionary overrides), specifically **optimizeDictionaryForMetrics**, mentioned in this doc. Perhaps these types of auto tuning logic (to enable/disable) should also belong to IndexType interface ?
https://docs.pinot.apache.org/configuration-reference/table#table-index-config
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org