You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Jackie-Jiang (via GitHub)" <gi...@apache.org> on 2025/12/18 23:09:08 UTC
[I] Enhance table config validation on index type compatibility [pinot]
Jackie-Jiang opened a new issue, #17398:
URL: https://github.com/apache/pinot/issues/17398
We should enhance index type compatibility check in table config validation. E.g. inverted index enabled column must have dictionary enabled.
We should list down all supported index types, and do an exhaustive check on all incompatible combinations.
For code reference, table config validation is performed in `TableConfigUtils.validate(TableConfig tableConfig, Schema schema)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Enhance table config validation on index type compatibility [pinot]
Posted by "Jaideep-C (via GitHub)" <gi...@apache.org>.
Jaideep-C commented on issue #17398:
URL: https://github.com/apache/pinot/issues/17398#issuecomment-3691653650
Hi @Jackie-Jiang, I would like to work on this issue. I have reviewed the `TableConfigUtils.validate(TableConfig tableConfig, Schema schema)` method and the [Data and Index Types Compatibility Matrix](https://docs.pinot.apache.org/basics/indexing#data-and-index-types-compatibility-matrix) documentation.
My plan is to audit the `IndexType::validate` method implementations across all index types and add the missing validation conditions based on the compatibility matrix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Enhance table config validation on index type compatibility [pinot]
Posted by "rsrkpatwari1234 (via GitHub)" <gi...@apache.org>.
rsrkpatwari1234 commented on issue #17398:
URL: https://github.com/apache/pinot/issues/17398#issuecomment-3765709999
We can divide this task into below validation groups -
1. Ensure columns with inverted/range/fst/ifst index are not set as noDictionaryColumns
2. Ensure sorted columns do not have inverted/bloom/range index set as the later will be redundant
3. Ensure if column has noDictionaryColumns, then only valid indexes are text/json/h3/vector/bloom
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Enhance table config validation on index type compatibility [pinot]
Posted by "RajaVinoth96 (via GitHub)" <gi...@apache.org>.
RajaVinoth96 commented on issue #17398:
URL: https://github.com/apache/pinot/issues/17398#issuecomment-4593920836
Can you please assign this ticket to me?. @Jackie-Jiang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Enhance table config validation on index type compatibility [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #17398:
URL: https://github.com/apache/pinot/issues/17398#issuecomment-3775707497
Thanks @rsrkpatwari1234 for taking this up.
Most validation logic is already available under `IndexType.validate()`. We just need to add the missing ones. Take a look at `RangeIndexType.validate()` as an example
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Enhance table config validation on index type compatibility [pinot]
Posted by "RajaVinoth96 (via GitHub)" <gi...@apache.org>.
RajaVinoth96 commented on issue #17398:
URL: https://github.com/apache/pinot/issues/17398#issuecomment-4692786447
@Jackie-Jiang Can you please review the change in https://github.com/apache/pinot/pull/18751
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org