You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "saurabhd336 (via GitHub)" <gi...@apache.org> on 2023/05/03 09:26:16 UTC
[GitHub] [pinot] saurabhd336 opened a new issue, #10713: Adding sorted / columns with inv index to noDictionaryColumns followed by a reload causes query failures
saurabhd336 opened a new issue, #10713:
URL: https://github.com/apache/pinot/issues/10713
Adding any columns to `noDictionaryColumns` list that aren't eligible for removing dictionary (like sorted, columns with inv index etc https://github.com/apache/pinot/blob/master/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/ForwardIndexHandler.java#L362), followed by reload causes query failures since the DictionaryReader isn't created even though the segment metadata still says it has a dictionary.
The indexReader creator seems to honour the current table config over what the physical segment metadata says about the available indexes https://github.com/apache/pinot/blob/master/pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/IndexReaderFactory.java#L60
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] saurabhd336 commented on issue #10713: Adding sorted / columns with inv index to noDictionaryColumns followed by a reload causes query failures
Posted by "saurabhd336 (via GitHub)" <gi...@apache.org>.
saurabhd336 commented on issue #10713:
URL: https://github.com/apache/pinot/issues/10713#issuecomment-1532711076
@gortiz
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] gortiz commented on issue #10713: Adding sorted / columns with inv index to noDictionaryColumns followed by a reload causes query failures
Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on issue #10713:
URL: https://github.com/apache/pinot/issues/10713#issuecomment-1533123219
Here we have a case where the user explicitly say they don't want to have a dictionary but some other index requires the dictionary. Historically, in this situation we decided to ignore user request and create the dictionary anyway, but the new isn't consistent: It doesn't fail at validation time and it fails at runtime as the code follows a path we don't expect.
It is clear that we need to either go back to the previous behavior or break compatibility. My suggestion is to introduce these special cases in `DictionaryIndex.getConfig` (or more specifically in `createDeserializer`). Other alternatives are focused on changing the behavior at reading time, but that would break the key idea introduced in `index-spi` where getConfig returns the actual configuration the index should use and therefore the invariant that FieldIndexConfigs can be seen as the source of trust.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] egalpin commented on issue #10713: Adding sorted / columns with inv index to noDictionaryColumns followed by a reload causes query failures
Posted by "egalpin (via GitHub)" <gi...@apache.org>.
egalpin commented on issue #10713:
URL: https://github.com/apache/pinot/issues/10713#issuecomment-1585154450
I've just confirmed that #10851 also fixes #10875. I validated locally using the same approach which was previously resulting in error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang closed issue #10713: Adding sorted / columns with inv index to noDictionaryColumns followed by a reload causes query failures
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang closed issue #10713: Adding sorted / columns with inv index to noDictionaryColumns followed by a reload causes query failures
URL: https://github.com/apache/pinot/issues/10713
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org