You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "gortiz (via GitHub)" <gi...@apache.org> on 2023/04/20 07:30:02 UTC
[GitHub] [pinot] gortiz opened a new issue, #10652: Support selective nullable columns
gortiz opened a new issue, #10652:
URL: https://github.com/apache/pinot/issues/10652
Pinot support for null support is configured at table level by setting `tableIndexConfg.nullHandlingEnabled` to true in TableConfig.
By setting this to true, all columns in the table are treated as nullable.
Given that nullable columns negatively impact in the heap and/or other optimizations, ideally, we should let users to individually decide the nullability of each column.
In order to do that, we have to:
* Change NullValueIndexType.createDeserializer to actually read `indexes` like other indexes does.
* Change places where `tableIndexConfg.nullHandlingEnabled` is read (directly or indirectly) in order to use `getIndex(StandardIndexes.nullValueVector()).isEnabled()`. This includes
* SegmentColumnarIndexCreator
* BaseDefaultColumnHandler.createDefaultValueColumnV1Indices
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #10652: Support selective nullable columns
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #10652:
URL: https://github.com/apache/pinot/issues/10652#issuecomment-1524119019
When null handling is enabled, the engine should pull the null value vector from the column, and if the vector doesn't exist or is empty, we handle it as non-null. Basically using the actual index as source of choose when processing the queries
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Support selective nullable columns [pinot]
Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on issue #10652:
URL: https://github.com/apache/pinot/issues/10652#issuecomment-1746367623
See https://github.com/apache/pinot/pull/10653 and https://github.com/apache/pinot/issues/10381
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #10652: Support selective nullable columns
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #10652:
URL: https://github.com/apache/pinot/issues/10652#issuecomment-1522324788
From the config perspective, instead of treating null value vector as an index similar to inverted index, I'd suggest putting it as part of the field spec (nullable). See #10428 for prior discussions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] gortiz commented on issue #10652: Support selective nullable columns
Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on issue #10652:
URL: https://github.com/apache/pinot/issues/10652#issuecomment-1522865970
The configuration shouldn't be a problem. IndexType.getConfig receives the schema as an argument, so it should be easy to read the field spec there. I guess I'm biased given that I don't have enough knowledge in that area, but I think the challenge will be in actually modifying the engine behavior in order to understand that the nullability granularity is at column level instead of table level.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org