You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/05/14 01:10:33 UTC
[GitHub] [pinot] snleee opened a new issue, #8701: BaseChunkSVForwardIndexWriter's default version can end up storing the overflowed chunkOffset in the header
snleee opened a new issue, #8701:
URL: https://github.com/apache/pinot/issues/8701
BaseChunkSVForwardIndexWriter's default version is V2. V2 uses 4bytes to store chunkOffset. When there are a lot of rows for "noDictionaryColumn", it can actually store overflowed int value for chunkOffset in the header. From V3, we store long value so we won't face this issue.
IMO, the correct behavior should be throwing the exception when we generate the segment if we face the int overflow for chunk Offset.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] richardstartin commented on issue #8701: BaseChunkSVForwardIndexWriter's default version can end up storing the overflowed chunkOffset in the header
Posted by GitBox <gi...@apache.org>.
richardstartin commented on issue #8701:
URL: https://github.com/apache/pinot/issues/8701#issuecomment-1126917997
V4 would also avoid this, has some other advantages discussed [here](https://docs.google.com/document/d/1Y7MyQGmDD2fI7brOOFQtToxd8ML837qRuc3IlNYFvCw/edit) but has a size limit of 4GiB.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] snleee closed issue #8701: BaseChunkSVForwardIndexWriter's default version can end up storing the overflowed chunkOffset in the header
Posted by GitBox <gi...@apache.org>.
snleee closed issue #8701: BaseChunkSVForwardIndexWriter's default version can end up storing the overflowed chunkOffset in the header
URL: https://github.com/apache/pinot/issues/8701
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org