You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Jackie-Jiang (via GitHub)" <gi...@apache.org> on 2023/05/24 17:36:08 UTC
[GitHub] [pinot] Jackie-Jiang opened a new issue, #10800: Upsert snapshot should be taken before a new consuming segment is created
Jackie-Jiang opened a new issue, #10800:
URL: https://github.com/apache/pinot/issues/10800
Currently, when snapshot is enabled, snapshot is taken when shutting down the server. This will cause inconsistency because the changes made by the new consuming segment has already been reflected into the snapshots, but when we restart the servers and reload all the segments, the consuming segment will be re-consumed.
This problem is more critical for partial-upsert table because the snapshot will mark the old record as invalid, but the new record still sit in the consuming segment thus not persisted.
The fix should be to persist the snapshot when a new consuming segment is created but before any record is consumed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang closed issue #10800: Upsert snapshot should be taken before a new consuming segment is created
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang closed issue #10800: Upsert snapshot should be taken before a new consuming segment is created
URL: https://github.com/apache/pinot/issues/10800
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #10800: Upsert snapshot should be taken before a new consuming segment is created
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #10800:
URL: https://github.com/apache/pinot/issues/10800#issuecomment-1561678393
cc @ankitsultana we need to consider this issue when solving #10552
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org