You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "rahul8383 (via GitHub)" <gi...@apache.org> on 2023/05/16 06:47:11 UTC
[GitHub] [pinot] rahul8383 opened a new issue, #10769: Remove Primary Key from Upsert Metadata when a Segment is removed
rahul8383 opened a new issue, #10769:
URL: https://github.com/apache/pinot/issues/10769
The Upsert Metadata of each partition is Maintained in a ConcurrentHashMap with Key as the PrimaryKey and Value as the RecordLocation.
When a segment is expired due to retention, the RecordLocation(value in the map) is removed but the Primary Key is still maintained in the map. Maintaining the expired key in the map will cause Memory overhead which can be reduced if the expired Primary Key is removed.
[code link](https://github.com/apache/pinot/blob/ca37a1e90982b2495632720645ff6a5414508afb/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java#L169)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] rahul8383 closed issue #10769: Remove Primary Key from Upsert Metadata when a Segment is removed
Posted by "rahul8383 (via GitHub)" <gi...@apache.org>.
rahul8383 closed issue #10769: Remove Primary Key from Upsert Metadata when a Segment is removed
URL: https://github.com/apache/pinot/issues/10769
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] rahul8383 commented on issue #10769: Remove Primary Key from Upsert Metadata when a Segment is removed
Posted by "rahul8383 (via GitHub)" <gi...@apache.org>.
rahul8383 commented on issue #10769:
URL: https://github.com/apache/pinot/issues/10769#issuecomment-1549104334
Tested that ConcurrentHashMap.computeIfPresent() removes the key when null is returned in the BiFunction.
This is not an issue. Closing the ticket.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang closed issue #10769: Expire Keys from Upsert Metadata when a Segment is removed due to retention
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang closed issue #10769: Expire Keys from Upsert Metadata when a Segment is removed due to retention
URL: https://github.com/apache/pinot/issues/10769
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org