You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/06/05 03:12:04 UTC
[GitHub] [pulsar] 315157973 opened a new pull request, #15936: Support shrink for TripleLongPriorityQueue
315157973 opened a new pull request, #15936:
URL: https://github.com/apache/pulsar/pull/15936
### Motivation
Support shrinkage in TripleLongPriorityQueue. Avoid memory waste.
### Modifications
When the actual memory usage is less than the threshold, scale down.
Since the thresholds for expansion and contraction may be the same, in order to avoid frequent expansion and contraction, a 10% buffer is reserved.
### Verifying this change
Verify that it can shrink normally
### Documentation
- [ x ] `doc-not-needed`
It is a internal class
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on a diff in pull request #15936: Support shrink for TripleLongPriorityQueue
Posted by GitBox <gi...@apache.org>.
merlimat commented on code in PR #15936:
URL: https://github.com/apache/pulsar/pull/15936#discussion_r889712113
##########
pulsar-common/src/main/java/org/apache/pulsar/common/util/collections/TripleLongPriorityQueue.java:
##########
@@ -152,6 +174,21 @@ private void increaseCapacity() {
buffer.capacity(this.capacity * TUPLE_SIZE);
}
+ private void shrinkCapacity() {
+ if (capacity > initialCapacity && size < capacity * shrinkFactor) {
Review Comment:
We could cache the value of `capacity * shrinkFactor` to avoid computing it each time we're removing an item
##########
pulsar-common/src/main/java/org/apache/pulsar/common/util/collections/TripleLongPriorityQueue.java:
##########
@@ -152,6 +174,21 @@ private void increaseCapacity() {
buffer.capacity(this.capacity * TUPLE_SIZE);
}
+ private void shrinkCapacity() {
+ if (capacity > initialCapacity && size < capacity * shrinkFactor) {
+ int decreasingSize = (int) (capacity * shrinkFactor * RESERVATION_FACTOR);
+ if (decreasingSize <= 0) {
+ return;
+ }
+ if (capacity - decreasingSize <= initialCapacity) {
+ this.capacity = initialCapacity;
+ } else {
+ this.capacity = capacity - decreasingSize;
+ }
+ buffer.capacity(this.capacity * TUPLE_SIZE);
Review Comment:
I think this will truncate the buffer, but it will not release the underlying memory back to the pool: https://netty.io/4.1/api/io/netty/buffer/ByteBuf.html#capacity-int-
We'd probably have to allocate a new buffer and copy the content into it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Jason918 commented on a diff in pull request #15936: Support shrink for TripleLongPriorityQueue
Posted by GitBox <gi...@apache.org>.
Jason918 commented on code in PR #15936:
URL: https://github.com/apache/pulsar/pull/15936#discussion_r889788102
##########
pulsar-common/src/main/java/org/apache/pulsar/common/util/collections/TripleLongPriorityQueue.java:
##########
@@ -49,14 +62,21 @@ public TripleLongPriorityQueue() {
this(DEFAULT_INITIAL_CAPACITY);
}
+ public TripleLongPriorityQueue(int initialCapacity, float shrinkFactor) {
+ checkArgument(shrinkFactor > 0);
+ this.initialCapacity = initialCapacity;
+ this.capacity = initialCapacity;
+ this.buffer = PooledByteBufAllocator.DEFAULT.directBuffer(initialCapacity * ITEMS_COUNT * SIZE_OF_LONG);
Review Comment:
`TUPLE_SIZE` can be used instead of `ITEMS_COUNT * SIZE_OF_LONG`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat merged pull request #15936: Support shrink for TripleLongPriorityQueue
Posted by GitBox <gi...@apache.org>.
merlimat merged PR #15936:
URL: https://github.com/apache/pulsar/pull/15936
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org