You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@celeborn.apache.org by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/10/10 16:37:09 UTC

[PR] [CELEBORN-1640] NettyMemoryMetrics supports numHeapArenas, numDirectArenas, tinyCacheSize, smallCacheSize, normalCacheSize, numThreadLocalCaches and chunkSize [celeborn]

SteNicholas opened a new pull request, #2802:
URL: https://github.com/apache/celeborn/pull/2802

   ### What changes were proposed in this pull request?
   
   `NettyMemoryMetrics` supports `numHeapArenas`, `numDirectArenas`, `tinyCacheSize`, `smallCacheSize`, `normalCacheSize`, `numThreadLocalCaches` and `chunkSize` from `PooledByteBufAllocatorMetric`.
   
   ### Why are the changes needed?
   
   `PooledByteBufAllocatorMetric` provides the following API to support netty memory metrics:
   
   ```
   public int numHeapArenas() {
     return this.allocator.numHeapArenas();
   }
   
   public int numDirectArenas() {
     return this.allocator.numDirectArenas();
   }
   
   public List<PoolArenaMetric> heapArenas() {
     return this.allocator.heapArenas();
   }
   
   public List<PoolArenaMetric> directArenas() {
     return this.allocator.directArenas();
   }
   
   public int numThreadLocalCaches() {
     return this.allocator.numThreadLocalCaches();
   }
   
   public int tinyCacheSize() {
     return this.allocator.tinyCacheSize();
   }
   
   public int smallCacheSize() {
     return this.allocator.smallCacheSize();
   }
   
   public int normalCacheSize() {
     return this.allocator.normalCacheSize();
   }
   
   public int chunkSize() {
     return this.allocator.chunkSize();
   }
   
   public long usedHeapMemory() {
     return this.allocator.usedHeapMemory();
   }
   
   public long usedDirectMemory() {
     return this.allocator.usedDirectMemory();
   }
   ```
   
   `NettyMemoryMetrics` only supports `usedHeapMemory` and `usedDirectMemory`, which could support `numHeapArenas`, `numDirectArenas`, `tinyCacheSize`, `smallCacheSize`, `normalCacheSize`, `numThreadLocalCaches` and `chunkSize` from `PooledByteBufAllocatorMetric`.
   
   ### Does this PR introduce _any_ user-facing change?
   
   No.
   
   ### How was this patch tested?
   
   [Celeborn Grafana Dashboard](https://stenicholas.grafana.net/public-dashboards/a520ca36a33843a38bbde28387023f97)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [CELEBORN-1640] NettyMemoryMetrics supports numHeapArenas, numDirectArenas, tinyCacheSize, smallCacheSize, normalCacheSize, numThreadLocalCaches and chunkSize [celeborn]

Posted by "FMX (via GitHub)" <gi...@apache.org>.
FMX closed pull request #2802: [CELEBORN-1640] NettyMemoryMetrics supports numHeapArenas, numDirectArenas, tinyCacheSize, smallCacheSize, normalCacheSize, numThreadLocalCaches and chunkSize
URL: https://github.com/apache/celeborn/pull/2802


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [CELEBORN-1640] NettyMemoryMetrics supports numHeapArenas, numDirectArenas, tinyCacheSize, smallCacheSize, normalCacheSize, numThreadLocalCaches and chunkSize [celeborn]

Posted by "turboFei (via GitHub)" <gi...@apache.org>.
turboFei commented on PR #2802:
URL: https://github.com/apache/celeborn/pull/2802#issuecomment-3101151186

   Hi @SteNicholas , the dashboard does not work.
   
   the metrics look likes:
   ```
   metrics_shared_pool_1_chunkSize_Value{instance="...",role="Worker"} 4194304 1753162553591
   ```
   
   instead of `metrics_fetch_chunkSize_Value` or `metrics_push_chunkSize_Value`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [CELEBORN-1640] NettyMemoryMetrics supports numHeapArenas, numDirectArenas, tinyCacheSize, smallCacheSize, normalCacheSize, numThreadLocalCaches and chunkSize [celeborn]

Posted by "SteNicholas (via GitHub)" <gi...@apache.org>.
SteNicholas commented on PR #2802:
URL: https://github.com/apache/celeborn/pull/2802#issuecomment-2405588679

   Ping @FMX.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [CELEBORN-1640] NettyMemoryMetrics supports numHeapArenas, numDirectArenas, tinyCacheSize, smallCacheSize, normalCacheSize, numThreadLocalCaches and chunkSize [celeborn]

Posted by "SteNicholas (via GitHub)" <gi...@apache.org>.
SteNicholas commented on PR #2802:
URL: https://github.com/apache/celeborn/pull/2802#issuecomment-2416481564

   Ping @FMX, @RexXiong.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org