You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bookkeeper.apache.org by GitBox <gi...@apache.org> on 2021/06/10 14:07:03 UTC

[GitHub] [bookkeeper] thinker0 opened a new issue #2736: bookies hosts by metrics grow exponentially.

thinker0 opened a new issue #2736:
URL: https://github.com/apache/bookkeeper/issues/2736


   **BUG REPORT**
   There are many, many, many Prometheus Metrics names.
   
   ***Describe the bug***
   ```
   ^(bookkeeper_client_per_channel_bookie_client|bookkeeper_server_BookieHighPriorityThread|replication_bookkeeper_client_per_channel_bookie_client)_hostnames_port_.*
   ```
   
   bookies hosts by metrics grow exponentially.
   
   ***To Reproduce***
   
   Steps to reproduce the behavior:
   1. Go to '...'
   2. Click on '....'
   3. Scroll down to '....'
   4. See error
   
   ***Expected behavior***
   Prometheus OOM (200Gbyte)
   
   ***Screenshots***
   ![image](https://user-images.githubusercontent.com/357785/121538948-2ad12780-ca40-11eb-92e4-4041b5efe785.png)
   ![image](https://user-images.githubusercontent.com/357785/121539143-5e13b680-ca40-11eb-9883-fdf8874853e5.png)
   
   ***Additional context***
   
   Add any other context about the problem here.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] sijie commented on issue #2736: bookies hosts by metrics grow exponentially.

Posted by GitBox <gi...@apache.org>.
sijie commented on issue #2736:
URL: https://github.com/apache/bookkeeper/issues/2736#issuecomment-859701094


   @thinker0 Can you create an issue in Pulsar? We can upgrade the bookkeeper version for Pulsar 2.7.3 release.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] thinker0 commented on issue #2736: bookies hosts by metrics grow exponentially.

Posted by GitBox <gi...@apache.org>.
thinker0 commented on issue #2736:
URL: https://github.com/apache/bookkeeper/issues/2736#issuecomment-858660542


   - pulsar-2.7.2
   - 120ea bookies
   - yes (apache mesos/aurora)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] sursingh commented on issue #2736: bookies hosts by metrics grow exponentially.

Posted by GitBox <gi...@apache.org>.
sursingh commented on issue #2736:
URL: https://github.com/apache/bookkeeper/issues/2736#issuecomment-858780231


   I think this has already been addressed: https://github.com/apache/bookkeeper/blame/master/bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java#L407
   
   4.14.1 release should have this change
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] thinker0 edited a comment on issue #2736: bookies hosts by metrics grow exponentially.

Posted by GitBox <gi...@apache.org>.
thinker0 edited a comment on issue #2736:
URL: https://github.com/apache/bookkeeper/issues/2736#issuecomment-858660542


   - pulsar-2.7.2
   - 120ea bookies
   - yes (apache mesos/aurora)
   ![image](https://user-images.githubusercontent.com/357785/121541222-12620c80-ca42-11eb-9f3f-686b25616a01.png)
   
   Change the metrics as shown below.
   ```
   bookkeeper_client_per_channel_bookie_client...{instance="hostname:port"}
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] eolivelli commented on issue #2736: bookies hosts by metrics grow exponentially.

Posted by GitBox <gi...@apache.org>.
eolivelli commented on issue #2736:
URL: https://github.com/apache/bookkeeper/issues/2736#issuecomment-858659554


   which version of BK are you using ?
   how many bookies ?
   do bookies have a dynamic address ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] thinker0 edited a comment on issue #2736: bookies hosts by metrics grow exponentially.

Posted by GitBox <gi...@apache.org>.
thinker0 edited a comment on issue #2736:
URL: https://github.com/apache/bookkeeper/issues/2736#issuecomment-858669838


   Like this ? https://github.com/apache/incubator-heron/pull/3493
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] thinker0 commented on issue #2736: bookies hosts by metrics grow exponentially.

Posted by GitBox <gi...@apache.org>.
thinker0 commented on issue #2736:
URL: https://github.com/apache/bookkeeper/issues/2736#issuecomment-859151887






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] thinker0 commented on issue #2736: bookies hosts by metrics grow exponentially.

Posted by GitBox <gi...@apache.org>.
thinker0 commented on issue #2736:
URL: https://github.com/apache/bookkeeper/issues/2736#issuecomment-858669838


   Can change this ?
   https://github.com/apache/incubator-heron/pull/3493
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] sursingh commented on issue #2736: bookies hosts by metrics grow exponentially.

Posted by GitBox <gi...@apache.org>.
sursingh commented on issue #2736:
URL: https://github.com/apache/bookkeeper/issues/2736#issuecomment-859170870






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org