You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/03/13 10:08:35 UTC
[GitHub] [pulsar] vmarchaud opened a new issue #9904: Crash when retrieving prometheus metrics
vmarchaud opened a new issue #9904:
URL: https://github.com/apache/pulsar/issues/9904
**Describe the bug**
When trying to retrieve prometheus metrics from a broker, i get:
```
mars 13 09:32:26 pulsar-hosts-2 pulsar[15176]: 09:32:26.679 [pulsar-web-49-4] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [13/Mar/2021:09:31:56 +0000] "GET /metrics/ HTTP/1.1" 500 553 "-" "curl/7.6
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: ... 11 more
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: at org.apache.pulsar.broker.stats.prometheus.PrometheusMetricsGenerator.<clinit>(PrometheusMetricsGenerator.java:61) ~[org.apache.pulsar-pulsar-broker-2.7.0
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: at io.prometheus.client.hotspot.DefaultExports.initialize(DefaultExports.java:23) ~[io.prometheus-simpleclient_hotspot-0.5.0.jar:?]
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: at io.prometheus.client.Collector.register(Collector.java:132) ~[io.prometheus-simpleclient-0.5.0.jar:?]
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: at io.prometheus.client.Collector.register(Collector.java:139) ~[io.prometheus-simpleclient-0.5.0.jar:?]
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: at io.prometheus.client.CollectorRegistry.register(CollectorRegistry.java:54) ~[io.prometheus-simpleclient-0.5.0.jar:?]
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: Caused by: java.lang.IllegalArgumentException: Collector already registered that provides name: process_cpu_seconds_total
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.51.Final.jar:4.1.51.Final]
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275]
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275]
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_275]
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_275]
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_275]
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_275]
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.12.0.jar:4.12.0]
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) ~[org.apache.pulsar-managed-ledger-2.7.0.jar:2.7.0]
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: at org.apache.pulsar.broker.stats.prometheus.PrometheusMetricsServlet.lambda$doGet$0(PrometheusMetricsServlet.java:70) ~[org.apache.pulsar-pulsar-broker-2.7
mars 13 09:31:56 pulsar-hosts-2 pulsar[15176]: java.lang.ExceptionInInitializerError: null
```
**To Reproduce**
Steps to reproduce the behavior:
1. Launch broker, version 2.7.0
2. Run `curl localhost:8080/metrics/
4. See error
**Expected behavior**
Retrieve metrics for the broker in prometheus format
**Desktop (please complete the following information):**
- OS: Debian 10
- Broker Config: https://textbin.net/raw/v5zhwqjoim
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] vmarchaud commented on issue #9904: Crash when retrieving prometheus metrics
Posted by GitBox <gi...@apache.org>.
vmarchaud commented on issue #9904:
URL: https://github.com/apache/pulsar/issues/9904#issuecomment-803580510
Found the issue, it was because the option `enableRunBookieAutoRecoveryTogether` was set to `true` even though i wasn't in standalone (bookies were running independently on my setup)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] vmarchaud closed issue #9904: Crash when retrieving prometheus metrics
Posted by GitBox <gi...@apache.org>.
vmarchaud closed issue #9904:
URL: https://github.com/apache/pulsar/issues/9904
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org