You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/09/29 09:24:35 UTC
[GitHub] [pulsar] liangyepianzhou opened a new pull request, #17888: [fix][conf] Fix port conflict in bookkeeper.conf
liangyepianzhou opened a new pull request, #17888:
URL: https://github.com/apache/pulsar/pull/17888
### Motivation
prometheusStatsHttpPort=8000
and httpServerPort=8000, there will be a port conflict when deploy a cluster.
### Modification
Change the prometheusStatsHttpPort=8001
### Documentation
- [x] `doc-not-needed`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on pull request #17888: [fix][storage] Fix port conflict in bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
liangyepianzhou commented on PR #17888:
URL: https://github.com/apache/pulsar/pull/17888#issuecomment-1278448515
@hangc0276 Thanks for your feedback!
I'm rebasing to the latest version.
I will redeploy the cluster after installation to see whether the problem can be reproduced.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on pull request #17888: [fix][storage] Fix port conflict in bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
liangyepianzhou commented on PR #17888:
URL: https://github.com/apache/pulsar/pull/17888#issuecomment-1278521551
@hangc0276
I deployed the Pulsar cluster with the latest master branch and got this exception.
Configurations in broker.conf:
> metadataStoreUrl=127.0.0.1:2181
> clusterName=cluster-a
> managedLedgerDefaultEnsembleSize=1
> managedLedgerDefaultWriteQuorum=1
> managedLedgerDefaultAckQuorum=1
And the exception is
> 2022-10-14T13:52:34,154+0800 [main] ERROR org.apache.bookkeeper.common.component.AbstractLifecycleComponent - Failed to start Component: stats-provider
java.lang.RuntimeException: java.io.IOException: Failed to bind to /0.0.0.0:8000
at org.apache.bookkeeper.stats.prometheus.PrometheusMetricsProvider.start(PrometheusMetricsProvider.java:119) ~[prometheus-metrics-provider-4.15.1.jar:4.15.1]
at org.apache.bookkeeper.server.service.StatsProviderService.doStart(StatsProviderService.java:51) ~[bookkeeper-server-4.15.1.jar:4.15.1]
at org.apache.bookkeeper.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:83) ~[bookkeeper-common-4.15.1.jar:4.15.1]
at org.apache.bookkeeper.common.component.LifecycleComponentStack.lambda$start$4(LifecycleComponentStack.java:144) ~[bookkeeper-common-4.15.1.jar:4.15.1]
at com.google.common.collect.ImmutableList.forEach(ImmutableList.java:422) ~[guava-31.0.1-jre.jar:?]
at org.apache.bookkeeper.common.component.LifecycleComponentStack.start(LifecycleComponentStack.java:144) ~[bookkeeper-common-4.15.1.jar:4.15.1]
at org.apache.bookkeeper.common.component.ComponentStarter.startComponent(ComponentStarter.java:85) ~[bookkeeper-common-4.15.1.jar:4.15.1]
at org.apache.bookkeeper.server.Main.doMain(Main.java:280) ~[bookkeeper-server-4.15.1.jar:4.15.1]
at org.apache.bookkeeper.server.Main.main(Main.java:255) ~[bookkeeper-server-4.15.1.jar:4.15.1]
Caused by: java.io.IOException: Failed to bind to /0.0.0.0:8000
at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:349) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:310) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:234) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.server.Server.doStart(Server.java:401) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.apache.bookkeeper.stats.prometheus.PrometheusMetricsProvider.start(PrometheusMetricsProvider.java:116) ~[prometheus-metrics-provider-4.15.1.jar:4.15.1]
... 8 more
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method) ~[?:?]
at sun.nio.ch.Net.bind(Net.java:555) ~[?:?]
at sun.nio.ch.ServerSocketChannelImpl.netBind(ServerSocketChannelImpl.java:337) ~[?:?]
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:294) ~[?:?]
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:89) ~[?:?]
at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:344) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:310) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:234) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.server.Server.doStart(Server.java:401) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.apache.bookkeeper.stats.prometheus.PrometheusMetricsProvider.start(PrometheusMetricsProvider.java:116) ~[prometheus-metrics-provider-4.15.1.jar:4.15.1]
... 8 more
2022-10-14T13:52:34,301+0800 [main] ERROR org.apache.bookkeeper.common.component.AbstractLifecycleComponent - Calling uncaughtExceptionHandler
2022-10-14T13:52:34,302+0800 [main] ERROR org.apache.bookkeeper.common.component.ComponentStarter - Triggered exceptionHandler of Component: bookie-server because of Exception in Thread: Thread[main,5,main]
java.lang.RuntimeException: java.io.IOException: Failed to bind to /0.0.0.0:8000
at org.apache.bookkeeper.stats.prometheus.PrometheusMetricsProvider.start(PrometheusMetricsProvider.java:119) ~[prometheus-metrics-provider-4.15.1.jar:4.15.1]
at org.apache.bookkeeper.server.service.StatsProviderService.doStart(StatsProviderService.java:51) ~[bookkeeper-server-4.15.1.jar:4.15.1]
at org.apache.bookkeeper.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:83) ~[bookkeeper-common-4.15.1.jar:4.15.1]
at org.apache.bookkeeper.common.component.LifecycleComponentStack.lambda$start$4(LifecycleComponentStack.java:144) ~[bookkeeper-common-4.15.1.jar:4.15.1]
at com.google.common.collect.ImmutableList.forEach(ImmutableList.java:422) ~[guava-31.0.1-jre.jar:?]
at org.apache.bookkeeper.common.component.LifecycleComponentStack.start(LifecycleComponentStack.java:144) ~[bookkeeper-common-4.15.1.jar:4.15.1]
at org.apache.bookkeeper.common.component.ComponentStarter.startComponent(ComponentStarter.java:85) ~[bookkeeper-common-4.15.1.jar:4.15.1]
at org.apache.bookkeeper.server.Main.doMain(Main.java:280) ~[bookkeeper-server-4.15.1.jar:4.15.1]
at org.apache.bookkeeper.server.Main.main(Main.java:255) ~[bookkeeper-server-4.15.1.jar:4.15.1]
Caused by: java.io.IOException: Failed to bind to /0.0.0.0:8000
at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:349) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:310) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:234) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.server.Server.doStart(Server.java:401) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.apache.bookkeeper.stats.prometheus.PrometheusMetricsProvider.start(PrometheusMetricsProvider.java:116) ~[prometheus-metrics-provider-4.15.1.jar:4.15.1]
... 8 more
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method) ~[?:?]
at sun.nio.ch.Net.bind(Net.java:555) ~[?:?]
at sun.nio.ch.ServerSocketChannelImpl.netBind(ServerSocketChannelImpl.java:337) ~[?:?]
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:294) ~[?:?]
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:89) ~[?:?]
at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:344) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:310) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:234) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.server.Server.doStart(Server.java:401) ~[jetty-server-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
at org.apache.bookkeeper.stats.prometheus.PrometheusMetricsProvider.start(PrometheusMetricsProvider.java:116) ~[prometheus-metrics-provider-4.15.1.jar:4.15.1]
... 8 more
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou closed pull request #17888: [fix][storage] Fix port conflict in bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
liangyepianzhou closed pull request #17888: [fix][storage] Fix port conflict in bookkeeper.conf
URL: https://github.com/apache/pulsar/pull/17888
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on a diff in pull request #17888: [fix][storage] Fix port conflict in bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
liangyepianzhou commented on code in PR #17888:
URL: https://github.com/apache/pulsar/pull/17888#discussion_r994146763
##########
conf/bookkeeper.conf:
##########
@@ -694,7 +694,7 @@ httpServerEnabled=false
# The http server port to listen on. Default value is 8080.
# Use `8000` as the port to keep it consistent with prometheus stats provider
-httpServerPort=8000
+httpServerPort=8001
Review Comment:
Thanks for your feedback!
When deploying the cluster before, there were often conflicts due to the port of prometheusStatsHttpPort=8000.
Then I found that there are two configurations whose port is 8000 in the bookkeeper.
If not for this reason I will redeploy the cluster and find out the real reason for the port conflict later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hangc0276 commented on pull request #17888: [fix][storage] Fix port conflict in bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on PR #17888:
URL: https://github.com/apache/pulsar/pull/17888#issuecomment-1278636539
> @hangc0276 Thanks for your feedback! I'm rebasing to the latest version. I will redeploy the cluster after installation to see whether the problem can be reproduced.
Zookeeper Prometheus metrics use `8000`, does the BK Prometheus metric conflict with ZK?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on a diff in pull request #17888: [fix][storage] Fix port conflict in bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on code in PR #17888:
URL: https://github.com/apache/pulsar/pull/17888#discussion_r994142418
##########
conf/bookkeeper.conf:
##########
@@ -694,7 +694,7 @@ httpServerEnabled=false
# The http server port to listen on. Default value is 8080.
# Use `8000` as the port to keep it consistent with prometheus stats provider
-httpServerPort=8000
+httpServerPort=8001
Review Comment:
@liangyepianzhou If the bookie side enabled the http server, prometheus should also use the http server port, not a separate port. Can you reproduce the issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou closed pull request #17888: [fix][conf] Fix port conflict in bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
liangyepianzhou closed pull request #17888: [fix][conf] Fix port conflict in bookkeeper.conf
URL: https://github.com/apache/pulsar/pull/17888
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] congbobo184 commented on pull request #17888: [fix][conf] Fix port conflict in bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
congbobo184 commented on PR #17888:
URL: https://github.com/apache/pulsar/pull/17888#issuecomment-1272703742
it's better to change `httpServerPort`, `httpServerEnabled` default is false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on pull request #17888: [fix][conf] Fix port conflict in bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on PR #17888:
URL: https://github.com/apache/pulsar/pull/17888#issuecomment-1272734095
> @Anonymitaet @hangc0276 now we encounter the first PR title issue. `conf` is not in our scope candidates, if we don't sort this patch in to `conf`, shall we sort it into...`storage` (since it's config of bookkeeper)?
>
> See the failure report at: https://github.com/apache/pulsar/actions/runs/3216489256/jobs/5258426561
`storage` makes sense to me because `conf` involves various areas, eg, `io conf`, `fn conf`, `client conf`, `cli conf`, ....
Each of them should belong to `io`, `fn`, `client`, `cli` rather than `conf`.
Details see [[Guide] Pulsar Pull Request Naming Convention](https://docs.google.com/document/d/1d8Pw6ZbWk-_pCKdOmdvx9rnhPiyuxwq60_TrD68d7BA/edit#bookmark=id.y8943h392zno)
<img width="746" alt="image" src="https://user-images.githubusercontent.com/50226895/194794252-9f2f91e9-eb28-4073-bd08-8cba977ca186.png">
cc @D-2-Ed @DaveDuggins @momo-jun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hangc0276 commented on pull request #17888: [fix][storage] Fix port conflict in bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on PR #17888:
URL: https://github.com/apache/pulsar/pull/17888#issuecomment-1278649924
> > Zookeeper Prometheus metrics use 8000, does the BK Prometheus metric conflict with ZK?
>
> Yes, It is the BK Prometheus metric port that conflict with the ZK Prometheus metric port. Should we modify one of them or make one of them with no default value?
I prefer not to modify anything. Most of the cases, we won't deploy ZK and bookie in the same node. If users want to do that, he just needs to change one of the port numbers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on a diff in pull request #17888: [fix][storage] Fix port conflict in bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on code in PR #17888:
URL: https://github.com/apache/pulsar/pull/17888#discussion_r995278130
##########
conf/bookkeeper.conf:
##########
@@ -694,7 +694,7 @@ httpServerEnabled=false
# The http server port to listen on. Default value is 8080.
# Use `8000` as the port to keep it consistent with prometheus stats provider
-httpServerPort=8000
+httpServerPort=8001
Review Comment:
If you are deploying the bookkeeper and zookeeper on the same node, it should conflict with the port used by Zookeeper
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on pull request #17888: [fix][storage] Fix port conflict in bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
liangyepianzhou commented on PR #17888:
URL: https://github.com/apache/pulsar/pull/17888#issuecomment-1278644335
> Zookeeper Prometheus metrics use 8000, does the BK Prometheus metric conflict with ZK?
Yes, It is the BK Prometheus metric port that conflict with the ZK Prometheus metric port.
Should we modify one of them or make one of them with no default value?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on pull request #17888: [fix][storage] Fix port conflict in bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
liangyepianzhou commented on PR #17888:
URL: https://github.com/apache/pulsar/pull/17888#issuecomment-1278655983
> I prefer not to modify anything. Most of the cases, we won't deploy ZK and bookie in the same node. If users want to do that, he just needs to change one of the port numbers
OK, Thanks for your feedback!
I will close this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tisonkun commented on pull request #17888: [fix][conf] Fix port conflict in bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
tisonkun commented on PR #17888:
URL: https://github.com/apache/pulsar/pull/17888#issuecomment-1272727265
@Anonymitaet @hangc0276 now we encounter the first PR title issue. `conf` is not in our scope candidates, if we don't sort this patch in to `conf`, shall we sort it into...`storage` (since it's config of bookkeeper)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org