You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/03/17 10:04:53 UTC
[GitHub] [pulsar] bpereto opened a new issue #14732: IPv6 Support for broker and proxy with bindAddress
bpereto opened a new issue #14732:
URL: https://github.com/apache/pulsar/issues/14732
**Describe the bug**
the default setting for binding in broker.conf and proxy.conf is `bindAddress=0.0.0.0`.
https://github.com/apache/pulsar/blob/3e065718939d028a970c6b03b96da94e78217583/conf/proxy.conf#L53
the specified default `0.0.0.0` binds to ipv4 only, so the service is not reachable over ipv6.
```
[root@proxy pulsar]# netstat -tulpen
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State User Inode PID/Program name
tcp 0 0 0.0.0.0:6650 0.0.0.0:* LISTEN 41186 8351118 2470721/java
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 41186 8349589 2470721/java
```
this setting does not accept a a ipv6 address `[::0]`, `::0` or a ipv6 address with or without brackets.
available error in logs:
```
2022-03-17T10:52:55,986+0100 [sun.misc.Launcher$AppClassLoader@1d56ce6a] error Uncaught exception in thread main: Failed to bind Pulsar Proxy on port 6650
```
**To Reproduce**
Steps to reproduce the behavior:
1. Set a ipv6 address in broker.conf or proxy.conf for bindAdress: `bindAddress=[::0]`
2. restart pulsar broker or proxy
3. check listening ports - no pulsar listen on `::6650`
**Expected behavior**
pulsar broker and proxy listens on ipv6 address.
**Desktop (please complete the following information):**
- OS: RedHat 8
- Pulsar v2.9.1
**Additional context**
* related issue: https://github.com/apache/pulsar/issues/8231
* There was a related merge request for the "advertised service urls", which claims to resolve the issue, but its only resolved for service urls, not the binding/listening: https://github.com/apache/pulsar/issues/954
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] bpereto commented on issue #14732: IPv6 Support for broker and proxy with bindAddress
Posted by GitBox <gi...@apache.org>.
bpereto commented on issue #14732:
URL: https://github.com/apache/pulsar/issues/14732#issuecomment-1070907778
error originating from:
https://github.com/apache/pulsar/blob/3a337b8952239136cb77302e22021b3d79154f1c/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/ProxyService.java#L210
I think this is the relevant part for binding in netty:
https://github.com/netty/netty/blob/d34212439068091bcec29a8fad4df82f0a82c638/transport/src/main/java/io/netty/bootstrap/AbstractBootstrap.java#L252
references here for the properties, where lobmok is used:
https://github.com/apache/pulsar/blob/branch-2.9/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/ProxyConfiguration.java#L182
convert with `SocketUtils.socketAddress` string to InetSocketAddress:
https://github.com/netty/netty/blob/40382fb1af275e931a765c5001f4b6bd8061991e/common/src/main/java/io/netty/util/internal/SocketUtils.java#L177
but i don't see the thrown exception somewhere in the output or log, but there should be one from the thrown `IOException`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on issue #14732: IPv6 Support for broker and proxy with bindAddress
Posted by GitBox <gi...@apache.org>.
eolivelli commented on issue #14732:
URL: https://github.com/apache/pulsar/issues/14732#issuecomment-1070942935
I believe that it is only because support for IPV6 is not fully available in many libraries (like parsing addresses..) and no one dares to enable it.
If you are able to successfully run all Pulsar components without issues, then please advertise on user@pulsar or dev@pulsar and we can remove that flag, or make it configurable somehow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] bpereto commented on issue #14732: IPv6 Support for broker and proxy with bindAddress
Posted by GitBox <gi...@apache.org>.
bpereto commented on issue #14732:
URL: https://github.com/apache/pulsar/issues/14732#issuecomment-1070813893
unforenately, setting log levels to debug with env vars:
```
PULSAR_LOG_LEVEL=debug
PULSAR_LOG_ROOT_LEVEL=debug
```
does show debug logs, but no additional errors or hints for this specific problem.
is jetty used for this?
the default bind address is set to `0.0.0.0` in the codebase:
https://github.com/apache/pulsar/blob/342a5df24ca9dcf142e471ac0aafebc253042ceb/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/ProxyConfiguration.java#L227
and there is a jetty discussion, if the bind address is not defined, both stacks (4 & 6) are considered by default:
https://www.eclipse.org/lists/jetty-users/msg09643.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] bpereto edited a comment on issue #14732: IPv6 Support for broker and proxy with bindAddress
Posted by GitBox <gi...@apache.org>.
bpereto edited a comment on issue #14732:
URL: https://github.com/apache/pulsar/issues/14732#issuecomment-1070907778
error originating from:
https://github.com/apache/pulsar/blob/3a337b8952239136cb77302e22021b3d79154f1c/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/ProxyService.java#L210
I think this is the relevant part for binding in netty:
https://github.com/netty/netty/blob/d34212439068091bcec29a8fad4df82f0a82c638/transport/src/main/java/io/netty/bootstrap/AbstractBootstrap.java#L252
references here for the properties, where lobmok is used:
https://github.com/apache/pulsar/blob/3a337b8952239136cb77302e22021b3d79154f1c/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/ProxyConfiguration.java#L143
convert with `SocketUtils.socketAddress` string to InetSocketAddress:
https://github.com/netty/netty/blob/40382fb1af275e931a765c5001f4b6bd8061991e/common/src/main/java/io/netty/util/internal/SocketUtils.java#L177
but i don't see the thrown exception somewhere in the output or log, but there should be one from the thrown `IOException`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on issue #14732: IPv6 Support for broker and proxy with bindAddress
Posted by GitBox <gi...@apache.org>.
eolivelli commented on issue #14732:
URL: https://github.com/apache/pulsar/issues/14732#issuecomment-1070817125
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] bpereto edited a comment on issue #14732: IPv6 Support for broker and proxy with bindAddress
Posted by GitBox <gi...@apache.org>.
bpereto edited a comment on issue #14732:
URL: https://github.com/apache/pulsar/issues/14732#issuecomment-1070813893
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] bpereto commented on issue #14732: IPv6 Support for broker and proxy with bindAddress
Posted by GitBox <gi...@apache.org>.
bpereto commented on issue #14732:
URL: https://github.com/apache/pulsar/issues/14732#issuecomment-1070813893
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] wangjialing218 commented on issue #14732: IPv6 Support for broker and proxy with bindAddress
Posted by GitBox <gi...@apache.org>.
wangjialing218 commented on issue #14732:
URL: https://github.com/apache/pulsar/issues/14732#issuecomment-1071971887
I have already test IPV6 connections between pulasr client and broker, as long as kafka client and kop, with `-Djava.net.preferIPv4Stack=false`, there seems no problem.
And I'm going to have a IPV6 test between broker and bookie and zookeeper, after than we could make it default false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] wangjialing218 commented on issue #14732: IPv6 Support for broker and proxy with bindAddress
Posted by GitBox <gi...@apache.org>.
wangjialing218 commented on issue #14732:
URL: https://github.com/apache/pulsar/issues/14732#issuecomment-1071971887
I have already test IPV6 connections between pulasr client and broker, as long as kafka client and kop, with `-Djava.net.preferIPv4Stack=false`, there seems no problem.
And I'm going to have a IPV6 test between broker and bookie and zookeeper, after than we could make it default false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on issue #14732: IPv6 Support for broker and proxy with bindAddress
Posted by GitBox <gi...@apache.org>.
eolivelli commented on issue #14732:
URL: https://github.com/apache/pulsar/issues/14732#issuecomment-1070916536
if I remove all the occurrences of "-Djava.net.preferIPv4Stack=true" inside "bin" (or you can set it to "false") then I am able to boot Pulsar standalone and the proxy on my Mac with JDK11
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on issue #14732: IPv6 Support for broker and proxy with bindAddress
Posted by GitBox <gi...@apache.org>.
eolivelli commented on issue #14732:
URL: https://github.com/apache/pulsar/issues/14732#issuecomment-1070817125
we are using Jetty for the REST API part and Netty for the Binary part
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] bpereto edited a comment on issue #14732: IPv6 Support for broker and proxy with bindAddress
Posted by GitBox <gi...@apache.org>.
bpereto edited a comment on issue #14732:
URL: https://github.com/apache/pulsar/issues/14732#issuecomment-1070813893
unfortunately, setting log levels to debug with env vars:
```
PULSAR_LOG_LEVEL=debug
PULSAR_LOG_ROOT_LEVEL=debug
```
does show debug logs, but no additional errors or hints for this specific problem.
is jetty used for this?
the default bind address is set to `0.0.0.0` in the codebase:
https://github.com/apache/pulsar/blob/342a5df24ca9dcf142e471ac0aafebc253042ceb/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/ProxyConfiguration.java#L227
and there is a jetty discussion, if the bind address is not defined, both stacks (4 & 6) are considered by default:
https://www.eclipse.org/lists/jetty-users/msg09643.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] bpereto commented on issue #14732: IPv6 Support for broker and proxy with bindAddress
Posted by GitBox <gi...@apache.org>.
bpereto commented on issue #14732:
URL: https://github.com/apache/pulsar/issues/14732#issuecomment-1070939771
jup, this works.
set `bindAddress=0.0.0.0` in proxy.conf and removing `-Djava.net.preferIPv4Stack=true` in `bin/pulsar` is indeed working.
why is the setting `java.net.preferIPv4Stack` even in place? Should this not be as the default (false) and enabling dualstack?
IMHO this should be standard today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on issue #14732: IPv6 Support for broker and proxy with bindAddress
Posted by GitBox <gi...@apache.org>.
eolivelli commented on issue #14732:
URL: https://github.com/apache/pulsar/issues/14732#issuecomment-1070785617
are you able to troubleshot why this is happening ?
like putting some debug
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org