You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "usernameisnull (via GitHub)" <gi...@apache.org> on 2023/12/22 02:31:48 UTC
[PR] [ISSUE #202 #159]when the broker uses hostNetwork, the IP registered by the broker to the nameserver is not the podIP [rocketmq-operator]
usernameisnull opened a new pull request, #203:
URL: https://github.com/apache/rocketmq-operator/pull/203
…ver is the IP of the k8s node where the broker is located.
## What is the purpose of the change
when the broker uses hostNetwork, the broker's IP returned by NameServer is the IP of the k8s node where the broker is located.
This is because the broker uses the cluster ip（or other ip） instead of its own pod ip when registering.
The root cause is that when using hostnetwork, the host's network is used, including /etc/hosts. When generating the `brokerIP1 `configuration project, use the `hostname -i` command to obtain the ip, but in many cases this command does not correctly obtain the correct ip.
for exampels:
```
#/etc/hosts
127.0.0.1 localhost
127.0.1.1 master01
```
bash-4.4# hostname -i
127.0.1.1 10.6.178.178
when use this to register, the nameserver will select the IP based on its own algorithm. Most of the time it is not the status.podIP of the pod we want: https://github.com/apache/rocketmq/blob/2c898c9b31bf195174cf1e3a626a7c61f7576381/common/src/main/java/org/apache/rocketmq/common/utils/NetworkUtil.java#L91
```
#/etc/hosts
127.0.0.1 localhost
127.0.1.1 master01
10.6.178.178 master01
```
bash-4.4# hostname -i
127.0.1.1 10.6.178.178
check the log, it seems to be successful
but when we used the mqadmin command to check the registered broker address, we found that this was not the case.
## Brief changelog
When setting up hostNetwork, we explicitly add an environment variable HOST_NETWORK_POD_IP on the pod. When obtaining brokerIP1, we obtain it from this environment variable and register it with the nameserver.
## Verifying this change
the log from the pod:
the podIP:
use mqadmin see the broker address:
**Please go through this checklist to help us incorporate your contribution quickly and easily.**
Notice: `It would be helpful if you could finish the following checklist (the last one is not necessary) before request the community to review your PR`.
- [x] Make sure there is a [Github issue](https://github.com/apache/rocketmq-operator/issues) filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
- [x] Format the pull request title like `[ISSUE #123] Fix UnknownException when host config not exist`. Each commit in the pull request should have a meaningful subject line and body.
- [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
- [x] Check RBAC rights for Kubernetes roles.
- [ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist.
- [ ] Run `make docker-build` to build docker image for operator, try your changes from Pod inside your Kubernetes cluster, **not just locally**. Also provide screenshots to show that the RocketMQ cluster is healthy after the changes.
- [ ] Before committing your changes, remember to run `make manifests` to make sure the CRD files are updated.
- [ ] Update documentation if necessary.
- [ ] If this contribution is large, please file an [Apache Individual Contributor License Agreement](http://www.apache.org/licenses/#clas).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ISSUE #202 #159]when the broker uses hostNetwork, the IP registered by the broker to the nameserver is not the podIP [rocketmq-operator]
Posted by "usernameisnull (via GitHub)" <gi...@apache.org>.
usernameisnull commented on PR #203:
URL: https://github.com/apache/rocketmq-operator/pull/203#issuecomment-1867156779
PTAL @caigy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ISSUE #202 #159]when the broker uses hostNetwork, the IP registered by the broker to the nameserver is not the podIP [rocketmq-operator]
Posted by "usernameisnull (via GitHub)" <gi...@apache.org>.
usernameisnull commented on PR #203:
URL: https://github.com/apache/rocketmq-operator/pull/203#issuecomment-1879715897
> ```shell
> BROKER_NETWORK_DEV=`/sbin/route |awk '/^default/{print $NF}'`
> BROKER_IP=`ip a s ${BROKER_NETWORK_DEV} |awk -F' +|/' '/ inet /{print $3}'`
> echo "brokerIP1=${BROKER_IP}" >> $BROKER_CONFIG_FILE
> ```
>
> You can try this instead, that doesn't require manual IP specification, it works for me.
thank you, I will try this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ISSUE #202 #159]when the broker uses hostNetwork, the IP registered by the broker to the nameserver is not the podIP [rocketmq-operator]
Posted by "usernameisnull (via GitHub)" <gi...@apache.org>.
usernameisnull commented on PR #203:
URL: https://github.com/apache/rocketmq-operator/pull/203#issuecomment-1909253674
PTAL @caigy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ISSUE #202 #159]when the broker uses hostNetwork, the IP registered by the broker to the nameserver is not the podIP [rocketmq-operator]
Posted by "shangjin92 (via GitHub)" <gi...@apache.org>.
shangjin92 commented on PR #203:
URL: https://github.com/apache/rocketmq-operator/pull/203#issuecomment-1874975139
```bash
BROKER_NETWORK_DEV=`/sbin/route |awk '/^default/{print $NF}'`
BROKER_IP=`ip a s ${BROKER_NETWORK_DEV} |awk -F' +|/' '/ inet /{print $3}'`
echo "brokerIP1=${BROKER_IP}" >> $BROKER_CONFIG_FILE
```
You can try this instead, that doesn't require manual IP specification, it works for me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ISSUE #202 #159]when the broker uses hostNetwork, the IP registered by the broker to the nameserver is not the podIP [rocketmq-operator]
Posted by "johnshawn-david (via GitHub)" <gi...@apache.org>.
johnshawn-david commented on PR #203:
URL: https://github.com/apache/rocketmq-operator/pull/203#issuecomment-1970321202
目前使用最新提交的代码 `echo "brokerIP1=${POD_IP}" >> $BROKER_CONFIG_FILE`获取到的brokerIP1为空
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ISSUE #202 #159]when the broker uses hostNetwork, the IP registered by the broker to the nameserver is not the podIP [rocketmq-operator]
Posted by "caigy (via GitHub)" <gi...@apache.org>.
caigy commented on code in PR #203:
URL: https://github.com/apache/rocketmq-operator/pull/203#discussion_r1434724183
##########
images/broker/alpine/brokerGenConfig.sh:
##########
@@ -27,7 +27,12 @@ function create_config() {
echo "brokerClusterName=$BROKER_CLUSTER_NAME" >> $BROKER_CONFIG_FILE
echo "brokerName=$BROKER_NAME" >> $BROKER_CONFIG_FILE
echo "brokerId=$BROKER_ID" >> $BROKER_CONFIG_FILE
- echo "brokerIP1=`hostname -i`" >> $BROKER_CONFIG_FILE
+ # If the STATUS_POD_IP ip is not empty, the pod uses hostNetwork: true
+ if [ -n "${HOST_NETWORK_POD_IP}" ]; then
+ echo "brokerIP1=${HOST_NETWORK_POD_IP}" >> ${BROKER_CONFIG_FILE}
+ else
+ echo "brokerIP1=`hostname -i`" >> $BROKER_CONFIG_FILE
+ fi
Review Comment:
Should we always use `status.podIP` as `brokerIP1` instead of the output of `hostname`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ISSUE #202 #159]when the broker uses hostNetwork, the IP registered by the broker to the nameserver is not the podIP [rocketmq-operator]
Posted by "caigy (via GitHub)" <gi...@apache.org>.
caigy merged PR #203:
URL: https://github.com/apache/rocketmq-operator/pull/203
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ISSUE #202 #159]when the broker uses hostNetwork, the IP registered by the broker to the nameserver is not the podIP [rocketmq-operator]
Posted by "usernameisnull (via GitHub)" <gi...@apache.org>.
usernameisnull commented on code in PR #203:
URL: https://github.com/apache/rocketmq-operator/pull/203#discussion_r1463286448
##########
images/broker/alpine/brokerGenConfig.sh:
##########
@@ -27,7 +27,12 @@ function create_config() {
echo "brokerClusterName=$BROKER_CLUSTER_NAME" >> $BROKER_CONFIG_FILE
echo "brokerName=$BROKER_NAME" >> $BROKER_CONFIG_FILE
echo "brokerId=$BROKER_ID" >> $BROKER_CONFIG_FILE
- echo "brokerIP1=`hostname -i`" >> $BROKER_CONFIG_FILE
+ # If the STATUS_POD_IP ip is not empty, the pod uses hostNetwork: true
+ if [ -n "${HOST_NETWORK_POD_IP}" ]; then
+ echo "brokerIP1=${HOST_NETWORK_POD_IP}" >> ${BROKER_CONFIG_FILE}
+ else
+ echo "brokerIP1=`hostname -i`" >> $BROKER_CONFIG_FILE
+ fi
Review Comment:
ok,I add a POD_IP env, and we get this in the brokerGenConfig.sh,and use this ip to register
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org