You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "tjzheng1002 (via GitHub)" <gi...@apache.org> on 2023/02/20 09:45:45 UTC
[GitHub] [skywalking] tjzheng1002 opened a new issue, #10414: [Bug] Consul reported an error as the configuration center
tjzheng1002 opened a new issue, #10414:
URL: https://github.com/apache/skywalking/issues/10414
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar issues.
### Apache SkyWalking Component
OAP server (apache/skywalking)
### What happened
When i use consul as the configuration center,skywalking Sync config center error!
```
2023-02-20 17:35:10,661 org.apache.skywalking.oap.server.configuration.api.ConfigWatcherRegister 85 [pool-5-thread-1] ERROR [] - [9.4.0-SNAPSHOT-71a3282] Sync config center error.
java.lang.IllegalArgumentException: Cache watchInterval=10sec >= networkClientReadTimeout=10000ms. It can cause issues
at com.orbitz.consul.cache.ConsulCache.checkWatch(ConsulCache.java:413) ~[consul-client-1.5.3.jar:?]
at com.orbitz.consul.cache.KVCache.lambda$new$0(KVCache.java:24) ~[consul-client-1.5.3.jar:?]
at com.orbitz.consul.cache.ConsulCache.runCallback(ConsulCache.java:236) ~[consul-client-1.5.3.jar:?]
at com.orbitz.consul.cache.ConsulCache.start(ConsulCache.java:209) ~[consul-client-1.5.3.jar:?]
at org.apache.skywalking.oap.server.configuration.consul.ConsulConfigurationWatcherRegister.lambda$registerKeyListeners$7(ConsulConfigurationWatcherRegister.java:130) ~[classes/:?]
at java.lang.Iterable.forEach(Iterable.java:75) ~[?:1.8.0_181]
at org.apache.skywalking.oap.server.configuration.consul.ConsulConfigurationWatcherRegister.registerKeyListeners(ConsulConfigurationWatcherRegister.java:120) ~[classes/:?]
at org.apache.skywalking.oap.server.configuration.consul.ConsulConfigurationWatcherRegister.readConfig(ConsulConfigurationWatcherRegister.java:81) ~[classes/:?]
at org.apache.skywalking.oap.server.configuration.api.ConfigWatcherRegister.singleConfigsSync(ConfigWatcherRegister.java:95) ~[classes/:?]
at org.apache.skywalking.oap.server.configuration.api.ConfigWatcherRegister.configSync(ConfigWatcherRegister.java:90) ~[classes/:?]
at org.apache.skywalking.oap.server.library.util.RunnableWithExceptionProtection.run(RunnableWithExceptionProtection.java:33) [classes/:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_181]
at java.util.concurrent.FutureTask.runAndReset$$$capture(FutureTask.java:308) [?:1.8.0_181]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java) [?:1.8.0_181]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_181]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
```
**application.yml** is as follows:
```
configuration:
selector: ${SW_CONFIGURATION:consul}
consul:
# Consul host and ports, separated by comma, e.g. 1.2.3.4:8500,2.3.4.5:8500
hostAndPorts: ${SW_CONFIG_CONSUL_HOST_AND_PORTS:192.168.126.143:8500}
# Sync period in seconds. Defaults to 60 seconds.
period: ${SW_CONFIG_CONSUL_PERIOD:60}
# Consul aclToken
aclToken: ${SW_CONFIG_CONSUL_ACL_TOKEN:""}
```
My consul server do not need Consul aclToken.
### What you expected to happen
I expecte Skywalking sync config from consul successfully.
### How to reproduce
Just modify the application.yml to specify consul as the configuration center
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] tjzheng1002 commented on issue #10414: [Bug] Consul reported an error as the configuration center
Posted by "tjzheng1002 (via GitHub)" <gi...@apache.org>.
tjzheng1002 commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1436661194
> I can see from source codes, aclToken is only picked up when it is set to not empty
>
> https://github.com/apache/skywalking/blob/af3889b47eabc8e63ce54c5415d6ec6c4528c53a/oap-server/server-configuration/configuration-consul/src/main/java/org/apache/skywalking/oap/server/configuration/consul/ConsulConfigurationWatcherRegister.java#L70-L72
>
> As you have the env, it is better you could debug on your env.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] tjzheng1002 closed issue #10414: [Bug] Consul reported an error as the configuration center
Posted by "tjzheng1002 (via GitHub)" <gi...@apache.org>.
tjzheng1002 closed issue #10414: [Bug] Consul reported an error as the configuration center
URL: https://github.com/apache/skywalking/issues/10414
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #10414: [Bug] Consul reported an error as the configuration center
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1437738849
That is not my point. This is an open source community, we don't debug for everyone, we can't afford that. As you are only mentioning a plugin, it is better you could address the issue directly and submit a fix if you want.
If there is an issue in your case, it should be not hard to address in your own env. We don't understand that error too. The client jar is simply from Consul community too.
You could search at Consul side, maybe some versions(client/server) mismatch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #10414: [Bug] Consul reported an error as the configuration center
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1436737431
I mean, you should debug and try to address issues on your env.
What you described doesn't match the codes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #10414: [Bug] Consul reported an error as the configuration center
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1437791480
Would you submit a pull request to fix this?
About the version, plugin module should not manage the version, it should be in the bom project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #10414: [Bug] Consul reported an error as the configuration center
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1436655888
I can see from source codes, aclToken is only picked up when it is set to not empty
https://github.com/apache/skywalking/blob/af3889b47eabc8e63ce54c5415d6ec6c4528c53a/oap-server/server-cluster-plugin/cluster-consul-plugin/src/main/java/org/apache/skywalking/oap/server/cluster/plugin/consul/ClusterModuleConsulProvider.java#L86-L88
As you have the env, it is better you could debug on your env.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng closed issue #10414: [Bug] Consul reported an error as the configuration center
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng closed issue #10414: [Bug] Consul reported an error as the configuration center
URL: https://github.com/apache/skywalking/issues/10414
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #10414: [Bug] Consul reported an error as the configuration center
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1436673482
So, what causes your error actually? From git commit log, last time, only this is added for acl support.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] tjzheng1002 commented on issue #10414: [Bug] Consul reported an error as the configuration center
Posted by "tjzheng1002 (via GitHub)" <gi...@apache.org>.
tjzheng1002 commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1437776997
> That is not my point. This is an open source community, we don't debug for everyone, we can't afford that. As you are only mentioning a plugin, it is better you could address the issue directly and submit a fix if you want. If there is an issue in your case, it should be not hard to address in your own env. We don't understand that error too. The client jar is simply from Consul community too. You could search at Consul side, maybe some versions(client/server) mismatch.
Okay, I've identified the problem,
The networkReadMillis alway equals cacheWatchSeconds,because networkReadMillis and cacheWatchSeconds from KeyValueClient are both 10 senconds by default.
![image](https://user-images.githubusercontent.com/6240715/220230465-b2588659-0ff0-476a-9090-4cd3eb9ff9e6.png)
We need to modify the ReadTimeoutMillis to ensue that networkReadMillis is greater than cacheWatchSeconds when initializing Consul.Then the error is fixed.
![image](https://user-images.githubusercontent.com/6240715/220231946-c2944257-cdff-4219-a11c-277a6952219a.png)
One more question:consul.client.version from configuration-consul is 1.2.6,but it is 1.5.3 from cluster-consul-plugin.Why are the versions different? Are there any compatibility issues?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] tjzheng1002 commented on issue #10414: [Bug] Consul reported an error as the configuration center
Posted by "tjzheng1002 (via GitHub)" <gi...@apache.org>.
tjzheng1002 commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1436664401
> I can see from source codes, aclToken is only picked up when it is set to not empty
>
> https://github.com/apache/skywalking/blob/af3889b47eabc8e63ce54c5415d6ec6c4528c53a/oap-server/server-configuration/configuration-consul/src/main/java/org/apache/skywalking/oap/server/configuration/consul/ConsulConfigurationWatcherRegister.java#L70-L72
>
> As you have the env, it is better you could debug on your env.
I have debugged on my env,it is true that the aclToken is empty and not used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] tjzheng1002 commented on issue #10414: [Bug] Consul reported an error as the configuration center
Posted by "tjzheng1002 (via GitHub)" <gi...@apache.org>.
tjzheng1002 commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1436702405
> So, what causes your error actually? From git commit log, last time, only this is added for acl support.
I didn't use ACLToken, I just changed the configuration center to the address of the Consul service, and the Consul service did not set the ACLtoken
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] tjzheng1002 commented on issue #10414: [Bug] Consul reported an error as the configuration center
Posted by "tjzheng1002 (via GitHub)" <gi...@apache.org>.
tjzheng1002 commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1437737291
> I mean, you should debug and try to address issues on your env. What you described doesn't match the codes.
But consul as the cluster register,it runned without any error.I think my env shoud be collect.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org