You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "tjzheng1002 (via GitHub)" <gi...@apache.org> on 2023/02/20 09:45:45 UTC

[GitHub] [skywalking] tjzheng1002 opened a new issue, #10414: [Bug] Consul reported an error as the configuration center

tjzheng1002 opened a new issue, #10414:
URL: https://github.com/apache/skywalking/issues/10414

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### Apache SkyWalking Component
   
   OAP server (apache/skywalking)
   
   ### What happened
   
   When i use consul as the configuration center,skywalking Sync config center error!
   ```
   2023-02-20 17:35:10,661 org.apache.skywalking.oap.server.configuration.api.ConfigWatcherRegister 85 [pool-5-thread-1] ERROR [] - [9.4.0-SNAPSHOT-71a3282] Sync config center error.
   java.lang.IllegalArgumentException: Cache watchInterval=10sec >= networkClientReadTimeout=10000ms. It can cause issues
   	at com.orbitz.consul.cache.ConsulCache.checkWatch(ConsulCache.java:413) ~[consul-client-1.5.3.jar:?]
   	at com.orbitz.consul.cache.KVCache.lambda$new$0(KVCache.java:24) ~[consul-client-1.5.3.jar:?]
   	at com.orbitz.consul.cache.ConsulCache.runCallback(ConsulCache.java:236) ~[consul-client-1.5.3.jar:?]
   	at com.orbitz.consul.cache.ConsulCache.start(ConsulCache.java:209) ~[consul-client-1.5.3.jar:?]
   	at org.apache.skywalking.oap.server.configuration.consul.ConsulConfigurationWatcherRegister.lambda$registerKeyListeners$7(ConsulConfigurationWatcherRegister.java:130) ~[classes/:?]
   	at java.lang.Iterable.forEach(Iterable.java:75) ~[?:1.8.0_181]
   	at org.apache.skywalking.oap.server.configuration.consul.ConsulConfigurationWatcherRegister.registerKeyListeners(ConsulConfigurationWatcherRegister.java:120) ~[classes/:?]
   	at org.apache.skywalking.oap.server.configuration.consul.ConsulConfigurationWatcherRegister.readConfig(ConsulConfigurationWatcherRegister.java:81) ~[classes/:?]
   	at org.apache.skywalking.oap.server.configuration.api.ConfigWatcherRegister.singleConfigsSync(ConfigWatcherRegister.java:95) ~[classes/:?]
   	at org.apache.skywalking.oap.server.configuration.api.ConfigWatcherRegister.configSync(ConfigWatcherRegister.java:90) ~[classes/:?]
   	at org.apache.skywalking.oap.server.library.util.RunnableWithExceptionProtection.run(RunnableWithExceptionProtection.java:33) [classes/:?]
   	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_181]
   	at java.util.concurrent.FutureTask.runAndReset$$$capture(FutureTask.java:308) [?:1.8.0_181]
   	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java) [?:1.8.0_181]
   	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_181]
   	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_181]
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
   	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
   ```
   **application.yml** is  as follows: 
   ```
   configuration:
     selector: ${SW_CONFIGURATION:consul}
     consul:
       # Consul host and ports, separated by comma, e.g. 1.2.3.4:8500,2.3.4.5:8500
       hostAndPorts: ${SW_CONFIG_CONSUL_HOST_AND_PORTS:192.168.126.143:8500}
       # Sync period in seconds. Defaults to 60 seconds.
       period: ${SW_CONFIG_CONSUL_PERIOD:60}
       # Consul aclToken
       aclToken: ${SW_CONFIG_CONSUL_ACL_TOKEN:""}
   ```
   My consul server do not need Consul aclToken.
   
   ### What you expected to happen
   
   I expecte Skywalking sync config from consul successfully.
   
   ### How to reproduce
   
   Just modify the application.yml to specify consul as the configuration center
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [skywalking] tjzheng1002 commented on issue #10414: [Bug] Consul reported an error as the configuration center

Posted by "tjzheng1002 (via GitHub)" <gi...@apache.org>.
tjzheng1002 commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1436661194

   > I can see from source codes, aclToken is only picked up when it is set to not empty
   > 
   > https://github.com/apache/skywalking/blob/af3889b47eabc8e63ce54c5415d6ec6c4528c53a/oap-server/server-configuration/configuration-consul/src/main/java/org/apache/skywalking/oap/server/configuration/consul/ConsulConfigurationWatcherRegister.java#L70-L72
   > 
   > As you have the env, it is better you could debug on your env.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [skywalking] tjzheng1002 closed issue #10414: [Bug] Consul reported an error as the configuration center

Posted by "tjzheng1002 (via GitHub)" <gi...@apache.org>.
tjzheng1002 closed issue #10414: [Bug] Consul reported an error as the configuration center
URL: https://github.com/apache/skywalking/issues/10414


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [skywalking] wu-sheng commented on issue #10414: [Bug] Consul reported an error as the configuration center

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1437738849

   That is not my point. This is an open source community, we don't debug for everyone, we can't afford that. As you are only mentioning a plugin, it is better you could address the issue directly and submit a fix if you want.
   If there is an issue in your case, it should be not hard to address in your own env. We don't understand that error too. The client jar is simply from Consul community too. 
   You could search at Consul side, maybe some versions(client/server) mismatch.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [skywalking] wu-sheng commented on issue #10414: [Bug] Consul reported an error as the configuration center

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1436737431

   I mean, you should debug and try to address issues on your env.
   What you described doesn't match the codes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [skywalking] wu-sheng commented on issue #10414: [Bug] Consul reported an error as the configuration center

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1437791480

   Would you submit a pull request to fix this? 
   
   About the version, plugin module should not manage the version, it should be in the bom project.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [skywalking] wu-sheng commented on issue #10414: [Bug] Consul reported an error as the configuration center

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1436655888

   I can see from source codes, aclToken is only picked up when it is set to not empty
   
   https://github.com/apache/skywalking/blob/af3889b47eabc8e63ce54c5415d6ec6c4528c53a/oap-server/server-cluster-plugin/cluster-consul-plugin/src/main/java/org/apache/skywalking/oap/server/cluster/plugin/consul/ClusterModuleConsulProvider.java#L86-L88
   
   As you have the env, it is better you could debug on your env.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [skywalking] wu-sheng closed issue #10414: [Bug] Consul reported an error as the configuration center

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng closed issue #10414: [Bug] Consul reported an error as the configuration center
URL: https://github.com/apache/skywalking/issues/10414


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [skywalking] wu-sheng commented on issue #10414: [Bug] Consul reported an error as the configuration center

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1436673482

   So, what causes your error actually? From git commit log, last time, only this is added for acl support.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [skywalking] tjzheng1002 commented on issue #10414: [Bug] Consul reported an error as the configuration center

Posted by "tjzheng1002 (via GitHub)" <gi...@apache.org>.
tjzheng1002 commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1437776997

   > That is not my point. This is an open source community, we don't debug for everyone, we can't afford that. As you are only mentioning a plugin, it is better you could address the issue directly and submit a fix if you want. If there is an issue in your case, it should be not hard to address in your own env. We don't understand that error too. The client jar is simply from Consul community too. You could search at Consul side, maybe some versions(client/server) mismatch.
   
   Okay, I've identified the problem,
   The networkReadMillis alway equals cacheWatchSeconds,because networkReadMillis and cacheWatchSeconds from KeyValueClient are both 10 senconds by default.
   ![image](https://user-images.githubusercontent.com/6240715/220230465-b2588659-0ff0-476a-9090-4cd3eb9ff9e6.png)
   We need to modify the ReadTimeoutMillis to ensue that networkReadMillis is greater than cacheWatchSeconds when initializing Consul.Then the error is fixed.
   ![image](https://user-images.githubusercontent.com/6240715/220231946-c2944257-cdff-4219-a11c-277a6952219a.png)
   One more question:consul.client.version from configuration-consul is 1.2.6,but it is 1.5.3 from cluster-consul-plugin.Why are the versions different? Are there any compatibility issues? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [skywalking] tjzheng1002 commented on issue #10414: [Bug] Consul reported an error as the configuration center

Posted by "tjzheng1002 (via GitHub)" <gi...@apache.org>.
tjzheng1002 commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1436664401

   > I can see from source codes, aclToken is only picked up when it is set to not empty
   > 
   > https://github.com/apache/skywalking/blob/af3889b47eabc8e63ce54c5415d6ec6c4528c53a/oap-server/server-configuration/configuration-consul/src/main/java/org/apache/skywalking/oap/server/configuration/consul/ConsulConfigurationWatcherRegister.java#L70-L72
   > 
   > As you have the env, it is better you could debug on your env.
   
   I have debugged on my env,it is true that the aclToken is empty and not used.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [skywalking] tjzheng1002 commented on issue #10414: [Bug] Consul reported an error as the configuration center

Posted by "tjzheng1002 (via GitHub)" <gi...@apache.org>.
tjzheng1002 commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1436702405

   > So, what causes your error actually? From git commit log, last time, only this is added for acl support.
   
   I didn't use ACLToken, I just changed the configuration center to the address of the Consul service, and the Consul service did not set the ACLtoken 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [skywalking] tjzheng1002 commented on issue #10414: [Bug] Consul reported an error as the configuration center

Posted by "tjzheng1002 (via GitHub)" <gi...@apache.org>.
tjzheng1002 commented on issue #10414:
URL: https://github.com/apache/skywalking/issues/10414#issuecomment-1437737291

   > I mean, you should debug and try to address issues on your env. What you described doesn't match the codes.
   
   But consul as the cluster register,it runned without any error.I think my env shoud be collect.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org