You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@servicecomb.apache.org by GitBox <gi...@apache.org> on 2021/12/03 08:48:41 UTC
[GitHub] [servicecomb-java-chassis] jessefjxm opened a new issue #2653: Nuwa-CSE 启动时有小概率出现数组越界错误，导致内部接口调用失败
jessefjxm opened a new issue #2653:
URL: https://github.com/apache/servicecomb-java-chassis/issues/2653
**背景**
我们的微服务基于女娲框架开发，在启动时CSE加载完成后会调用一些微服务间CSE接口，用以初始化缓存
最近我们注意到，在开发环境和测试环境，启动时有大约5%的概率出现servicecomb相关的数组越界异常，导致部分内部接口调用失败。
完整错误栈位于底部，似乎是CSE在初始化被调用接口时失败了，但是看不出来具体是什么原因。由于这一问题是概率出现的，且最近一个月内仅有3~5天遇到，当天会有大概50%的概率在启动时遇到此错误，所以感觉跟配置项无关。
**产品**
Nuwa 3.0.8.103
（Nuwa-CSE插件）ServiceComb Java-Chassis 1.3.6
**错误栈**
```
2021-12-02 10:13:58.763 [main] ERROR - [MicroserviceVersions.java:safeSetInstances:182] - [] - Failed to setInstances, appId=MapOpenService, microserviceName=MapOpenService:MapOpenKitAccessService.
java.lang.IndexOutOfBoundsException: Index: 5, Size: 5
at java.util.ArrayList.rangeCheck(ArrayList.java:659) ~[?:1.8.0_272]
at java.util.ArrayList.get(ArrayList.java:435) ~[?:1.8.0_272]
at org.apache.servicecomb.common.rest.definition.RestOperationMeta.init(RestOperationMeta.java:105) ~[common-rest-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.common.rest.locator.ServicePathManager.addSchema(ServicePathManager.java:86) ~[common-rest-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.common.rest.RestEngineSchemaListener.onSchemaLoaded(RestEngineSchemaListener.java:54) ~[common-rest-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.core.definition.loader.SchemaListenerManager.notifySchemaListener(SchemaListenerManager.java:59) ~[java-chassis-core-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.core.definition.loader.SchemaListenerManager.notifySchemaListener(SchemaListenerManager.java:48) ~[java-chassis-core-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.core.definition.MicroserviceVersionMeta.<init>(MicroserviceVersionMeta.java:46) ~[java-chassis-core-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.core.definition.MicroserviceVersionMetaFactory.create(MicroserviceVersionMetaFactory.java:38) ~[java-chassis-core-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.serviceregistry.consumer.MicroserviceVersions.lambda$setInstances$1(MicroserviceVersions.java:200) ~[service-registry-1.3.6.jar!/:1.3.6]
at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) ~[?:1.8.0_272]
at org.apache.servicecomb.foundation.common.concurrent.ConcurrentHashMapEx.computeIfAbsent(ConcurrentHashMapEx.java:56) ~[foundation-common-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.serviceregistry.consumer.MicroserviceVersions.setInstances(MicroserviceVersions.java:199) ~[service-registry-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.serviceregistry.consumer.MicroserviceVersions.safeSetInstances(MicroserviceVersions.java:178) ~[service-registry-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.serviceregistry.consumer.MicroserviceVersions.pullInstances(MicroserviceVersions.java:164) ~[service-registry-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.serviceregistry.consumer.MicroserviceVersions.submitPull(MicroserviceVersions.java:136) ~[service-registry-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.serviceregistry.consumer.MicroserviceManager.lambda$getOrCreateMicroserviceVersions$0(MicroserviceManager.java:55) ~[service-registry-1.3.6.jar!/:1.3.6]
at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) ~[?:1.8.0_272]
at org.apache.servicecomb.foundation.common.concurrent.ConcurrentHashMapEx.computeIfAbsent(ConcurrentHashMapEx.java:56) ~[foundation-common-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.serviceregistry.consumer.MicroserviceManager.getOrCreateMicroserviceVersions(MicroserviceManager.java:53) ~[service-registry-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.serviceregistry.consumer.MicroserviceManager.getOrCreateMicroserviceVersionRule(MicroserviceManager.java:80) ~[service-registry-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.serviceregistry.consumer.AppManager.getOrCreateMicroserviceVersionRule(AppManager.java:67) ~[service-registry-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.core.provider.consumer.ReferenceConfig.<init>(ReferenceConfig.java:40) ~[java-chassis-core-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.core.provider.consumer.ConsumerProviderManager.createReferenceConfig(ConsumerProviderManager.java:56) ~[java-chassis-core-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.core.provider.consumer.ConsumerProviderManager.createReferenceConfig(ConsumerProviderManager.java:87) ~[java-chassis-core-1.3.6.jar!/:1.3.6]
at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) ~[?:1.8.0_272]
at org.apache.servicecomb.foundation.common.concurrent.ConcurrentHashMapEx.computeIfAbsent(ConcurrentHashMapEx.java:56) ~[foundation-common-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.core.provider.consumer.ConsumerProviderManager.getReferenceConfig(ConsumerProviderManager.java:91) ~[java-chassis-core-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.core.SCBEngine.getReferenceConfigForInvoke(SCBEngine.java:398) ~[java-chassis-core-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.provider.pojo.Invoker.findReferenceConfig(Invoker.java:122) ~[provider-pojo-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.provider.pojo.Invoker.createInvokerMeta(Invoker.java:91) ~[provider-pojo-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.provider.pojo.Invoker.invoke(Invoker.java:132) ~[provider-pojo-1.3.6.jar!/:1.3.6]
at com.sun.proxy.$Proxy138.queryList(Unknown Source) ~[?:?]
at com.huawei.map.open.sdk.client.projectbypass.ProjectBypassClient.lambda$getProjectBypassList$0(ProjectBypassClient.java:50) ~[map-open-service-sdk-6.3.0.1-20211201.070340-29.jar!/:?]
at com.huawei.map.foundation.utils.olc.OLCUtil.run(OLCUtil.java:78) ~[map-foundation-6.3.0.1-20211201.064413-11.jar!/:?]
at com.huawei.map.foundation.utils.olc.OLCCaller$OLCCallerBuilder.call(OLCCaller.java:92) ~[map-foundation-6.3.0.1-20211201.064413-11.jar!/:?]
at com.huawei.map.open.sdk.utils.OpenOLCUtil.lambda$call$1(OpenOLCUtil.java:268) ~[map-open-service-sdk-6.3.0.1-20211201.070340-29.jar!/:?]
at com.huawei.map.foundation.utils.UncheckCaller.innerCall(UncheckCaller.java:88) ~[map-foundation-6.3.0.1-20211201.064413-11.jar!/:?]
at com.huawei.map.foundation.utils.UncheckCaller.callQuietly(UncheckCaller.java:74) ~[map-foundation-6.3.0.1-20211201.064413-11.jar!/:?]
at com.huawei.map.open.sdk.utils.OpenOLCUtil.call(OpenOLCUtil.java:268) ~[map-open-service-sdk-6.3.0.1-20211201.070340-29.jar!/:?]
at com.huawei.map.open.sdk.client.projectbypass.ProjectBypassClient.getProjectBypassList(ProjectBypassClient.java:49) ~[map-open-service-sdk-6.3.0.1-20211201.070340-29.jar!/:?]
at com.huawei.map.open.sdk.cache.ProjectBypassCache.loadData(ProjectBypassCache.java:60) ~[map-open-service-sdk-6.3.0.1-20211201.070340-29.jar!/:?]
at com.huawei.map.foundation.cache.PeriodicReloadCache.innerAddReloadJob(PeriodicReloadCache.java:168) ~[map-foundation-6.3.0.1-20211201.064413-11.jar!/:?]
at com.huawei.map.foundation.cache.PeriodicReloadCache.addReloadJob(PeriodicReloadCache.java:126) ~[map-foundation-6.3.0.1-20211201.064413-11.jar!/:?]
at com.huawei.map.foundation.service.SCBEngineBootListener.onBootEvent(SCBEngineBootListener.java:72) ~[map-foundation-6.3.0.1-20211201.064413-11.jar!/:?]
at org.apache.servicecomb.core.SCBEngine.triggerEvent(SCBEngine.java:163) ~[java-chassis-core-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.core.SCBEngine$1.afterRegistryInstance(SCBEngine.java:202) ~[java-chassis-core-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.foundation.common.event.SimpleSubscriber.syncDispatch(SimpleSubscriber.java:96) ~[foundation-common-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.foundation.common.event.SimpleSubscriber.dispatchEvent(SimpleSubscriber.java:88) ~[foundation-common-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.foundation.common.event.SimpleEventBus.post(SimpleEventBus.java:73) ~[foundation-common-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.serviceregistry.task.AbstractTask.run(AbstractTask.java:54) ~[service-registry-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.serviceregistry.task.CompositeTask.run(CompositeTask.java:35) ~[service-registry-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.serviceregistry.task.ServiceCenterTask.init(ServiceCenterTask.java:108) ~[service-registry-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.serviceregistry.registry.AbstractServiceRegistry.run(AbstractServiceRegistry.java:178) ~[service-registry-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.serviceregistry.registry.RemoteServiceRegistry.run(RemoteServiceRegistry.java:84) ~[?:1.3.6]
at org.apache.servicecomb.serviceregistry.RegistryUtils.run(RegistryUtils.java:70) ~[service-registry-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.core.SCBEngine.doInit(SCBEngine.java:287) ~[java-chassis-core-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.core.SCBEngine.init(SCBEngine.java:225) ~[java-chassis-core-1.3.6.jar!/:1.3.6]
at org.apache.servicecomb.core.CseApplicationListener.onApplicationEvent(CseApplicationListener.java:81) ~[java-chassis-core-1.3.6.jar!/:1.3.6]
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172) ~[spring-context-5.2.18.RELEASE.jar!/:5.2.18.RELEASE]
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165) ~[spring-context-5.2.18.RELEASE.jar!/:5.2.18.RELEASE]
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139) ~[spring-context-5.2.18.RELEASE.jar!/:5.2.18.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:404) ~[spring-context-5.2.18.RELEASE.jar!/:5.2.18.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:361) ~[spring-context-5.2.18.RELEASE.jar!/:5.2.18.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:898) ~[spring-context-5.2.18.RELEASE.jar!/:5.2.18.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:554) ~[spring-context-5.2.18.RELEASE.jar!/:5.2.18.RELEASE]
at com.huawei.nuwa.SpringInitializer.init(SpringInitializer.java:40) ~[nuwa-core-3.0.8.103.jar!/:3.0.8.103]
at com.huawei.nuwa.NuwaBootstrap.start(NuwaBootstrap.java:50) ~[nuwa-core-3.0.8.103.jar!/:3.0.8.103]
at com.huawei.nuwa.container.model.PluginModel.lambda$null$1(PluginModel.java:684) ~[nuwa-boot-container-3.0.8.103.jar!/:?]
at java.util.Optional.ifPresent(Optional.java:159) ~[?:1.8.0_272]
at com.huawei.nuwa.container.model.PluginModel.lambda$start$2(PluginModel.java:684) ~[nuwa-boot-container-3.0.8.103.jar!/:?]
at com.huawei.nuwa.boot.loader.util.ClassLoaderUtils.runWithClassLoader(ClassLoaderUtils.java:57) [nuwa-boot-loader-3.0.8.103.jar!/:?]
at com.huawei.nuwa.container.model.PluginModel.start(PluginModel.java:683) [nuwa-boot-container-3.0.8.103.jar!/:?]
at com.huawei.nuwa.container.service.plugin.PluginDeployServiceImpl.deploy(PluginDeployServiceImpl.java:31) [nuwa-boot-container-3.0.8.103.jar!/:?]
at com.huawei.nuwa.container.pipeline.DeployPluginStage.process(DeployPluginStage.java:33) [nuwa-boot-container-3.0.8.103.jar!/:?]
at com.huawei.nuwa.container.pipeline.StandardPipeline.process(StandardPipeline.java:58) [nuwa-boot-container-3.0.8.103.jar!/:?]
at com.huawei.nuwa.container.NuwaContainer.start(NuwaContainer.java:99) [nuwa-boot-container-3.0.8.103.jar!/:?]
at com.huawei.nuwa.container.NuwaContainer.main(NuwaContainer.java:65) [nuwa-boot-container-3.0.8.103.jar!/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_272]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_272]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_272]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_272]
at com.huawei.nuwa.boot.loader.MainMethodRunner.run(MainMethodRunner.java:41) [nuwa-boot-container-full-3.0.8.103.jar:3.0.8.103]
at com.huawei.nuwa.boot.loader.AbstractLauncher.launch(AbstractLauncher.java:105) [nuwa-boot-container-full-3.0.8.103.jar:3.0.8.103]
at com.huawei.nuwa.boot.loader.AbstractLauncher.launch(AbstractLauncher.java:52) [nuwa-boot-container-full-3.0.8.103.jar:3.0.8.103]
at com.huawei.nuwa.boot.loader.NuwaRuntimeLauncher.main(NuwaRuntimeLauncher.java:28) [nuwa-boot-container-full-3.0.8.103.jar:3.0.8.103]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@servicecomb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org