You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2021/04/15 03:28:19 UTC
[GitHub] [skywalking] yckbilly1929 commented on issue #6513: receiver.envoy.als.Fetcher show error log: code:0 header:null body:null
yckbilly1929 commented on issue #6513:
URL: https://github.com/apache/skywalking/issues/6513#issuecomment-820030114
> @wu-sheng @kezhenxu94 Sorry to interrupt again, i reinstall skywalking 8.4.0 using the following command：
>
> ```
> helm install 8.4.0 skywalking -n istio-system \
> --set oap.env.SW_ENVOY_METRIC_ALS_HTTP_ANALYSIS=k8s-mesh \
> --set fullnameOverride=skywalking \
> --set oap.envoy.als.enabled=true \
> --set ui.image.tag=8.4.0 \
> --set oap.image.tag=8.4.0-es6 \
> --set oap.storageType=elasticsearch
> ```
>
> but the oap show me error logs:
>
> ```
> 2021-03-08 10:20:44,303 - org.elasticsearch.client.RestClient -10628 [I/O dispatcher 1] WARN [] - request [HEAD http://elasticsearch-master:9200/_template/instance_jvm_thread_peak_count] returned 1 warnings: [299 Elasticsearch-6.8.6-3d9f765 "[types removal] The parameter include_type_name should be explicitly specified in get template requests to prepare for 7.0. In 7.0 include_type_name will default to 'false', which means responses will omit the type name in mapping definitions."]
> 2021-03-08 10:20:44,305 - org.apache.skywalking.oap.server.core.analysis.DispatcherManager -10630 [main] INFO [] - Dispatcher org.apache.skywalking.oap.server.core.source.oal.rt.dispatcher.ServiceInstanceJVMMemoryDispatcher is added into DefaultScopeDefine 9.
> 2021-03-08 10:20:44,306 - org.apache.skywalking.oap.server.core.analysis.DispatcherManager -10631 [main] INFO [] - Dispatcher org.apache.skywalking.oap.server.core.source.oal.rt.dispatcher.ServiceInstanceJVMGCDispatcher is added into DefaultScopeDefine 11.
> 2021-03-08 10:20:44,306 - org.apache.skywalking.oap.server.core.analysis.DispatcherManager -10631 [main] INFO [] - Dispatcher org.apache.skywalking.oap.server.core.source.oal.rt.dispatcher.ServiceInstanceJVMThreadDispatcher is added into DefaultScopeDefine 33.
> 2021-03-08 10:20:44,306 - org.apache.skywalking.oap.server.core.analysis.DispatcherManager -10631 [main] INFO [] - Dispatcher org.apache.skywalking.oap.server.core.source.oal.rt.dispatcher.ServiceInstanceJVMCPUDispatcher is added into DefaultScopeDefine 8.
> 2021-03-08 10:20:44,308 - org.apache.skywalking.oap.server.library.server.grpc.GRPCServer -10633 [main] INFO [] - Bind handler JVMMetricReportServiceHandler into gRPC server 0.0.0.0:11800
> 2021-03-08 10:20:44,310 - org.apache.skywalking.oap.server.library.server.grpc.GRPCServer -10635 [main] INFO [] - Bind handler JVMMetricReportServiceHandlerCompat into gRPC server 0.0.0.0:11800
> 2021-03-08 10:20:44,312 - org.apache.skywalking.oap.server.library.module.BootstrapFlow -10637 [main] INFO [] - start the provider default in receiver-meter module.
> 2021-03-08 10:20:44,312 - org.apache.skywalking.oap.server.library.server.grpc.GRPCServer -10637 [main] INFO [] - Bind handler MeterServiceHandler into gRPC server 0.0.0.0:11800
> 2021-03-08 10:20:44,315 - org.apache.skywalking.oap.server.library.server.grpc.GRPCServer -10640 [main] INFO [] - Bind handler MeterServiceHandlerCompat into gRPC server 0.0.0.0:11800
> 2021-03-08 10:20:44,465 - org.apache.skywalking.oap.server.library.server.jetty.JettyServer -10790 [main] INFO [] - start server, host: 0.0.0.0, port: 12800
> 2021-03-08 10:20:44,470 - org.eclipse.jetty.server.Server -10795 [main] INFO [] - jetty-9.4.28.v20200408; built: 2020-04-08T17:49:39.557Z; git: ab228fde9e55e9164c738d7fa121f8ac5acd51c9; jvm 11.0.9+11
> 2021-03-08 10:20:44,511 - org.eclipse.jetty.server.handler.ContextHandler -10836 [main] INFO [] - Started o.e.j.s.ServletContextHandler@1455d0f5{/,null,AVAILABLE}
> 2021-03-08 10:20:44,521 - org.eclipse.jetty.server.AbstractConnector -10846 [main] INFO [] - Started ServerConnector@1a0d96a5{HTTP/1.1, (http/1.1)}{0.0.0.0:12800}
> 2021-03-08 10:20:44,522 - org.eclipse.jetty.server.Server -10847 [main] INFO [] - Started @10895ms
> 2021-03-08 10:20:44,523 - org.apache.skywalking.oap.server.core.storage.PersistenceTimer -10848 [main] INFO [] - persistence timer start
> 2021-03-08 10:20:44,526 - org.apache.skywalking.oap.server.core.cache.CacheUpdateTimer -10851 [main] INFO [] - Cache updateServiceInventory timer start
> 2021-03-08 10:20:44,887 - io.kubernetes.client.informer.cache.ProcessorListener -11212 [pool-12-thread-1] ERROR [] - failed invoking ADD event handler: null
> 2021-03-08 10:20:44,887 - io.kubernetes.client.informer.cache.ProcessorListener -11212 [pool-12-thread-1] ERROR [] - failed invoking ADD event handler: null
> 2021-03-08 10:20:44,887 - io.kubernetes.client.informer.cache.ProcessorListener -11212 [pool-12-thread-1] ERROR [] - failed invoking ADD event handler: null
> 2021-03-08 10:20:45,018 - io.kubernetes.client.informer.cache.ProcessorListener -11343 [pool-15-thread-1] ERROR [] - failed invoking ADD event handler: null
> 2021-03-08 10:20:45,521 - io.kubernetes.client.informer.cache.ProcessorListener -11846 [pool-12-thread-1] ERROR [] - failed invoking UPDATE event handler: null
> 2021-03-08 10:20:47,801 - io.kubernetes.client.informer.cache.ProcessorListener -14126 [pool-12-thread-1] ERROR [] - failed invoking UPDATE event handler: null
> ```
i'm now testing on istio 1.9.2, elastic-search 7.12.0 and oap 8.5.0
metrics working but there are no traces, wonder if this is related issues
below are the stacktrace:
```
2021-04-15 03:09:43,893 - io.kubernetes.client.informer.cache.ProcessorListener -199718 [pool-15-thread-1] ERROR [] - failed invoking ADD event handler: {}
java.lang.NullPointerException: null
at java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011) ~[?:?]
at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006) ~[?:?]
at org.apache.skywalking.oap.server.receiver.envoy.als.k8s.K8SServiceRegistry.lambda$addPod$5(K8SServiceRegistry.java:218) ~[envoy-metrics-receiver-plugin-8.5.0.jar:8.5.0]
at java.util.Optional.ifPresent(Optional.java:183) ~[?:?]
at org.apache.skywalking.oap.server.receiver.envoy.als.k8s.K8SServiceRegistry.addPod(K8SServiceRegistry.java:217) ~[envoy-metrics-receiver-plugin-8.5.0.jar:8.5.0]
at org.apache.skywalking.oap.server.receiver.envoy.als.k8s.K8SServiceRegistry$3.onAdd(K8SServiceRegistry.java:187) ~[envoy-metrics-receiver-plugin-8.5.0.jar:8.5.0]
at org.apache.skywalking.oap.server.receiver.envoy.als.k8s.K8SServiceRegistry$3.onAdd(K8SServiceRegistry.java:184) ~[envoy-metrics-receiver-plugin-8.5.0.jar:8.5.0]
at io.kubernetes.client.informer.cache.ProcessorListener.run(ProcessorListener.java:71) [client-java-10.0.0.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
2021-04-15 03:09:43,932 - io.kubernetes.client.informer.cache.ProcessorListener -199757 [pool-15-thread-1] ERROR [] - failed invoking UPDATE event handler: {}
java.lang.NullPointerException: null
at java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011) ~[?:?]
at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006) ~[?:?]
at org.apache.skywalking.oap.server.receiver.envoy.als.k8s.K8SServiceRegistry.lambda$addPod$5(K8SServiceRegistry.java:218) ~[envoy-metrics-receiver-plugin-8.5.0.jar:8.5.0]
at java.util.Optional.ifPresent(Optional.java:183) ~[?:?]
at org.apache.skywalking.oap.server.receiver.envoy.als.k8s.K8SServiceRegistry.addPod(K8SServiceRegistry.java:217) ~[envoy-metrics-receiver-plugin-8.5.0.jar:8.5.0]
at org.apache.skywalking.oap.server.receiver.envoy.als.k8s.K8SServiceRegistry$3.onUpdate(K8SServiceRegistry.java:192) ~[envoy-metrics-receiver-plugin-8.5.0.jar:8.5.0]
at org.apache.skywalking.oap.server.receiver.envoy.als.k8s.K8SServiceRegistry$3.onUpdate(K8SServiceRegistry.java:184) ~[envoy-metrics-receiver-plugin-8.5.0.jar:8.5.0]
at io.kubernetes.client.informer.cache.ProcessorListener.run(ProcessorListener.java:61) [client-java-10.0.0.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org