You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2021/03/29 01:31:39 UTC
[GitHub] [skywalking] goooogs opened a new issue #6635: swctl checkHealth passed but failed with -javaagent (grpc over https)
goooogs opened a new issue #6635:
URL: https://github.com/apache/skywalking/issues/6635
Please answer these questions before submitting your issue.
- Why do you submit this issue?
- [x] Question or discussion
- [x] Bug
- [ ] Requirement
- [ ] Feature or performance improvement
___
### Question
- What do you want to know?
I have two isolated VPC: VPC1 and VPC2.
skywalking was deployed in VPC1 (kubernetes + istio),
and I'm located in VPC2, access skywalking with these addresses:
* UI: https://skywalking-ui.example.com
* GRPC over https: skywalking-oap-grpc.example.com:443
* REST: https://skywalking-oap-rest.example.com
before startup my java application, i use swctl to check skywalking status:
```
swctl --base-url https://skywalking-oap-rest.example.com/graphql --grpcAddr skywalking-oap-grpc.example.com:443 checkHealth --grpcTLS
INFO OAP modules are healthy
INFO OAP gRPC is healthy
```
it looks fine. Then i startup my java application:
```
SW_AGENT_COLLECTOR_BACKEND_SERVICES=skywalking-oap-grpc.example.com:443 \
SW_AGENT_NAME=test::app1 \
SW_AGENT_FORCE_TLS=true \
java -javaagent:/agent/skywalking-agent.jar -jar /app1.jar
```
But i got apm agent error
```
ERROR 2021-03-28 10:30:27:764 SkywalkingAgent-7-JVMService-consume-0 JVMMetricsSender : send JVM metrics to Collector fail.
org.apache.skywalking.apm.dependencies.io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
at org.apache.skywalking.apm.dependencies.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:262)
at org.apache.skywalking.apm.dependencies.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:243)
at org.apache.skywalking.apm.dependencies.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:156)
at org.apache.skywalking.apm.network.language.agent.v3.JVMMetricReportServiceGrpc$JVMMetricReportServiceBlockingStub.collect(JVMMetricReportServiceGrpc.java:181)
at org.apache.skywalking.apm.agent.core.jvm.JVMMetricsSender.run(JVMMetricsSender.java:82)
at org.apache.skywalking.apm.util.RunnableWithExceptionProtection.run(RunnableWithExceptionProtection.java:33)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.io.IOException: Connection reset by peer
at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at java.base/sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276)
at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:233)
at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:223)
at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:358)
at org.apache.skywalking.apm.dependencies.io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:253)
at org.apache.skywalking.apm.dependencies.io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133)
at org.apache.skywalking.apm.dependencies.io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
at org.apache.skywalking.apm.dependencies.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148)
at org.apache.skywalking.apm.dependencies.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at org.apache.skywalking.apm.dependencies.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at org.apache.skywalking.apm.dependencies.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at org.apache.skywalking.apm.dependencies.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at org.apache.skywalking.apm.dependencies.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at org.apache.skywalking.apm.dependencies.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at org.apache.skywalking.apm.dependencies.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 more
```
Why swctl checkHealth can pass, but it really not with grpc over https?
I hope skywalking grpc over https works fine for apm agent.
___
### Bug
- Which version of SkyWalking, OS, and JRE?
SkyWalking: apache-skywalking-apm-es6-8.4.0
swctl: version 0.6.0
OS: MacOS Catalina 10.15.4
JRE: openjdk version "11.0.8" 2020-07-14, 64-Bit
- Which company or project?
- What happened?
If possible, provide a way to reproduce the error. e.g. demo application, component version.
```yaml
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
annotations:
name: my-gateway
namespace: istio-system
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- '*.example.com'
port:
name: http
number: 80
protocol: HTTP
- hosts:
- '*.example.com'
port:
name: https
number: 443
protocol: HTTPS
tls:
mode: SIMPLE
privateKey: /etc/istio/ingressgateway-certs/tls.key
serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: skywalking-oap-grpc
spec:
hosts:
- skywalking-oap-grpc.example.com
gateways:
- istio-system/my-gateway
http:
- route:
- destination:
host: skywalking-oap.default.svc.cluster.local
port:
number: 11800
```
___
### Requirement or improvement
- Please describe your requirements or improvement suggestions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng closed issue #6635: swctl checkHealth passed but failed with -javaagent (grpc over https)
Posted by GitBox <gi...@apache.org>.
wu-sheng closed issue #6635:
URL: https://github.com/apache/skywalking/issues/6635
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #6635: swctl checkHealth passed but failed with -javaagent (grpc over https)
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #6635:
URL: https://github.com/apache/skywalking/issues/6635#issuecomment-809030216
healthcheck has nothing related to agentconnecting. It is for oap internal status only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org