You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2022/11/25 09:12:47 UTC
[GitHub] [skywalking] chengfq5 opened a new issue, #10028: [Bug] skywakling hanged up request under multi-concurrent requests on spring cloud gateway
chengfq5 opened a new issue, #10028:
URL: https://github.com/apache/skywalking/issues/10028
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar issues.
### Apache SkyWalking Component
Java Agent (apache/skywalking-java)
### What happened
skywakling hanged up request under multi-concurrent requests on spring cloud gateway
### What you expected to happen
scg handed up request, and can recur stably when assign parameter of "-c " greatter than 1
ab -c 4 -n 400 -T 'application/json;charset=UTF-8' 'http://127.0.0.1:30000/api/hello'
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
apr_pollset_poll: The timeout specified has expired (70007)
Total of 398 requests completed
spring-cloud-gateway throwed excepiton :
2022-11-25 17:10:42.491 ERROR 44595 --- [ctor-http-nio-6] reactor.core.publisher.Operators : Operator called default onErrorDropped
java.lang.NullPointerException: null
at org.apache.skywalking.apm.plugin.spring.cloud.gateway.v20x.HttpClientRequestInterceptor$2.accept(HttpClientRequestInterceptor.java:98) ~[na:na]
at org.apache.skywalking.apm.plugin.spring.cloud.gateway.v20x.HttpClientRequestInterceptor$2.accept(HttpClientRequestInterceptor.java:94) ~[na:na]
at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onNext(MonoPeekTerminal.java:189) ~[reactor-core-3.1.10.RELEASE.jar:3.1.10.RELEASE]
at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onNext(FluxMap.java:201) [reactor-core-3.1.10.RELEASE.jar:3.1.10.RELEASE]
at reactor.core.publisher.FluxRetryPredicate$RetryPredicateSubscriber.onNext(FluxRetryPredicate.java:81) [reactor-core-3.1.10.RELEASE.jar:3.1.10.RELEASE]
at reactor.core.publisher.MonoCreate$DefaultMonoSink.success(MonoCreate.java:146) [reactor-core-3.1.10.RELEASE.jar:3.1.10.RELEASE]
at reactor.ipc.netty.channel.PooledClientContextHandler.fireContextActive(PooledClientContextHandler.java:87) [reactor-netty-0.7.10.RELEASE.jar:0.7.10.RELEASE]
at reactor.ipc.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:584) [reactor-netty-0.7.10.RELEASE.jar:0.7.10.RELEASE]
at reactor.ipc.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:138) [reactor-netty-0.7.10.RELEASE.jar:0.7.10.RELEASE]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.29.Final.jar:4.1.29.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) [netty-codec-4.1.29.Final.jar:4.1.29.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) [netty-codec-4.1.29.Final.jar:4.1.29.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:628) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:563) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:480) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:442) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884) [netty-common-4.1.29.Final.jar:4.1.29.Final]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_301]
2022-11-25 17:10:42.509 ERROR 44595 --- [ctor-http-nio-6] r.ipc.netty.channel.ContextHandler : Error cannot be forwarded to user-facing Mono
reactor.core.Exceptions$BubblingException: java.lang.NullPointerException
at reactor.core.Exceptions.bubble(Exceptions.java:154) ~[reactor-core-3.1.10.RELEASE.jar:3.1.10.RELEASE]
at reactor.core.publisher.Operators.onErrorDropped(Operators.java:263) ~[reactor-core-3.1.10.RELEASE.jar:3.1.10.RELEASE]
at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onNext(MonoPeekTerminal.java:193) ~[reactor-core-3.1.10.RELEASE.jar:3.1.10.RELEASE]
at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onNext(FluxMap.java:201) ~[reactor-core-3.1.10.RELEASE.jar:3.1.10.RELEASE]
at reactor.core.publisher.FluxRetryPredicate$RetryPredicateSubscriber.onNext(FluxRetryPredicate.java:81) ~[reactor-core-3.1.10.RELEASE.jar:3.1.10.RELEASE]
at reactor.core.publisher.MonoCreate$DefaultMonoSink.success(MonoCreate.java:146) ~[reactor-core-3.1.10.RELEASE.jar:3.1.10.RELEASE]
at reactor.ipc.netty.channel.PooledClientContextHandler.fireContextActive(PooledClientContextHandler.java:87) ~[reactor-netty-0.7.10.RELEASE.jar:0.7.10.RELEASE]
at reactor.ipc.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:584) ~[reactor-netty-0.7.10.RELEASE.jar:0.7.10.RELEASE]
at reactor.ipc.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:138) ~[reactor-netty-0.7.10.RELEASE.jar:0.7.10.RELEASE]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.29.Final.jar:4.1.29.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) [netty-codec-4.1.29.Final.jar:4.1.29.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) [netty-codec-4.1.29.Final.jar:4.1.29.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:628) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:563) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:480) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:442) [netty-transport-4.1.29.Final.jar:4.1.29.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884) [netty-common-4.1.29.Final.jar:4.1.29.Final]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_301]
Caused by: java.lang.NullPointerException: null
at org.apache.skywalking.apm.plugin.spring.cloud.gateway.v20x.HttpClientRequestInterceptor$2.accept(HttpClientRequestInterceptor.java:98) ~[na:na]
at org.apache.skywalking.apm.plugin.spring.cloud.gateway.v20x.HttpClientRequestInterceptor$2.accept(HttpClientRequestInterceptor.java:94) ~[na:na]
at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onNext(MonoPeekTerminal.java:189) ~[reactor-core-3.1.10.RELEASE.jar:3.1.10.RELEASE]
... 29 common frames omitted
### How to reproduce
spring-cloud-gateway: 2.0.2.RELEASE
skywalking-version: apache-skywalking-java-agent-8.13.0
test scg demo: https://github.com/chengfq5/scg_demo.git
jvm options: -javaagent:/XXX/skywalking-agent/skywalking-agent.jar
ab -c 4 -n 400 -T 'application/json;charset=UTF-8' 'http://127.0.0.1:30000/api/hello'
/hello is a any api in another servcie need to mock by yourself
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng closed issue #10028: [Bug] skywakling hanged up request under multi-concurrent requests on spring cloud gateway
Posted by GitBox <gi...@apache.org>.
wu-sheng closed issue #10028: [Bug] skywakling hanged up request under multi-concurrent requests on spring cloud gateway
URL: https://github.com/apache/skywalking/issues/10028
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org