You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@dubbo.apache.org by "ghsau (GitHub)" <gi...@apache.org> on 2018/12/19 10:18:36 UTC

[GitHub] [incubator-dubbo] ghsau commented on issue #351: Hession反序列化导致CPU占用飙高

> 我们在生产环境也遇到这个问题,**现象是服务提供者耗时就1~2ms,但消费者耗时偶发性的2s超时。最开始通过链路跟踪看到耗时在网络**,怀疑是网络重传包引起,TCP监控看到有一些异常但不明显,通过`tcpdump`没发现问题。然后有同事观察到**CPU占用偶发性高,与超时时间保持一致**,同时观察到下面的异常日志每天有1.5w上下。
> 
> * dubbo-2.5.8
> * org.jboss.netty:netty-3.2.5.Final
> 
> 完整的异常日志:
> 
> ```
> 2018-12-13 00:00:02,863 WARN  [New I/O client worker #1-5] c.a.c.c.h.io.SerializerFactory:613 - [] [] [] Hessian/Burlap: 'com.xxx.ucenter.constant.AccountAuthLoginTypeEnum' is an unknown class in TomcatEmbeddedWebappClassLoader
>   context: ROOT
>   delegate: true
> ----------> Parent Classloader:
> org.springframework.boot.loader.LaunchedURLClassLoader@442d9b6e
> :
> java.lang.ClassNotFoundException: com.xxx.ucenter.constant.AccountAuthLoginTypeEnum
> ```
> 从异常日志看到是Dubbo消费者的Netty客户端工作者线程(`New I/O client worker`,`NioWorker`)在反序列化时一直找不到某个类型(`ClassNotFoundException: AccountAuthLoginTypeEnum`),然后不停地加载类引起。修复这个异常后,偶发性超时就没有了。
> 
> 我们准备在`SerializerFactory`里对`ClassNotFoundException`异常做本地缓存,避免重复地加载找不到的类。

嗯,就是这样解决

[ Full content available at: https://github.com/apache/incubator-dubbo/issues/351 ]
This message was relayed via gitbox.apache.org for notifications@dubbo.apache.org