You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@dubbo.apache.org by "xlq20080808 (via GitHub)" <gi...@apache.org> on 2023/06/11 16:52:52 UTC
[GitHub] [dubbo] xlq20080808 opened a new issue, #12493: 未及时释放FutureContext造成频繁GC问题
xlq20080808 opened a new issue, #12493:
URL: https://github.com/apache/dubbo/issues/12493
<!-- If you need to report a security issue please visit https://github.com/apache/dubbo/security/policy -->
- [x] I have searched the [issues](https://github.com/apache/dubbo/issues) of this repository and believe that this is not a duplicate.
### Environment
* Dubbo version: 3.0.12 3.1.8
* Java version: 1.8
### Steps to reproduce this issue
1. consumer及provider 通过环境变量，设置参数 future.sync.set=false
2. provider返回的报文较大
3. 3.0.12版本的调用，dubbo协议不会产生频繁GC问题，triple会产生
4. 3.1.8版本调用，dubbo协议及triple协议都会产生频繁GC问题
Pls. provide [GitHub address] to reproduce this issue.
### Expected Behavior
Consumer通过dubbo或triple请求，设置了future.sync.set=false 不会在IntenalThreadLocal中保存返回结果，不再产生频繁GC问题
<!-- What do you expect from the above steps？-->
### Actual Behavior
统一设置了参数 future.sync.set=false
**3.0.12版本:**
> Dubbo协议调用
及时清除了FutureContext，生效原因
<img width="1135" alt="image" src="https://github.com/apache/dubbo/assets/33999699/cca05d9d-dacd-4cf9-95d9-a72f9e20533d">
并在之后响应过程中清除了FutureContext
<img width="1088" alt="image" src="https://github.com/apache/dubbo/assets/33999699/c63dff14-6814-41ac-8ad2-d35968f4cd8a">
且因参数future.sync.set=false 之后也不再设置FutureContext，从而能够解决频发GC的情况
> Tiple协议调用
Tiple未及时清除了FutureContext原因
<img width="1027" alt="image" src="https://github.com/apache/dubbo/assets/33999699/2a71560a-2053-4d9e-8f88-07e976e7c713">
且没有在接收到响应后清除InternalThreadLocal
虽参数future.sync.set=false，但已经将FutureContext设置进了InternalThreadLocal，会造成频繁GC的情况
**3.1.8版本:**
Dubbo协议、Triple协议都没有及时清除FutureContext
> Dubbo协议调用
<img width="1147" alt="image" src="https://github.com/apache/dubbo/assets/33999699/6b170820-1088-423f-8634-563944f6b00b">
> Tiple协议调用
Triple未清除原因 同3.0.12
<!-- What actually happens? -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] xlq20080808 commented on issue #12493: 未及时释放FutureContext造成频繁GC问题
Posted by "xlq20080808 (via GitHub)" <gi...@apache.org>.
xlq20080808 commented on issue #12493:
URL: https://github.com/apache/dubbo/issues/12493#issuecomment-1586929863
> >
>
> 1. dubbo3.0.12中 直接清除的逻辑就不太合适，会把其他值线程其他变量都清理掉,比如服务端返回的 RpcContext 相关的值，如开启了 future 设置，在响应的时候被清除掉是不合理的。future 的清理 和 处理服务端返回的 attch 参数信息一样。 框架没有合适的处理时机。只能在下次使用的时候清理。或者用户手动清理掉。
> 2. `FutureContext.getContext().setCompatibleFuture` 这个不是为了 适配2.6.x 相关的么, 这个代码目前存在会导致 `future.sync.set` 这个不起作用。 npe 是由因为clearAll了导致的
是的，如果dubbo协议及triple协议中的`FutureContext.getContext().setCompatibleFuture`仅是适配2.6.x 相关，可删除的话，是可以解决该问题的
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] AlbumenJ closed issue #12493: 未及时释放FutureContext造成频繁GC问题
Posted by "AlbumenJ (via GitHub)" <gi...@apache.org>.
AlbumenJ closed issue #12493: 未及时释放FutureContext造成频繁GC问题
URL: https://github.com/apache/dubbo/issues/12493
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] liufeiyu1002 commented on issue #12493: 未及时释放FutureContext造成频繁GC问题
Posted by "liufeiyu1002 (via GitHub)" <gi...@apache.org>.
liufeiyu1002 commented on issue #12493:
URL: https://github.com/apache/dubbo/issues/12493#issuecomment-1586948839
你可以先写个filter之类的fix 一下， 手动把线程变量里的 future 移除掉，如果你不用到它的话。 因为你返回的响应体较大，多个业务线程比较容易造成内存爆涨 导致gc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] xlq20080808 commented on issue #12493: 未及时释放FutureContext造成频繁GC问题
Posted by "xlq20080808 (via GitHub)" <gi...@apache.org>.
xlq20080808 commented on issue #12493:
URL: https://github.com/apache/dubbo/issues/12493#issuecomment-1586465800
> > 3.1.8版本:
> > Dubbo协议、Triple协议都没有及时清除FutureContext
>
> 这个 getAndRemote 之后只是为了还原信息，需要看下第一个把环境污染的行为是什么，不然每一次 get 都应该是空的
应该是不同协议分别在这两处第一次将future设置进了InternalThreadlocal
Triple调用
org.apache.dubbo.rpc.protocol.tri.TripleInvoker#invokeUnary
<img width="1140" alt="image" src="https://github.com/apache/dubbo/assets/33999699/24b7446c-f4cd-435b-8af9-20a99ba5c2a8">
Dubbo调用
org.apache.dubbo.rpc.protocol.dubbo.DubboInvoker#doInvoke
<img width="1152" alt="image" src="https://github.com/apache/dubbo/assets/33999699/73e11d27-ea4e-4726-89a8-48290d65c8b1">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] AlbumenJ commented on issue #12493: 未及时释放FutureContext造成频繁GC问题
Posted by "AlbumenJ (via GitHub)" <gi...@apache.org>.
AlbumenJ commented on issue #12493:
URL: https://github.com/apache/dubbo/issues/12493#issuecomment-1595968973
> @AlbumenJ 我理解将future放入到InternalThreadlocal是为了兼容2.6.x版本trace功能或其他自定义filter设置回调函数。请教下如果我们这确认没有使用自定义回调，通过filter去除threadlocal中的future是否还有其他风险
如果没有使用可以直接删除
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] AlbumenJ commented on issue #12493: 未及时释放FutureContext造成频繁GC问题
Posted by "AlbumenJ (via GitHub)" <gi...@apache.org>.
AlbumenJ commented on issue #12493:
URL: https://github.com/apache/dubbo/issues/12493#issuecomment-1586445242
这里使用 get 的原因是需要保证业务自己的 biz 线程池的缓存还有效
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] liufeiyu1002 commented on issue #12493: 未及时释放FutureContext造成频繁GC问题
Posted by "liufeiyu1002 (via GitHub)" <gi...@apache.org>.
liufeiyu1002 commented on issue #12493:
URL: https://github.com/apache/dubbo/issues/12493#issuecomment-1586911277
>
1. dubbo3.0.12中 直接清除的逻辑就不太合适，会把其他值线程其他变量都清理掉,比如服务端返回的 RpcContext 相关的值，如开启了 future 设置，在响应的时候被清除掉是不合理的。future 的清理 和 处理服务端返回的 attch 参数信息一样。 框架没有合适的处理时机。只能在下次使用的时候清理。或者用户手动清理掉。
2. `FutureContext.getContext().setCompatibleFuture` 这个不是为了 适配2.6.x 相关的么, 这个代码目前存在会导致 `future.sync.set` 这个不起作用。 npe 是由因为clearAll了导致的
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] xlq20080808 commented on issue #12493: 未及时释放FutureContext造成频繁GC问题
Posted by "xlq20080808 (via GitHub)" <gi...@apache.org>.
xlq20080808 commented on issue #12493:
URL: https://github.com/apache/dubbo/issues/12493#issuecomment-1590359985
@AlbumenJ 我理解将future放入到InternalThreadlocal是为了兼容2.6.x版本trace功能或其他自定义filter设置回调函数。请教下如果我们这确认没有使用自定义回调，通过filter去除threadlocal中的future是否还有其他风险
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] liufeiyu1002 commented on issue #12493: 未及时释放FutureContext造成频繁GC问题
Posted by "liufeiyu1002 (via GitHub)" <gi...@apache.org>.
liufeiyu1002 commented on issue #12493:
URL: https://github.com/apache/dubbo/issues/12493#issuecomment-1586759904
在同步调用时。在消费端 `FutureContext.getContext()` 是作用在业务线程上的。整个消费调用都是用业务线程上执行的。所以 `FutureContext.getContext()` 不会被清除，除非手动清除掉 或者业务线程下次调用会覆盖掉旧值。 还有就是在每次调用以后 就 执行 `FutureContext.getContext().setCompatibleFuture(appResponseFuture)` 注释`// save for 2.6.x compatibility, for example, TraceFilter in Zipkin uses com.alibaba.xxx.FutureAdapter
` 写的是 为了适配 2.6.x，现在 到了3.x 了是不可以考虑移除这段代码了？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] xlq20080808 commented on issue #12493: 未及时释放FutureContext造成频繁GC问题
Posted by "xlq20080808 (via GitHub)" <gi...@apache.org>.
xlq20080808 commented on issue #12493:
URL: https://github.com/apache/dubbo/issues/12493#issuecomment-1586793574
> 在同步调用时。在消费端 `FutureContext.getContext()` 是作用在业务线程上的。整个消费调用都是用业务线程上执行的。所以 `FutureContext.getContext()` 不会被清除，除非手动清除掉 或者业务线程下次调用会覆盖掉旧值。 还有就是在每次调用以后 就 执行 `FutureContext.getContext().setCompatibleFuture(appResponseFuture)` 注释`// save for 2.6.x compatibility, for example, TraceFilter in Zipkin uses com.alibaba.xxx.FutureAdapter ` 写的是 为了适配 2.6.x，现在 到了3.x 了是不可以考虑移除这段代码了？
在dubbo3.0.12中，同步调用 dubbo协议在接收到响应后会由业务线程继续处理，是有清除掉线程中的InternalThreadLocal逻辑的
`FutureContext.getContext().setCompatibleFuture`逻辑在3.x的triple调用中仍存在，如果直接去掉该逻辑，可能会引发NPE #11443
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] AlbumenJ commented on issue #12493: 未及时释放FutureContext造成频繁GC问题
Posted by "AlbumenJ (via GitHub)" <gi...@apache.org>.
AlbumenJ commented on issue #12493:
URL: https://github.com/apache/dubbo/issues/12493#issuecomment-1586469893
但是这个在 Runnable 的 finally 里面应该删除了呀
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] ZPerling commented on issue #12493: 未及时释放FutureContext造成频繁GC问题
Posted by "ZPerling (via GitHub)" <gi...@apache.org>.
ZPerling commented on issue #12493:
URL: https://github.com/apache/dubbo/issues/12493#issuecomment-1590348117
@AlbumenJ 我们也遇到了类似的问题，严重影响性能
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] AlbumenJ commented on issue #12493: 未及时释放FutureContext造成频繁GC问题
Posted by "AlbumenJ (via GitHub)" <gi...@apache.org>.
AlbumenJ commented on issue #12493:
URL: https://github.com/apache/dubbo/issues/12493#issuecomment-1586444860
> 3.1.8版本:
> Dubbo协议、Triple协议都没有及时清除FutureContext
这个 getAndRemote 之后只是为了还原信息，需要看下第一个把环境污染的行为是什么，不然每一次 get 都应该是空的
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] xlq20080808 commented on issue #12493: 未及时释放FutureContext造成频繁GC问题
Posted by "xlq20080808 (via GitHub)" <gi...@apache.org>.
xlq20080808 commented on issue #12493:
URL: https://github.com/apache/dubbo/issues/12493#issuecomment-1586508125
>
大佬能否辛苦告知下是在哪清除的呀，我跟debug，一直到dubbo调用完成，业务线程返回结果，FutureContext.getContext()还是能够获取到包含返回结果的future
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org