You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@dubbo.apache.org by GitBox <gi...@apache.org> on 2021/09/08 05:14:09 UTC
[GitHub] [dubbo] zrlw opened a new issue #8725: [3.0] 部分单元测试的测试方法非线程安全，不能并行执行
zrlw opened a new issue #8725:
URL: https://github.com/apache/dubbo/issues/8725
### Environment
* Dubbo version: 3.0
很多测试方法结束后执行DubboBootstrap.reset操作，但是：
1. all registries will be destroyed by DubboBootstrap.reset(),
2. if running these test case concurrently, DubboBootstrap.start() and DubboBootstrap.reset() might be called at the same time, a RuntimeException might be thrown out of MetadataServiceNameMapping.map() because of registerServiceAppMapping method failure.
比如DubboBootstrapMultiInstanceTest：
```
2021-09-06T23:17:04.3783873Z [06/09/21 23:17:04:369 UTC] main INFO support.AbstractRegistryFactory: [DUBBO] Close all registries [org.apache.dubbo.registry.client.ServiceDiscoveryRegistry@6faf0232, zookeeper://127.0.0.1:34385/org.apache.dubbo.registry.RegistryService?application=consumer-app&dubbo=2.0.2&environment=product&file.cache=false&interface=org.apache.dubbo.registry.RegistryService&pid=9244&release=3.0.3-SNAPSHOT], dubbo version: 3.0.3-SNAPSHOT, current host: 10.1.0.234
2021-09-06T23:17:04.3788871Z [06/09/21 23:17:04:369 UTC] main INFO zookeeper.ZookeeperRegistry: [DUBBO] Destroy registry:zookeeper://127.0.0.1:34385/org.apache.dubbo.registry.RegistryService?application=consumer-app&dubbo=2.0.2&environment=product&file.cache=false&interface=org.apache.dubbo.registry.RegistryService&pid=9244&release=3.0.3-SNAPSHOT, dubbo version: 3.0.3-SNAPSHOT, current host: 10.1.0.234
2021-09-06T23:17:04.3798680Z [06/09/21 23:17:04:372 UTC] DubboSaveMetadataReport-thread-1 ERROR zookeeper.ZookeeperMetadataReport: [DUBBO] Failed to put consumer metadata org.apache.dubbo.metadata.report.identifier.MetadataIdentifier@27cfcb4f; org.apache.dubbo.common.url.component.URLParam$URLParamMap@c7a36a43, cause: Expected state [STARTED] was [STOPPED], dubbo version: 3.0.3-SNAPSHOT, current host: 10.1.0.234
2021-09-06T23:17:04.3801771Z java.lang.IllegalStateException: Expected state [STARTED] was [STOPPED]
2021-09-06T23:17:04.3803955Z at org.apache.dubbo.remoting.zookeeper.curator.CuratorZookeeperClient.deletePath(CuratorZookeeperClient.java:193)
2021-09-06T23:17:04.3806956Z at org.apache.dubbo.remoting.zookeeper.AbstractZookeeperClient.delete(AbstractZookeeperClient.java:66)
2021-09-06T23:17:04.3809616Z at org.apache.dubbo.remoting.zookeeper.AbstractZookeeperClient.create(AbstractZookeeperClient.java:170)
2021-09-06T23:17:04.3814840Z at org.apache.dubbo.metadata.store.zookeeper.ZookeeperMetadataReport.storeMetadata(ZookeeperMetadataReport.java:134)
2021-09-06T23:17:04.3818645Z at org.apache.dubbo.metadata.store.zookeeper.ZookeeperMetadataReport.doStoreConsumerMetadata(ZookeeperMetadataReport.java:96)
2021-09-06T23:17:04.3822638Z at org.apache.dubbo.metadata.report.support.AbstractMetadataReport.storeConsumerMetadataTask(AbstractMetadataReport.java:289)
2021-09-06T23:17:04.3826257Z at org.apache.dubbo.metadata.report.support.AbstractMetadataReport.lambda$storeConsumerMetadata$1(AbstractMetadataReport.java:275)
2021-09-06T23:17:04.3831984Z at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
2021-09-06T23:17:04.3833764Z at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
2021-09-06T23:17:04.3835097Z at java.base/java.lang.Thread.run(Thread.java:829)
2021-09-06T23:17:04.3836142Z Caused by: java.lang.IllegalStateException: Expected state [STARTED] was [STOPPED]
2021-09-06T23:17:04.3838157Z at org.apache.curator.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:721)
2021-09-06T23:17:04.3840779Z at org.apache.curator.framework.imps.CuratorFrameworkImpl.checkState(CuratorFrameworkImpl.java:426)
2021-09-06T23:17:04.3843184Z at org.apache.curator.framework.imps.CuratorFrameworkImpl.delete(CuratorFrameworkImpl.java:446)
2021-09-06T23:17:04.3846081Z at org.apache.dubbo.remoting.zookeeper.curator.CuratorZookeeperClient.deletePath(CuratorZookeeperClient.java:190)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] zrlw edited a comment on issue #8725: [3.0] 部分单元测试的测试方法非线程安全，不能并行执行
Posted by GitBox <gi...@apache.org>.
zrlw edited a comment on issue #8725:
URL: https://github.com/apache/dubbo/issues/8725#issuecomment-914926497
看了一下代码，如果采用测试类加@net.jcip.annotations.NotThreadSafe注解单线程执行，几乎大部分测试类都需要加，一是改动量有点大，二是那样github构建时间可能要奔向2小时了，不好搞。
DubboBootstrap.reset操作修改了ConfigUtils、DubboBootstrap、MetadataReportInstance、AbstractRegistryFactory、FrameworkModel、ApplicationModel、ShutdownHookCallbacks类成员，这个要搞并发支持，要改的内容应该非常丰厚，
可能不是一个小小的PR能搞得定了。
@AlbumenJ 该你出手了
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] zrlw removed a comment on issue #8725: [3.0] 部分单元测试的测试方法非线程安全，不能并行执行
Posted by GitBox <gi...@apache.org>.
zrlw removed a comment on issue #8725:
URL: https://github.com/apache/dubbo/issues/8725#issuecomment-914926497
看了一下代码，如果采用测试类加@net.jcip.annotations.NotThreadSafe注解单线程执行，几乎大部分测试类都需要加，一是改动量有点大，二是那样github构建时间可能要奔向2小时了，不好搞。
DubboBootstrap.reset操作修改了ConfigUtils、DubboBootstrap、MetadataReportInstance、AbstractRegistryFactory、FrameworkModel、ApplicationModel、ShutdownHookCallbacks类成员，这个要搞并发支持，要改的内容应该非常丰厚，
可能不是一个小小的PR能搞得定了。
@AlbumenJ 该你出手了
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] zrlw commented on issue #8725: [3.0] 部分单元测试的测试方法非线程安全，不能并行执行
Posted by GitBox <gi...@apache.org>.
zrlw commented on issue #8725:
URL: https://github.com/apache/dubbo/issues/8725#issuecomment-915679045
我对forkMode的理解有错误，
forkMode=once 对应的是 forkCount=1 && reuseForks=true 创建一个新进程用于每个测试（复用）
forkMode=always 对应的是 forkCount=1 && reuseForks=false 为每一个测试创建一个新进程
从日志内容看，once forkMode方式下AfterEach的reset关闭zk连接的动作竟然在和下一个测试方法并行执行
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] zrlw commented on issue #8725: [3.0] 部分单元测试的测试方法非线程安全，不能并行执行
Posted by GitBox <gi...@apache.org>.
zrlw commented on issue #8725:
URL: https://github.com/apache/dubbo/issues/8725#issuecomment-914926497
看了一下代码，如果采用测试类加@net.jcip.annotations.NotThreadSafe注解单线程执行，几乎大部分测试类都需要加，不可行。
把AbstractRegistryFactory加个引用计数吧，就像ReferenceCountExchangeClient那样，计数降到0之后再做destroy，如何？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] zrlw edited a comment on issue #8725: [3.0] 部分单元测试的测试方法非线程安全，不能并行执行
Posted by GitBox <gi...@apache.org>.
zrlw edited a comment on issue #8725:
URL: https://github.com/apache/dubbo/issues/8725#issuecomment-915679045
我对forkMode的理解有错误，
forkMode=once 对应的是 forkCount=1 && reuseForks=true 创建一个新进程用于每个测试（复用）
forkMode=always 对应的是 forkCount=1 && reuseForks=false 为每一个测试创建一个新进程
从日志内容看，once forkMode方式下AfterEach的reset关闭zk连接的动作时，DubboSaveMetadataReport-thread-1 还在工作，然后下一个测试方法在main线程执行start操作进行zk注册时出异常了。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] zrlw commented on issue #8725: [3.0] 部分单元测试的测试方法非线程安全，不能并行执行
Posted by GitBox <gi...@apache.org>.
zrlw commented on issue #8725:
URL: https://github.com/apache/dubbo/issues/8725#issuecomment-914930397
提个建议哈，要么干脆把parent的pom里面forkMode从once改成always算了，然后把UT的timeout调得大大大大。。。。的
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] zrlw closed issue #8725: [3.0] 单元测试AfterEach执行DubboBootstrap.reset时与dubbo后台线程存在冲突，可能是后续单元测试运行失败的诱因
Posted by GitBox <gi...@apache.org>.
zrlw closed issue #8725:
URL: https://github.com/apache/dubbo/issues/8725
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] zrlw edited a comment on issue #8725: [3.0] 部分单元测试的测试方法非线程安全，不能并行执行
Posted by GitBox <gi...@apache.org>.
zrlw edited a comment on issue #8725:
URL: https://github.com/apache/dubbo/issues/8725#issuecomment-914930397
提个建议哈，要么干脆把parent的pom里面forkMode从once改成always算了
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] zrlw removed a comment on issue #8725: [3.0] 单元测试AfterEach执行DubboBootstrap.reset时与dubbo后台线程存在冲突
Posted by GitBox <gi...@apache.org>.
zrlw removed a comment on issue #8725:
URL: https://github.com/apache/dubbo/issues/8725#issuecomment-914930397
提个建议哈，要么干脆把parent的pom里面forkMode从once改成always算了
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org