You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@dubbo.apache.org by "Agorguy (via GitHub)" <gi...@apache.org> on 2023/05/15 18:59:11 UTC
[GitHub] [dubbo] Agorguy opened a new issue, #12320: Some Flaky tests
Agorguy opened a new issue, #12320:
URL: https://github.com/apache/dubbo/issues/12320
<!-- If you need to report a security issue please visit https://github.com/apache/dubbo/security/policy -->
- [ X ] I have searched the [issues](https://github.com/apache/dubbo/issues) of this repository and believe that this is not a duplicate.
### Environment
* Dubbo version: xxx
* Operating System version: ubuntu 22.04
* Java version: 11
### Comments
Hello, We tried running your project and discovered that it contains some flaky tests (i.e., tests that nondeterministically pass and fail). We found these tests to fail more frequently when running them on certain machines of ours.
To prevent others from running this project and its tests in machines that may result in flaky tests, we suggest adding information to the README.md file indicating the minimum resource configuration for running the tests of this project as to prevent observation of test flakiness.
If we run this project in a machine with 1cpu and 1gb ram, we observe flaky tests. We found that the tests in this project did not have any flaky tests when we ran it on machines with 2cpu and 2gb ram.
Here is a list of the tests we have identified and their likelihood of failure on a system with less than the recommended 2 CPUs and 2 GB RAM.
1. org.apache.dubbo.rpc.cluster.support.MergeableClusterInvokerTest#testGetMenuSuccessfully (4 out 10)
2. org.apache.dubbo.rpc.protocol.dubbo.RpcFilterTest#testRpcFilter (1 out 10)
3. org.apache.dubbo.rpc.filter.ExecuteLimitFilterTest#testMoreThanExecuteLimitInvoke ( 2 out 10)
4. org.apache.dubbo.rpc.filter.ActiveLimitFilterTest#testInvokeGreaterActives ( 1 out 10)
5. org.apache.dubbo.rpc.protocol.rest.SpringMvcRestProtocolTest#testRestProtocol ( 7 out 10)
6. org.apache.dubbo.rpc.protocol.rest.SpringMvcRestProtocolTest#testRestProtocolWithContextPath ( 7 out 10)
7. org.apache.dubbo.rpc.protocol.dubbo.MultiThreadTest#testDubboMultiThreadInvoke ( 4 out 10)
Please let me know if you would like us to create a pull request on this matter (possibly to the readme of this project).
Thank you for your attention to this matter. We hope that our recommendations will be helpful in improving the quality and performance of your project, especially for others to use.
## Reproducing
~~~Dockerfile
FROM maven:3.5.4-jdk-11
WORKDIR /home/
RUN git clone https://github.com/apache/dubbo && \
cd dubbo && \
git checkout acd4212f597813411b6634377f96ac2bb3888a07
WORKDIR /home/dubbo
RUN mvn install -DskipTests
ENTRYPOINT ["mvn", "test", "-fn"]
~~~
Build the image:
~~~bash
$> mkdir tmp
$> cp Dockerfile tmp
$> cd tmp
$> docker build -t dubbo . # estimated time of build 3m
~~~
Running:
this configuration likely prevents flakiness (no flakiness in 10 runs)
~~~bash
$> docker run --rm --memory=2g --cpus=2 --memory-swap=-1 dubbo | tee output.txt
$> grep "Failures:" output.txt # checking results
~~~
this other configuration –similar to the previous– can’t prevent flaky tests (observation in 10 runs)
~~~bash
$> docker run --rm --memory=1g --cpus=1 --memory-swap=-1 dubbo | tee output2.txt
$> grep "Failures:" output2.txt # checking results
~~~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] Agorguy commented on issue #12320: Some Flaky tests
Posted by "Agorguy (via GitHub)" <gi...@apache.org>.
Agorguy commented on issue #12320:
URL: https://github.com/apache/dubbo/issues/12320#issuecomment-1560017111
@AlbumenJ I created the pr, can you take a look? Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] AlbumenJ commented on issue #12320: Some Flaky tests
Posted by "AlbumenJ (via GitHub)" <gi...@apache.org>.
AlbumenJ commented on issue #12320:
URL: https://github.com/apache/dubbo/issues/12320#issuecomment-1552676029
Nice job. Cound you please help update it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
[GitHub] [dubbo] AlbumenJ closed issue #12320: Some Flaky tests
Posted by "AlbumenJ (via GitHub)" <gi...@apache.org>.
AlbumenJ closed issue #12320: Some Flaky tests
URL: https://github.com/apache/dubbo/issues/12320
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org