You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "Abacn (via GitHub)" <gi...@apache.org> on 2023/04/07 16:59:50 UTC
[GitHub] [beam] Abacn opened a new issue, #26175: [Failing Test]: JmsIOIT.testPublishingThenReadingAll highly flaky
Abacn opened a new issue, #26175:
URL: https://github.com/apache/beam/issues/26175
### What happened?
Affecting https://ci-beam.apache.org/view/PostCommit/job/beam_PreCommit_Java_Jms_IO_Direct_Cron/
```
java.lang.AssertionError: actual number of records 14220 smaller than expected: 100000.
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.apache.beam.sdk.io.jms.JmsIOIT.testPublishingThenReadingAll(JmsIOIT.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:323)
```
We should probably decrease the number of element when test is running locally
### Issue Failure
Failure: Test is flaky
### Issue Priority
Priority: 2 (backlog / disabled test but we think the product is healthy)
### Issue Components
- [ ] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Amraneze commented on issue #26175: [Failing Test]: JmsIOIT.testPublishingThenReadingAll highly flaky
Posted by "Amraneze (via GitHub)" <gi...@apache.org>.
Amraneze commented on issue #26175:
URL: https://github.com/apache/beam/issues/26175#issuecomment-1500533557
I will run it with gradle until the test fails to see if I can reproduce it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Amraneze commented on issue #26175: [Failing Test]: JmsIOIT.testPublishingThenReadingAll highly flaky
Posted by "Amraneze (via GitHub)" <gi...@apache.org>.
Amraneze commented on issue #26175:
URL: https://github.com/apache/beam/issues/26175#issuecomment-1500498104
And I can see in the logs of the [failing test](https://ci-beam.apache.org/view/PostCommit/job/beam_PreCommit_Java_Jms_IO_Direct_Cron/lastCompletedBuild/testReport/org.apache.beam.sdk.io.jms/JmsIOIT/testPublishingThenReadingAll_with_client_class_class_org_apache_qpid_jms_JmsConnectionFactory_/) that there is an issue with connection.
We have some ghost connections and the JmsIO is creating new connections but for reading not publishing.
``JmsIO$UnboundedJmsReader.closeConnection(JmsIO.java:649)`` Should we look into it too ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Amraneze commented on issue #26175: [Failing Test]: JmsIOIT.testPublishingThenReadingAll highly flaky
Posted by "Amraneze (via GitHub)" <gi...@apache.org>.
Amraneze commented on issue #26175:
URL: https://github.com/apache/beam/issues/26175#issuecomment-1500495792
> Related to #25945. CC: @Amraneze
I made a workaround to cancel the pipeline because it was running for more than 30 mins even though all messages were published & received. What do you think it would be the best approach to do ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Amraneze commented on issue #26175: [Failing Test]: JmsIOIT.testPublishingThenReadingAll highly flaky
Posted by "Amraneze (via GitHub)" <gi...@apache.org>.
Amraneze commented on issue #26175:
URL: https://github.com/apache/beam/issues/26175#issuecomment-1500513606
> @Amraneze from the log you linked there are lots of connection gets created. This is because the number of DoFn instance can be many in streaming. Defer to connect when first element is received may mitigate the ghost connection issue. Connection pool is a long term solution.
Yeah the instance of the DoFn is created over and over because in the code we throw the exception and DoFn catches it to run `TearDown` function. But, I'm not sure if the connection is closed for sure. I'm trying to find time to work on the connection pool in the next few weeks. We also use finalize method which is deprecated and we already call the function `doClose` in the overridden close function of `UnboundedReader`. I guess it's better to remove it. What do you think ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #26175: [Failing Test]: JmsIOIT.testPublishingThenReadingAll highly flaky
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #26175:
URL: https://github.com/apache/beam/issues/26175#issuecomment-1500505101
@Amraneze from the log you linked there are lots of connection gets created. This is because the number of DoFn instance can be many in streaming. Defer to connect when first element is received may mitigate the ghost connection issue. Connection pool is a long term solution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #26175: [Failing Test]: JmsIOIT.testPublishingThenReadingAll highly flaky
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #26175:
URL: https://github.com/apache/beam/issues/26175#issuecomment-1500464742
Related to #25945. CC: @Amraneze
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #26175: [Failing Test]: JmsIOIT.testPublishingThenReadingAll highly flaky
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #26175:
URL: https://github.com/apache/beam/issues/26175#issuecomment-1500516548
I have opened #26179 to see if it works. I tested that the integration test passed locally but on Jenkins it has higher possibility of failure. This may be due to the ci nodes have higher nodes and higher possibility of connection issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org