You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Denes Arvay <de...@cloudera.com> on 2016/10/04 11:51:50 UTC
Review Request 52511: FLUME-3003: testSourceCounter in
TestSyslogUdpSource is flaky
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52511/
-----------------------------------------------------------
Review request for Flume, Bal�zs Don�t Bessenyei, Mike Percy, and Attila Simon.
Bugs: FLUME-3003
https://issues.apache.org/jira/browse/FLUME-3003
Repository: flume-git
Description
-------
The event processing in `SyslogUDPSource` is handled on a separate thread by Netty so message delivery, thus the `sourceCounter`'s increment can be delayed resulting in a flaky test
Diffs
-----
flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUdpSource.java 6ba7512
Diff: https://reviews.apache.org/r/52511/diff/
Testing
-------
tests in `flume-ng-core` pass, checkstyle verification done.
Thanks,
Denes Arvay
Re: Review Request 52511: FLUME-3003: testSourceCounter in
TestSyslogUdpSource is flaky
Posted by Lior Zeno <li...@gmail.com>.
> On Oct. 4, 2016, noon, Lior Zeno wrote:
> > Shouldn't you simply call source.close()?
>
> Denes Arvay wrote:
> Thanks for the suggestion, it'd be a cleaner solution for sure but I tried it and unfortunately doesn't work. It might flush the netty channel but it returns before the `SyslogUDPSource.syslogHandler#messageReceived()` method handles the message on Netty's IO thread, so the `eventAcceptedCount` might still be 0 when the `assertEquals` check happens.
So do we need to add this patch to all `SyslogUDPSource` tests?
- Lior
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52511/#review151322
-----------------------------------------------------------
On Oct. 4, 2016, 11:51 a.m., Denes Arvay wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52511/
> -----------------------------------------------------------
>
> (Updated Oct. 4, 2016, 11:51 a.m.)
>
>
> Review request for Flume, Bal�zs Don�t Bessenyei, Mike Percy, and Attila Simon.
>
>
> Bugs: FLUME-3003
> https://issues.apache.org/jira/browse/FLUME-3003
>
>
> Repository: flume-git
>
>
> Description
> -------
>
> The event processing in `SyslogUDPSource` is handled on a separate thread by Netty so message delivery, thus the `sourceCounter`'s increment can be delayed resulting in a flaky test
>
>
> Diffs
> -----
>
> flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUdpSource.java 6ba7512
>
> Diff: https://reviews.apache.org/r/52511/diff/
>
>
> Testing
> -------
>
> tests in `flume-ng-core` pass, checkstyle verification done.
>
>
> Thanks,
>
> Denes Arvay
>
>
Re: Review Request 52511: FLUME-3003: testSourceCounter in
TestSyslogUdpSource is flaky
Posted by Denes Arvay <de...@cloudera.com>.
> On Oct. 4, 2016, noon, Lior Zeno wrote:
> > Shouldn't you simply call source.close()?
Thanks for the suggestion, it'd be a cleaner solution for sure but I tried it and unfortunately doesn't work. It might flush the netty channel but it returns before the `SyslogUDPSource.syslogHandler#messageReceived()` method handles the message on Netty's IO thread, so the `eventAcceptedCount` might still be 0 when the `assertEquals` check happens.
- Denes
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52511/#review151322
-----------------------------------------------------------
On Oct. 4, 2016, 11:51 a.m., Denes Arvay wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52511/
> -----------------------------------------------------------
>
> (Updated Oct. 4, 2016, 11:51 a.m.)
>
>
> Review request for Flume, Bal�zs Don�t Bessenyei, Mike Percy, and Attila Simon.
>
>
> Bugs: FLUME-3003
> https://issues.apache.org/jira/browse/FLUME-3003
>
>
> Repository: flume-git
>
>
> Description
> -------
>
> The event processing in `SyslogUDPSource` is handled on a separate thread by Netty so message delivery, thus the `sourceCounter`'s increment can be delayed resulting in a flaky test
>
>
> Diffs
> -----
>
> flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUdpSource.java 6ba7512
>
> Diff: https://reviews.apache.org/r/52511/diff/
>
>
> Testing
> -------
>
> tests in `flume-ng-core` pass, checkstyle verification done.
>
>
> Thanks,
>
> Denes Arvay
>
>
Re: Review Request 52511: FLUME-3003: testSourceCounter in
TestSyslogUdpSource is flaky
Posted by Denes Arvay <de...@cloudera.com>.
> On Oct. 4, 2016, noon, Lior Zeno wrote:
> > Shouldn't you simply call source.close()?
>
> Denes Arvay wrote:
> Thanks for the suggestion, it'd be a cleaner solution for sure but I tried it and unfortunately doesn't work. It might flush the netty channel but it returns before the `SyslogUDPSource.syslogHandler#messageReceived()` method handles the message on Netty's IO thread, so the `eventAcceptedCount` might still be 0 when the `assertEquals` check happens.
>
> Lior Zeno wrote:
> So do we need to add this patch to all `SyslogUDPSource` tests?
I haven't checked the others but they don't seem to be flaky while the `testSourceCounter` fails ~7-8 times of 10 runs on a jenkins running on a m3.medium ec2 instance. (I wasn't able to reproduce it locally on a core i7 macbook pro)
- Denes
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52511/#review151322
-----------------------------------------------------------
On Oct. 4, 2016, 11:51 a.m., Denes Arvay wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52511/
> -----------------------------------------------------------
>
> (Updated Oct. 4, 2016, 11:51 a.m.)
>
>
> Review request for Flume, Bal�zs Don�t Bessenyei, Mike Percy, and Attila Simon.
>
>
> Bugs: FLUME-3003
> https://issues.apache.org/jira/browse/FLUME-3003
>
>
> Repository: flume-git
>
>
> Description
> -------
>
> The event processing in `SyslogUDPSource` is handled on a separate thread by Netty so message delivery, thus the `sourceCounter`'s increment can be delayed resulting in a flaky test
>
>
> Diffs
> -----
>
> flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUdpSource.java 6ba7512
>
> Diff: https://reviews.apache.org/r/52511/diff/
>
>
> Testing
> -------
>
> tests in `flume-ng-core` pass, checkstyle verification done.
>
>
> Thanks,
>
> Denes Arvay
>
>
Re: Review Request 52511: FLUME-3003: testSourceCounter in
TestSyslogUdpSource is flaky
Posted by Lior Zeno <li...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52511/#review151322
-----------------------------------------------------------
Shouldn't you simply call source.close()?
- Lior Zeno
On Oct. 4, 2016, 11:51 a.m., Denes Arvay wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52511/
> -----------------------------------------------------------
>
> (Updated Oct. 4, 2016, 11:51 a.m.)
>
>
> Review request for Flume, Bal�zs Don�t Bessenyei, Mike Percy, and Attila Simon.
>
>
> Bugs: FLUME-3003
> https://issues.apache.org/jira/browse/FLUME-3003
>
>
> Repository: flume-git
>
>
> Description
> -------
>
> The event processing in `SyslogUDPSource` is handled on a separate thread by Netty so message delivery, thus the `sourceCounter`'s increment can be delayed resulting in a flaky test
>
>
> Diffs
> -----
>
> flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUdpSource.java 6ba7512
>
> Diff: https://reviews.apache.org/r/52511/diff/
>
>
> Testing
> -------
>
> tests in `flume-ng-core` pass, checkstyle verification done.
>
>
> Thanks,
>
> Denes Arvay
>
>
Re: Review Request 52511: FLUME-3003: testSourceCounter in
TestSyslogUdpSource is flaky
Posted by Balázs Donát Bessenyei <be...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52511/#review151960
-----------------------------------------------------------
Ship it!
Ship It!
- Bal�zs Don�t Bessenyei
On Oct. 4, 2016, 11:51 a.m., Denes Arvay wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52511/
> -----------------------------------------------------------
>
> (Updated Oct. 4, 2016, 11:51 a.m.)
>
>
> Review request for Flume, Bal�zs Don�t Bessenyei, Mike Percy, and Attila Simon.
>
>
> Bugs: FLUME-3003
> https://issues.apache.org/jira/browse/FLUME-3003
>
>
> Repository: flume-git
>
>
> Description
> -------
>
> The event processing in `SyslogUDPSource` is handled on a separate thread by Netty so message delivery, thus the `sourceCounter`'s increment can be delayed resulting in a flaky test
>
>
> Diffs
> -----
>
> flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUdpSource.java 6ba7512
>
> Diff: https://reviews.apache.org/r/52511/diff/
>
>
> Testing
> -------
>
> tests in `flume-ng-core` pass, checkstyle verification done.
>
>
> Thanks,
>
> Denes Arvay
>
>