You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Nicolò Boschi <bo...@gmail.com> on 2023/04/04 10:01:31 UTC
Pulsar Flaky test report 2023-04-03 for PR builds in CI
Dear community,
Here's a report of the flaky tests in Pulsar CI during the observation
period of 2023-03-27 - 2023-04-03
https://github.com/nicoloboschi/pulsar-flakes/tree/master/2023-03-27-to-2023-04-03
Most flaky tests:
org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.stateCheck
9
org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.testSplitBundleAdminAPI
4
org.apache.bookkeeper.mledger.impl.ManagedLedgerErrorsTest.recoverLongTimeAfterMultipleWriteErrors
4
org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.initializeState
3
org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.testAssign
2
More flaky test issues:
https://github.com/apache/pulsar/issues?q=flaky+sort%3Aupdated-desc+is%3Aopen+is:issue
To coordinate the work,
1) please search for an existing issues or search for all flaky issues with
"flaky" or the test class name (without package) in the search:
https://github.com/apache/pulsar/issues?q=is%3Aopen+flaky+sort%3Aupdated-desc
2) If there isn't an issue for a particular flaky test failure that you'd
like to fix, please create an issue using the "Flaky test" template at
https://github.com/apache/pulsar/issues/new/choose
3) Please comment on the issue that you are working on.
More contributions are welcome!
Nicolò Boschi
Re: Pulsar Flaky test report 2023-04-03 for PR builds in CI
Posted by Devin Bost <de...@gmail.com>.
Great team work on all the flaky tests. It's much appreciated.
- Devin
On Wed, Apr 5, 2023, 4:39 PM Michael Marshall <mm...@apache.org> wrote:
> Thanks for fixing that test Heesung, and thanks for the report Nicolo!
>
> - Michael
>
> On Wed, Apr 5, 2023 at 2:17 PM Lari Hotari <lh...@apache.org> wrote:
> >
> > Thank you Heesung for resolving the problem so quickly.
> >
> > -Lari
> >
> > ke 5. huhtik. 2023 klo 20.23 Heesung Sohn
> > <he...@streamnative.io.invalid> kirjoitti:
> >
> > > Hi,
> > >
> > > The fix PR( #19945 <https://github.com/apache/pulsar/pull/19945>) has
> been
> > > merged(I also notified here,
> > > https://github.com/apache/pulsar/issues/20007#issuecomment-1497679571
> ).
> > > If this test continues to be flaky, I agree with quarantining it not to
> > > harm builds for the time being.
> > >
> > > Regards,
> > > Heesung
> > >
> > >
> > >
> > > On Wed, Apr 5, 2023 at 6:24 AM Lari Hotari <lh...@apache.org> wrote:
> > >
> > > > I think we might want to move ExtensibleLoadManagerImplTest to the
> flaky
> > > > group until the fixes have been completed. Most builds fail now
> because
> > > of
> > > > the flakiness of that test. That is blocking our progress and moving
> to
> > > > flaky group would resolve that issue.
> > > > Once that it is proven to be stable, we can make a simple update to
> > > change
> > > > the test group.
> > > > I'll proceed with a PR for this soon.
> > > >
> > > > -Lari
> > > >
> > > > On 2023/04/04 16:39:21 Heesung Sohn wrote:
> > > > > Hi,
> > > > >
> > > > > Thank you for the report.
> > > > >
> > > > > We are fixing the errors from ExtensibleLoadManagerImplTest.
> > > > >
> > > > > Regards,
> > > > > Heesung
> > > > >
> > > > > On Tue, Apr 4, 2023 at 3:01 AM Nicolò Boschi <boschi1997@gmail.com
> >
> > > > wrote:
> > > > >
> > > > > > Dear community,
> > > > > > Here's a report of the flaky tests in Pulsar CI during the
> > > observation
> > > > > > period of 2023-03-27 - 2023-04-03
> > > > > >
> > > > > >
> > > > > >
> > > >
> > >
> https://github.com/nicoloboschi/pulsar-flakes/tree/master/2023-03-27-to-2023-04-03
> > > > > >
> > > > > >
> > > > > > Most flaky tests:
> > > > > >
> > > > > >
> > > >
> > >
> org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.stateCheck
> > > > > > 9
> > > > > >
> > > > > >
> > > >
> > >
> org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.testSplitBundleAdminAPI
> > > > > > 4
> > > > > >
> > > > > >
> > > >
> > >
> org.apache.bookkeeper.mledger.impl.ManagedLedgerErrorsTest.recoverLongTimeAfterMultipleWriteErrors
> > > > > > 4
> > > > > >
> > > > > >
> > > >
> > >
> org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.initializeState
> > > > > > 3
> > > > > >
> > > > > >
> > > >
> > >
> org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.testAssign
> > > > > > 2
> > > > > > More flaky test issues:
> > > > > >
> > > > > >
> > > >
> > >
> https://github.com/apache/pulsar/issues?q=flaky+sort%3Aupdated-desc+is%3Aopen+is:issue
> > > > > >
> > > > > >
> > > > > > To coordinate the work,
> > > > > > 1) please search for an existing issues or search for all flaky
> > > issues
> > > > with
> > > > > > "flaky" or the test class name (without package) in the search:
> > > > > >
> > > > > >
> > > >
> > >
> https://github.com/apache/pulsar/issues?q=is%3Aopen+flaky+sort%3Aupdated-desc
> > > > > > 2) If there isn't an issue for a particular flaky test failure
> that
> > > > you'd
> > > > > > like to fix, please create an issue using the "Flaky test"
> template
> > > at
> > > > > > https://github.com/apache/pulsar/issues/new/choose
> > > > > > 3) Please comment on the issue that you are working on.
> > > > > >
> > > > > >
> > > > > > More contributions are welcome!
> > > > > > Nicolò Boschi
> > > > > >
> > > > >
> > > >
> > >
>
Re: Pulsar Flaky test report 2023-04-03 for PR builds in CI
Posted by Michael Marshall <mm...@apache.org>.
Thanks for fixing that test Heesung, and thanks for the report Nicolo!
- Michael
On Wed, Apr 5, 2023 at 2:17 PM Lari Hotari <lh...@apache.org> wrote:
>
> Thank you Heesung for resolving the problem so quickly.
>
> -Lari
>
> ke 5. huhtik. 2023 klo 20.23 Heesung Sohn
> <he...@streamnative.io.invalid> kirjoitti:
>
> > Hi,
> >
> > The fix PR( #19945 <https://github.com/apache/pulsar/pull/19945>) has been
> > merged(I also notified here,
> > https://github.com/apache/pulsar/issues/20007#issuecomment-1497679571).
> > If this test continues to be flaky, I agree with quarantining it not to
> > harm builds for the time being.
> >
> > Regards,
> > Heesung
> >
> >
> >
> > On Wed, Apr 5, 2023 at 6:24 AM Lari Hotari <lh...@apache.org> wrote:
> >
> > > I think we might want to move ExtensibleLoadManagerImplTest to the flaky
> > > group until the fixes have been completed. Most builds fail now because
> > of
> > > the flakiness of that test. That is blocking our progress and moving to
> > > flaky group would resolve that issue.
> > > Once that it is proven to be stable, we can make a simple update to
> > change
> > > the test group.
> > > I'll proceed with a PR for this soon.
> > >
> > > -Lari
> > >
> > > On 2023/04/04 16:39:21 Heesung Sohn wrote:
> > > > Hi,
> > > >
> > > > Thank you for the report.
> > > >
> > > > We are fixing the errors from ExtensibleLoadManagerImplTest.
> > > >
> > > > Regards,
> > > > Heesung
> > > >
> > > > On Tue, Apr 4, 2023 at 3:01 AM Nicolò Boschi <bo...@gmail.com>
> > > wrote:
> > > >
> > > > > Dear community,
> > > > > Here's a report of the flaky tests in Pulsar CI during the
> > observation
> > > > > period of 2023-03-27 - 2023-04-03
> > > > >
> > > > >
> > > > >
> > >
> > https://github.com/nicoloboschi/pulsar-flakes/tree/master/2023-03-27-to-2023-04-03
> > > > >
> > > > >
> > > > > Most flaky tests:
> > > > >
> > > > >
> > >
> > org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.stateCheck
> > > > > 9
> > > > >
> > > > >
> > >
> > org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.testSplitBundleAdminAPI
> > > > > 4
> > > > >
> > > > >
> > >
> > org.apache.bookkeeper.mledger.impl.ManagedLedgerErrorsTest.recoverLongTimeAfterMultipleWriteErrors
> > > > > 4
> > > > >
> > > > >
> > >
> > org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.initializeState
> > > > > 3
> > > > >
> > > > >
> > >
> > org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.testAssign
> > > > > 2
> > > > > More flaky test issues:
> > > > >
> > > > >
> > >
> > https://github.com/apache/pulsar/issues?q=flaky+sort%3Aupdated-desc+is%3Aopen+is:issue
> > > > >
> > > > >
> > > > > To coordinate the work,
> > > > > 1) please search for an existing issues or search for all flaky
> > issues
> > > with
> > > > > "flaky" or the test class name (without package) in the search:
> > > > >
> > > > >
> > >
> > https://github.com/apache/pulsar/issues?q=is%3Aopen+flaky+sort%3Aupdated-desc
> > > > > 2) If there isn't an issue for a particular flaky test failure that
> > > you'd
> > > > > like to fix, please create an issue using the "Flaky test" template
> > at
> > > > > https://github.com/apache/pulsar/issues/new/choose
> > > > > 3) Please comment on the issue that you are working on.
> > > > >
> > > > >
> > > > > More contributions are welcome!
> > > > > Nicolò Boschi
> > > > >
> > > >
> > >
> >
Re: Pulsar Flaky test report 2023-04-03 for PR builds in CI
Posted by Lari Hotari <lh...@apache.org>.
Thank you Heesung for resolving the problem so quickly.
-Lari
ke 5. huhtik. 2023 klo 20.23 Heesung Sohn
<he...@streamnative.io.invalid> kirjoitti:
> Hi,
>
> The fix PR( #19945 <https://github.com/apache/pulsar/pull/19945>) has been
> merged(I also notified here,
> https://github.com/apache/pulsar/issues/20007#issuecomment-1497679571).
> If this test continues to be flaky, I agree with quarantining it not to
> harm builds for the time being.
>
> Regards,
> Heesung
>
>
>
> On Wed, Apr 5, 2023 at 6:24 AM Lari Hotari <lh...@apache.org> wrote:
>
> > I think we might want to move ExtensibleLoadManagerImplTest to the flaky
> > group until the fixes have been completed. Most builds fail now because
> of
> > the flakiness of that test. That is blocking our progress and moving to
> > flaky group would resolve that issue.
> > Once that it is proven to be stable, we can make a simple update to
> change
> > the test group.
> > I'll proceed with a PR for this soon.
> >
> > -Lari
> >
> > On 2023/04/04 16:39:21 Heesung Sohn wrote:
> > > Hi,
> > >
> > > Thank you for the report.
> > >
> > > We are fixing the errors from ExtensibleLoadManagerImplTest.
> > >
> > > Regards,
> > > Heesung
> > >
> > > On Tue, Apr 4, 2023 at 3:01 AM Nicolò Boschi <bo...@gmail.com>
> > wrote:
> > >
> > > > Dear community,
> > > > Here's a report of the flaky tests in Pulsar CI during the
> observation
> > > > period of 2023-03-27 - 2023-04-03
> > > >
> > > >
> > > >
> >
> https://github.com/nicoloboschi/pulsar-flakes/tree/master/2023-03-27-to-2023-04-03
> > > >
> > > >
> > > > Most flaky tests:
> > > >
> > > >
> >
> org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.stateCheck
> > > > 9
> > > >
> > > >
> >
> org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.testSplitBundleAdminAPI
> > > > 4
> > > >
> > > >
> >
> org.apache.bookkeeper.mledger.impl.ManagedLedgerErrorsTest.recoverLongTimeAfterMultipleWriteErrors
> > > > 4
> > > >
> > > >
> >
> org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.initializeState
> > > > 3
> > > >
> > > >
> >
> org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.testAssign
> > > > 2
> > > > More flaky test issues:
> > > >
> > > >
> >
> https://github.com/apache/pulsar/issues?q=flaky+sort%3Aupdated-desc+is%3Aopen+is:issue
> > > >
> > > >
> > > > To coordinate the work,
> > > > 1) please search for an existing issues or search for all flaky
> issues
> > with
> > > > "flaky" or the test class name (without package) in the search:
> > > >
> > > >
> >
> https://github.com/apache/pulsar/issues?q=is%3Aopen+flaky+sort%3Aupdated-desc
> > > > 2) If there isn't an issue for a particular flaky test failure that
> > you'd
> > > > like to fix, please create an issue using the "Flaky test" template
> at
> > > > https://github.com/apache/pulsar/issues/new/choose
> > > > 3) Please comment on the issue that you are working on.
> > > >
> > > >
> > > > More contributions are welcome!
> > > > Nicolò Boschi
> > > >
> > >
> >
>
Re: Pulsar Flaky test report 2023-04-03 for PR builds in CI
Posted by Heesung Sohn <he...@streamnative.io.INVALID>.
Hi,
The fix PR( #19945 <https://github.com/apache/pulsar/pull/19945>) has been
merged(I also notified here,
https://github.com/apache/pulsar/issues/20007#issuecomment-1497679571).
If this test continues to be flaky, I agree with quarantining it not to
harm builds for the time being.
Regards,
Heesung
On Wed, Apr 5, 2023 at 6:24 AM Lari Hotari <lh...@apache.org> wrote:
> I think we might want to move ExtensibleLoadManagerImplTest to the flaky
> group until the fixes have been completed. Most builds fail now because of
> the flakiness of that test. That is blocking our progress and moving to
> flaky group would resolve that issue.
> Once that it is proven to be stable, we can make a simple update to change
> the test group.
> I'll proceed with a PR for this soon.
>
> -Lari
>
> On 2023/04/04 16:39:21 Heesung Sohn wrote:
> > Hi,
> >
> > Thank you for the report.
> >
> > We are fixing the errors from ExtensibleLoadManagerImplTest.
> >
> > Regards,
> > Heesung
> >
> > On Tue, Apr 4, 2023 at 3:01 AM Nicolò Boschi <bo...@gmail.com>
> wrote:
> >
> > > Dear community,
> > > Here's a report of the flaky tests in Pulsar CI during the observation
> > > period of 2023-03-27 - 2023-04-03
> > >
> > >
> > >
> https://github.com/nicoloboschi/pulsar-flakes/tree/master/2023-03-27-to-2023-04-03
> > >
> > >
> > > Most flaky tests:
> > >
> > >
> org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.stateCheck
> > > 9
> > >
> > >
> org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.testSplitBundleAdminAPI
> > > 4
> > >
> > >
> org.apache.bookkeeper.mledger.impl.ManagedLedgerErrorsTest.recoverLongTimeAfterMultipleWriteErrors
> > > 4
> > >
> > >
> org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.initializeState
> > > 3
> > >
> > >
> org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.testAssign
> > > 2
> > > More flaky test issues:
> > >
> > >
> https://github.com/apache/pulsar/issues?q=flaky+sort%3Aupdated-desc+is%3Aopen+is:issue
> > >
> > >
> > > To coordinate the work,
> > > 1) please search for an existing issues or search for all flaky issues
> with
> > > "flaky" or the test class name (without package) in the search:
> > >
> > >
> https://github.com/apache/pulsar/issues?q=is%3Aopen+flaky+sort%3Aupdated-desc
> > > 2) If there isn't an issue for a particular flaky test failure that
> you'd
> > > like to fix, please create an issue using the "Flaky test" template at
> > > https://github.com/apache/pulsar/issues/new/choose
> > > 3) Please comment on the issue that you are working on.
> > >
> > >
> > > More contributions are welcome!
> > > Nicolò Boschi
> > >
> >
>
Re: Pulsar Flaky test report 2023-04-03 for PR builds in CI
Posted by Lari Hotari <lh...@apache.org>.
I think we might want to move ExtensibleLoadManagerImplTest to the flaky group until the fixes have been completed. Most builds fail now because of the flakiness of that test. That is blocking our progress and moving to flaky group would resolve that issue.
Once that it is proven to be stable, we can make a simple update to change the test group.
I'll proceed with a PR for this soon.
-Lari
On 2023/04/04 16:39:21 Heesung Sohn wrote:
> Hi,
>
> Thank you for the report.
>
> We are fixing the errors from ExtensibleLoadManagerImplTest.
>
> Regards,
> Heesung
>
> On Tue, Apr 4, 2023 at 3:01 AM Nicolò Boschi <bo...@gmail.com> wrote:
>
> > Dear community,
> > Here's a report of the flaky tests in Pulsar CI during the observation
> > period of 2023-03-27 - 2023-04-03
> >
> >
> > https://github.com/nicoloboschi/pulsar-flakes/tree/master/2023-03-27-to-2023-04-03
> >
> >
> > Most flaky tests:
> >
> > org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.stateCheck
> > 9
> >
> > org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.testSplitBundleAdminAPI
> > 4
> >
> > org.apache.bookkeeper.mledger.impl.ManagedLedgerErrorsTest.recoverLongTimeAfterMultipleWriteErrors
> > 4
> >
> > org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.initializeState
> > 3
> >
> > org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.testAssign
> > 2
> > More flaky test issues:
> >
> > https://github.com/apache/pulsar/issues?q=flaky+sort%3Aupdated-desc+is%3Aopen+is:issue
> >
> >
> > To coordinate the work,
> > 1) please search for an existing issues or search for all flaky issues with
> > "flaky" or the test class name (without package) in the search:
> >
> > https://github.com/apache/pulsar/issues?q=is%3Aopen+flaky+sort%3Aupdated-desc
> > 2) If there isn't an issue for a particular flaky test failure that you'd
> > like to fix, please create an issue using the "Flaky test" template at
> > https://github.com/apache/pulsar/issues/new/choose
> > 3) Please comment on the issue that you are working on.
> >
> >
> > More contributions are welcome!
> > Nicolò Boschi
> >
>
Re: Pulsar Flaky test report 2023-04-03 for PR builds in CI
Posted by Heesung Sohn <he...@streamnative.io.INVALID>.
Hi,
Thank you for the report.
We are fixing the errors from ExtensibleLoadManagerImplTest.
Regards,
Heesung
On Tue, Apr 4, 2023 at 3:01 AM Nicolò Boschi <bo...@gmail.com> wrote:
> Dear community,
> Here's a report of the flaky tests in Pulsar CI during the observation
> period of 2023-03-27 - 2023-04-03
>
>
> https://github.com/nicoloboschi/pulsar-flakes/tree/master/2023-03-27-to-2023-04-03
>
>
> Most flaky tests:
>
> org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.stateCheck
> 9
>
> org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.testSplitBundleAdminAPI
> 4
>
> org.apache.bookkeeper.mledger.impl.ManagedLedgerErrorsTest.recoverLongTimeAfterMultipleWriteErrors
> 4
>
> org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.initializeState
> 3
>
> org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.testAssign
> 2
> More flaky test issues:
>
> https://github.com/apache/pulsar/issues?q=flaky+sort%3Aupdated-desc+is%3Aopen+is:issue
>
>
> To coordinate the work,
> 1) please search for an existing issues or search for all flaky issues with
> "flaky" or the test class name (without package) in the search:
>
> https://github.com/apache/pulsar/issues?q=is%3Aopen+flaky+sort%3Aupdated-desc
> 2) If there isn't an issue for a particular flaky test failure that you'd
> like to fix, please create an issue using the "Flaky test" template at
> https://github.com/apache/pulsar/issues/new/choose
> 3) Please comment on the issue that you are working on.
>
>
> More contributions are welcome!
> Nicolò Boschi
>