Discuss about closing Pulsar resource

[guo jiwei <te...@apache.org>]

Hi community:
    In the past few days, when investigating the problem of pulsar close
being hung, I saw that many resources are called synchronously and have no
timeout when they are closed. To solve this problem,
patch-15638 <https://github.com/apache/pulsar/pull/15638>, add the async
close method and timeout.
patch-15680 <https://github.com/apache/pulsar/pull/15680>, uniformly
increase the timeout in the pulsar close method.
patch-15777 <https://github.com/apache/pulsar/pull/15777>, add timeout in
the close method.
However, neither increasing the async interface nor the timeout is
guaranteed to shut down the pulsar service within `brokerShutdownTimeoutMs`.

In order to ensure the uniformity of resource closure, I hope to discuss
with you a relatively reasonable solution.


Regards
Jiwei Guo (Tboy)

Re: Discuss about closing Pulsar resource

[Nicolò Boschi <bo...@gmail.com>]

Hi,

the initial intention of https://github.com/apache/pulsar/pull/15638 was
about fixing thread leaks in tests which lead to Mockito weird behaviours.
Now a few executors do not wait for thread termination and it makes the
tests unstable, mainly with mocked objects. The AsyncCloseable addresses a
different problem so I will create another pull for that.

In order to respect the brokerShutdownTimeoutMs we definitely need that
every component handled by the BrokerService support the async closing.
Other than adding the interface and making all those classes implement it,
we need to ensure the order of components being closed will not lead to
deadlocks (which is a current problem) and it's not trivial.
Another important aspect is that now brokerShutdownTimeoutMs has a default
value which respects all the components handled in the async executor but
there are other components not included in this shutdown time. So it is
possible that we'd need to set a higher default value.

A good plan would be to create an ad-hoc pull request for the
"AsyncCloneable" interface where all the work is addressed in one shot.

Nicolò Boschi


Il giorno gio 26 mag 2022 alle ore 03:57 guo jiwei <te...@apache.org>
ha scritto:

> Hi community:
>     In the past few days, when investigating the problem of pulsar close
> being hung, I saw that many resources are called synchronously and have no
> timeout when they are closed. To solve this problem,
> patch-15638 <https://github.com/apache/pulsar/pull/15638>, add the async
> close method and timeout.
> patch-15680 <https://github.com/apache/pulsar/pull/15680>, uniformly
> increase the timeout in the pulsar close method.
> patch-15777 <https://github.com/apache/pulsar/pull/15777>, add timeout in
> the close method.
> However, neither increasing the async interface nor the timeout is
> guaranteed to shut down the pulsar service within
> `brokerShutdownTimeoutMs`.
>
> In order to ensure the uniformity of resource closure, I hope to discuss
> with you a relatively reasonable solution.
>
>
> Regards
> Jiwei Guo (Tboy)
>