Producer Graceful Shutdown issue in Container (Kafka version 0.8.x.x)

[Bhavesh Mistry <mi...@gmail.com>]

Hi Kafka Team,

We are running multiple webapps in tomcat container, and we have producer
which are managed by the ServletContextListener (Lifecycle).  Upon
contextInitialized we create and on contextDestroyed we call the
producer.close() but underlying Metric Lib does not shutdown.  So we have
thread leak due to this issue.  I had to call
Metrics.defaultRegistry().shutdown() to resolve this issue.  is this know
issue ? I know the metric lib have JVM Shutdown hook, but it will not be
invoke since the contain thread is un-deploying the web app and class
loader goes way and leaking thread does not find the under lying Kafka
class.    Because of this tomcat, it not shutting down gracefully.

Are you guys planing to un-register metrics when Producer close is called
or shutdown Metrics pool for client.id ?


SEVERE: The web application [  ] appears to have started a thread named [
*metrics-meter-tick-thread-1*] but has failed to stop it. This is very
likely to create a memory leak.
SEVERE: The web application [] appears to have started a thread named [
*metrics-meter-tick-thread-2*] but has failed to stop it. This is very
likely to create a memory leak.

Thanks,

Bhavesh

Re: Producer Graceful Shutdown issue in Container (Kafka version 0.8.x.x)

[Bhavesh Mistry <mi...@gmail.com>]

Hi Guozhang,

Thank you for your quick response.

This is version 0.8.0 producer package
* kafka.javaapi.producer.Producer.*
Thanks,
Bhavesh


On Thu, Jul 3, 2014 at 11:04 AM, Guozhang Wang <wa...@gmail.com> wrote:

> Hi Bhavesh,
>
> Is this the new producer under clients or the original producer under core?
>
> Guozhang
>
>
> On Thu, Jul 3, 2014 at 10:52 AM, Bhavesh Mistry <
> mistry.p.bhavesh@gmail.com>
> wrote:
>
> > Hi Kafka Team,
> >
> > We are running multiple webapps in tomcat container, and we have producer
> > which are managed by the ServletContextListener (Lifecycle).  Upon
> > contextInitialized we create and on contextDestroyed we call the
> > producer.close() but underlying Metric Lib does not shutdown.  So we have
> > thread leak due to this issue.  I had to call
> > Metrics.defaultRegistry().shutdown() to resolve this issue.  is this know
> > issue ? I know the metric lib have JVM Shutdown hook, but it will not be
> > invoke since the contain thread is un-deploying the web app and class
> > loader goes way and leaking thread does not find the under lying Kafka
> > class.    Because of this tomcat, it not shutting down gracefully.
> >
> > Are you guys planing to un-register metrics when Producer close is called
> > or shutdown Metrics pool for client.id ?
> >
> >
> > SEVERE: The web application [  ] appears to have started a thread named [
> > *metrics-meter-tick-thread-1*] but has failed to stop it. This is very
> > likely to create a memory leak.
> > SEVERE: The web application [] appears to have started a thread named [
> > *metrics-meter-tick-thread-2*] but has failed to stop it. This is very
> > likely to create a memory leak.
> >
> > Thanks,
> >
> > Bhavesh
> >
>
>
>
> --
> -- Guozhang
>

Re: Producer Graceful Shutdown issue in Container (Kafka version 0.8.x.x)

[Guozhang Wang <wa...@gmail.com>]

Hi Bhavesh,

Is this the new producer under clients or the original producer under core?

Guozhang


On Thu, Jul 3, 2014 at 10:52 AM, Bhavesh Mistry <mi...@gmail.com>
wrote:

> Hi Kafka Team,
>
> We are running multiple webapps in tomcat container, and we have producer
> which are managed by the ServletContextListener (Lifecycle).  Upon
> contextInitialized we create and on contextDestroyed we call the
> producer.close() but underlying Metric Lib does not shutdown.  So we have
> thread leak due to this issue.  I had to call
> Metrics.defaultRegistry().shutdown() to resolve this issue.  is this know
> issue ? I know the metric lib have JVM Shutdown hook, but it will not be
> invoke since the contain thread is un-deploying the web app and class
> loader goes way and leaking thread does not find the under lying Kafka
> class.    Because of this tomcat, it not shutting down gracefully.
>
> Are you guys planing to un-register metrics when Producer close is called
> or shutdown Metrics pool for client.id ?
>
>
> SEVERE: The web application [  ] appears to have started a thread named [
> *metrics-meter-tick-thread-1*] but has failed to stop it. This is very
> likely to create a memory leak.
> SEVERE: The web application [] appears to have started a thread named [
> *metrics-meter-tick-thread-2*] but has failed to stop it. This is very
> likely to create a memory leak.
>
> Thanks,
>
> Bhavesh
>



-- 
-- Guozhang