You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Hema Bhatia <He...@apollo.edu> on 2015/09/22 21:56:40 UTC
ZkClient throwing NPEs
Hi,
I keep getting the below exception when Kafka tries to connect to zookeeper and zookeeper is momentarily not able to connect. After that, connection does not restore unless we restart the servers.
This may be connected to this issue : https://issues.apache.org/jira/browse/KAFKA-824 But I am already using zkclient-0.5, and still seeing this issue.
java.lang.NullPointerException: null
at org.I0Itec.zkclient.ZkConnection.exists(ZkConnection.java:107) ~[zkclient-0.5.jar:0.5]
at org.I0Itec.zkclient.ZkClient$3.call(ZkClient.java:631) ~[zkclient-0.5.jar:0.5]
at org.I0Itec.zkclient.ZkClient$3.call(ZkClient.java:628) ~[zkclient-0.5.jar:0.5]
at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:883) ~[zkclient-0.5.jar:0.5]
at org.I0Itec.zkclient.ZkClient.exists(ZkClient.java:628) ~[zkclient-0.5.jar:0.5]
at org.I0Itec.zkclient.ZkClient.exists(ZkClient.java:637) ~[zkclient-0.5.jar:0.5]
at kafka.admin.AdminUtils$.topicExists(AdminUtils.scala:163) ~[kafka_2.10-0.8.2.0.jar:na]
at kafka.admin.AdminUtils.topicExists(AdminUtils.scala) ~[kafka_2.10-0.8.2.0.jar:na]
Any ideas of how to solve this issue?
Thanks,
Hema
RE: ZkClient throwing NPEs
Posted by Hema Bhatia <He...@apollo.edu>.
Thanks Jason!
This exception is occurring mostly in pre-prod environment where we have more network blips. However, I expected that application should be back up gracefully and running. Requiring restarts is kind of bad. For now to handle this scenario, I plan to monitor the zk connection timely and kill & re-create a new one if required, but it is little ugly solution.
Having a patch/solution will be great as it would help everybody including those who have not still faced this problem.
-----Original Message-----
From: Jason Gustafson [mailto:jason@confluent.io]
Sent: Thursday, September 24, 2015 1:35 PM
To: users@kafka.apache.org
Subject: Re: ZkClient throwing NPEs
Hey Hema,
I'm a little surprised you're seeing this so frequently. Do you know why the zookeeper connection is so unstable? If not, then it might be worthwhile investigating a little bit to see if there are other ways to mitigate the problem. Other than that, maybe we can give the ZkClient devs a little time to investigate the issue and work on a solution. If we don't hear anything, I may be able to help with a patch since I think I have an idea how to fix the problem.
-Jason
On Thu, Sep 24, 2015 at 12:39 PM, Hema Bhatia <He...@apollo.edu>
wrote:
> Thanks Jason.
>
> So what is the temporary workaround for this until its fixed? For now,
> we just restart the app server having this issue, but we keep seeing
> this issue time and again.
>
>
> -----Original Message-----
> From: Jason Gustafson [mailto:jason@confluent.io]
> Sent: Thursday, September 24, 2015 12:21 PM
> To: users@kafka.apache.org
> Subject: Re: ZkClient throwing NPEs
>
> Hey Hema,
>
> I'm not too familiar with ZkClient, but I took a look at the code and
> it seems like there may still be a race condition around reconnects
> which could cause the NPE you're seeing. I left a comment on the
> github issue on the slim chance I'm not wrong:
> https://github.com/sgroschupf/zkclient/issues/25.
>
> -Jason
>
> On Thu, Sep 24, 2015 at 10:09 AM, Hema Bhatia <He...@apollo.edu>
> wrote:
>
> > Can anyone help with the below issue...
> > Thanks.
> >
> > -----Original Message-----
> > From: Hema Bhatia [mailto:Hema.Bhatia@apollo.edu]
> > Sent: Tuesday, September 22, 2015 12:57 PM
> > To: users@kafka.apache.org
> > Subject: ZkClient throwing NPEs
> >
> > Hi,
> >
> > I keep getting the below exception when Kafka tries to connect to
> > zookeeper and zookeeper is momentarily not able to connect. After
> > that, connection does not restore unless we restart the servers.
> >
> > This may be connected to this issue :
> > https://issues.apache.org/jira/browse/KAFKA-824 But I am already
> > using zkclient-0.5, and still seeing this issue.
> >
> >
> > java.lang.NullPointerException: null
> >
> > at
> > org.I0Itec.zkclient.ZkConnection.exists(ZkConnection.java:107)
> > ~[zkclient-0.5.jar:0.5]
> >
> > at
> > org.I0Itec.zkclient.ZkClient$3.call(ZkClient.java:631)
> > ~[zkclient-0.5.jar:0.5]
> >
> > at
> > org.I0Itec.zkclient.ZkClient$3.call(ZkClient.java:628)
> > ~[zkclient-0.5.jar:0.5]
> >
> > at
> > org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:883)
> > ~[zkclient-0.5.jar:0.5]
> >
> > at
> > org.I0Itec.zkclient.ZkClient.exists(ZkClient.java:628)
> > ~[zkclient-0.5.jar:0.5]
> >
> > at
> > org.I0Itec.zkclient.ZkClient.exists(ZkClient.java:637)
> > ~[zkclient-0.5.jar:0.5]
> >
> > at
> > kafka.admin.AdminUtils$.topicExists(AdminUtils.scala:163)
> > ~[kafka_2.10-0.8.2.0.jar:na]
> >
> > at
> > kafka.admin.AdminUtils.topicExists(AdminUtils.scala)
> > ~[kafka_2.10-0.8.2.0.jar:na]
> >
> > Any ideas of how to solve this issue?
> >
> > Thanks,
> > Hema
> >
> >
>
Re: ZkClient throwing NPEs
Posted by Jason Gustafson <ja...@confluent.io>.
Hey Hema,
I'm a little surprised you're seeing this so frequently. Do you know why
the zookeeper connection is so unstable? If not, then it might be
worthwhile investigating a little bit to see if there are other ways to
mitigate the problem. Other than that, maybe we can give the ZkClient devs
a little time to investigate the issue and work on a solution. If we don't
hear anything, I may be able to help with a patch since I think I have an
idea how to fix the problem.
-Jason
On Thu, Sep 24, 2015 at 12:39 PM, Hema Bhatia <He...@apollo.edu>
wrote:
> Thanks Jason.
>
> So what is the temporary workaround for this until its fixed? For now, we
> just restart the app server having this issue, but we keep seeing this
> issue time and again.
>
>
> -----Original Message-----
> From: Jason Gustafson [mailto:jason@confluent.io]
> Sent: Thursday, September 24, 2015 12:21 PM
> To: users@kafka.apache.org
> Subject: Re: ZkClient throwing NPEs
>
> Hey Hema,
>
> I'm not too familiar with ZkClient, but I took a look at the code and it
> seems like there may still be a race condition around reconnects which
> could cause the NPE you're seeing. I left a comment on the github issue on
> the slim chance I'm not wrong:
> https://github.com/sgroschupf/zkclient/issues/25.
>
> -Jason
>
> On Thu, Sep 24, 2015 at 10:09 AM, Hema Bhatia <He...@apollo.edu>
> wrote:
>
> > Can anyone help with the below issue...
> > Thanks.
> >
> > -----Original Message-----
> > From: Hema Bhatia [mailto:Hema.Bhatia@apollo.edu]
> > Sent: Tuesday, September 22, 2015 12:57 PM
> > To: users@kafka.apache.org
> > Subject: ZkClient throwing NPEs
> >
> > Hi,
> >
> > I keep getting the below exception when Kafka tries to connect to
> > zookeeper and zookeeper is momentarily not able to connect. After
> > that, connection does not restore unless we restart the servers.
> >
> > This may be connected to this issue :
> > https://issues.apache.org/jira/browse/KAFKA-824 But I am already using
> > zkclient-0.5, and still seeing this issue.
> >
> >
> > java.lang.NullPointerException: null
> >
> > at
> > org.I0Itec.zkclient.ZkConnection.exists(ZkConnection.java:107)
> > ~[zkclient-0.5.jar:0.5]
> >
> > at
> > org.I0Itec.zkclient.ZkClient$3.call(ZkClient.java:631)
> > ~[zkclient-0.5.jar:0.5]
> >
> > at
> > org.I0Itec.zkclient.ZkClient$3.call(ZkClient.java:628)
> > ~[zkclient-0.5.jar:0.5]
> >
> > at
> > org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:883)
> > ~[zkclient-0.5.jar:0.5]
> >
> > at
> > org.I0Itec.zkclient.ZkClient.exists(ZkClient.java:628)
> > ~[zkclient-0.5.jar:0.5]
> >
> > at
> > org.I0Itec.zkclient.ZkClient.exists(ZkClient.java:637)
> > ~[zkclient-0.5.jar:0.5]
> >
> > at
> > kafka.admin.AdminUtils$.topicExists(AdminUtils.scala:163)
> > ~[kafka_2.10-0.8.2.0.jar:na]
> >
> > at
> > kafka.admin.AdminUtils.topicExists(AdminUtils.scala)
> > ~[kafka_2.10-0.8.2.0.jar:na]
> >
> > Any ideas of how to solve this issue?
> >
> > Thanks,
> > Hema
> >
> >
>
RE: ZkClient throwing NPEs
Posted by Hema Bhatia <He...@apollo.edu>.
Thanks Jason.
So what is the temporary workaround for this until its fixed? For now, we just restart the app server having this issue, but we keep seeing this issue time and again.
-----Original Message-----
From: Jason Gustafson [mailto:jason@confluent.io]
Sent: Thursday, September 24, 2015 12:21 PM
To: users@kafka.apache.org
Subject: Re: ZkClient throwing NPEs
Hey Hema,
I'm not too familiar with ZkClient, but I took a look at the code and it seems like there may still be a race condition around reconnects which could cause the NPE you're seeing. I left a comment on the github issue on the slim chance I'm not wrong:
https://github.com/sgroschupf/zkclient/issues/25.
-Jason
On Thu, Sep 24, 2015 at 10:09 AM, Hema Bhatia <He...@apollo.edu>
wrote:
> Can anyone help with the below issue...
> Thanks.
>
> -----Original Message-----
> From: Hema Bhatia [mailto:Hema.Bhatia@apollo.edu]
> Sent: Tuesday, September 22, 2015 12:57 PM
> To: users@kafka.apache.org
> Subject: ZkClient throwing NPEs
>
> Hi,
>
> I keep getting the below exception when Kafka tries to connect to
> zookeeper and zookeeper is momentarily not able to connect. After
> that, connection does not restore unless we restart the servers.
>
> This may be connected to this issue :
> https://issues.apache.org/jira/browse/KAFKA-824 But I am already using
> zkclient-0.5, and still seeing this issue.
>
>
> java.lang.NullPointerException: null
>
> at
> org.I0Itec.zkclient.ZkConnection.exists(ZkConnection.java:107)
> ~[zkclient-0.5.jar:0.5]
>
> at
> org.I0Itec.zkclient.ZkClient$3.call(ZkClient.java:631)
> ~[zkclient-0.5.jar:0.5]
>
> at
> org.I0Itec.zkclient.ZkClient$3.call(ZkClient.java:628)
> ~[zkclient-0.5.jar:0.5]
>
> at
> org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:883)
> ~[zkclient-0.5.jar:0.5]
>
> at
> org.I0Itec.zkclient.ZkClient.exists(ZkClient.java:628)
> ~[zkclient-0.5.jar:0.5]
>
> at
> org.I0Itec.zkclient.ZkClient.exists(ZkClient.java:637)
> ~[zkclient-0.5.jar:0.5]
>
> at
> kafka.admin.AdminUtils$.topicExists(AdminUtils.scala:163)
> ~[kafka_2.10-0.8.2.0.jar:na]
>
> at
> kafka.admin.AdminUtils.topicExists(AdminUtils.scala)
> ~[kafka_2.10-0.8.2.0.jar:na]
>
> Any ideas of how to solve this issue?
>
> Thanks,
> Hema
>
>
Re: ZkClient throwing NPEs
Posted by Jason Gustafson <ja...@confluent.io>.
Hey Hema,
I'm not too familiar with ZkClient, but I took a look at the code and it
seems like there may still be a race condition around reconnects which
could cause the NPE you're seeing. I left a comment on the github issue on
the slim chance I'm not wrong:
https://github.com/sgroschupf/zkclient/issues/25.
-Jason
On Thu, Sep 24, 2015 at 10:09 AM, Hema Bhatia <He...@apollo.edu>
wrote:
> Can anyone help with the below issue...
> Thanks.
>
> -----Original Message-----
> From: Hema Bhatia [mailto:Hema.Bhatia@apollo.edu]
> Sent: Tuesday, September 22, 2015 12:57 PM
> To: users@kafka.apache.org
> Subject: ZkClient throwing NPEs
>
> Hi,
>
> I keep getting the below exception when Kafka tries to connect to
> zookeeper and zookeeper is momentarily not able to connect. After that,
> connection does not restore unless we restart the servers.
>
> This may be connected to this issue :
> https://issues.apache.org/jira/browse/KAFKA-824 But I am already using
> zkclient-0.5, and still seeing this issue.
>
>
> java.lang.NullPointerException: null
>
> at
> org.I0Itec.zkclient.ZkConnection.exists(ZkConnection.java:107)
> ~[zkclient-0.5.jar:0.5]
>
> at org.I0Itec.zkclient.ZkClient$3.call(ZkClient.java:631)
> ~[zkclient-0.5.jar:0.5]
>
> at org.I0Itec.zkclient.ZkClient$3.call(ZkClient.java:628)
> ~[zkclient-0.5.jar:0.5]
>
> at
> org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:883)
> ~[zkclient-0.5.jar:0.5]
>
> at org.I0Itec.zkclient.ZkClient.exists(ZkClient.java:628)
> ~[zkclient-0.5.jar:0.5]
>
> at org.I0Itec.zkclient.ZkClient.exists(ZkClient.java:637)
> ~[zkclient-0.5.jar:0.5]
>
> at
> kafka.admin.AdminUtils$.topicExists(AdminUtils.scala:163)
> ~[kafka_2.10-0.8.2.0.jar:na]
>
> at kafka.admin.AdminUtils.topicExists(AdminUtils.scala)
> ~[kafka_2.10-0.8.2.0.jar:na]
>
> Any ideas of how to solve this issue?
>
> Thanks,
> Hema
>
>
RE: ZkClient throwing NPEs
Posted by Hema Bhatia <He...@apollo.edu>.
Can anyone help with the below issue...
Thanks.
-----Original Message-----
From: Hema Bhatia [mailto:Hema.Bhatia@apollo.edu]
Sent: Tuesday, September 22, 2015 12:57 PM
To: users@kafka.apache.org
Subject: ZkClient throwing NPEs
Hi,
I keep getting the below exception when Kafka tries to connect to zookeeper and zookeeper is momentarily not able to connect. After that, connection does not restore unless we restart the servers.
This may be connected to this issue : https://issues.apache.org/jira/browse/KAFKA-824 But I am already using zkclient-0.5, and still seeing this issue.
java.lang.NullPointerException: null
at org.I0Itec.zkclient.ZkConnection.exists(ZkConnection.java:107) ~[zkclient-0.5.jar:0.5]
at org.I0Itec.zkclient.ZkClient$3.call(ZkClient.java:631) ~[zkclient-0.5.jar:0.5]
at org.I0Itec.zkclient.ZkClient$3.call(ZkClient.java:628) ~[zkclient-0.5.jar:0.5]
at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:883) ~[zkclient-0.5.jar:0.5]
at org.I0Itec.zkclient.ZkClient.exists(ZkClient.java:628) ~[zkclient-0.5.jar:0.5]
at org.I0Itec.zkclient.ZkClient.exists(ZkClient.java:637) ~[zkclient-0.5.jar:0.5]
at kafka.admin.AdminUtils$.topicExists(AdminUtils.scala:163) ~[kafka_2.10-0.8.2.0.jar:na]
at kafka.admin.AdminUtils.topicExists(AdminUtils.scala) ~[kafka_2.10-0.8.2.0.jar:na]
Any ideas of how to solve this issue?
Thanks,
Hema