You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by Alexandre Vermeerbergen <av...@gmail.com> on 2019/09/18 15:03:38 UTC
Any hope for a fix of STORM-3032 ?
Hello,
I have seen couple of occurrences of the NullPointerException in Storm
Kafka Spout with same stack trace as in
https://issues.apache.org/jira/browse/STORM-3032
Using Storm 1.2.3, with Kafka Client at 2.0.1 version, connected to a
cluster of Kafka Brokers at 2.2.1 version.
Since the stack traces seems to tell that the NullPointerException
occurs in some code related to statistics computation, wouldn't it be
better to fix the issue by catching the NullPointerException to avoid
blocking Kafka consumption ?
Kind regards,
Alexandre Vermeerbergen
Re: Any hope for a fix of STORM-3032 ?
Posted by Stig Rohde Døssing <st...@gmail.com>.
Sounds good, thanks for letting other people know about the JDK part.
Den ons. 25. sep. 2019 kl. 09.09 skrev Alexandre Vermeerbergen <
avermeerbergen@gmail.com>:
> Hello Stig,
>
> I did not realize that these API exist in Storm 1.2.x, if so then I
> can start working on this adaptation to prepare 1.x/2.x transition of
> our topologies.
> On a side note, I realized that I could only reproduce STORM-3032 only
> if running Supervisor processes using AdoptOpenJDK 8 update 222 with
> OpenJ9 ; whereas I don't get these exceptions when using AdopOpenJDK 8
> update 222 with HotSpot
> => let me update the JIRA with this piece of information
>
> Kind regards,
> Alexandre
>
>
> Le mer. 25 sept. 2019 à 08:21, Stig Rohde Døssing
> <st...@gmail.com> a écrit :
> >
> > I believe those API changes are in Storm 1.2.0+ as well. You can upgrade
> to
> > one of those versions, and update your topologies so they don't use
> > deprecated methods. Then the topologies should be compatible with 2.x
> > without rebuilding.
> >
> > Den tir. 24. sep. 2019 kl. 22.34 skrev Alexandre Vermeerbergen <
> > avermeerbergen@gmail.com>:
> >
> > > Hello Stig,
> > >
> > > The API changes are the ones mentionned in this thread:
> > >
> > >
> http://mail-archives.apache.org/mod_mbox/storm-dev/201905.mbox/%3cCAG09ER3iDfXY7+SixSsCDo=A+H0z-sD6hJwNFNFp+zfAW-RAmg@mail.gmail.com%3e
> > >
> > > My problem is that if I migrate my topologies to 2.x to try it, then I
> > > have to duplicate some code to keep our topologies compatible with
> > > Storm 1.x until we have 100% tested our migration.
> > >
> > > Unless I am missing something? Is there a way to run Storm
> > > 1.x-compiled topologies on Storm 2.x without rebuilding?
> > >
> > > Kind regards,
> > > Alexandre
> > >
> > >
> > > Le dim. 22 sept. 2019 à 22:56, Stig Rohde Døssing
> > > <st...@gmail.com> a écrit :
> > > >
> > > > I thought the API changes for Storm 2.x were mostly minor. Which API
> > > > changes are you being blocked by?
> > > >
> > > > Den søn. 22. sep. 2019 kl. 21.27 skrev Alexandre Vermeerbergen <
> > > > avermeerbergen@gmail.com>:
> > > >
> > > > > Hello Stig,
> > > > >
> > > > > Thank you very much for your answer.
> > > > >
> > > > > Upgrading to Storm 2.x is something I would like to try, but I know
> > > > > that building with Storm 2.x API requires changing our topologies
> > > > > since some API have changes (I used to try, and was kind of
> blocked by
> > > > > the un-reversible changes that were required).
> > > > >
> > > > > Question: could our Storm 1.x topology work with Storm 2.x ?
> > > > >
> > > > > Kind regards,
> > > > > Alexandre
> > > > >
> > > > > Le jeu. 19 sept. 2019 à 19:05, Stig Rohde Døssing
> > > > > <st...@gmail.com> a écrit :
> > > > > >
> > > > > > Looking at the stack trace, the issue is not in the Kafka bolt,
> but
> > > in
> > > > > some
> > > > > > internal Storm statistics code. It seems likely to me that it has
> > > been
> > > > > > fixed in 2.0.0, as we haven't heard of anyone encounter it there.
> > > Are you
> > > > > > able to upgrade to 2.x to see if that fixes this?
> > > > > >
> > > > > > If not, the exception is coming from
> > > > > >
> > > > >
> > >
> https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L131
> > > > > .
> > > > > > Looks like either "amt" or " (stats-rate stats) " in that line is
> > > null.
> > > > > My
> > > > > > guess would be stats. The equivalent code in 2.x uses primitive
> > > ints, so
> > > > > > won't be throwing NPEs
> > > > > >
> > > > >
> > >
> https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/stats/CommonStats.java#L63
> > > > > >
> > > > > > If you have the ability to upgrade to 2.0.0 that would be best.
> > > > > >
> > > > > > I don't think simply catching the NPE is a good solution, we
> should
> > > > > prevent
> > > > > > the NPE from being thrown in the first place. If you want to
> take a
> > > look
> > > > > at
> > > > > > it, feel free to raise a PR. May want to also lobby to get a
> 1.2.4
> > > > > release
> > > > > > out after that.
> > > > > >
> > > > > > Den ons. 18. sep. 2019 kl. 17.04 skrev Alexandre Vermeerbergen <
> > > > > > avermeerbergen@gmail.com>:
> > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > I have seen couple of occurrences of the NullPointerException
> in
> > > Storm
> > > > > > > Kafka Spout with same stack trace as in
> > > > > > > https://issues.apache.org/jira/browse/STORM-3032
> > > > > > >
> > > > > > > Using Storm 1.2.3, with Kafka Client at 2.0.1 version,
> connected
> > > to a
> > > > > > > cluster of Kafka Brokers at 2.2.1 version.
> > > > > > >
> > > > > > > Since the stack traces seems to tell that the
> NullPointerException
> > > > > > > occurs in some code related to statistics computation,
> wouldn't it
> > > be
> > > > > > > better to fix the issue by catching the NullPointerException to
> > > avoid
> > > > > > > blocking Kafka consumption ?
> > > > > > >
> > > > > > > Kind regards,
> > > > > > > Alexandre Vermeerbergen
> > > > > > >
> > > > >
> > >
>
Re: Any hope for a fix of STORM-3032 ?
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Hello Stig,
I did not realize that these API exist in Storm 1.2.x, if so then I
can start working on this adaptation to prepare 1.x/2.x transition of
our topologies.
On a side note, I realized that I could only reproduce STORM-3032 only
if running Supervisor processes using AdoptOpenJDK 8 update 222 with
OpenJ9 ; whereas I don't get these exceptions when using AdopOpenJDK 8
update 222 with HotSpot
=> let me update the JIRA with this piece of information
Kind regards,
Alexandre
Le mer. 25 sept. 2019 à 08:21, Stig Rohde Døssing
<st...@gmail.com> a écrit :
>
> I believe those API changes are in Storm 1.2.0+ as well. You can upgrade to
> one of those versions, and update your topologies so they don't use
> deprecated methods. Then the topologies should be compatible with 2.x
> without rebuilding.
>
> Den tir. 24. sep. 2019 kl. 22.34 skrev Alexandre Vermeerbergen <
> avermeerbergen@gmail.com>:
>
> > Hello Stig,
> >
> > The API changes are the ones mentionned in this thread:
> >
> > http://mail-archives.apache.org/mod_mbox/storm-dev/201905.mbox/%3cCAG09ER3iDfXY7+SixSsCDo=A+H0z-sD6hJwNFNFp+zfAW-RAmg@mail.gmail.com%3e
> >
> > My problem is that if I migrate my topologies to 2.x to try it, then I
> > have to duplicate some code to keep our topologies compatible with
> > Storm 1.x until we have 100% tested our migration.
> >
> > Unless I am missing something? Is there a way to run Storm
> > 1.x-compiled topologies on Storm 2.x without rebuilding?
> >
> > Kind regards,
> > Alexandre
> >
> >
> > Le dim. 22 sept. 2019 à 22:56, Stig Rohde Døssing
> > <st...@gmail.com> a écrit :
> > >
> > > I thought the API changes for Storm 2.x were mostly minor. Which API
> > > changes are you being blocked by?
> > >
> > > Den søn. 22. sep. 2019 kl. 21.27 skrev Alexandre Vermeerbergen <
> > > avermeerbergen@gmail.com>:
> > >
> > > > Hello Stig,
> > > >
> > > > Thank you very much for your answer.
> > > >
> > > > Upgrading to Storm 2.x is something I would like to try, but I know
> > > > that building with Storm 2.x API requires changing our topologies
> > > > since some API have changes (I used to try, and was kind of blocked by
> > > > the un-reversible changes that were required).
> > > >
> > > > Question: could our Storm 1.x topology work with Storm 2.x ?
> > > >
> > > > Kind regards,
> > > > Alexandre
> > > >
> > > > Le jeu. 19 sept. 2019 à 19:05, Stig Rohde Døssing
> > > > <st...@gmail.com> a écrit :
> > > > >
> > > > > Looking at the stack trace, the issue is not in the Kafka bolt, but
> > in
> > > > some
> > > > > internal Storm statistics code. It seems likely to me that it has
> > been
> > > > > fixed in 2.0.0, as we haven't heard of anyone encounter it there.
> > Are you
> > > > > able to upgrade to 2.x to see if that fixes this?
> > > > >
> > > > > If not, the exception is coming from
> > > > >
> > > >
> > https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L131
> > > > .
> > > > > Looks like either "amt" or " (stats-rate stats) " in that line is
> > null.
> > > > My
> > > > > guess would be stats. The equivalent code in 2.x uses primitive
> > ints, so
> > > > > won't be throwing NPEs
> > > > >
> > > >
> > https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/stats/CommonStats.java#L63
> > > > >
> > > > > If you have the ability to upgrade to 2.0.0 that would be best.
> > > > >
> > > > > I don't think simply catching the NPE is a good solution, we should
> > > > prevent
> > > > > the NPE from being thrown in the first place. If you want to take a
> > look
> > > > at
> > > > > it, feel free to raise a PR. May want to also lobby to get a 1.2.4
> > > > release
> > > > > out after that.
> > > > >
> > > > > Den ons. 18. sep. 2019 kl. 17.04 skrev Alexandre Vermeerbergen <
> > > > > avermeerbergen@gmail.com>:
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I have seen couple of occurrences of the NullPointerException in
> > Storm
> > > > > > Kafka Spout with same stack trace as in
> > > > > > https://issues.apache.org/jira/browse/STORM-3032
> > > > > >
> > > > > > Using Storm 1.2.3, with Kafka Client at 2.0.1 version, connected
> > to a
> > > > > > cluster of Kafka Brokers at 2.2.1 version.
> > > > > >
> > > > > > Since the stack traces seems to tell that the NullPointerException
> > > > > > occurs in some code related to statistics computation, wouldn't it
> > be
> > > > > > better to fix the issue by catching the NullPointerException to
> > avoid
> > > > > > blocking Kafka consumption ?
> > > > > >
> > > > > > Kind regards,
> > > > > > Alexandre Vermeerbergen
> > > > > >
> > > >
> >
Re: Any hope for a fix of STORM-3032 ?
Posted by Stig Rohde Døssing <st...@gmail.com>.
I believe those API changes are in Storm 1.2.0+ as well. You can upgrade to
one of those versions, and update your topologies so they don't use
deprecated methods. Then the topologies should be compatible with 2.x
without rebuilding.
Den tir. 24. sep. 2019 kl. 22.34 skrev Alexandre Vermeerbergen <
avermeerbergen@gmail.com>:
> Hello Stig,
>
> The API changes are the ones mentionned in this thread:
>
> http://mail-archives.apache.org/mod_mbox/storm-dev/201905.mbox/%3cCAG09ER3iDfXY7+SixSsCDo=A+H0z-sD6hJwNFNFp+zfAW-RAmg@mail.gmail.com%3e
>
> My problem is that if I migrate my topologies to 2.x to try it, then I
> have to duplicate some code to keep our topologies compatible with
> Storm 1.x until we have 100% tested our migration.
>
> Unless I am missing something? Is there a way to run Storm
> 1.x-compiled topologies on Storm 2.x without rebuilding?
>
> Kind regards,
> Alexandre
>
>
> Le dim. 22 sept. 2019 à 22:56, Stig Rohde Døssing
> <st...@gmail.com> a écrit :
> >
> > I thought the API changes for Storm 2.x were mostly minor. Which API
> > changes are you being blocked by?
> >
> > Den søn. 22. sep. 2019 kl. 21.27 skrev Alexandre Vermeerbergen <
> > avermeerbergen@gmail.com>:
> >
> > > Hello Stig,
> > >
> > > Thank you very much for your answer.
> > >
> > > Upgrading to Storm 2.x is something I would like to try, but I know
> > > that building with Storm 2.x API requires changing our topologies
> > > since some API have changes (I used to try, and was kind of blocked by
> > > the un-reversible changes that were required).
> > >
> > > Question: could our Storm 1.x topology work with Storm 2.x ?
> > >
> > > Kind regards,
> > > Alexandre
> > >
> > > Le jeu. 19 sept. 2019 à 19:05, Stig Rohde Døssing
> > > <st...@gmail.com> a écrit :
> > > >
> > > > Looking at the stack trace, the issue is not in the Kafka bolt, but
> in
> > > some
> > > > internal Storm statistics code. It seems likely to me that it has
> been
> > > > fixed in 2.0.0, as we haven't heard of anyone encounter it there.
> Are you
> > > > able to upgrade to 2.x to see if that fixes this?
> > > >
> > > > If not, the exception is coming from
> > > >
> > >
> https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L131
> > > .
> > > > Looks like either "amt" or " (stats-rate stats) " in that line is
> null.
> > > My
> > > > guess would be stats. The equivalent code in 2.x uses primitive
> ints, so
> > > > won't be throwing NPEs
> > > >
> > >
> https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/stats/CommonStats.java#L63
> > > >
> > > > If you have the ability to upgrade to 2.0.0 that would be best.
> > > >
> > > > I don't think simply catching the NPE is a good solution, we should
> > > prevent
> > > > the NPE from being thrown in the first place. If you want to take a
> look
> > > at
> > > > it, feel free to raise a PR. May want to also lobby to get a 1.2.4
> > > release
> > > > out after that.
> > > >
> > > > Den ons. 18. sep. 2019 kl. 17.04 skrev Alexandre Vermeerbergen <
> > > > avermeerbergen@gmail.com>:
> > > >
> > > > > Hello,
> > > > >
> > > > > I have seen couple of occurrences of the NullPointerException in
> Storm
> > > > > Kafka Spout with same stack trace as in
> > > > > https://issues.apache.org/jira/browse/STORM-3032
> > > > >
> > > > > Using Storm 1.2.3, with Kafka Client at 2.0.1 version, connected
> to a
> > > > > cluster of Kafka Brokers at 2.2.1 version.
> > > > >
> > > > > Since the stack traces seems to tell that the NullPointerException
> > > > > occurs in some code related to statistics computation, wouldn't it
> be
> > > > > better to fix the issue by catching the NullPointerException to
> avoid
> > > > > blocking Kafka consumption ?
> > > > >
> > > > > Kind regards,
> > > > > Alexandre Vermeerbergen
> > > > >
> > >
>
Re: Any hope for a fix of STORM-3032 ?
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Hello Stig,
The API changes are the ones mentionned in this thread:
http://mail-archives.apache.org/mod_mbox/storm-dev/201905.mbox/%3cCAG09ER3iDfXY7+SixSsCDo=A+H0z-sD6hJwNFNFp+zfAW-RAmg@mail.gmail.com%3e
My problem is that if I migrate my topologies to 2.x to try it, then I
have to duplicate some code to keep our topologies compatible with
Storm 1.x until we have 100% tested our migration.
Unless I am missing something? Is there a way to run Storm
1.x-compiled topologies on Storm 2.x without rebuilding?
Kind regards,
Alexandre
Le dim. 22 sept. 2019 à 22:56, Stig Rohde Døssing
<st...@gmail.com> a écrit :
>
> I thought the API changes for Storm 2.x were mostly minor. Which API
> changes are you being blocked by?
>
> Den søn. 22. sep. 2019 kl. 21.27 skrev Alexandre Vermeerbergen <
> avermeerbergen@gmail.com>:
>
> > Hello Stig,
> >
> > Thank you very much for your answer.
> >
> > Upgrading to Storm 2.x is something I would like to try, but I know
> > that building with Storm 2.x API requires changing our topologies
> > since some API have changes (I used to try, and was kind of blocked by
> > the un-reversible changes that were required).
> >
> > Question: could our Storm 1.x topology work with Storm 2.x ?
> >
> > Kind regards,
> > Alexandre
> >
> > Le jeu. 19 sept. 2019 à 19:05, Stig Rohde Døssing
> > <st...@gmail.com> a écrit :
> > >
> > > Looking at the stack trace, the issue is not in the Kafka bolt, but in
> > some
> > > internal Storm statistics code. It seems likely to me that it has been
> > > fixed in 2.0.0, as we haven't heard of anyone encounter it there. Are you
> > > able to upgrade to 2.x to see if that fixes this?
> > >
> > > If not, the exception is coming from
> > >
> > https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L131
> > .
> > > Looks like either "amt" or " (stats-rate stats) " in that line is null.
> > My
> > > guess would be stats. The equivalent code in 2.x uses primitive ints, so
> > > won't be throwing NPEs
> > >
> > https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/stats/CommonStats.java#L63
> > >
> > > If you have the ability to upgrade to 2.0.0 that would be best.
> > >
> > > I don't think simply catching the NPE is a good solution, we should
> > prevent
> > > the NPE from being thrown in the first place. If you want to take a look
> > at
> > > it, feel free to raise a PR. May want to also lobby to get a 1.2.4
> > release
> > > out after that.
> > >
> > > Den ons. 18. sep. 2019 kl. 17.04 skrev Alexandre Vermeerbergen <
> > > avermeerbergen@gmail.com>:
> > >
> > > > Hello,
> > > >
> > > > I have seen couple of occurrences of the NullPointerException in Storm
> > > > Kafka Spout with same stack trace as in
> > > > https://issues.apache.org/jira/browse/STORM-3032
> > > >
> > > > Using Storm 1.2.3, with Kafka Client at 2.0.1 version, connected to a
> > > > cluster of Kafka Brokers at 2.2.1 version.
> > > >
> > > > Since the stack traces seems to tell that the NullPointerException
> > > > occurs in some code related to statistics computation, wouldn't it be
> > > > better to fix the issue by catching the NullPointerException to avoid
> > > > blocking Kafka consumption ?
> > > >
> > > > Kind regards,
> > > > Alexandre Vermeerbergen
> > > >
> >
Re: Any hope for a fix of STORM-3032 ?
Posted by Stig Rohde Døssing <st...@gmail.com>.
I thought the API changes for Storm 2.x were mostly minor. Which API
changes are you being blocked by?
Den søn. 22. sep. 2019 kl. 21.27 skrev Alexandre Vermeerbergen <
avermeerbergen@gmail.com>:
> Hello Stig,
>
> Thank you very much for your answer.
>
> Upgrading to Storm 2.x is something I would like to try, but I know
> that building with Storm 2.x API requires changing our topologies
> since some API have changes (I used to try, and was kind of blocked by
> the un-reversible changes that were required).
>
> Question: could our Storm 1.x topology work with Storm 2.x ?
>
> Kind regards,
> Alexandre
>
> Le jeu. 19 sept. 2019 à 19:05, Stig Rohde Døssing
> <st...@gmail.com> a écrit :
> >
> > Looking at the stack trace, the issue is not in the Kafka bolt, but in
> some
> > internal Storm statistics code. It seems likely to me that it has been
> > fixed in 2.0.0, as we haven't heard of anyone encounter it there. Are you
> > able to upgrade to 2.x to see if that fixes this?
> >
> > If not, the exception is coming from
> >
> https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L131
> .
> > Looks like either "amt" or " (stats-rate stats) " in that line is null.
> My
> > guess would be stats. The equivalent code in 2.x uses primitive ints, so
> > won't be throwing NPEs
> >
> https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/stats/CommonStats.java#L63
> >
> > If you have the ability to upgrade to 2.0.0 that would be best.
> >
> > I don't think simply catching the NPE is a good solution, we should
> prevent
> > the NPE from being thrown in the first place. If you want to take a look
> at
> > it, feel free to raise a PR. May want to also lobby to get a 1.2.4
> release
> > out after that.
> >
> > Den ons. 18. sep. 2019 kl. 17.04 skrev Alexandre Vermeerbergen <
> > avermeerbergen@gmail.com>:
> >
> > > Hello,
> > >
> > > I have seen couple of occurrences of the NullPointerException in Storm
> > > Kafka Spout with same stack trace as in
> > > https://issues.apache.org/jira/browse/STORM-3032
> > >
> > > Using Storm 1.2.3, with Kafka Client at 2.0.1 version, connected to a
> > > cluster of Kafka Brokers at 2.2.1 version.
> > >
> > > Since the stack traces seems to tell that the NullPointerException
> > > occurs in some code related to statistics computation, wouldn't it be
> > > better to fix the issue by catching the NullPointerException to avoid
> > > blocking Kafka consumption ?
> > >
> > > Kind regards,
> > > Alexandre Vermeerbergen
> > >
>
Re: Any hope for a fix of STORM-3032 ?
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Hello Stig,
Thank you very much for your answer.
Upgrading to Storm 2.x is something I would like to try, but I know
that building with Storm 2.x API requires changing our topologies
since some API have changes (I used to try, and was kind of blocked by
the un-reversible changes that were required).
Question: could our Storm 1.x topology work with Storm 2.x ?
Kind regards,
Alexandre
Le jeu. 19 sept. 2019 à 19:05, Stig Rohde Døssing
<st...@gmail.com> a écrit :
>
> Looking at the stack trace, the issue is not in the Kafka bolt, but in some
> internal Storm statistics code. It seems likely to me that it has been
> fixed in 2.0.0, as we haven't heard of anyone encounter it there. Are you
> able to upgrade to 2.x to see if that fixes this?
>
> If not, the exception is coming from
> https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L131.
> Looks like either "amt" or " (stats-rate stats) " in that line is null. My
> guess would be stats. The equivalent code in 2.x uses primitive ints, so
> won't be throwing NPEs
> https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/stats/CommonStats.java#L63
>
> If you have the ability to upgrade to 2.0.0 that would be best.
>
> I don't think simply catching the NPE is a good solution, we should prevent
> the NPE from being thrown in the first place. If you want to take a look at
> it, feel free to raise a PR. May want to also lobby to get a 1.2.4 release
> out after that.
>
> Den ons. 18. sep. 2019 kl. 17.04 skrev Alexandre Vermeerbergen <
> avermeerbergen@gmail.com>:
>
> > Hello,
> >
> > I have seen couple of occurrences of the NullPointerException in Storm
> > Kafka Spout with same stack trace as in
> > https://issues.apache.org/jira/browse/STORM-3032
> >
> > Using Storm 1.2.3, with Kafka Client at 2.0.1 version, connected to a
> > cluster of Kafka Brokers at 2.2.1 version.
> >
> > Since the stack traces seems to tell that the NullPointerException
> > occurs in some code related to statistics computation, wouldn't it be
> > better to fix the issue by catching the NullPointerException to avoid
> > blocking Kafka consumption ?
> >
> > Kind regards,
> > Alexandre Vermeerbergen
> >
Re: Any hope for a fix of STORM-3032 ?
Posted by Stig Rohde Døssing <st...@gmail.com>.
Looking at the stack trace, the issue is not in the Kafka bolt, but in some
internal Storm statistics code. It seems likely to me that it has been
fixed in 2.0.0, as we haven't heard of anyone encounter it there. Are you
able to upgrade to 2.x to see if that fixes this?
If not, the exception is coming from
https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L131.
Looks like either "amt" or " (stats-rate stats) " in that line is null. My
guess would be stats. The equivalent code in 2.x uses primitive ints, so
won't be throwing NPEs
https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/stats/CommonStats.java#L63
If you have the ability to upgrade to 2.0.0 that would be best.
I don't think simply catching the NPE is a good solution, we should prevent
the NPE from being thrown in the first place. If you want to take a look at
it, feel free to raise a PR. May want to also lobby to get a 1.2.4 release
out after that.
Den ons. 18. sep. 2019 kl. 17.04 skrev Alexandre Vermeerbergen <
avermeerbergen@gmail.com>:
> Hello,
>
> I have seen couple of occurrences of the NullPointerException in Storm
> Kafka Spout with same stack trace as in
> https://issues.apache.org/jira/browse/STORM-3032
>
> Using Storm 1.2.3, with Kafka Client at 2.0.1 version, connected to a
> cluster of Kafka Brokers at 2.2.1 version.
>
> Since the stack traces seems to tell that the NullPointerException
> occurs in some code related to statistics computation, wouldn't it be
> better to fix the issue by catching the NullPointerException to avoid
> blocking Kafka consumption ?
>
> Kind regards,
> Alexandre Vermeerbergen
>