You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@twill.apache.org by Keith Turner <ke...@deenlo.com> on 2016/06/14 17:59:41 UTC
Help tracking down an issue.
At runtime I changes my dependencies and I started getting an error in
Twill. I was seeing an NPE at the following line.
org.apache.twill.yarn.YarnTwillRunnerService.getApplicationId(YarnTwillRunnerService.java:542)
~[twill-yarn-0.6.0-incubating.jar:0.6.0-incubating]
Through zookeeper and twill logging, I tracked the cause down to the
following data being in zookeeper.
The following path :
/fluo-app-phrasecount/instances/2e0b9fe4-6e1c-4ecc-a454-d821e8453cb9
had the following data :
{"members":{"data":{"appId":1,"appIdClusterTime":1465862775942,"containerId":"container_1465862775942_0001_01_000001"}}}
When I run with a set of dependencies that works, I see the following data
in zookeeper. This has a root of data in the json as opposed to the root
of members.
{"data":{"appId":1,"appIdClusterTime":1465866930685,"containerId":"container_1465866930685_0001_01_000001"}}
Could someone help me locate the code that sets the value of
/fluo-app-phrasecount/instances/2e0b9fe4-6e1c-4ecc-a454-d821e8453cb9 in
zookeeper? If I can locate that code, maybe I can understand why changing
my deps changed the value in ZK to have a root of members.
Thanks,
Keith
Re: Help tracking down an issue.
Posted by Terence Yim <ch...@gmail.com>.
Hi Keith,
Glad that it helps. It's probably a bug that they fixed between 2.2.2 and
2.2.4, although I don't see it get mentioned in the release note.
Terence
On Wed, Jun 15, 2016 at 10:18 AM, Keith Turner <ke...@deenlo.com> wrote:
> On Wed, Jun 15, 2016 at 1:21 AM, Terence Yim <ch...@gmail.com> wrote:
>
> > Hi Keith,
> >
> > It is constructed in here:
> >
> >
> >
> https://github.com/apache/incubator-twill/blob/master/twill-core/src/main/java/org/apache/twill/internal/AbstractTwillService.java#L194
> >
> > Basically it is explicitly constructed with a JsonObject (from Gson
> > library) that has a "data" member. I don't see how it can have the
> "member"
> > level added. Is it caused by a different Gson version? If that's the
> case,
> > what version of Gson are you using?
> >
>
> It was caused by a different version of gson. Finally tracked it down t
> GSon version 2.2.2 causing the problem. When I forced Gson 2.2.4 onto the
> classpath, everything worked fine.
>
> Copying some of the code you referenced, I was able to write standalone
> code that would end up with members in the json when using 2.2.2.
>
> Thanks for the feedback, it was very helpful in tracking this down.
>
>
> >
> > Terence
> >
> > On Tue, Jun 14, 2016 at 10:59 AM, Keith Turner <ke...@deenlo.com> wrote:
> >
> > > At runtime I changes my dependencies and I started getting an error in
> > > Twill. I was seeing an NPE at the following line.
> > >
> > >
> > >
> >
> org.apache.twill.yarn.YarnTwillRunnerService.getApplicationId(YarnTwillRunnerService.java:542)
> > >
> > > ~[twill-yarn-0.6.0-incubating.jar:0.6.0-incubating]
> > >
> > >
> > > Through zookeeper and twill logging, I tracked the cause down to the
> > > following data being in zookeeper.
> > >
> > > The following path :
> > >
> > > /fluo-app-phrasecount/instances/2e0b9fe4-6e1c-4ecc-a454-d821e8453cb9
> > >
> > > had the following data :
> > >
> > >
> >
> {"members":{"data":{"appId":1,"appIdClusterTime":1465862775942,"containerId":"container_1465862775942_0001_01_000001"}}}
> > >
> > >
> > > When I run with a set of dependencies that works, I see the following
> > data
> > > in zookeeper. This has a root of data in the json as opposed to the
> root
> > > of members.
> > >
> > >
> > >
> >
> {"data":{"appId":1,"appIdClusterTime":1465866930685,"containerId":"container_1465866930685_0001_01_000001"}}
> > >
> > >
> > > Could someone help me locate the code that sets the value of
> > > /fluo-app-phrasecount/instances/2e0b9fe4-6e1c-4ecc-a454-d821e8453cb9 in
> > > zookeeper? If I can locate that code, maybe I can understand why
> > changing
> > > my deps changed the value in ZK to have a root of members.
> > >
> > > Thanks,
> > >
> > > Keith
> > >
> >
>
Re: Help tracking down an issue.
Posted by Keith Turner <ke...@deenlo.com>.
On Wed, Jun 15, 2016 at 1:21 AM, Terence Yim <ch...@gmail.com> wrote:
> Hi Keith,
>
> It is constructed in here:
>
>
> https://github.com/apache/incubator-twill/blob/master/twill-core/src/main/java/org/apache/twill/internal/AbstractTwillService.java#L194
>
> Basically it is explicitly constructed with a JsonObject (from Gson
> library) that has a "data" member. I don't see how it can have the "member"
> level added. Is it caused by a different Gson version? If that's the case,
> what version of Gson are you using?
>
It was caused by a different version of gson. Finally tracked it down t
GSon version 2.2.2 causing the problem. When I forced Gson 2.2.4 onto the
classpath, everything worked fine.
Copying some of the code you referenced, I was able to write standalone
code that would end up with members in the json when using 2.2.2.
Thanks for the feedback, it was very helpful in tracking this down.
>
> Terence
>
> On Tue, Jun 14, 2016 at 10:59 AM, Keith Turner <ke...@deenlo.com> wrote:
>
> > At runtime I changes my dependencies and I started getting an error in
> > Twill. I was seeing an NPE at the following line.
> >
> >
> >
> org.apache.twill.yarn.YarnTwillRunnerService.getApplicationId(YarnTwillRunnerService.java:542)
> >
> > ~[twill-yarn-0.6.0-incubating.jar:0.6.0-incubating]
> >
> >
> > Through zookeeper and twill logging, I tracked the cause down to the
> > following data being in zookeeper.
> >
> > The following path :
> >
> > /fluo-app-phrasecount/instances/2e0b9fe4-6e1c-4ecc-a454-d821e8453cb9
> >
> > had the following data :
> >
> >
> {"members":{"data":{"appId":1,"appIdClusterTime":1465862775942,"containerId":"container_1465862775942_0001_01_000001"}}}
> >
> >
> > When I run with a set of dependencies that works, I see the following
> data
> > in zookeeper. This has a root of data in the json as opposed to the root
> > of members.
> >
> >
> >
> {"data":{"appId":1,"appIdClusterTime":1465866930685,"containerId":"container_1465866930685_0001_01_000001"}}
> >
> >
> > Could someone help me locate the code that sets the value of
> > /fluo-app-phrasecount/instances/2e0b9fe4-6e1c-4ecc-a454-d821e8453cb9 in
> > zookeeper? If I can locate that code, maybe I can understand why
> changing
> > my deps changed the value in ZK to have a root of members.
> >
> > Thanks,
> >
> > Keith
> >
>
Re: Help tracking down an issue.
Posted by Terence Yim <ch...@gmail.com>.
Hi Keith,
It is constructed in here:
https://github.com/apache/incubator-twill/blob/master/twill-core/src/main/java/org/apache/twill/internal/AbstractTwillService.java#L194
Basically it is explicitly constructed with a JsonObject (from Gson
library) that has a "data" member. I don't see how it can have the "member"
level added. Is it caused by a different Gson version? If that's the case,
what version of Gson are you using?
Terence
On Tue, Jun 14, 2016 at 10:59 AM, Keith Turner <ke...@deenlo.com> wrote:
> At runtime I changes my dependencies and I started getting an error in
> Twill. I was seeing an NPE at the following line.
>
>
> org.apache.twill.yarn.YarnTwillRunnerService.getApplicationId(YarnTwillRunnerService.java:542)
>
> ~[twill-yarn-0.6.0-incubating.jar:0.6.0-incubating]
>
>
> Through zookeeper and twill logging, I tracked the cause down to the
> following data being in zookeeper.
>
> The following path :
>
> /fluo-app-phrasecount/instances/2e0b9fe4-6e1c-4ecc-a454-d821e8453cb9
>
> had the following data :
>
> {"members":{"data":{"appId":1,"appIdClusterTime":1465862775942,"containerId":"container_1465862775942_0001_01_000001"}}}
>
>
> When I run with a set of dependencies that works, I see the following data
> in zookeeper. This has a root of data in the json as opposed to the root
> of members.
>
>
> {"data":{"appId":1,"appIdClusterTime":1465866930685,"containerId":"container_1465866930685_0001_01_000001"}}
>
>
> Could someone help me locate the code that sets the value of
> /fluo-app-phrasecount/instances/2e0b9fe4-6e1c-4ecc-a454-d821e8453cb9 in
> zookeeper? If I can locate that code, maybe I can understand why changing
> my deps changed the value in ZK to have a root of members.
>
> Thanks,
>
> Keith
>