You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Adrien Carreira <ac...@reportlinker.com> on 2017/01/11 16:29:59 UTC

Bug in MetricConsumer

Hi Guys,

I'm running a topology on 3 supervisor. I've registred a custom
MetricsConsumer class that output metrics on Elasticsearch.

My bug is that the taskInfo.srcWorkerHost that always the same values as
the first supervisor (nimbus as well), Even when I put a parallelismHint to
3...

I'm using storm 1.0.2

Thanks for your feedback

Re: Bug in MetricConsumer

Posted by Jungtaek Lim <ka...@gmail.com>.
Just filed https://issues.apache.org/jira/browse/STORM-2293 and submitted a
patch for master, 1.x, 1.0.x branches.

I manually tested this so I think the patch will fix your problem, but I'd
be still appreciated when you can verify my patch.

- Jungtaek Lim (HeartSaVioR)

2017년 1월 13일 (금) 오후 6:44, Jungtaek Lim <ka...@gmail.com>님이 작성:

> Cool :)
>
> Btw, I've craft a patch for this, and will submit a pull request after you
> filed to issue. It should be great if you could help me verifying the patch
> if you really don't mind.
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> 2017년 1월 13일 (금) 오후 5:33, Adrien Carreira <ac...@reportlinker.com>님이 작성:
>
> I've set up storm.local.hostname on storm.yaml that is used for nimbus
> and supervisor. I've just removed this one, restarted nimbus and all
> supervisor and it's working :D
>
> I will submit a bug :-)
>
> 2017-01-12 17:54 GMT+01:00 Jungtaek Lim <ka...@gmail.com>:
>
> Did you set up storm.local.hostname only for storm clusters, or also the
> node you're submitting jar from?
>
> Anyway I think this is not intended behavior (a bug).
> Could you file an issue to JIRA?
> https://issues.apache.org/jira/browse/STORM
> Please let me know if you want me to file this instead.
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> 2017년 1월 13일 (금) 오전 1:36, Adrien Carreira <ac...@reportlinker.com>님이 작성:
>
> You are right, My topology config contains :
> storm.local.hostname
>
> "crwl-agent1"
>
>
> But why ? I haven't set up this value, just in each storm.yaml, because
> either I see localhost insteed of the hostname...
>
> 2017-01-12 10:26 GMT+01:00 Jungtaek Lim <ka...@gmail.com>:
>
> Hi Adrien,
>
> could you check topology config has 'storm.local.hostname'?
> srcWorkerHost uses the value of 'storm.local.hostname' if available. It's
> intended to read that value for each node but there might be edge case like
> reading topology config instead of node's config value.
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> 2017년 1월 12일 (목) 오후 5:24, Adrien Carreira <ac...@reportlinker.com>님이 작성:
>
> Nope :/
>
>
>
> 2017-01-11 18:28 GMT+01:00 Jacob Johansen <jo...@gmail.com>:
>
>
>
> Do you have the hostname on all the supervisors machines named the same?
>
> On January 11, 2017 at 10:30:03 AM, Adrien Carreira (aca@reportlinker.com)
> wrote:
>
> Hi Guys,
>
> I'm running a topology on 3 supervisor. I've registred a custom
> MetricsConsumer class that output metrics on Elasticsearch.
>
> My bug is that the taskInfo.srcWorkerHost that always the same values as
> the first supervisor (nimbus as well), Even when I put a parallelismHint to
> 3...
>
> I'm using storm 1.0.2
>
> Thanks for your feedback
>
>
>
>
>

Re: Bug in MetricConsumer

Posted by Jungtaek Lim <ka...@gmail.com>.
Cool :)

Btw, I've craft a patch for this, and will submit a pull request after you
filed to issue. It should be great if you could help me verifying the patch
if you really don't mind.

Thanks,
Jungtaek Lim (HeartSaVioR)

2017년 1월 13일 (금) 오후 5:33, Adrien Carreira <ac...@reportlinker.com>님이 작성:

I've set up storm.local.hostname on storm.yaml that is used for nimbus and
supervisor. I've just removed this one, restarted nimbus and all supervisor
and it's working :D

I will submit a bug :-)

2017-01-12 17:54 GMT+01:00 Jungtaek Lim <ka...@gmail.com>:

Did you set up storm.local.hostname only for storm clusters, or also the
node you're submitting jar from?

Anyway I think this is not intended behavior (a bug).
Could you file an issue to JIRA? https://issues.apache.org/jira/browse/STORM
Please let me know if you want me to file this instead.

Thanks,
Jungtaek Lim (HeartSaVioR)

2017년 1월 13일 (금) 오전 1:36, Adrien Carreira <ac...@reportlinker.com>님이 작성:

You are right, My topology config contains :
storm.local.hostname

"crwl-agent1"


But why ? I haven't set up this value, just in each storm.yaml, because
either I see localhost insteed of the hostname...

2017-01-12 10:26 GMT+01:00 Jungtaek Lim <ka...@gmail.com>:

Hi Adrien,

could you check topology config has 'storm.local.hostname'?
srcWorkerHost uses the value of 'storm.local.hostname' if available. It's
intended to read that value for each node but there might be edge case like
reading topology config instead of node's config value.

Thanks,
Jungtaek Lim (HeartSaVioR)

2017년 1월 12일 (목) 오후 5:24, Adrien Carreira <ac...@reportlinker.com>님이 작성:

Nope :/



2017-01-11 18:28 GMT+01:00 Jacob Johansen <jo...@gmail.com>:



Do you have the hostname on all the supervisors machines named the same?

On January 11, 2017 at 10:30:03 AM, Adrien Carreira (aca@reportlinker.com)
wrote:

Hi Guys,

I'm running a topology on 3 supervisor. I've registred a custom
MetricsConsumer class that output metrics on Elasticsearch.

My bug is that the taskInfo.srcWorkerHost that always the same values as
the first supervisor (nimbus as well), Even when I put a parallelismHint to
3...

I'm using storm 1.0.2

Thanks for your feedback

Re: Bug in MetricConsumer

Posted by Adrien Carreira <ac...@reportlinker.com>.
I've set up storm.local.hostname on storm.yaml that is used for nimbus and
supervisor. I've just removed this one, restarted nimbus and all supervisor
and it's working :D

I will submit a bug :-)

2017-01-12 17:54 GMT+01:00 Jungtaek Lim <ka...@gmail.com>:

> Did you set up storm.local.hostname only for storm clusters, or also the
> node you're submitting jar from?
>
> Anyway I think this is not intended behavior (a bug).
> Could you file an issue to JIRA? https://issues.apache.org/
> jira/browse/STORM
> Please let me know if you want me to file this instead.
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> 2017년 1월 13일 (금) 오전 1:36, Adrien Carreira <ac...@reportlinker.com>님이 작성:
>
>> You are right, My topology config contains :
>> storm.local.hostname
>>
>> "crwl-agent1"
>>
>>
>> But why ? I haven't set up this value, just in each storm.yaml, because
>> either I see localhost insteed of the hostname...
>>
>> 2017-01-12 10:26 GMT+01:00 Jungtaek Lim <ka...@gmail.com>:
>>
>> Hi Adrien,
>>
>> could you check topology config has 'storm.local.hostname'?
>> srcWorkerHost uses the value of 'storm.local.hostname' if available. It's
>> intended to read that value for each node but there might be edge case like
>> reading topology config instead of node's config value.
>>
>> Thanks,
>> Jungtaek Lim (HeartSaVioR)
>>
>> 2017년 1월 12일 (목) 오후 5:24, Adrien Carreira <ac...@reportlinker.com>님이 작성:
>>
>> Nope :/
>>
>>
>>
>> 2017-01-11 18:28 GMT+01:00 Jacob Johansen <jo...@gmail.com>:
>>
>>
>>
>> Do you have the hostname on all the supervisors machines named the same?
>>
>> On January 11, 2017 at 10:30:03 AM, Adrien Carreira (aca@reportlinker.com)
>> wrote:
>>
>> Hi Guys,
>>
>> I'm running a topology on 3 supervisor. I've registred a custom
>> MetricsConsumer class that output metrics on Elasticsearch.
>>
>> My bug is that the taskInfo.srcWorkerHost that always the same values as
>> the first supervisor (nimbus as well), Even when I put a parallelismHint to
>> 3...
>>
>> I'm using storm 1.0.2
>>
>> Thanks for your feedback
>>
>>
>>
>>

Re: Bug in MetricConsumer

Posted by Jungtaek Lim <ka...@gmail.com>.
Did you set up storm.local.hostname only for storm clusters, or also the
node you're submitting jar from?

Anyway I think this is not intended behavior (a bug).
Could you file an issue to JIRA? https://issues.apache.org/jira/browse/STORM
Please let me know if you want me to file this instead.

Thanks,
Jungtaek Lim (HeartSaVioR)

2017년 1월 13일 (금) 오전 1:36, Adrien Carreira <ac...@reportlinker.com>님이 작성:

> You are right, My topology config contains :
> storm.local.hostname
>
> "crwl-agent1"
>
>
> But why ? I haven't set up this value, just in each storm.yaml, because
> either I see localhost insteed of the hostname...
>
> 2017-01-12 10:26 GMT+01:00 Jungtaek Lim <ka...@gmail.com>:
>
> Hi Adrien,
>
> could you check topology config has 'storm.local.hostname'?
> srcWorkerHost uses the value of 'storm.local.hostname' if available. It's
> intended to read that value for each node but there might be edge case like
> reading topology config instead of node's config value.
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> 2017년 1월 12일 (목) 오후 5:24, Adrien Carreira <ac...@reportlinker.com>님이 작성:
>
> Nope :/
>
>
>
> 2017-01-11 18:28 GMT+01:00 Jacob Johansen <jo...@gmail.com>:
>
>
>
> Do you have the hostname on all the supervisors machines named the same?
>
> On January 11, 2017 at 10:30:03 AM, Adrien Carreira (aca@reportlinker.com)
> wrote:
>
> Hi Guys,
>
> I'm running a topology on 3 supervisor. I've registred a custom
> MetricsConsumer class that output metrics on Elasticsearch.
>
> My bug is that the taskInfo.srcWorkerHost that always the same values as
> the first supervisor (nimbus as well), Even when I put a parallelismHint to
> 3...
>
> I'm using storm 1.0.2
>
> Thanks for your feedback
>
>
>
>

Re: Bug in MetricConsumer

Posted by Adrien Carreira <ac...@reportlinker.com>.
You are right, My topology config contains :
storm.local.hostname

"crwl-agent1"


But why ? I haven't set up this value, just in each storm.yaml, because
either I see localhost insteed of the hostname...

2017-01-12 10:26 GMT+01:00 Jungtaek Lim <ka...@gmail.com>:

> Hi Adrien,
>
> could you check topology config has 'storm.local.hostname'?
> srcWorkerHost uses the value of 'storm.local.hostname' if available. It's
> intended to read that value for each node but there might be edge case like
> reading topology config instead of node's config value.
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> 2017년 1월 12일 (목) 오후 5:24, Adrien Carreira <ac...@reportlinker.com>님이 작성:
>
>> Nope :/
>>
>>
>>
>> 2017-01-11 18:28 GMT+01:00 Jacob Johansen <jo...@gmail.com>:
>>
>>
>>
>> Do you have the hostname on all the supervisors machines named the same?
>>
>> On January 11, 2017 at 10:30:03 AM, Adrien Carreira (aca@reportlinker.com)
>> wrote:
>>
>> Hi Guys,
>>
>> I'm running a topology on 3 supervisor. I've registred a custom
>> MetricsConsumer class that output metrics on Elasticsearch.
>>
>> My bug is that the taskInfo.srcWorkerHost that always the same values as
>> the first supervisor (nimbus as well), Even when I put a parallelismHint to
>> 3...
>>
>> I'm using storm 1.0.2
>>
>> Thanks for your feedback
>>
>>
>>

Re: Bug in MetricConsumer

Posted by Jungtaek Lim <ka...@gmail.com>.
Hi Adrien,

could you check topology config has 'storm.local.hostname'?
srcWorkerHost uses the value of 'storm.local.hostname' if available. It's
intended to read that value for each node but there might be edge case like
reading topology config instead of node's config value.

Thanks,
Jungtaek Lim (HeartSaVioR)

2017년 1월 12일 (목) 오후 5:24, Adrien Carreira <ac...@reportlinker.com>님이 작성:

> Nope :/
>
>
>
> 2017-01-11 18:28 GMT+01:00 Jacob Johansen <jo...@gmail.com>:
>
>
>
> Do you have the hostname on all the supervisors machines named the same?
>
> On January 11, 2017 at 10:30:03 AM, Adrien Carreira (aca@reportlinker.com)
> wrote:
>
> Hi Guys,
>
> I'm running a topology on 3 supervisor. I've registred a custom
> MetricsConsumer class that output metrics on Elasticsearch.
>
> My bug is that the taskInfo.srcWorkerHost that always the same values as
> the first supervisor (nimbus as well), Even when I put a parallelismHint to
> 3...
>
> I'm using storm 1.0.2
>
> Thanks for your feedback
>
>
>

Re: Bug in MetricConsumer

Posted by Adrien Carreira <ac...@reportlinker.com>.
Nope :/



2017-01-11 18:28 GMT+01:00 Jacob Johansen <jo...@gmail.com>:

>
>
> Do you have the hostname on all the supervisors machines named the same?
>
> On January 11, 2017 at 10:30:03 AM, Adrien Carreira (aca@reportlinker.com)
> wrote:
>
> Hi Guys,
>
> I'm running a topology on 3 supervisor. I've registred a custom
> MetricsConsumer class that output metrics on Elasticsearch.
>
> My bug is that the taskInfo.srcWorkerHost that always the same values as
> the first supervisor (nimbus as well), Even when I put a parallelismHint to
> 3...
>
> I'm using storm 1.0.2
>
> Thanks for your feedback
>
>

Re: Bug in MetricConsumer

Posted by Jacob Johansen <jo...@gmail.com>.
Do you have the hostname on all the supervisors machines named the same?

On January 11, 2017 at 10:30:03 AM, Adrien Carreira (aca@reportlinker.com)
wrote:

Hi Guys,

I'm running a topology on 3 supervisor. I've registred a custom
MetricsConsumer class that output metrics on Elasticsearch.

My bug is that the taskInfo.srcWorkerHost that always the same values as
the first supervisor (nimbus as well), Even when I put a parallelismHint to
3...

I'm using storm 1.0.2

Thanks for your feedback