[Storm 2.0.1] topologies are in danger of being unscheduled

[Pavithra Gunasekara <th...@gmail.com>]

Hi Folks,

We are in the process of upgrading Storm version from 1.0.3 to 2.0.1

We were able to build topology jars with version 2.0.1 and bring up the
Storm cluster, so far 2 topologies are running in Dev environment.

However, we see a new owner section in Storm UI, which shows following
message in a red banner.

This user's topologies are in danger of being unscheduled due to the
owner's over-use of cluster resources.
Please keep this user's resource consumption within guaranteed bounds to
ensure topologies for this user will continue to run.

[image: Screen Shot 2020-04-09 at 4.54.49 PM.png]
Could someone please help us understand why message appears? Is it required
to specify memory/cpu usage for each user(in our case root)? If yes, what
should be the way to do that?

Thanks

-- 
-Pavithra

Re: [Storm 2.0.1] topologies are in danger of being unscheduled

[Pavithra Gunasekara <th...@gmail.com>]

Thanks Ethan for the explanation.

Regards,
-Pavithra

On Thu, Apr 9, 2020 at 7:13 PM Ethan Li <et...@gmail.com> wrote:

> With resource aware scheduler being used, the scheduler is aware of
> user resource quota and requested resources by topologies.
>
> https://github.com/apache/storm/blob/master/docs/Resource_Aware_Scheduler_overview.md
>
> If the requested resources by the submitted topologies is larger than the
> user quota (in your case, probably it's not configured), UI will show a red
> banner. This is fine if you are not using ResourceAwareScheduler.
>
>
> Best,
> Ethan
>
> On Thu, Apr 9, 2020 at 7:12 PM Pavithra Gunasekara <
> thilini.gunasekara@gmail.com> wrote:
>
>> Hi Folks,
>>
>> We are in the process of upgrading Storm version from 1.0.3 to 2.0.1
>>
>> We were able to build topology jars with version 2.0.1 and bring up the
>> Storm cluster, so far 2 topologies are running in Dev environment.
>>
>> However, we see a new owner section in Storm UI, which shows following
>> message in a red banner.
>>
>> This user's topologies are in danger of being unscheduled due to the
>> owner's over-use of cluster resources.
>> Please keep this user's resource consumption within guaranteed bounds to
>> ensure topologies for this user will continue to run.
>>
>> [image: Screen Shot 2020-04-09 at 4.54.49 PM.png]
>> Could someone please help us understand why message appears? Is it
>> required to specify memory/cpu usage for each user(in our case root)? If
>> yes, what should be the way to do that?
>>
>> Thanks
>>
>> --
>> -Pavithra
>>
> --
-Pavithra

Re: [Storm 2.0.1] topologies are in danger of being unscheduled

[Ethan Li <et...@gmail.com>]

With resource aware scheduler being used, the scheduler is aware of
user resource quota and requested resources by topologies.
https://github.com/apache/storm/blob/master/docs/Resource_Aware_Scheduler_overview.md

If the requested resources by the submitted topologies is larger than the
user quota (in your case, probably it's not configured), UI will show a red
banner. This is fine if you are not using ResourceAwareScheduler.


Best,
Ethan

On Thu, Apr 9, 2020 at 7:12 PM Pavithra Gunasekara <
thilini.gunasekara@gmail.com> wrote:

> Hi Folks,
>
> We are in the process of upgrading Storm version from 1.0.3 to 2.0.1
>
> We were able to build topology jars with version 2.0.1 and bring up the
> Storm cluster, so far 2 topologies are running in Dev environment.
>
> However, we see a new owner section in Storm UI, which shows following
> message in a red banner.
>
> This user's topologies are in danger of being unscheduled due to the
> owner's over-use of cluster resources.
> Please keep this user's resource consumption within guaranteed bounds to
> ensure topologies for this user will continue to run.
>
> [image: Screen Shot 2020-04-09 at 4.54.49 PM.png]
> Could someone please help us understand why message appears? Is it
> required to specify memory/cpu usage for each user(in our case root)? If
> yes, what should be the way to do that?
>
> Thanks
>
> --
> -Pavithra
>