You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Andre Luiz via user <us...@flink.apache.org> on 2023/03/01 13:31:14 UTC
Re: Flink UI showing multiple entries of the same operator on Overview tab.
Hi Weihua Hu,
Thanks for your reply, I found a way of reproducing the problem with the
Flink examples:
1 - cp ./examples/streaming/TopSpeedWindowing.jar lib/
2 - ./bin/standalone-job.sh start -Dscheduler-mode=reactive
-Dexecution.checkpointing.interval="10s" -j
org.apache.flink.streaming.examples.windowing.TopSpeedWindowing
3 - execute 2 or more times ./bin/taskmanager.sh start
If you go to overview of the job it will look like this:
[image: image.png]
Many Thanks,
André Diniz
On Wed, Mar 1, 2023 at 7:27 AM Weihua Hu <hu...@gmail.com> wrote:
> Hi, Andre
>
> I found this ticket[1] similar to your description, and this ticket is
> closed because it was not reproduced.
>
> Could you please upload some detailed jobmanager.log or dump the memory of
> JM?
>
> [1]https://issues.apache.org/jira/browse/FLINK-29852
>
> Best,
> Weihua
>
>
> On Tue, Feb 28, 2023 at 10:54 PM Andre Luiz via user <
> user@flink.apache.org> wrote:
>
>> Hello everyone, how are you?
>>
>> We recently updated Flink from version 1.15.3 to 1.16.1, and we've
>> noticed a strange behavior when visualizing the job's plan in Flink UI.
>> Specifically, multiple entries of the same operator are appearing. We've
>> observed that the number of entries corresponds to the degree of
>> parallelism, and this issue seems to occur when Reactive Mode is enabled in
>> Application mode.
>>
>>
>> I have attached a file containing the JSON response from the API, where
>> you can see in the `plan` section that each operator has multiple entries.
>>
>> Do you happen to know whether this issue is a result of a
>> misconfiguration or a bug in the API?
>>
>> Thanks in advance,
>>
>> André Diniz
>>
>> ------------------------------
>> Confidentiality note: This e-mail may contain confidential information
>> from Nu Holdings Ltd and/or its affiliates. If you have received it by
>> mistake, please let us know by e-mail reply and delete it from your system;
>> you may not copy this message or disclose its contents to anyone; for
>> details about what personal information we collect and why, please refer to
>> our privacy policy
>> <https://api.mziq.com/mzfilemanager/v2/d/59a081d2-0d63-4bb5-b786-4c07ae26bc74/6f4939b9-5f74-a528-1835-596b481dca54>
>> .
>>
>
--
Confidentiality note: This e-mail may contain confidential information
from Nu Holdings Ltd and/or its affiliates. If you have received it by
mistake, please let us know by e-mail reply and delete it from your system;
you may not copy this message or disclose its contents to anyone; for
details about what personal information we collect and why, please refer to
our privacy policy
<https://api.mziq.com/mzfilemanager/v2/d/59a081d2-0d63-4bb5-b786-4c07ae26bc74/6f4939b9-5f74-a528-1835-596b481dca54>.
Re: Flink UI showing multiple entries of the same operator on Overview tab.
Posted by Weihua Hu <hu...@gmail.com>.
Thanks André for bringing the reproducing way.
This is because the adaptive scheduler updates the wrong execution plan.
I'm requesting to re-open this ticket[1] and fix it.
[1] https://issues.apache.org/jira/browse/FLINK-29852
Best,
Weihua
On Wed, Mar 1, 2023 at 9:31 PM Andre Luiz <an...@nubank.com.br> wrote:
> Hi Weihua Hu,
>
> Thanks for your reply, I found a way of reproducing the problem with the
> Flink examples:
>
> 1 - cp ./examples/streaming/TopSpeedWindowing.jar lib/
> 2 - ./bin/standalone-job.sh start -Dscheduler-mode=reactive
> -Dexecution.checkpointing.interval="10s" -j
> org.apache.flink.streaming.examples.windowing.TopSpeedWindowing
> 3 - execute 2 or more times ./bin/taskmanager.sh start
>
> If you go to overview of the job it will look like this:
> [image: image.png]
>
> Many Thanks,
>
> André Diniz
>
> On Wed, Mar 1, 2023 at 7:27 AM Weihua Hu <hu...@gmail.com> wrote:
>
>> Hi, Andre
>>
>> I found this ticket[1] similar to your description, and this ticket is
>> closed because it was not reproduced.
>>
>> Could you please upload some detailed jobmanager.log or dump the memory
>> of JM?
>>
>> [1]https://issues.apache.org/jira/browse/FLINK-29852
>>
>> Best,
>> Weihua
>>
>>
>> On Tue, Feb 28, 2023 at 10:54 PM Andre Luiz via user <
>> user@flink.apache.org> wrote:
>>
>>> Hello everyone, how are you?
>>>
>>> We recently updated Flink from version 1.15.3 to 1.16.1, and we've
>>> noticed a strange behavior when visualizing the job's plan in Flink UI.
>>> Specifically, multiple entries of the same operator are appearing. We've
>>> observed that the number of entries corresponds to the degree of
>>> parallelism, and this issue seems to occur when Reactive Mode is enabled in
>>> Application mode.
>>>
>>>
>>> I have attached a file containing the JSON response from the API, where
>>> you can see in the `plan` section that each operator has multiple entries.
>>>
>>> Do you happen to know whether this issue is a result of a
>>> misconfiguration or a bug in the API?
>>>
>>> Thanks in advance,
>>>
>>> André Diniz
>>>
>>> ------------------------------
>>> Confidentiality note: This e-mail may contain confidential information
>>> from Nu Holdings Ltd and/or its affiliates. If you have received it by
>>> mistake, please let us know by e-mail reply and delete it from your system;
>>> you may not copy this message or disclose its contents to anyone; for
>>> details about what personal information we collect and why, please refer to
>>> our privacy policy
>>> <https://api.mziq.com/mzfilemanager/v2/d/59a081d2-0d63-4bb5-b786-4c07ae26bc74/6f4939b9-5f74-a528-1835-596b481dca54>
>>> .
>>>
>>
> ------------------------------
> Confidentiality note: This e-mail may contain confidential information
> from Nu Holdings Ltd and/or its affiliates. If you have received it by
> mistake, please let us know by e-mail reply and delete it from your system;
> you may not copy this message or disclose its contents to anyone; for
> details about what personal information we collect and why, please refer to
> our privacy policy
> <https://api.mziq.com/mzfilemanager/v2/d/59a081d2-0d63-4bb5-b786-4c07ae26bc74/6f4939b9-5f74-a528-1835-596b481dca54>
> .
>