Flink operator stuck on created

[Dave Maughan <da...@gmail.com>]

Hi,

I have a Flink job on EMR with an operator stuck on CREATED. The subtasks
are not being assigned to task manager slots. The previous operator is
running and has non-zero Bytes Sent and Records Sent. When the job started
the Job manager requested new workers to start a bunch of the operators but
it's not requesting any more so the available slots is 0 and the job just
seems to have stalled. Any pointers on what I might be doing wrong?

I'm specifying parallelism 24 and that's how many task slots are being
requested by the job manager but also how many subtasks are being created
each operator. Should I (if so how) specify these two numbers separately?

Thanks,
Dave

RE: Flink operator stuck on created

[Schwalbe Matthias <Ma...@viseca.ch>]

Hi Dave,

In batch mode an operator/task only starts running once all input tasks are finished. So without further detail this is perfectly in line with what you describe.

Thias


From: Dave Maughan <da...@gmail.com>
Sent: Montag, 20. September 2021 13:15
To: user@flink.apache.org
Subject: Re: Flink operator stuck on created

I should note - this job is being run in batch mode. Could there be a deadlock related to FLINK-16430?

On Mon, 20 Sept 2021 at 11:26, Dave Maughan <da...@gmail.com>> wrote:
Hi,

I have a Flink job on EMR with an operator stuck on CREATED. The subtasks are not being assigned to task manager slots. The previous operator is running and has non-zero Bytes Sent and Records Sent. When the job started the Job manager requested new workers to start a bunch of the operators but it's not requesting any more so the available slots is 0 and the job just seems to have stalled. Any pointers on what I might be doing wrong?

I'm specifying parallelism 24 and that's how many task slots are being requested by the job manager but also how many subtasks are being created each operator. Should I (if so how) specify these two numbers separately?

Thanks,
Dave
Diese Nachricht ist ausschliesslich für den Adressaten bestimmt und beinhaltet unter Umständen vertrauliche Mitteilungen. Da die Vertraulichkeit von e-Mail-Nachrichten nicht gewährleistet werden kann, übernehmen wir keine Haftung für die Gewährung der Vertraulichkeit und Unversehrtheit dieser Mitteilung. Bei irrtümlicher Zustellung bitten wir Sie um Benachrichtigung per e-Mail und um Löschung dieser Nachricht sowie eventueller Anhänge. Jegliche unberechtigte Verwendung oder Verbreitung dieser Informationen ist streng verboten.

This message is intended only for the named recipient and may contain confidential or privileged information. As the confidentiality of email communication cannot be guaranteed, we do not accept any responsibility for the confidentiality and the intactness of this message. If you have received it in error, please advise the sender by return e-mail and delete this message and any attachments. Any unauthorised use or dissemination of this information is strictly prohibited.

Re: Flink operator stuck on created

[Dave Maughan <da...@gmail.com>]

I should note - this job is being run in batch mode. Could there be a
deadlock related to FLINK-16430?

On Mon, 20 Sept 2021 at 11:26, Dave Maughan <da...@gmail.com> wrote:

> Hi,
>
> I have a Flink job on EMR with an operator stuck on CREATED. The subtasks
> are not being assigned to task manager slots. The previous operator is
> running and has non-zero Bytes Sent and Records Sent. When the job started
> the Job manager requested new workers to start a bunch of the operators but
> it's not requesting any more so the available slots is 0 and the job just
> seems to have stalled. Any pointers on what I might be doing wrong?
>
> I'm specifying parallelism 24 and that's how many task slots are being
> requested by the job manager but also how many subtasks are being created
> each operator. Should I (if so how) specify these two numbers separately?
>
> Thanks,
> Dave
>