Re: Setting Parallelsim for implicit key extractor operator/NoResourceAvailableException: Not enough free slots available to run the job.

[Fabian Hueske <fh...@gmail.com>]

Hi Regina,

Thanks for reporting this issue!
This is a bug and should be fixed. I've created a JIRA issue [1].

As a workaround, you can set the default parallelism of the
ExecutionEnvironment:

env.setParallelism(2);

The generated operator will then have the configured default parallelism.

Best,
Fabian

[1] https://issues.apache.org/jira/browse/FLINK-9289

2018-05-01 0:27 GMT+02:00 Chan, Regina <Re...@gs.com>:

> Hi,
>
>
>
> I’m currently running multiple jobs in the same yarn session and
> controlling how I submit jobs based off what my calculated parallelism
> should be. For example, I have 4 TMs, 2 slots, so I should be able run
> multiple parallelism 1 jobs at the same time up to 8. However, what I’m
> seeing is that the implicit key extractor that’s going into the cogroup has
> parallelism 8 instead of having the same parallelism of the cogroup 1. This
> causes the other jobs to fail with:
>
>
>
> org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException:
> Not enough free slots available to run the job. You can decrease the
> operator parallelism or increase the number of slots per TaskManager in the
> configuration.
>
> This isn’t an issue when it chains but is one when the key extractor
> becomes its own operator. Is there a way I can force the key extractor to
> have the correct parallelism? What’s the best way around this?
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *Regina Chan*
>
> *Goldman Sachs* *–* Enterprise Platforms, Data Architecture
>
> *30 Hudson Street, 37th floor | Jersey City, NY 07302
> <https://maps.google.com/?q=30+Hudson+Street,+37th+floor+%7C+Jersey+City,+NY+07302&entry=gmail&source=g>*
> (  (212) 902-5697
>
>
>