You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Madan D <ma...@yahoo.com.au> on 2022/04/18 21:28:47 UTC
Adpative scheduler is not recognizing by Flink 1.14.0
Hello Team,
I am trying to use adaptive schedule with flink 1.14 to run flink job based on available resources instead of waiting for required parallelism (scaling) but I don't see flink is getting recognize adaptive schedule.
Ex: flink run -m yarn-cluster -ynm jobName -p 128 -D jobmanager.scheduler=adaptive -D cluster.declarative-resource-management.enabled=true -c className JarName
Reference : Elastic Scaling
|
|
| |
Elastic Scaling
Elastic Scaling # Apache Flink allows you to rescale your jobs. You can do this manually by stopping the job an...
|
|
|
Caused by: java.util.concurrent.CompletionException: java.util.concurrent.CompletionException: org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Slot request bulk is not fulfillable! Could not allocate the required slot within slot request timeout at org.apache.flink.runtime.scheduler.DefaultScheduler.lambda$assignResource$8(DefaultScheduler.java:515) ... 37 moreCaused by: java.util.concurrent.CompletionException: org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Slot request bulk is not fulfillable! Could not allocate the required slot within slot request timeout at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:607) at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591) ... 35 more
Can you please help me on this.
Regards,
Madan
Re: Adpative scheduler is not recognizing by Flink 1.14.0
Posted by Zhanghao Chen <zh...@outlook.com>.
Hi Madan,
The reason might be that the -D parameters are not recognized since you used the old-fashioned YARN CLI command, where you need to add the -y prefix for command options. Use -yD instead of -D or use "flink run -t yarn-per-job -Dyarn.application.name=jobname" instead of "flink run -m yarn-cluster -ynm jobName".
I suggest you allocating enough resources first and lanuching the JM. Then check on the Flink WebUI - JobManager - Configuration page if the configurations are recognized and try again.
Best,
Zhanghao Chen
________________________________
From: Madan D <ma...@yahoo.com.au>
Sent: Tuesday, April 19, 2022 5:28
To: user@flink.apache.org <us...@flink.apache.org>
Subject: Adpative scheduler is not recognizing by Flink 1.14.0
Hello Team,
I am trying to use adaptive schedule with flink 1.14 to run flink job based on available resources instead of waiting for required parallelism (scaling) but I don't see flink is getting recognize adaptive schedule.
Ex: flink run -m yarn-cluster -ynm jobName -p 128 -D jobmanager.scheduler=adaptive
-D cluster.declarative-resource-management.enabled=true -c className JarName
Reference : Elastic Scaling<https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/elastic_scaling/>
<https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/elastic_scaling/>
Elastic Scaling
Elastic Scaling # Apache Flink allows you to rescale your jobs. You can do this manually by stopping the job an...
Caused by: java.util.concurrent.CompletionException: java.util.concurrent.CompletionException: org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Slot request bulk is not fulfillable! Could not allocate the required slot within slot request timeout
at org.apache.flink.runtime.scheduler.DefaultScheduler.lambda$assignResource$8(DefaultScheduler.java:515)
... 37 more
Caused by: java.util.concurrent.CompletionException: org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Slot request bulk is not fulfillable! Could not allocate the required slot within slot request timeout
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:607)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
... 35 more
Can you please help me on this.
Regards,
Madan