You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Jared Stehler <ja...@intellifylearning.com> on 2017/10/27 18:30:33 UTC
Programmatic way to determine required number of slots for a job?
Is there a way to determine via an API the required number of task manager slots for a particular job graph? I’ve looked at "flink.getStreamGraph().getJobGraph().getNumberOfVertices()” but it doesn’t seem to correspond with what the job actually ends up using.
--
Jared Stehler
Chief Architect - Intellify Learning
o: 617.701.6330 x703
Re: Programmatic way to determine required number of slots for a job?
Posted by Till Rohrmann <tr...@apache.org>.
Hi Jared,
I think by adding up the parallelism of the operator with the maximum
degree of parallelism in each slot sharing group, you will end up with the
correct number of required slots (given that all operators have to run
concurrently).
Cheers,
Till
On Fri, Oct 27, 2017 at 8:34 PM, Jared Stehler <
jared.stehler@intellifylearning.com> wrote:
> I found in the “Flip6LocalStreamEnvironment” the following code, but with
> the caveat that it doesn’t include slot reuse?
>
> *int* slotsCount = 0;
> *for* (org.apache.flink.runtime.jobgraph.JobVertex jobVertex :
> flink.getStreamGraph().getJobGraph().getVertices()) {
> slotsCount += jobVertex.getParallelism();
> }
>
> --
> Jared Stehler
> Chief Architect - Intellify Learning
> o: 617.701.6330 x703 <(617)%20701-6330>
>
>
>
> On Oct 27, 2017, at 2:30 PM, Jared Stehler <jared.stehler@
> intellifylearning.com> wrote:
>
> Is there a way to determine via an API the required number of task manager
> slots for a particular job graph? I’ve looked at "flink.getStreamGraph().
> getJobGraph().getNumberOfVertices()” but it doesn’t seem to correspond
> with what the job actually ends up using.
>
> --
> Jared Stehler
> Chief Architect - Intellify Learning
> o: 617.701.6330 x703 <(617)%20701-6330>
>
>
>
>
>
Re: Programmatic way to determine required number of slots for a job?
Posted by Jared Stehler <ja...@intellifylearning.com>.
I found in the “Flip6LocalStreamEnvironment” the following code, but with the caveat that it doesn’t include slot reuse?
int slotsCount = 0;
for (org.apache.flink.runtime.jobgraph.JobVertex jobVertex : flink.getStreamGraph().getJobGraph().getVertices()) {
slotsCount += jobVertex.getParallelism();
}
--
Jared Stehler
Chief Architect - Intellify Learning
o: 617.701.6330 x703
> On Oct 27, 2017, at 2:30 PM, Jared Stehler <ja...@intellifylearning.com> wrote:
>
> Is there a way to determine via an API the required number of task manager slots for a particular job graph? I’ve looked at "flink.getStreamGraph().getJobGraph().getNumberOfVertices()” but it doesn’t seem to correspond with what the job actually ends up using.
>
> --
> Jared Stehler
> Chief Architect - Intellify Learning
> o: 617.701.6330 x703
>
>
>