You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Vitaliy Semochkin <vi...@gmail.com> on 2020/03/23 21:59:26 UTC

usae of ClusterSpecificationBuilder.taskManagerMemoryMB

Hi,

what ClusterSpecificationBuilder.taskManagerMemoryMB is for in flink 1.10?
It's only usage I see is in YarnCluserDescriptor.validateClusterResources
and I do not get the meaning of it.
How is it different from taskmanager.memory.process.size?
And what's the point of having it, if it's not used anywhere?

Regards,
Vitaliy

Re: usae of ClusterSpecificationBuilder.taskManagerMemoryMB

Posted by Vitaliy Semochkin <vi...@gmail.com>.

Got it, thank you very much for the reply.
So far we can not avoid using ClusterSpecification because
 clusterDescriptor.deployJobCluster(clusterSpecification, jobGraph... )
depends on it.

Best Regards,
Vitaliy

On Tue, Mar 24, 2020 at 5:24 AM Xintong Song <to...@gmail.com> wrote:

> Hi Vitality,
>
> After FLIP-49, ClusterSpecification.taskManagerMemoryMB is no longer
> necessary. It can be completely replaced by
> `taskmanager.memory.process.size`. It is kept merely for legacy reasons.
>
> I'm actually thinking about removing ClusterSpecification, maybe after
> finishing FLIP-116 [1], which replaces masterMemoryMB with
> `jobmanager.memory.process.size`. That would also involve refactoring
> YarnClusterDescriptor, which is not in good shape (e.g. the method
> startAppMaster has more than 400 lines) and is closely coupled with
> ClusterSpecification.
>
>
> Thank you~
>
> Xintong Song
>
>
>
> On Tue, Mar 24, 2020 at 5:59 AM Vitaliy Semochkin <vi...@gmail.com>
> wrote:
>
>> Hi,
>>
>> what ClusterSpecificationBuilder.taskManagerMemoryMB is for in flink 1.10?
>> It's only usage I see is in YarnCluserDescriptor.validateClusterResources
>> and I do not get the meaning of it.
>> How is it different from taskmanager.memory.process.size?
>> And what's the point of having it, if it's not used anywhere?
>>
>> Regards,
>> Vitaliy
>>
>

Re: usae of ClusterSpecificationBuilder.taskManagerMemoryMB

Posted by Xintong Song <to...@gmail.com>.

Hi Vitality,

After FLIP-49, ClusterSpecification.taskManagerMemoryMB is no longer
necessary. It can be completely replaced by
`taskmanager.memory.process.size`. It is kept merely for legacy reasons.

I'm actually thinking about removing ClusterSpecification, maybe after
finishing FLIP-116 [1], which replaces masterMemoryMB with
`jobmanager.memory.process.size`. That would also involve refactoring
YarnClusterDescriptor, which is not in good shape (e.g. the method
startAppMaster has more than 400 lines) and is closely coupled with
ClusterSpecification.

Thank you~

Xintong Song

On Tue, Mar 24, 2020 at 5:59 AM Vitaliy Semochkin <vi...@gmail.com>
wrote:

> Hi,
>
> what ClusterSpecificationBuilder.taskManagerMemoryMB is for in flink 1.10?
> It's only usage I see is in YarnCluserDescriptor.validateClusterResources
> and I do not get the meaning of it.
> How is it different from taskmanager.memory.process.size?
> And what's the point of having it, if it's not used anywhere?
>
> Regards,
> Vitaliy
>