You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Xintong Song (Jira)" <ji...@apache.org> on 2020/01/15 10:24:01 UTC

[jira] [Created] (FLINK-15598) Memory accuracy loss in YarnClusterDescriptor may lead to deployment failure.

Xintong Song created FLINK-15598:
------------------------------------

             Summary: Memory accuracy loss in YarnClusterDescriptor may lead to deployment failure.
                 Key: FLINK-15598
                 URL: https://issues.apache.org/jira/browse/FLINK-15598
             Project: Flink
          Issue Type: Bug
          Components: Deployment / YARN
            Reporter: Xintong Song
             Fix For: 1.10.0


Currently, YarnClusterDescriptor parses/derives TM process memory size from configuration, store it in ClusterSpecification and validate ClusterSpecification, then overwrite the memory size back to configuration.

This logic is unnecessary. The memory validation is already covered by creating TaskExecutorResourceSpec from configuration in TaskExecutorResourceUtils.

Moreover, the memory size is stored in MB in ClusterSpecification. The accuracy loss may lead to memory validation failure, which prevent the cluster from being deployed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)