You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2019/12/06 15:09:08 UTC
[GitHub] [flink] tillrohrmann commented on a change in pull request #10462:
[FLINK-15031][runtime] Calculate required shuffle memory before allocating
slots if resources are specified
tillrohrmann commented on a change in pull request #10462: [FLINK-15031][runtime] Calculate required shuffle memory before allocating slots if resources are specified
URL: https://github.com/apache/flink/pull/10462#discussion_r354855860
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/shuffle/NettyShuffleMaster.java
##########
@@ -32,12 +35,18 @@
*/
public class NettyShuffleMaster implements ShuffleMaster<NettyShuffleDescriptor> {
+ private final int networkBuffersPerInputChannel;
+
+ private final int networkBufferSize;
+
@VisibleForTesting
public NettyShuffleMaster() {
this(new Configuration());
}
public NettyShuffleMaster(final Configuration configuration) {
+ networkBuffersPerInputChannel = configuration.getInteger(NettyShuffleEnvironmentOptions.NETWORK_BUFFERS_PER_CHANNEL);
+ networkBufferSize = ConfigurationParserUtils.getPageSize(configuration);
Review comment:
What we could do is to introduce two static methods `int getNetworkBuffersPerInputChannel(Configuration)` and `int getNetworkBufferSize(Configuration)` which we use here and in the `NettyShuffleEnvironmentConfiguration.fromConfiguration()`. That way there is only a single place where we access the `NETWORK_BUFFERS_PER_CHANNEL` config constant.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services