You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/03/02 13:27:06 UTC
[GitHub] [flink] xintongsong opened a new pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
xintongsong opened a new pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284
## What is the purpose of the change
This PR makes Flink work with NAT by introducing separated configuration options for address/port and bind-address/bind-port of JM/TM RPC services and Netty Shuffle Service.
## Brief change log
- 592a79c26d047920e5d7ba20ba99c01c926797ec: Minor code clean-ups.
- c7f8860864951b33aef27404b1666f0648a26f8b: Refactor to create `AkkaRpsService` with builder class. Currently there are too many nested methods for creating a `AkkaRpsService`, making it hard to understand which changes affect which code paths.
- 24fb21896bc3bc95532c925057cca5031a1721a3: Separate config options for JM/TM hostname/bind-hostname and RPC port/bind-port.
- 00808b0c3814cf7bd028b069e5b7022e92569c18: Separate config options for Netty Shuffle Service data port/bind-port.
- 26db673d7bf7505bd46186fe2c69ffb11c4f76f9: Introduce e2e test for Flink over NAT.
- 4683aaeae4a1ee29a403bf6495f2065c539b224f: **DO_NOT_MERGE_THIS** Add the NAT e2e test to pre-commit tests, for the convenience of review.
## Verifying this change
Added a docker-based e2e test for verifying Flink works over NAT.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (yes)
- If yes, how is the feature documented? (not documented)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395435948
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -351,21 +351,21 @@ public static TaskExecutor startTaskManager(
LOG.info("Starting TaskManager with ResourceID: {}", resourceID);
- InetAddress remoteAddress = InetAddress.getByName(rpcService.getAddress());
+ InetAddress externalAddress = InetAddress.getByName(rpcService.getAddress());
Review comment:
True. I'll try to fix the problem and update the e2e test case.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395430171
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/ClusterEntrypoint.java
##########
@@ -248,10 +247,14 @@ protected void initializeServices(Configuration configuration) throws Exception
LOG.info("Initializing cluster services.");
synchronized (lock) {
- final String bindAddress = configuration.getString(JobManagerOptions.ADDRESS);
- final String portRange = getRPCPortRange(configuration);
-
- commonRpcService = createRpcService(configuration, bindAddress, portRange);
+ commonRpcService = AkkaRpcServiceUtils.createRemoteRpcService(
+ configuration,
+ configuration.getString(JobManagerOptions.ADDRESS),
+ getRPCPortRange(configuration),
+ configuration.getString(JobManagerOptions.BIND_HOST),
+ configuration.contains(JobManagerOptions.RPC_BIND_PORT) ?
+ configuration.getInteger(JobManagerOptions.RPC_BIND_PORT) :
+ null);
Review comment:
Same here. I also think using Optional will express the contract better. The question is do we need to strictly follow the code style guide.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386769074
##########
File path: flink-end-to-end-tests/test-scripts/container-scripts/docker-compose.nat.yml
##########
@@ -0,0 +1,66 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+# Docker compose file for a Flink job cluster deployment with NAT network.
+#
+# Parameters:
+# * FLINK_DOCKER_IMAGE_NAME - Image name to use for the deployment (default: flink-job:latest)
+# * FLINK_JOB - Name of the Flink job to execute (default: none)
+# * FLINK_JOB_ARGUMENTS - Additional arguments which will be passed to the job cluster (default: none)
+#
+# * HOST_ADDR - Address of the docker host. Will be used for JM/TM external addresses.
+# * JM_RPC_EX_PORT - External RPC port for JM.
+# * JM_RPC_IN_PORT - Internal RPC port for JM.
+# * TM_1_RPC_EX_PORT - External RPC port for TM 1.
+# * TM_2_RPC_EX_PORT - External RPC port for TM 2.
+# * TM_RPC_IN_PORT - Internal RPC port for both TMs.
+# * TM_1_DATA_EX_PORT - External data port for TM 1.
+# * TM_2_DATA_EX_PORT - External data port for TM 2.
+# * TM_DATA_IN_PORT - Internal data port for both TMs.
Review comment:
Minor: We may also list `INPUT_VOLUME` and `OUTPUT_VOLUME` here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386807612
##########
File path: flink-end-to-end-tests/test-scripts/test_nat.sh
##########
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+source "$(dirname "$0")"/common.sh
+
+DOCKER_MODULE_DIR=${END_TO_END_DIR}/../flink-container/docker
+DOCKER_SCRIPTS=${END_TO_END_DIR}/test-scripts/container-scripts
+DOCKER_IMAGE_BUILD_RETRIES=3
+BUILD_BACKOFF_TIME=5
+
+export FLINK_JOB=org.apache.flink.examples.java.wordcount.WordCount
+export FLINK_DOCKER_IMAGE_NAME=test_nat
+export INPUT_VOLUME=${END_TO_END_DIR}/test-scripts/test-data
+export OUTPUT_VOLUME=${TEST_DATA_DIR}/out
+export INPUT_PATH=/data/test/input
+export OUTPUT_PATH=/data/test/output
+
+export HOST_ADDR=$(get_node_ip | awk '{print $1}')
+export JM_RPC_EX_PORT=10000
+export JM_RPC_IN_PORT=10000
+export TM_1_RPC_EX_PORT=10001
+export TM_2_RPC_EX_PORT=10002
+export TM_RPC_IN_PORT=10000
+export TM_1_DATA_EX_PORT=11001
+export TM_2_DATA_EX_PORT=11002
+export TM_DATA_IN_PORT=11000
+
+RESULT_HASH="72a690412be8928ba239c2da967328a5"
+INPUT_ARGS="--input ${INPUT_PATH}/words"
+
+export FLINK_JOB_ARGUMENTS="${INPUT_ARGS} --output ${OUTPUT_PATH}/docker_wc_out"
+
+build_image() {
+ ./build.sh --from-local-dist --job-artifacts ${FLINK_DIR}/examples/batch/WordCount.jar --image-name ${FLINK_DOCKER_IMAGE_NAME}
+}
+
+# user inside the container must be able to create files, this is a workaround in-container permissions
+mkdir -p $OUTPUT_VOLUME
+chmod 777 $OUTPUT_VOLUME
Review comment:
Turns out `666` does not work. Flink throws the following error:
> Output directory '/data/test/output/docker_wc_out' could not be created.
And I found from google that:
> For directories, execute permission allows you to enter the directory (i.e., cd into it), and to access any of its files. http://web.mit.edu/sipb/doc/working/afs/html/subsection3.1.html
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #11284: [FLINK-15911][runtime]
Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot commented on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* aa1386ad6c2ed2774d659263d7684e777eaccd2d UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386801810
##########
File path: flink-end-to-end-tests/test-scripts/test_nat.sh
##########
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+source "$(dirname "$0")"/common.sh
+
+DOCKER_MODULE_DIR=${END_TO_END_DIR}/../flink-container/docker
+DOCKER_SCRIPTS=${END_TO_END_DIR}/test-scripts/container-scripts
+DOCKER_IMAGE_BUILD_RETRIES=3
+BUILD_BACKOFF_TIME=5
+
+export FLINK_JOB=org.apache.flink.examples.java.wordcount.WordCount
+export FLINK_DOCKER_IMAGE_NAME=test_nat
+export INPUT_VOLUME=${END_TO_END_DIR}/test-scripts/test-data
+export OUTPUT_VOLUME=${TEST_DATA_DIR}/out
+export INPUT_PATH=/data/test/input
+export OUTPUT_PATH=/data/test/output
+
+export HOST_ADDR=$(get_node_ip | awk '{print $1}')
+export JM_RPC_EX_PORT=10000
+export JM_RPC_IN_PORT=10000
+export TM_1_RPC_EX_PORT=10001
+export TM_2_RPC_EX_PORT=10002
+export TM_RPC_IN_PORT=10000
+export TM_1_DATA_EX_PORT=11001
+export TM_2_DATA_EX_PORT=11002
+export TM_DATA_IN_PORT=11000
+
+RESULT_HASH="72a690412be8928ba239c2da967328a5"
+INPUT_ARGS="--input ${INPUT_PATH}/words"
+
+export FLINK_JOB_ARGUMENTS="${INPUT_ARGS} --output ${OUTPUT_PATH}/docker_wc_out"
+
+build_image() {
+ ./build.sh --from-local-dist --job-artifacts ${FLINK_DIR}/examples/batch/WordCount.jar --image-name ${FLINK_DOCKER_IMAGE_NAME}
+}
+
+# user inside the container must be able to create files, this is a workaround in-container permissions
+mkdir -p $OUTPUT_VOLUME
+chmod 777 $OUTPUT_VOLUME
Review comment:
It does not infect the correctness of this test case. My point is we need to be careful when setting the permission of the directory mount in the container, in case anything happens unexpectedly while mutely.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395428193
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/JobManagerOptions.java
##########
@@ -54,6 +54,16 @@
" leader-election service (like ZooKeeper) is used to elect and discover the JobManager" +
" leader from potentially multiple standby JobManagers.");
+ /**
+ * The local address of the network interface that the job manager binds to.
+ */
+ public static final ConfigOption<String> BIND_HOST =
+ key("jobmanager.bind-host")
Review comment:
This address is not only used by the RPC service, but also shared by the blob server, rest server, and potentially any other service in future that needs to bind to the network interface.
To that end, I think the config option for TM ("taskmanager.host") makes more sense. I guess that might be the reason we use "host" in the config key at the first place.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395426615
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniCluster.java
##########
@@ -261,26 +258,30 @@ public void start() throws Exception {
// bring up all the RPC services
LOG.info("Starting RPC Service(s)");
- AkkaRpcServiceConfiguration akkaRpcServiceConfig = AkkaRpcServiceConfiguration.fromConfiguration(configuration);
-
final RpcServiceFactory dispatcherResourceManagreComponentRpcServiceFactory;
if (useSingleRpcService) {
// we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, false, null);
+ commonRpcService = createLocalRpcService(configuration);
final CommonRpcServiceFactory commonRpcServiceFactory = new CommonRpcServiceFactory(commonRpcService);
taskManagerRpcServiceFactory = commonRpcServiceFactory;
dispatcherResourceManagreComponentRpcServiceFactory = commonRpcServiceFactory;
} else {
- // we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, true, null);
// start a new service per component, possibly with custom bind addresses
final String jobManagerBindAddress = miniClusterConfiguration.getJobManagerBindAddress();
final String taskManagerBindAddress = miniClusterConfiguration.getTaskManagerBindAddress();
+ final String jobManagerBindPort = miniClusterConfiguration.getJobManagerBindPortRange();
Review comment:
No, and I don't think we should.
Whenever the user needs to specify a bind port apart form the external port, the user also needs to set port forwarding rules in the environment, e.g. NAT gateway. For each external port, it can only be forwarded to one internal port. Therefore, it does not make sense to support a range of bind ports, because the user needs to know which internal port is bound anyway.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395425038
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
##########
@@ -82,92 +82,80 @@
.build();
/**
- * Starts an ActorSystem with the given configuration listening at the address/ports.
+ * Starts a remote ActorSystem at given address and specific port range.
* @param configuration The Flink configuration
- * @param listeningAddress The address to listen at.
- * @param portRangeDefinition The port range to choose a port from.
+ * @param externalAddress The external address to access the ActorSystem.
+ * @param externalPortRange The choosing range of the external port to access the ActorSystem.
* @param logger The logger to output log information.
* @return The ActorSystem which has been started
* @throws Exception Thrown when actor system cannot be started in specified port range
*/
- public static ActorSystem startActorSystem(
+ @VisibleForTesting
+ public static ActorSystem startRemoteActorSystem(
Configuration configuration,
- String listeningAddress,
- String portRangeDefinition,
+ String externalAddress,
+ String externalPortRange,
Logger logger) throws Exception {
- return startActorSystem(
- configuration,
- listeningAddress,
- portRangeDefinition,
- logger,
- ForkJoinExecutorConfiguration.fromConfiguration(configuration));
- }
-
- /**
- * Starts an ActorSystem with the given configuration listening at the address/ports.
- *
- * @param configuration The Flink configuration
- * @param listeningAddress The address to listen at.
- * @param portRangeDefinition The port range to choose a port from.
- * @param logger The logger to output log information.
- * @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
- * @return The ActorSystem which has been started
- * @throws Exception Thrown when actor system cannot be started in specified port range
- */
- public static ActorSystem startActorSystem(
- Configuration configuration,
- String listeningAddress,
- String portRangeDefinition,
- Logger logger,
- @Nonnull ActorSystemExecutorConfiguration actorSystemExecutorConfiguration) throws Exception {
- return startActorSystem(
+ return startRemoteActorSystem(
configuration,
AkkaUtils.getFlinkActorSystemName(),
- listeningAddress,
- portRangeDefinition,
+ externalAddress,
+ externalPortRange,
+ NetUtils.getWildcardIPAddress(),
+ -1,
logger,
- actorSystemExecutorConfiguration);
+ ForkJoinExecutorConfiguration.fromConfiguration(configuration),
+ null);
}
/**
- * Starts an ActorSystem with the given configuration listening at the address/ports.
+ * Starts a remote ActorSystem at given address and specific port range.
*
* @param configuration The Flink configuration
* @param actorSystemName Name of the started {@link ActorSystem}
- * @param listeningAddress The address to listen at.
- * @param portRangeDefinition The port range to choose a port from.
+ * @param externalAddress The external address to access the ActorSystem.
+ * @param externalPortRange The choosing range of the external port to access the ActorSystem.
+ * @param bindAddress The local address to bind to.
+ * @param bindPort The local port to bind to. If negative, external port will be used.
* @param logger The logger to output log information.
* @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
+ * @param customConfig Custom Akka config to be combined with the config derived from Flink configuration.
* @return The ActorSystem which has been started
* @throws Exception Thrown when actor system cannot be started in specified port range
*/
- public static ActorSystem startActorSystem(
+ public static ActorSystem startRemoteActorSystem(
Configuration configuration,
String actorSystemName,
- String listeningAddress,
- String portRangeDefinition,
+ String externalAddress,
+ String externalPortRange,
+ String bindAddress,
+ int bindPort,
Review comment:
Actually, I also thought about making the bindPort Optional.
However I also noticed that the code style guide suggests to use Optional only for return values of API/public methods, but not for class fields and method arguments.
Not sure if it's ok to not strictly following the guide. We do have a good reason to use Optional here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-602522424
@tillrohrmann
I'm ok with supporting the scenarios where address of `t` can be resolved by all other `t'` except for itself, and have updated the PR to address this issue.After all, it is indeed an improvement that we are able to support more use cases without introducing any problem to exist ones.
It's just how tricky I have to simulate this scenario in the e2e test that makes me suspect something might be wrong. Or put it another way, why users want to use Flink like this? Have we truly understood what they need?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395528291
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##########
@@ -94,19 +94,28 @@
" or if it has been quarantined by another actor system.");
/**
- * The config parameter defining the task manager's hostname.
+ * The external address of the network interface where the TaskManager is exposed.
* Overrides {@link #HOST_BIND_POLICY} automatic address binding.
*/
@Documentation.Section({Documentation.Sections.COMMON_HOST_PORT, Documentation.Sections.ALL_TASK_MANAGER})
public static final ConfigOption<String> HOST =
key("taskmanager.host")
.stringType()
.noDefaultValue()
- .withDescription("The address of the network interface that the TaskManager binds to." +
- " This option can be used to define explicitly a binding address. Because" +
- " different TaskManagers need different values for this option, usually it is specified in an" +
+ .withDescription("The external address of the network interface where the TaskManager is exposed." +
+ " Because different TaskManagers need different values for this option, usually it is specified in an" +
" additional non-shared TaskManager-specific config file.");
+ /**
+ * The local address of the network interface that the task manager binds to.
+ */
+ public static final ConfigOption<String> BIND_HOST =
+ key("taskmanager.bind-host")
Review comment:
ok this makes sense.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395528589
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/ClusterEntrypoint.java
##########
@@ -248,10 +247,14 @@ protected void initializeServices(Configuration configuration) throws Exception
LOG.info("Initializing cluster services.");
synchronized (lock) {
- final String bindAddress = configuration.getString(JobManagerOptions.ADDRESS);
- final String portRange = getRPCPortRange(configuration);
-
- commonRpcService = createRpcService(configuration, bindAddress, portRange);
+ commonRpcService = AkkaRpcServiceUtils.createRemoteRpcService(
+ configuration,
+ configuration.getString(JobManagerOptions.ADDRESS),
+ getRPCPortRange(configuration),
+ configuration.getString(JobManagerOptions.BIND_HOST),
+ configuration.contains(JobManagerOptions.RPC_BIND_PORT) ?
+ configuration.getInteger(JobManagerOptions.RPC_BIND_PORT) :
+ null);
Review comment:
Since it is an internal method, we can diverge a bit from the code style here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396554259
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerServicesConfiguration.java
##########
@@ -235,8 +235,8 @@ public static TaskManagerServicesConfiguration fromConfiguration(
final int externalDataPort = configuration.getInteger(NettyShuffleEnvironmentOptions.DATA_PORT);
- String bindAddr = configuration.getString(TaskManagerOptions.BIND_HOST);
- InetAddress bindAddress = bindAddr == null ? externalAddress : InetAddress.getByName(bindAddr);
+ String bindAddr = configuration.getString(TaskManagerOptions.BIND_HOST, externalAddress);
+ InetAddress bindAddress = InetAddress.getByName(bindAddr);
Review comment:
Wouldn't this try to resolve `externalAddress` if `TaskManagerOptions#BIND_HOST` has not been configured. In this case, we should bind to `0.0.0.0`, right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395464715
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -351,21 +351,21 @@ public static TaskExecutor startTaskManager(
LOG.info("Starting TaskManager with ResourceID: {}", resourceID);
- InetAddress remoteAddress = InetAddress.getByName(rpcService.getAddress());
+ InetAddress externalAddress = InetAddress.getByName(rpcService.getAddress());
Review comment:
After looking more into this, I'm not entirely sure whether we should support unresolvable external addresses.
The prerequisites are unclear to me. If an external address cannot be resolved by the TaskManager itself, then where can it be resolved? Do we assume all other components (JobManager and other TaskManagers) except for itself can resolve the external address? That sounds like a strong assumption to me, which may not always be true. E.g., deploying a Flink cluster over machines from two different sub-network, each sub-network has multiple machines/TMs. TMs in the same sub-network might not be able to resolve each other's external address. Another example, multiple TMs on the same machine.
I don't see a good way to cover all such cases. If we can only support the special case where external addresses can be resolved by all other components, I'm not sure whether we should do it given the efforts needed. Alternatively, user might use IP addresses to workaround in such cases.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wangyang0918 commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396259803
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -351,21 +351,21 @@ public static TaskExecutor startTaskManager(
LOG.info("Starting TaskManager with ResourceID: {}", resourceID);
- InetAddress remoteAddress = InetAddress.getByName(rpcService.getAddress());
+ InetAddress externalAddress = InetAddress.getByName(rpcService.getAddress());
Review comment:
@tillrohrmann @xintongsong
FYI
I think we do not have the requirement that other `TMs` need to be able to resolve a `TM's ` address. For current standalone Flink cluster on K8s, all the `TMs` could not resolve other's hostname. However, it works pretty well. This is the case for current master code without external address.
When introducing the external address, i think it is same that we do not require other `TMs` need to be able to resolve a `TM's ` address.
Correct me if i am wrong somewhere.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "PENDING",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151875334",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6e0a577481f9300c87299b0629fab6dc1b3bd71a Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/151494400) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853)
* 3e9df27458782312e53586e53f49cd55c11f4df5 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/151875334) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann closed pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann closed pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395199512
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/JobManagerOptions.java
##########
@@ -54,6 +54,16 @@
" leader-election service (like ZooKeeper) is used to elect and discover the JobManager" +
" leader from potentially multiple standby JobManagers.");
+ /**
+ * The local address of the network interface that the job manager binds to.
+ */
+ public static final ConfigOption<String> BIND_HOST =
+ key("jobmanager.bind-host")
Review comment:
Would it be more consistent to name it `jobmanager.rpc.bind-address`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396554778
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskmanager/TaskManagerLocation.java
##########
@@ -90,6 +93,14 @@ public TaskManagerLocation(ResourceID resourceID, InetAddress inetAddress, int d
"%s @ %s (dataPort=%d)", resourceID, fqdnHostName, dataPort);
}
+ public static TaskManagerLocation fromUnresolvedLocation(final TaskManagerUnresolvedLocation unresolvedLocation)
+ throws UnknownHostException {
+ return new TaskManagerLocation(
+ unresolvedLocation.getResourceID(),
+ InetAddress.getByName(unresolvedLocation.getExternalAddress()),
+ unresolvedLocation.getDataPort());
+ }
Review comment:
Nice :-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395185710
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
##########
@@ -181,104 +169,115 @@ public static ActorSystem startActorSystem(
// if we come here, we have exhausted the port range
throw new BindException("Could not start actor system on any port in port range "
- + portRangeDefinition);
+ + externalPortRange);
}
/**
- * Starts an Actor System at a specific port.
- *
+ * Starts a remote Actor System at given address and specific port.
* @param configuration The Flink configuration.
- * @param listeningAddress The address to listen at.
- * @param listeningPort The port to listen at.
+ * @param actorSystemName Name of the started {@link ActorSystem}
+ * @param externalAddress The external address to access the ActorSystem.
+ * @param externalPort The external port to access the ActorSystem.
+ * @param bindAddress The local address to bind to.
+ * @param bindPort The local port to bind to.
* @param logger the logger to output log information.
+ * @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
+ * @param customConfig Custom Akka config to be combined with the config derived from Flink configuration.
* @return The ActorSystem which has been started.
* @throws Exception
*/
- public static ActorSystem startActorSystem(
+ private static ActorSystem startRemoteActorSystem(
Configuration configuration,
- String listeningAddress,
- int listeningPort,
- Logger logger) throws Exception {
- return startActorSystem(
- configuration,
- listeningAddress,
- listeningPort,
- logger,
- ForkJoinExecutorConfiguration.fromConfiguration(configuration));
- }
+ String actorSystemName,
+ String externalAddress,
+ int externalPort,
+ String bindAddress,
+ int bindPort,
+ Logger logger,
+ ActorSystemExecutorConfiguration actorSystemExecutorConfiguration,
+ Config customConfig) throws Exception {
- /**
- * Starts an Actor System at a specific port.
- * @param configuration The Flink configuration.
- * @param listeningAddress The address to listen at.
- * @param listeningPort The port to listen at.
- * @param logger the logger to output log information.
- * @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
- * @return The ActorSystem which has been started.
- * @throws Exception
- */
- public static ActorSystem startActorSystem(
- Configuration configuration,
- String listeningAddress,
- int listeningPort,
- Logger logger,
- ActorSystemExecutorConfiguration actorSystemExecutorConfiguration) throws Exception {
- return startActorSystem(
- configuration,
- AkkaUtils.getFlinkActorSystemName(),
- listeningAddress,
- listeningPort,
- logger,
- actorSystemExecutorConfiguration);
+ String externalHostPortUrl = NetUtils.unresolvedHostAndPortToNormalizedString(externalAddress, externalPort);
+ String bindHostPortUrl = NetUtils.unresolvedHostAndPortToNormalizedString(bindAddress, bindPort);
+ logger.info("Trying to start actor system, external address {}, bind address {}.", externalHostPortUrl, bindHostPortUrl);
+
+ try {
+ Config akkaConfig = AkkaUtils.getAkkaConfig(
+ configuration,
+ new Some<>(new Tuple2<>(externalAddress, externalPort)),
+ new Some<>(new Tuple2<>(bindAddress, bindPort)),
+ actorSystemExecutorConfiguration.getAkkaConfig());
+
+ if (customConfig != null) {
+ akkaConfig = customConfig.withFallback(akkaConfig);
+ }
+
+ return startActorSystem(akkaConfig, actorSystemName, logger);
+ }
+ catch (Throwable t) {
+ if (t instanceof ChannelException) {
+ Throwable cause = t.getCause();
+ if (cause != null && t.getCause() instanceof BindException) {
+ throw new IOException("Unable to create ActorSystem at address " + externalHostPortUrl +
+ " : " + cause.getMessage(), t);
+ }
+ }
+ throw new Exception("Could not create actor system", t);
+ }
}
/**
- * Starts an Actor System at a specific port.
+ * Starts a local Actor System.
* @param configuration The Flink configuration.
- * @param actorSystemName Name of the started {@link ActorSystem}
- * @param listeningAddress The address to listen at.
- * @param listeningPort The port to listen at.
- * @param logger the logger to output log information.
- * @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
+ * @param actorSystemName Name of the started ActorSystem.
+ * @param logger The logger to output log information.
+ * @param actorSystemExecutorConfiguration Configuration for the ActorSystem's underlying executor.
+ * @param customConfig Custom Akka config to be combined with the config derived from Flink configuration.
* @return The ActorSystem which has been started.
* @throws Exception
*/
- public static ActorSystem startActorSystem(
+ public static ActorSystem startLocalActorSystem(
Configuration configuration,
String actorSystemName,
- String listeningAddress,
- int listeningPort,
Logger logger,
- ActorSystemExecutorConfiguration actorSystemExecutorConfiguration) throws Exception {
+ ActorSystemExecutorConfiguration actorSystemExecutorConfiguration,
+ Config customConfig) throws Exception {
- String hostPortUrl = NetUtils.unresolvedHostAndPortToNormalizedString(listeningAddress, listeningPort);
- logger.info("Trying to start actor system at {}", hostPortUrl);
+ logger.info("Trying to start local actor");
try {
Config akkaConfig = AkkaUtils.getAkkaConfig(
configuration,
- new Some<>(new Tuple2<>(listeningAddress, listeningPort)),
+ scala.Option.apply(null),
+ scala.Option.apply(null),
Review comment:
```suggestion
scala.Option.empty(),
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396518337
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
##########
@@ -82,92 +83,80 @@
.build();
/**
- * Starts an ActorSystem with the given configuration listening at the address/ports.
+ * Starts a remote ActorSystem at given address and specific port range.
* @param configuration The Flink configuration
- * @param listeningAddress The address to listen at.
- * @param portRangeDefinition The port range to choose a port from.
+ * @param externalAddress The external address to access the ActorSystem.
+ * @param externalPortRange The choosing range of the external port to access the ActorSystem.
* @param logger The logger to output log information.
* @return The ActorSystem which has been started
* @throws Exception Thrown when actor system cannot be started in specified port range
*/
- public static ActorSystem startActorSystem(
+ @VisibleForTesting
+ public static ActorSystem startRemoteActorSystem(
Configuration configuration,
- String listeningAddress,
- String portRangeDefinition,
+ String externalAddress,
+ String externalPortRange,
Logger logger) throws Exception {
- return startActorSystem(
- configuration,
- listeningAddress,
- portRangeDefinition,
- logger,
- ForkJoinExecutorConfiguration.fromConfiguration(configuration));
- }
-
- /**
- * Starts an ActorSystem with the given configuration listening at the address/ports.
- *
- * @param configuration The Flink configuration
- * @param listeningAddress The address to listen at.
- * @param portRangeDefinition The port range to choose a port from.
- * @param logger The logger to output log information.
- * @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
- * @return The ActorSystem which has been started
- * @throws Exception Thrown when actor system cannot be started in specified port range
- */
- public static ActorSystem startActorSystem(
- Configuration configuration,
- String listeningAddress,
- String portRangeDefinition,
- Logger logger,
- @Nonnull ActorSystemExecutorConfiguration actorSystemExecutorConfiguration) throws Exception {
- return startActorSystem(
+ return startRemoteActorSystem(
configuration,
AkkaUtils.getFlinkActorSystemName(),
- listeningAddress,
- portRangeDefinition,
+ externalAddress,
+ externalPortRange,
+ NetUtils.getWildcardIPAddress(),
+ Optional.empty(),
logger,
- actorSystemExecutorConfiguration);
+ ForkJoinExecutorConfiguration.fromConfiguration(configuration),
+ null);
}
/**
- * Starts an ActorSystem with the given configuration listening at the address/ports.
+ * Starts a remote ActorSystem at given address and specific port range.
*
* @param configuration The Flink configuration
* @param actorSystemName Name of the started {@link ActorSystem}
- * @param listeningAddress The address to listen at.
- * @param portRangeDefinition The port range to choose a port from.
+ * @param externalAddress The external address to access the ActorSystem.
+ * @param externalPortRange The choosing range of the external port to access the ActorSystem.
+ * @param bindAddress The local address to bind to.
+ * @param bindPort The local port to bind to. If negative, external port will be used.
Review comment:
```suggestion
* @param bindPort The local port to bind to. If not present, then the external port will be used
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396985385
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/KvStateService.java
##########
@@ -181,7 +181,7 @@ public static KvStateService fromConfiguration(TaskManagerServicesConfiguration
int numStateServerQueryThreads = qsConfig.numStateQueryThreads() == 0 ?
taskManagerServicesConfiguration.getNumberOfSlots() : qsConfig.numStateQueryThreads();
kvStateServer = QueryableStateUtils.createKvStateServer(
Review comment:
Sounds good.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395185640
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
##########
@@ -181,104 +169,115 @@ public static ActorSystem startActorSystem(
// if we come here, we have exhausted the port range
throw new BindException("Could not start actor system on any port in port range "
- + portRangeDefinition);
+ + externalPortRange);
}
/**
- * Starts an Actor System at a specific port.
- *
+ * Starts a remote Actor System at given address and specific port.
* @param configuration The Flink configuration.
- * @param listeningAddress The address to listen at.
- * @param listeningPort The port to listen at.
+ * @param actorSystemName Name of the started {@link ActorSystem}
+ * @param externalAddress The external address to access the ActorSystem.
+ * @param externalPort The external port to access the ActorSystem.
+ * @param bindAddress The local address to bind to.
+ * @param bindPort The local port to bind to.
* @param logger the logger to output log information.
+ * @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
+ * @param customConfig Custom Akka config to be combined with the config derived from Flink configuration.
* @return The ActorSystem which has been started.
* @throws Exception
*/
- public static ActorSystem startActorSystem(
+ private static ActorSystem startRemoteActorSystem(
Configuration configuration,
- String listeningAddress,
- int listeningPort,
- Logger logger) throws Exception {
- return startActorSystem(
- configuration,
- listeningAddress,
- listeningPort,
- logger,
- ForkJoinExecutorConfiguration.fromConfiguration(configuration));
- }
+ String actorSystemName,
+ String externalAddress,
+ int externalPort,
+ String bindAddress,
+ int bindPort,
+ Logger logger,
+ ActorSystemExecutorConfiguration actorSystemExecutorConfiguration,
+ Config customConfig) throws Exception {
- /**
- * Starts an Actor System at a specific port.
- * @param configuration The Flink configuration.
- * @param listeningAddress The address to listen at.
- * @param listeningPort The port to listen at.
- * @param logger the logger to output log information.
- * @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
- * @return The ActorSystem which has been started.
- * @throws Exception
- */
- public static ActorSystem startActorSystem(
- Configuration configuration,
- String listeningAddress,
- int listeningPort,
- Logger logger,
- ActorSystemExecutorConfiguration actorSystemExecutorConfiguration) throws Exception {
- return startActorSystem(
- configuration,
- AkkaUtils.getFlinkActorSystemName(),
- listeningAddress,
- listeningPort,
- logger,
- actorSystemExecutorConfiguration);
+ String externalHostPortUrl = NetUtils.unresolvedHostAndPortToNormalizedString(externalAddress, externalPort);
+ String bindHostPortUrl = NetUtils.unresolvedHostAndPortToNormalizedString(bindAddress, bindPort);
+ logger.info("Trying to start actor system, external address {}, bind address {}.", externalHostPortUrl, bindHostPortUrl);
+
+ try {
+ Config akkaConfig = AkkaUtils.getAkkaConfig(
+ configuration,
+ new Some<>(new Tuple2<>(externalAddress, externalPort)),
+ new Some<>(new Tuple2<>(bindAddress, bindPort)),
+ actorSystemExecutorConfiguration.getAkkaConfig());
+
+ if (customConfig != null) {
+ akkaConfig = customConfig.withFallback(akkaConfig);
+ }
+
+ return startActorSystem(akkaConfig, actorSystemName, logger);
+ }
+ catch (Throwable t) {
+ if (t instanceof ChannelException) {
+ Throwable cause = t.getCause();
+ if (cause != null && t.getCause() instanceof BindException) {
+ throw new IOException("Unable to create ActorSystem at address " + externalHostPortUrl +
+ " : " + cause.getMessage(), t);
+ }
+ }
+ throw new Exception("Could not create actor system", t);
+ }
}
/**
- * Starts an Actor System at a specific port.
+ * Starts a local Actor System.
* @param configuration The Flink configuration.
- * @param actorSystemName Name of the started {@link ActorSystem}
- * @param listeningAddress The address to listen at.
- * @param listeningPort The port to listen at.
- * @param logger the logger to output log information.
- * @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
+ * @param actorSystemName Name of the started ActorSystem.
+ * @param logger The logger to output log information.
+ * @param actorSystemExecutorConfiguration Configuration for the ActorSystem's underlying executor.
+ * @param customConfig Custom Akka config to be combined with the config derived from Flink configuration.
* @return The ActorSystem which has been started.
* @throws Exception
*/
- public static ActorSystem startActorSystem(
+ public static ActorSystem startLocalActorSystem(
Configuration configuration,
String actorSystemName,
- String listeningAddress,
- int listeningPort,
Logger logger,
- ActorSystemExecutorConfiguration actorSystemExecutorConfiguration) throws Exception {
+ ActorSystemExecutorConfiguration actorSystemExecutorConfiguration,
+ Config customConfig) throws Exception {
- String hostPortUrl = NetUtils.unresolvedHostAndPortToNormalizedString(listeningAddress, listeningPort);
- logger.info("Trying to start actor system at {}", hostPortUrl);
+ logger.info("Trying to start local actor");
try {
Config akkaConfig = AkkaUtils.getAkkaConfig(
configuration,
- new Some<>(new Tuple2<>(listeningAddress, listeningPort)),
+ scala.Option.apply(null),
Review comment:
```suggestion
scala.Option.empty(),
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151875334",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154513428",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6492",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6507",
"triggerID" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"triggerType" : "PUSH"
}, {
"hash" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154538899",
"triggerID" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0be3fd11e11d708b254036e9e91fc71f1fd02181 Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/154513428) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6492)
* ff670f742762f45214ca32df966e02f4dbe6f758 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/154538899) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6507)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396556851
##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/jobmaster/JobMasterPartitionReleaseTest.java
##########
@@ -186,7 +186,7 @@ AllocationID takeAllocationId() {
private final TemporaryFolder temporaryFolder = new TemporaryFolder();
- private final LocalTaskManagerLocation taskManagerLocation = new LocalTaskManagerLocation();
+ private final LocalTaskManagerUnresolvedLocation localTaskManagerUnresolvedLocation = new LocalTaskManagerUnresolvedLocation();
Review comment:
If we rename `TaskManagerUnresolvedLocation` into `UnresolvedTaskManagerLocation`, then I would suggest to rename `LocalTaskManagerUnresolvedLocation` into `LocalUnresolvedTaskManagerLocation` as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395544062
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -351,21 +351,21 @@ public static TaskExecutor startTaskManager(
LOG.info("Starting TaskManager with ResourceID: {}", resourceID);
- InetAddress remoteAddress = InetAddress.getByName(rpcService.getAddress());
+ InetAddress externalAddress = InetAddress.getByName(rpcService.getAddress());
Review comment:
I thought that this is actually the problem we are trying to solve with this PR. Assume the `TM` is running in a Docker container which has an external name which is not resolvable from within the container. If all other Flink components can resolve this name, then it would work if the `TM` itself would not try to resolve the address.
At the moment we have the requirement that all `TM` components can resolve their own and the addresses of the other `TMs`. I think this requirement is too strict. What we actually need is that for a given `TM` `t` all other `TMs` `t'` can resolve the address of `t`.
Of course, this means that if the user deploys multiple `TMs` into the same network where they cannot resolve each others external names, then Flink would not work. But this is ok because it violates the requirement that other `TMs` need to be able to resolve a `TM's` address. It is then the user's responsibility to configure the cluster accordingly.
Moreover, one could still enforce the current model by configuring the `bind-host` to the external address. If this is the case, then a `TM` would only start if it can resolve its own address.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-601641483
True, I've responded two the other comments. I would suggest a simple renaming of the methods.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396984737
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/jobmaster/JobMaster.java
##########
@@ -571,9 +572,16 @@ private void releaseEmptyTaskManager(ResourceID resourceId) {
@Override
public CompletableFuture<RegistrationResponse> registerTaskManager(
final String taskManagerRpcAddress,
- final TaskManagerLocation taskManagerLocation,
+ final TaskManagerUnresolvedLocation taskManagerUnresolvedLocation,
final Time timeout) {
+ final TaskManagerLocation taskManagerLocation;
+ try {
+ taskManagerLocation = TaskManagerLocation.fromUnresolvedLocation(taskManagerUnresolvedLocation);
Review comment:
I guess it should be fine since the `TM` establishes the connection. But this is hairsplitting from my side. I'm good with keeping it as is.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396546590
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/jobmaster/JobMaster.java
##########
@@ -571,9 +572,16 @@ private void releaseEmptyTaskManager(ResourceID resourceId) {
@Override
public CompletableFuture<RegistrationResponse> registerTaskManager(
final String taskManagerRpcAddress,
- final TaskManagerLocation taskManagerLocation,
+ final TaskManagerUnresolvedLocation taskManagerUnresolvedLocation,
final Time timeout) {
+ final TaskManagerLocation taskManagerLocation;
+ try {
+ taskManagerLocation = TaskManagerLocation.fromUnresolvedLocation(taskManagerUnresolvedLocation);
+ } catch (Throwable throwable) {
+ return CompletableFuture.completedFuture(new RegistrationResponse.Decline(throwable.getMessage()));
Review comment:
I would add a log message saying that we could not accept the registration of a `TaskManager` because its address could not be resolved.
Maybe we could add this to the `Decline` message as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-601629677
Thanks for the clarification. It makes sense to me now :-)
Concerning the external address resolution on the `TaskManager`, I've responded to the respective comment. I think we need to clarify what we want to achieve with this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386771202
##########
File path: flink-end-to-end-tests/test-scripts/test_nat.sh
##########
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+source "$(dirname "$0")"/common.sh
+
+DOCKER_MODULE_DIR=${END_TO_END_DIR}/../flink-container/docker
+DOCKER_SCRIPTS=${END_TO_END_DIR}/test-scripts/container-scripts
+DOCKER_IMAGE_BUILD_RETRIES=3
+BUILD_BACKOFF_TIME=5
+
+export FLINK_JOB=org.apache.flink.examples.java.wordcount.WordCount
+export FLINK_DOCKER_IMAGE_NAME=test_nat
+export INPUT_VOLUME=${END_TO_END_DIR}/test-scripts/test-data
+export OUTPUT_VOLUME=${TEST_DATA_DIR}/out
+export INPUT_PATH=/data/test/input
+export OUTPUT_PATH=/data/test/output
+
+export HOST_ADDR=$(get_node_ip | awk '{print $1}')
+export JM_RPC_EX_PORT=10000
+export JM_RPC_IN_PORT=10000
+export TM_1_RPC_EX_PORT=10001
+export TM_2_RPC_EX_PORT=10002
+export TM_RPC_IN_PORT=10000
+export TM_1_DATA_EX_PORT=11001
+export TM_2_DATA_EX_PORT=11002
+export TM_DATA_IN_PORT=11000
+
+RESULT_HASH="72a690412be8928ba239c2da967328a5"
+INPUT_ARGS="--input ${INPUT_PATH}/words"
+
+export FLINK_JOB_ARGUMENTS="${INPUT_ARGS} --output ${OUTPUT_PATH}/docker_wc_out"
+
+build_image() {
+ ./build.sh --from-local-dist --job-artifacts ${FLINK_DIR}/examples/batch/WordCount.jar --image-name ${FLINK_DOCKER_IMAGE_NAME}
+}
+
+# user inside the container must be able to create files, this is a workaround in-container permissions
+mkdir -p $OUTPUT_VOLUME
+chmod 777 $OUTPUT_VOLUME
Review comment:
Why we need execution permission here? I think 666 is enough at the moment.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395435813
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniClusterConfiguration.java
##########
@@ -102,21 +102,21 @@ public int getNumTaskManagers() {
public String getJobManagerBindAddress() {
return commonBindAddress != null ?
commonBindAddress :
- configuration.getString(JobManagerOptions.ADDRESS, "localhost");
+ configuration.getString(JobManagerOptions.BIND_HOST, "localhost");
}
public String getTaskManagerBindAddress() {
return commonBindAddress != null ?
commonBindAddress :
- configuration.getString(TaskManagerOptions.HOST, "localhost");
+ configuration.getString(TaskManagerOptions.BIND_HOST, "localhost");
}
- public String getJobManagerBindPortRange() {
- return String.valueOf(configuration.getInteger(JobManagerOptions.PORT, 0));
+ public int getJobManagerBindPort() {
+ return configuration.getInteger(JobManagerOptions.RPC_BIND_PORT, 0);
}
- public String getTaskManagerBindPortRange() {
- return configuration.getString(TaskManagerOptions.RPC_PORT);
+ public int getTaskManagerBindPort() {
+ return configuration.getInteger(TaskManagerOptions.RPC_BIND_PORT, 0);
Review comment:
Not sure if I fully understand your concern.
I think the RPC services are meant to be used by Flink components internally. Then how would user be affected if the configured address is not used internally?
I have no problem preserving the external address/ports if configured. Just trying to understand.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386790513
##########
File path: flink-end-to-end-tests/test-scripts/test_nat.sh
##########
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+source "$(dirname "$0")"/common.sh
+
+DOCKER_MODULE_DIR=${END_TO_END_DIR}/../flink-container/docker
+DOCKER_SCRIPTS=${END_TO_END_DIR}/test-scripts/container-scripts
+DOCKER_IMAGE_BUILD_RETRIES=3
+BUILD_BACKOFF_TIME=5
+
+export FLINK_JOB=org.apache.flink.examples.java.wordcount.WordCount
+export FLINK_DOCKER_IMAGE_NAME=test_nat
+export INPUT_VOLUME=${END_TO_END_DIR}/test-scripts/test-data
+export OUTPUT_VOLUME=${TEST_DATA_DIR}/out
+export INPUT_PATH=/data/test/input
+export OUTPUT_PATH=/data/test/output
+
+export HOST_ADDR=$(get_node_ip | awk '{print $1}')
+export JM_RPC_EX_PORT=10000
+export JM_RPC_IN_PORT=10000
+export TM_1_RPC_EX_PORT=10001
+export TM_2_RPC_EX_PORT=10002
+export TM_RPC_IN_PORT=10000
+export TM_1_DATA_EX_PORT=11001
+export TM_2_DATA_EX_PORT=11002
+export TM_DATA_IN_PORT=11000
+
+RESULT_HASH="72a690412be8928ba239c2da967328a5"
+INPUT_ARGS="--input ${INPUT_PATH}/words"
+
+export FLINK_JOB_ARGUMENTS="${INPUT_ARGS} --output ${OUTPUT_PATH}/docker_wc_out"
+
+build_image() {
+ ./build.sh --from-local-dist --job-artifacts ${FLINK_DIR}/examples/batch/WordCount.jar --image-name ${FLINK_DOCKER_IMAGE_NAME}
+}
+
+# user inside the container must be able to create files, this is a workaround in-container permissions
+mkdir -p $OUTPUT_VOLUME
+chmod 777 $OUTPUT_VOLUME
+
+pushd "$DOCKER_MODULE_DIR"
+if ! retry_times $DOCKER_IMAGE_BUILD_RETRIES ${BUILD_BACKOFF_TIME} build_image; then
+ echo "Failed to build docker image. Aborting..."
+ exit 1
+fi
+popd
+
+docker-compose -f ${DOCKER_SCRIPTS}/docker-compose.nat.yml up --abort-on-container-exit --exit-code-from job-cluster &> /dev/null
+docker-compose -f ${DOCKER_SCRIPTS}/docker-compose.nat.yml logs job-cluster > ${FLINK_DIR}/log/jobmanager.log
+docker-compose -f ${DOCKER_SCRIPTS}/docker-compose.nat.yml logs taskmanager1 > ${FLINK_DIR}/log/taskmanager1.log
+docker-compose -f ${DOCKER_SCRIPTS}/docker-compose.nat.yml logs taskmanager2 > ${FLINK_DIR}/log/taskmanager2.log
+
+cat $OUTPUT_VOLUME/docker_wc_out/1 $OUTPUT_VOLUME/docker_wc_out/2 | sort > $OUTPUT_VOLUME/docker_wc_out/merged
Review comment:
`check_result_hash_no_exit` requires a common prefix of the output files, which the outputs of WordCount do not have.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396349218
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -351,21 +351,21 @@ public static TaskExecutor startTaskManager(
LOG.info("Starting TaskManager with ResourceID: {}", resourceID);
- InetAddress remoteAddress = InetAddress.getByName(rpcService.getAddress());
+ InetAddress externalAddress = InetAddress.getByName(rpcService.getAddress());
Review comment:
I think it currently works because the TM will resolve its own hostname and communicate the resolved IP address to the other TMs. If the TM cannot resolve it, then it will fail. If we introduce an external hostname, then this will no longer be possible.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395216054
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -351,21 +351,21 @@ public static TaskExecutor startTaskManager(
LOG.info("Starting TaskManager with ResourceID: {}", resourceID);
- InetAddress remoteAddress = InetAddress.getByName(rpcService.getAddress());
+ InetAddress externalAddress = InetAddress.getByName(rpcService.getAddress());
Review comment:
I think this does not work with unresolvable external addresses. Maybe we could change the e2e test to use a external hostname instead of an ip. Then the problem should hopefully surface.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386800915
##########
File path: flink-end-to-end-tests/test-scripts/test_nat.sh
##########
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+source "$(dirname "$0")"/common.sh
+
+DOCKER_MODULE_DIR=${END_TO_END_DIR}/../flink-container/docker
+DOCKER_SCRIPTS=${END_TO_END_DIR}/test-scripts/container-scripts
+DOCKER_IMAGE_BUILD_RETRIES=3
+BUILD_BACKOFF_TIME=5
+
+export FLINK_JOB=org.apache.flink.examples.java.wordcount.WordCount
+export FLINK_DOCKER_IMAGE_NAME=test_nat
+export INPUT_VOLUME=${END_TO_END_DIR}/test-scripts/test-data
+export OUTPUT_VOLUME=${TEST_DATA_DIR}/out
+export INPUT_PATH=/data/test/input
+export OUTPUT_PATH=/data/test/output
+
+export HOST_ADDR=$(get_node_ip | awk '{print $1}')
+export JM_RPC_EX_PORT=10000
+export JM_RPC_IN_PORT=10000
+export TM_1_RPC_EX_PORT=10001
+export TM_2_RPC_EX_PORT=10002
+export TM_RPC_IN_PORT=10000
+export TM_1_DATA_EX_PORT=11001
+export TM_2_DATA_EX_PORT=11002
+export TM_DATA_IN_PORT=11000
+
+RESULT_HASH="72a690412be8928ba239c2da967328a5"
+INPUT_ARGS="--input ${INPUT_PATH}/words"
+
+export FLINK_JOB_ARGUMENTS="${INPUT_ARGS} --output ${OUTPUT_PATH}/docker_wc_out"
+
+build_image() {
+ ./build.sh --from-local-dist --job-artifacts ${FLINK_DIR}/examples/batch/WordCount.jar --image-name ${FLINK_DOCKER_IMAGE_NAME}
+}
+
+# user inside the container must be able to create files, this is a workaround in-container permissions
+mkdir -p $OUTPUT_VOLUME
+chmod 777 $OUTPUT_VOLUME
+
+pushd "$DOCKER_MODULE_DIR"
+if ! retry_times $DOCKER_IMAGE_BUILD_RETRIES ${BUILD_BACKOFF_TIME} build_image; then
+ echo "Failed to build docker image. Aborting..."
+ exit 1
+fi
+popd
+
+docker-compose -f ${DOCKER_SCRIPTS}/docker-compose.nat.yml up --abort-on-container-exit --exit-code-from job-cluster &> /dev/null
+docker-compose -f ${DOCKER_SCRIPTS}/docker-compose.nat.yml logs job-cluster > ${FLINK_DIR}/log/jobmanager.log
+docker-compose -f ${DOCKER_SCRIPTS}/docker-compose.nat.yml logs taskmanager1 > ${FLINK_DIR}/log/taskmanager1.log
+docker-compose -f ${DOCKER_SCRIPTS}/docker-compose.nat.yml logs taskmanager2 > ${FLINK_DIR}/log/taskmanager2.log
+
+cat $OUTPUT_VOLUME/docker_wc_out/1 $OUTPUT_VOLUME/docker_wc_out/2 | sort > $OUTPUT_VOLUME/docker_wc_out/merged
Review comment:
The prefix could be "$OUTPUT_VOLUME/docker_wc_out/" here. BTW, we could extract `docker_wc_out` as a parameter called `OUTPUT_DIR` or `OUTPUT_FILE`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386753541
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/rpc/akka/AkkaRpcServiceBuilder.java
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.rpc.akka;
+
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.runtime.akka.AkkaUtils;
+import org.apache.flink.runtime.clusterframework.BootstrapTools;
+import org.apache.flink.util.NetUtils;
+import org.apache.flink.util.Preconditions;
+
+import akka.actor.ActorSystem;
+import com.typesafe.config.Config;
+import org.slf4j.Logger;
+
+import javax.annotation.Nullable;
+
+import java.net.InetAddress;
+
+/**
+ * Builder for {@link AkkaRpcService}.
+ */
+public class AkkaRpcServiceBuilder {
Review comment:
I think it's better to make the `Builder` inside the `AkkaRpcService `:
- We can make the constructor of `AkkaRpcService` private in case someone does not follow this pattern.
- We can make the constructor of this builder private. Instead, add two static methods `localRpcServiceBuilder` and `remoteRpcServiceBuilder` to `AkkaRpcService`. Compare to explain the usage scenario in the javadoc, it could improve readability.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "PENDING",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6e0a577481f9300c87299b0629fab6dc1b3bd71a Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/151494400) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151875334",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154513428",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6492",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6507",
"triggerID" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"triggerType" : "PUSH"
}, {
"hash" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154538899",
"triggerID" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"triggerType" : "PUSH"
}, {
"hash" : "57914142eb72e078d48ee32c2b3f89de734fe72a",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6555",
"triggerID" : "57914142eb72e078d48ee32c2b3f89de734fe72a",
"triggerType" : "PUSH"
}, {
"hash" : "57914142eb72e078d48ee32c2b3f89de734fe72a",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154724688",
"triggerID" : "57914142eb72e078d48ee32c2b3f89de734fe72a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ff670f742762f45214ca32df966e02f4dbe6f758 Travis: [SUCCESS](https://travis-ci.com/github/flink-ci/flink/builds/154538899) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6507)
* 57914142eb72e078d48ee32c2b3f89de734fe72a Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/154724688) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6555)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151875334",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3e9df27458782312e53586e53f49cd55c11f4df5 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/151875334) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942)
* 0be3fd11e11d708b254036e9e91fc71f1fd02181 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395429730
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##########
@@ -94,19 +94,28 @@
" or if it has been quarantined by another actor system.");
/**
- * The config parameter defining the task manager's hostname.
+ * The external address of the network interface where the TaskManager is exposed.
* Overrides {@link #HOST_BIND_POLICY} automatic address binding.
*/
@Documentation.Section({Documentation.Sections.COMMON_HOST_PORT, Documentation.Sections.ALL_TASK_MANAGER})
public static final ConfigOption<String> HOST =
key("taskmanager.host")
.stringType()
.noDefaultValue()
- .withDescription("The address of the network interface that the TaskManager binds to." +
- " This option can be used to define explicitly a binding address. Because" +
- " different TaskManagers need different values for this option, usually it is specified in an" +
+ .withDescription("The external address of the network interface where the TaskManager is exposed." +
+ " Because different TaskManagers need different values for this option, usually it is specified in an" +
" additional non-shared TaskManager-specific config file.");
+ /**
+ * The local address of the network interface that the task manager binds to.
+ */
+ public static final ConfigOption<String> BIND_HOST =
+ key("taskmanager.bind-host")
Review comment:
Same here. TM address and bind-address are shared by RPC service, netty shuffle service and queryable kv state.
I'm actually thinking about changing "jobmanager.rpc.address" to "jobmanager.host" as a follow up, to make the config keys consistently using "host".
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-601638616
Thanks for the responses, @tillrohrmann.
I'll try to address the comments asap, including the unresolvable hostname issue.
BTW, I think you overlooked two comments regarding the binding port range, which are in the "hidden conversations".
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396525633
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniClusterConfiguration.java
##########
@@ -106,6 +106,14 @@ public String getTaskManagerBindAddress() {
configuration.getString(TaskManagerOptions.HOST, "localhost");
}
+ public String getJobManagerBindPortRange() {
+ return String.valueOf(configuration.getInteger(JobManagerOptions.PORT, 0));
+ }
+
+ public String getTaskManagerBindPortRange() {
+ return configuration.getString(TaskManagerOptions.RPC_PORT);
+ }
Review comment:
Because of the above-mentioned problems, I think we should remove these methods and rely on the random port.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386804627
##########
File path: flink-end-to-end-tests/test-scripts/test_nat.sh
##########
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+source "$(dirname "$0")"/common.sh
+
+DOCKER_MODULE_DIR=${END_TO_END_DIR}/../flink-container/docker
+DOCKER_SCRIPTS=${END_TO_END_DIR}/test-scripts/container-scripts
+DOCKER_IMAGE_BUILD_RETRIES=3
+BUILD_BACKOFF_TIME=5
+
+export FLINK_JOB=org.apache.flink.examples.java.wordcount.WordCount
+export FLINK_DOCKER_IMAGE_NAME=test_nat
+export INPUT_VOLUME=${END_TO_END_DIR}/test-scripts/test-data
+export OUTPUT_VOLUME=${TEST_DATA_DIR}/out
+export INPUT_PATH=/data/test/input
+export OUTPUT_PATH=/data/test/output
+
+export HOST_ADDR=$(get_node_ip | awk '{print $1}')
+export JM_RPC_EX_PORT=10000
+export JM_RPC_IN_PORT=10000
+export TM_1_RPC_EX_PORT=10001
+export TM_2_RPC_EX_PORT=10002
+export TM_RPC_IN_PORT=10000
+export TM_1_DATA_EX_PORT=11001
+export TM_2_DATA_EX_PORT=11002
+export TM_DATA_IN_PORT=11000
+
+RESULT_HASH="72a690412be8928ba239c2da967328a5"
+INPUT_ARGS="--input ${INPUT_PATH}/words"
+
+export FLINK_JOB_ARGUMENTS="${INPUT_ARGS} --output ${OUTPUT_PATH}/docker_wc_out"
+
+build_image() {
+ ./build.sh --from-local-dist --job-artifacts ${FLINK_DIR}/examples/batch/WordCount.jar --image-name ${FLINK_DOCKER_IMAGE_NAME}
+}
+
+# user inside the container must be able to create files, this is a workaround in-container permissions
+mkdir -p $OUTPUT_VOLUME
+chmod 777 $OUTPUT_VOLUME
Review comment:
I would argue that this is a new created empty directory dedicated for the test case to use, and will be be cleaned up when the test is finished. But never mind, I'll make it `666` if it bothers you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396547183
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/jobmaster/JobMaster.java
##########
@@ -571,9 +572,16 @@ private void releaseEmptyTaskManager(ResourceID resourceId) {
@Override
public CompletableFuture<RegistrationResponse> registerTaskManager(
final String taskManagerRpcAddress,
- final TaskManagerLocation taskManagerLocation,
+ final TaskManagerUnresolvedLocation taskManagerUnresolvedLocation,
Review comment:
nit:
```suggestion
final UnresolvedTaskManagerLocation unresolvedTaskManagerLocation,
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151875334",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154513428",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6492",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3e9df27458782312e53586e53f49cd55c11f4df5 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/151875334) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942)
* 0be3fd11e11d708b254036e9e91fc71f1fd02181 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/154513428) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6492)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6e0a577481f9300c87299b0629fab6dc1b3bd71a Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/151494400) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-602404885
@flinkbot run travis re-run the last Travis build
@flinkbot run azure re-run the last Azure build
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396893625
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/jobmaster/JobMaster.java
##########
@@ -571,9 +572,16 @@ private void releaseEmptyTaskManager(ResourceID resourceId) {
@Override
public CompletableFuture<RegistrationResponse> registerTaskManager(
final String taskManagerRpcAddress,
- final TaskManagerLocation taskManagerLocation,
+ final TaskManagerUnresolvedLocation taskManagerUnresolvedLocation,
final Time timeout) {
+ final TaskManagerLocation taskManagerLocation;
+ try {
+ taskManagerLocation = TaskManagerLocation.fromUnresolvedLocation(taskManagerUnresolvedLocation);
Review comment:
For data transmission, yes. But I think the TM's address is required to be resolvable for JM process anyway. Because we still need them for RPC communications.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396891883
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/KvStateService.java
##########
@@ -181,7 +181,7 @@ public static KvStateService fromConfiguration(TaskManagerServicesConfiguration
int numStateServerQueryThreads = qsConfig.numStateQueryThreads() == 0 ?
taskManagerServicesConfiguration.getNumberOfSlots() : qsConfig.numStateQueryThreads();
kvStateServer = QueryableStateUtils.createKvStateServer(
Review comment:
I have no opinion on this.
I'm not familiar with how the queryable state works and its use cases. I've consulted @zhijiangW about this, and he suggested that queryable state does not have much use cases and may not be necessary for the NAT scenarios.
In addition, if we want to support this for queryable state, we could use some help from people who are more familiar with the queryable state.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395189642
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniCluster.java
##########
@@ -261,26 +258,30 @@ public void start() throws Exception {
// bring up all the RPC services
LOG.info("Starting RPC Service(s)");
- AkkaRpcServiceConfiguration akkaRpcServiceConfig = AkkaRpcServiceConfiguration.fromConfiguration(configuration);
-
final RpcServiceFactory dispatcherResourceManagreComponentRpcServiceFactory;
if (useSingleRpcService) {
// we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, false, null);
+ commonRpcService = createLocalRpcService(configuration);
final CommonRpcServiceFactory commonRpcServiceFactory = new CommonRpcServiceFactory(commonRpcService);
taskManagerRpcServiceFactory = commonRpcServiceFactory;
dispatcherResourceManagreComponentRpcServiceFactory = commonRpcServiceFactory;
} else {
- // we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, true, null);
// start a new service per component, possibly with custom bind addresses
final String jobManagerBindAddress = miniClusterConfiguration.getJobManagerBindAddress();
final String taskManagerBindAddress = miniClusterConfiguration.getTaskManagerBindAddress();
+ final String jobManagerBindPort = miniClusterConfiguration.getJobManagerBindPortRange();
+ final String taskManagerBindPort = miniClusterConfiguration.getTaskManagerBindPortRange();
Review comment:
Same here. Do we support bind port ranges?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151875334",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154513428",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6492",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6507",
"triggerID" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"triggerType" : "PUSH"
}, {
"hash" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154538899",
"triggerID" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ff670f742762f45214ca32df966e02f4dbe6f758 Travis: [SUCCESS](https://travis-ci.com/github/flink-ci/flink/builds/154538899) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6507)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wangyang0918 commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396385213
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -351,21 +351,21 @@ public static TaskExecutor startTaskManager(
LOG.info("Starting TaskManager with ResourceID: {}", resourceID);
- InetAddress remoteAddress = InetAddress.getByName(rpcService.getAddress());
+ InetAddress externalAddress = InetAddress.getByName(rpcService.getAddress());
Review comment:
I agree with you that if the `TM` could not resolve its own address, it will fail. Indeed, it is problem we need to fix in this PR. However, i am just wondering given that we have fix this and do not require the `TM` could resolve its own external hostname, do we still need other `TMs` could resolve the external hostname?
In my opinion every `TM` registers to the `JM` with hostname and ip. So all the `TMs` could get other `TM` hostname and ip. I am not sure about this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151875334",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154513428",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6492",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6507",
"triggerID" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"triggerType" : "PUSH"
}, {
"hash" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154538899",
"triggerID" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"triggerType" : "PUSH"
}, {
"hash" : "57914142eb72e078d48ee32c2b3f89de734fe72a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "57914142eb72e078d48ee32c2b3f89de734fe72a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ff670f742762f45214ca32df966e02f4dbe6f758 Travis: [SUCCESS](https://travis-ci.com/github/flink-ci/flink/builds/154538899) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6507)
* 57914142eb72e078d48ee32c2b3f89de734fe72a UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395426701
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniCluster.java
##########
@@ -261,26 +258,30 @@ public void start() throws Exception {
// bring up all the RPC services
LOG.info("Starting RPC Service(s)");
- AkkaRpcServiceConfiguration akkaRpcServiceConfig = AkkaRpcServiceConfiguration.fromConfiguration(configuration);
-
final RpcServiceFactory dispatcherResourceManagreComponentRpcServiceFactory;
if (useSingleRpcService) {
// we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, false, null);
+ commonRpcService = createLocalRpcService(configuration);
final CommonRpcServiceFactory commonRpcServiceFactory = new CommonRpcServiceFactory(commonRpcService);
taskManagerRpcServiceFactory = commonRpcServiceFactory;
dispatcherResourceManagreComponentRpcServiceFactory = commonRpcServiceFactory;
} else {
- // we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, true, null);
// start a new service per component, possibly with custom bind addresses
final String jobManagerBindAddress = miniClusterConfiguration.getJobManagerBindAddress();
final String taskManagerBindAddress = miniClusterConfiguration.getTaskManagerBindAddress();
+ final String jobManagerBindPort = miniClusterConfiguration.getJobManagerBindPortRange();
+ final String taskManagerBindPort = miniClusterConfiguration.getTaskManagerBindPortRange();
Review comment:
Please see my explanation above.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386828354
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/rpc/akka/AkkaRpcServiceBuilder.java
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.rpc.akka;
+
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.runtime.akka.AkkaUtils;
+import org.apache.flink.runtime.clusterframework.BootstrapTools;
+import org.apache.flink.util.NetUtils;
+import org.apache.flink.util.Preconditions;
+
+import akka.actor.ActorSystem;
+import com.typesafe.config.Config;
+import org.slf4j.Logger;
+
+import javax.annotation.Nullable;
+
+import java.net.InetAddress;
+
+/**
+ * Builder for {@link AkkaRpcService}.
+ */
+public class AkkaRpcServiceBuilder {
Review comment:
I moved `AkkaRpcServiceBuilder` into `AkkaRpcServiceUtils`, and made its constructors private. Also annotated constructor of `AkkaRpcService` `VisibleForTesting`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395200778
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##########
@@ -94,19 +94,28 @@
" or if it has been quarantined by another actor system.");
/**
- * The config parameter defining the task manager's hostname.
+ * The external address of the network interface where the TaskManager is exposed.
* Overrides {@link #HOST_BIND_POLICY} automatic address binding.
*/
@Documentation.Section({Documentation.Sections.COMMON_HOST_PORT, Documentation.Sections.ALL_TASK_MANAGER})
public static final ConfigOption<String> HOST =
key("taskmanager.host")
.stringType()
.noDefaultValue()
- .withDescription("The address of the network interface that the TaskManager binds to." +
- " This option can be used to define explicitly a binding address. Because" +
- " different TaskManagers need different values for this option, usually it is specified in an" +
+ .withDescription("The external address of the network interface where the TaskManager is exposed." +
+ " Because different TaskManagers need different values for this option, usually it is specified in an" +
" additional non-shared TaskManager-specific config file.");
+ /**
+ * The local address of the network interface that the task manager binds to.
+ */
+ public static final ConfigOption<String> BIND_HOST =
+ key("taskmanager.bind-host")
Review comment:
Maybe it would be more consistent wrt to the `jobmanager` if we used `taskmanager.rpc.bind-address`. The downside would be that it is not consistent with `taskmanager.host`. But we could change this option key as a follow up.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "PENDING",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* aa1386ad6c2ed2774d659263d7684e777eaccd2d Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/151359451) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386774149
##########
File path: flink-end-to-end-tests/test-scripts/test_nat.sh
##########
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+source "$(dirname "$0")"/common.sh
+
+DOCKER_MODULE_DIR=${END_TO_END_DIR}/../flink-container/docker
+DOCKER_SCRIPTS=${END_TO_END_DIR}/test-scripts/container-scripts
+DOCKER_IMAGE_BUILD_RETRIES=3
+BUILD_BACKOFF_TIME=5
+
+export FLINK_JOB=org.apache.flink.examples.java.wordcount.WordCount
+export FLINK_DOCKER_IMAGE_NAME=test_nat
+export INPUT_VOLUME=${END_TO_END_DIR}/test-scripts/test-data
+export OUTPUT_VOLUME=${TEST_DATA_DIR}/out
+export INPUT_PATH=/data/test/input
+export OUTPUT_PATH=/data/test/output
+
+export HOST_ADDR=$(get_node_ip | awk '{print $1}')
+export JM_RPC_EX_PORT=10000
+export JM_RPC_IN_PORT=10000
+export TM_1_RPC_EX_PORT=10001
+export TM_2_RPC_EX_PORT=10002
+export TM_RPC_IN_PORT=10000
+export TM_1_DATA_EX_PORT=11001
+export TM_2_DATA_EX_PORT=11002
+export TM_DATA_IN_PORT=11000
+
+RESULT_HASH="72a690412be8928ba239c2da967328a5"
+INPUT_ARGS="--input ${INPUT_PATH}/words"
+
+export FLINK_JOB_ARGUMENTS="${INPUT_ARGS} --output ${OUTPUT_PATH}/docker_wc_out"
+
+build_image() {
+ ./build.sh --from-local-dist --job-artifacts ${FLINK_DIR}/examples/batch/WordCount.jar --image-name ${FLINK_DOCKER_IMAGE_NAME}
+}
+
+# user inside the container must be able to create files, this is a workaround in-container permissions
+mkdir -p $OUTPUT_VOLUME
+chmod 777 $OUTPUT_VOLUME
+
+pushd "$DOCKER_MODULE_DIR"
+if ! retry_times $DOCKER_IMAGE_BUILD_RETRIES ${BUILD_BACKOFF_TIME} build_image; then
+ echo "Failed to build docker image. Aborting..."
+ exit 1
+fi
+popd
+
+docker-compose -f ${DOCKER_SCRIPTS}/docker-compose.nat.yml up --abort-on-container-exit --exit-code-from job-cluster &> /dev/null
+docker-compose -f ${DOCKER_SCRIPTS}/docker-compose.nat.yml logs job-cluster > ${FLINK_DIR}/log/jobmanager.log
+docker-compose -f ${DOCKER_SCRIPTS}/docker-compose.nat.yml logs taskmanager1 > ${FLINK_DIR}/log/taskmanager1.log
+docker-compose -f ${DOCKER_SCRIPTS}/docker-compose.nat.yml logs taskmanager2 > ${FLINK_DIR}/log/taskmanager2.log
+
+cat $OUTPUT_VOLUME/docker_wc_out/1 $OUTPUT_VOLUME/docker_wc_out/2 | sort > $OUTPUT_VOLUME/docker_wc_out/merged
Review comment:
I'm not sure of the necessity of this step. It seems the `check_result_hash_no_exit` function do this job for us.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151875334",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3e9df27458782312e53586e53f49cd55c11f4df5 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/151875334) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "PENDING",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* aa1386ad6c2ed2774d659263d7684e777eaccd2d Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/151359451) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826)
* 6e0a577481f9300c87299b0629fab6dc1b3bd71a Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/151494400) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386790141
##########
File path: flink-end-to-end-tests/test-scripts/test_nat.sh
##########
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+source "$(dirname "$0")"/common.sh
+
+DOCKER_MODULE_DIR=${END_TO_END_DIR}/../flink-container/docker
+DOCKER_SCRIPTS=${END_TO_END_DIR}/test-scripts/container-scripts
+DOCKER_IMAGE_BUILD_RETRIES=3
+BUILD_BACKOFF_TIME=5
+
+export FLINK_JOB=org.apache.flink.examples.java.wordcount.WordCount
+export FLINK_DOCKER_IMAGE_NAME=test_nat
+export INPUT_VOLUME=${END_TO_END_DIR}/test-scripts/test-data
+export OUTPUT_VOLUME=${TEST_DATA_DIR}/out
+export INPUT_PATH=/data/test/input
+export OUTPUT_PATH=/data/test/output
+
+export HOST_ADDR=$(get_node_ip | awk '{print $1}')
+export JM_RPC_EX_PORT=10000
+export JM_RPC_IN_PORT=10000
+export TM_1_RPC_EX_PORT=10001
+export TM_2_RPC_EX_PORT=10002
+export TM_RPC_IN_PORT=10000
+export TM_1_DATA_EX_PORT=11001
+export TM_2_DATA_EX_PORT=11002
+export TM_DATA_IN_PORT=11000
+
+RESULT_HASH="72a690412be8928ba239c2da967328a5"
+INPUT_ARGS="--input ${INPUT_PATH}/words"
+
+export FLINK_JOB_ARGUMENTS="${INPUT_ARGS} --output ${OUTPUT_PATH}/docker_wc_out"
+
+build_image() {
+ ./build.sh --from-local-dist --job-artifacts ${FLINK_DIR}/examples/batch/WordCount.jar --image-name ${FLINK_DOCKER_IMAGE_NAME}
+}
+
+# user inside the container must be able to create files, this is a workaround in-container permissions
+mkdir -p $OUTPUT_VOLUME
+chmod 777 $OUTPUT_VOLUME
Review comment:
I think here we simply give the container full access to the path, not assuming what it does with the path. What is the problem?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151875334",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154513428",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6492",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0be3fd11e11d708b254036e9e91fc71f1fd02181 Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/154513428) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6492)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-602522424
@tillrohrmann
I'm ok with supporting the scenarios where address of `t` can be resolved by all other `t'` except for itself, and have updated the PR to address this issue.After all, it is indeed an improvement that we are able to support more use cases without introducing any problem to exist ones.
It just how tricky I have to simulate this scenario in the e2e test that makes me suspect something might be wrong. Or put it another way, why users want to use Flink like this? Have we truly understood what they need?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395549344
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -351,21 +351,21 @@ public static TaskExecutor startTaskManager(
LOG.info("Starting TaskManager with ResourceID: {}", resourceID);
- InetAddress remoteAddress = InetAddress.getByName(rpcService.getAddress());
+ InetAddress externalAddress = InetAddress.getByName(rpcService.getAddress());
Review comment:
Alright, thanks for the explanation. I'll try to fix it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-594996261
@flinkbot run azure
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395425038
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
##########
@@ -82,92 +82,80 @@
.build();
/**
- * Starts an ActorSystem with the given configuration listening at the address/ports.
+ * Starts a remote ActorSystem at given address and specific port range.
* @param configuration The Flink configuration
- * @param listeningAddress The address to listen at.
- * @param portRangeDefinition The port range to choose a port from.
+ * @param externalAddress The external address to access the ActorSystem.
+ * @param externalPortRange The choosing range of the external port to access the ActorSystem.
* @param logger The logger to output log information.
* @return The ActorSystem which has been started
* @throws Exception Thrown when actor system cannot be started in specified port range
*/
- public static ActorSystem startActorSystem(
+ @VisibleForTesting
+ public static ActorSystem startRemoteActorSystem(
Configuration configuration,
- String listeningAddress,
- String portRangeDefinition,
+ String externalAddress,
+ String externalPortRange,
Logger logger) throws Exception {
- return startActorSystem(
- configuration,
- listeningAddress,
- portRangeDefinition,
- logger,
- ForkJoinExecutorConfiguration.fromConfiguration(configuration));
- }
-
- /**
- * Starts an ActorSystem with the given configuration listening at the address/ports.
- *
- * @param configuration The Flink configuration
- * @param listeningAddress The address to listen at.
- * @param portRangeDefinition The port range to choose a port from.
- * @param logger The logger to output log information.
- * @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
- * @return The ActorSystem which has been started
- * @throws Exception Thrown when actor system cannot be started in specified port range
- */
- public static ActorSystem startActorSystem(
- Configuration configuration,
- String listeningAddress,
- String portRangeDefinition,
- Logger logger,
- @Nonnull ActorSystemExecutorConfiguration actorSystemExecutorConfiguration) throws Exception {
- return startActorSystem(
+ return startRemoteActorSystem(
configuration,
AkkaUtils.getFlinkActorSystemName(),
- listeningAddress,
- portRangeDefinition,
+ externalAddress,
+ externalPortRange,
+ NetUtils.getWildcardIPAddress(),
+ -1,
logger,
- actorSystemExecutorConfiguration);
+ ForkJoinExecutorConfiguration.fromConfiguration(configuration),
+ null);
}
/**
- * Starts an ActorSystem with the given configuration listening at the address/ports.
+ * Starts a remote ActorSystem at given address and specific port range.
*
* @param configuration The Flink configuration
* @param actorSystemName Name of the started {@link ActorSystem}
- * @param listeningAddress The address to listen at.
- * @param portRangeDefinition The port range to choose a port from.
+ * @param externalAddress The external address to access the ActorSystem.
+ * @param externalPortRange The choosing range of the external port to access the ActorSystem.
+ * @param bindAddress The local address to bind to.
+ * @param bindPort The local port to bind to. If negative, external port will be used.
* @param logger The logger to output log information.
* @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
+ * @param customConfig Custom Akka config to be combined with the config derived from Flink configuration.
* @return The ActorSystem which has been started
* @throws Exception Thrown when actor system cannot be started in specified port range
*/
- public static ActorSystem startActorSystem(
+ public static ActorSystem startRemoteActorSystem(
Configuration configuration,
String actorSystemName,
- String listeningAddress,
- String portRangeDefinition,
+ String externalAddress,
+ String externalPortRange,
+ String bindAddress,
+ int bindPort,
Review comment:
Actually, I also thought about making the bindPort Optional, and maybe also bindAddress.
However I also noticed that the code style guide suggests to use Optional only for return values of API/public methods, but not for class fields and method arguments.
Not sure if it's ok to not strictly following the guide. We do have a good reason to use Optional here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386803470
##########
File path: flink-end-to-end-tests/test-scripts/test_nat.sh
##########
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+source "$(dirname "$0")"/common.sh
+
+DOCKER_MODULE_DIR=${END_TO_END_DIR}/../flink-container/docker
+DOCKER_SCRIPTS=${END_TO_END_DIR}/test-scripts/container-scripts
+DOCKER_IMAGE_BUILD_RETRIES=3
+BUILD_BACKOFF_TIME=5
+
+export FLINK_JOB=org.apache.flink.examples.java.wordcount.WordCount
+export FLINK_DOCKER_IMAGE_NAME=test_nat
+export INPUT_VOLUME=${END_TO_END_DIR}/test-scripts/test-data
+export OUTPUT_VOLUME=${TEST_DATA_DIR}/out
+export INPUT_PATH=/data/test/input
+export OUTPUT_PATH=/data/test/output
+
+export HOST_ADDR=$(get_node_ip | awk '{print $1}')
+export JM_RPC_EX_PORT=10000
+export JM_RPC_IN_PORT=10000
+export TM_1_RPC_EX_PORT=10001
+export TM_2_RPC_EX_PORT=10002
+export TM_RPC_IN_PORT=10000
+export TM_1_DATA_EX_PORT=11001
+export TM_2_DATA_EX_PORT=11002
+export TM_DATA_IN_PORT=11000
+
+RESULT_HASH="72a690412be8928ba239c2da967328a5"
+INPUT_ARGS="--input ${INPUT_PATH}/words"
+
+export FLINK_JOB_ARGUMENTS="${INPUT_ARGS} --output ${OUTPUT_PATH}/docker_wc_out"
+
+build_image() {
+ ./build.sh --from-local-dist --job-artifacts ${FLINK_DIR}/examples/batch/WordCount.jar --image-name ${FLINK_DOCKER_IMAGE_NAME}
+}
+
+# user inside the container must be able to create files, this is a workaround in-container permissions
+mkdir -p $OUTPUT_VOLUME
+chmod 777 $OUTPUT_VOLUME
+
+pushd "$DOCKER_MODULE_DIR"
+if ! retry_times $DOCKER_IMAGE_BUILD_RETRIES ${BUILD_BACKOFF_TIME} build_image; then
+ echo "Failed to build docker image. Aborting..."
+ exit 1
+fi
+popd
+
+docker-compose -f ${DOCKER_SCRIPTS}/docker-compose.nat.yml up --abort-on-container-exit --exit-code-from job-cluster &> /dev/null
+docker-compose -f ${DOCKER_SCRIPTS}/docker-compose.nat.yml logs job-cluster > ${FLINK_DIR}/log/jobmanager.log
+docker-compose -f ${DOCKER_SCRIPTS}/docker-compose.nat.yml logs taskmanager1 > ${FLINK_DIR}/log/taskmanager1.log
+docker-compose -f ${DOCKER_SCRIPTS}/docker-compose.nat.yml logs taskmanager2 > ${FLINK_DIR}/log/taskmanager2.log
+
+cat $OUTPUT_VOLUME/docker_wc_out/1 $OUTPUT_VOLUME/docker_wc_out/2 | sort > $OUTPUT_VOLUME/docker_wc_out/merged
Review comment:
You're right.
I think I tried this before, but forgot to add the '/' in the prefix so it failed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396896951
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/KvStateService.java
##########
@@ -170,7 +170,7 @@ public static KvStateService fromConfiguration(TaskManagerServicesConfiguration
int numProxyServerQueryThreads = qsConfig.numProxyQueryThreads() == 0 ?
taskManagerServicesConfiguration.getNumberOfSlots() : qsConfig.numProxyQueryThreads();
kvClientProxy = QueryableStateUtils.createKvStateClientProxy(
- taskManagerServicesConfiguration.getTaskManagerAddress(),
+ taskManagerServicesConfiguration.getExternalAddress(),
Review comment:
As mentioned in the JIRA ticket, I'd like to exclude queryable state from the scope of this PR. Because I'm not familiar with how it works, and it seems does not have many use cases.
I think this should not affect other use cases, because the code path should only be executed if queryable state is enabled.
I think we should mention in the release notes that the new NAT support does not cover queryable state.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* aa1386ad6c2ed2774d659263d7684e777eaccd2d Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/151359451) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395562961
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniCluster.java
##########
@@ -261,26 +258,30 @@ public void start() throws Exception {
// bring up all the RPC services
LOG.info("Starting RPC Service(s)");
- AkkaRpcServiceConfiguration akkaRpcServiceConfig = AkkaRpcServiceConfiguration.fromConfiguration(configuration);
-
final RpcServiceFactory dispatcherResourceManagreComponentRpcServiceFactory;
if (useSingleRpcService) {
// we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, false, null);
+ commonRpcService = createLocalRpcService(configuration);
final CommonRpcServiceFactory commonRpcServiceFactory = new CommonRpcServiceFactory(commonRpcService);
taskManagerRpcServiceFactory = commonRpcServiceFactory;
dispatcherResourceManagreComponentRpcServiceFactory = commonRpcServiceFactory;
} else {
- // we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, true, null);
// start a new service per component, possibly with custom bind addresses
final String jobManagerBindAddress = miniClusterConfiguration.getJobManagerBindAddress();
final String taskManagerBindAddress = miniClusterConfiguration.getTaskManagerBindAddress();
+ final String jobManagerBindPort = miniClusterConfiguration.getJobManagerBindPortRange();
Review comment:
Ok, then we should maybe change `getJobManagerBindPortRange` into `getJobManagerBindPort` to better reflect this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395534217
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniClusterConfiguration.java
##########
@@ -102,21 +102,21 @@ public int getNumTaskManagers() {
public String getJobManagerBindAddress() {
return commonBindAddress != null ?
commonBindAddress :
- configuration.getString(JobManagerOptions.ADDRESS, "localhost");
+ configuration.getString(JobManagerOptions.BIND_HOST, "localhost");
}
public String getTaskManagerBindAddress() {
return commonBindAddress != null ?
commonBindAddress :
- configuration.getString(TaskManagerOptions.HOST, "localhost");
+ configuration.getString(TaskManagerOptions.BIND_HOST, "localhost");
}
- public String getJobManagerBindPortRange() {
- return String.valueOf(configuration.getInteger(JobManagerOptions.PORT, 0));
+ public int getJobManagerBindPort() {
+ return configuration.getInteger(JobManagerOptions.RPC_BIND_PORT, 0);
}
- public String getTaskManagerBindPortRange() {
- return configuration.getString(TaskManagerOptions.RPC_PORT);
+ public int getTaskManagerBindPort() {
+ return configuration.getInteger(TaskManagerOptions.RPC_BIND_PORT, 0);
Review comment:
I think the problem I see is that we are using `RPC_BIND_PORT` and `BIND_HOST` as the external address and port when creating the `RpcService` in the `MiniCluster`. It might not be a big problem but it is a behaviour inconsistency.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395528362
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/JobManagerOptions.java
##########
@@ -54,6 +54,16 @@
" leader-election service (like ZooKeeper) is used to elect and discover the JobManager" +
" leader from potentially multiple standby JobManagers.");
+ /**
+ * The local address of the network interface that the job manager binds to.
+ */
+ public static final ConfigOption<String> BIND_HOST =
+ key("jobmanager.bind-host")
Review comment:
makes sense. Thanks for the clarification.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6e0a577481f9300c87299b0629fab6dc1b3bd71a Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/151494400) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853)
* 3e9df27458782312e53586e53f49cd55c11f4df5 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* aa1386ad6c2ed2774d659263d7684e777eaccd2d Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/151359451) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826)
* 6e0a577481f9300c87299b0629fab6dc1b3bd71a UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395527673
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
##########
@@ -82,92 +82,80 @@
.build();
/**
- * Starts an ActorSystem with the given configuration listening at the address/ports.
+ * Starts a remote ActorSystem at given address and specific port range.
* @param configuration The Flink configuration
- * @param listeningAddress The address to listen at.
- * @param portRangeDefinition The port range to choose a port from.
+ * @param externalAddress The external address to access the ActorSystem.
+ * @param externalPortRange The choosing range of the external port to access the ActorSystem.
* @param logger The logger to output log information.
* @return The ActorSystem which has been started
* @throws Exception Thrown when actor system cannot be started in specified port range
*/
- public static ActorSystem startActorSystem(
+ @VisibleForTesting
+ public static ActorSystem startRemoteActorSystem(
Configuration configuration,
- String listeningAddress,
- String portRangeDefinition,
+ String externalAddress,
+ String externalPortRange,
Logger logger) throws Exception {
- return startActorSystem(
- configuration,
- listeningAddress,
- portRangeDefinition,
- logger,
- ForkJoinExecutorConfiguration.fromConfiguration(configuration));
- }
-
- /**
- * Starts an ActorSystem with the given configuration listening at the address/ports.
- *
- * @param configuration The Flink configuration
- * @param listeningAddress The address to listen at.
- * @param portRangeDefinition The port range to choose a port from.
- * @param logger The logger to output log information.
- * @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
- * @return The ActorSystem which has been started
- * @throws Exception Thrown when actor system cannot be started in specified port range
- */
- public static ActorSystem startActorSystem(
- Configuration configuration,
- String listeningAddress,
- String portRangeDefinition,
- Logger logger,
- @Nonnull ActorSystemExecutorConfiguration actorSystemExecutorConfiguration) throws Exception {
- return startActorSystem(
+ return startRemoteActorSystem(
configuration,
AkkaUtils.getFlinkActorSystemName(),
- listeningAddress,
- portRangeDefinition,
+ externalAddress,
+ externalPortRange,
+ NetUtils.getWildcardIPAddress(),
+ -1,
logger,
- actorSystemExecutorConfiguration);
+ ForkJoinExecutorConfiguration.fromConfiguration(configuration),
+ null);
}
/**
- * Starts an ActorSystem with the given configuration listening at the address/ports.
+ * Starts a remote ActorSystem at given address and specific port range.
*
* @param configuration The Flink configuration
* @param actorSystemName Name of the started {@link ActorSystem}
- * @param listeningAddress The address to listen at.
- * @param portRangeDefinition The port range to choose a port from.
+ * @param externalAddress The external address to access the ActorSystem.
+ * @param externalPortRange The choosing range of the external port to access the ActorSystem.
+ * @param bindAddress The local address to bind to.
+ * @param bindPort The local port to bind to. If negative, external port will be used.
* @param logger The logger to output log information.
* @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
+ * @param customConfig Custom Akka config to be combined with the config derived from Flink configuration.
* @return The ActorSystem which has been started
* @throws Exception Thrown when actor system cannot be started in specified port range
*/
- public static ActorSystem startActorSystem(
+ public static ActorSystem startRemoteActorSystem(
Configuration configuration,
String actorSystemName,
- String listeningAddress,
- String portRangeDefinition,
+ String externalAddress,
+ String externalPortRange,
+ String bindAddress,
+ int bindPort,
Review comment:
I think we said that it is ok to use `Optional` as a parameter for internal functions if it makes things easier/clearer. Hence, it should be ok to use it here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-602514974
> @tillrohrmann, I have addressed your comments.
>
> However, I'm still trying to understand how the following is a common need.
>
> > What we actually need is that for a given TM `t` all other TMs `t'` can resolve the address of `t`.
>
> I find it not natural to simulate the above scenario with Docker in the e2e test. For each container I have to manually add the hostname-ip entries of all other containers except for itself.
>
> Say all the JM and TMs are running in Docker containers. If a given TM `t` cannot resolve its own external address from inside the container, then unless specifically configured like I did in the e2e test, how could the JM and all the other TMs `t'`, which are also inside containers, resolve the address of `t`? If `t'` can resolve the address of `t` by using the same DNS settings as its host, then shouldn't `t'` also use the same DNS settings as its host and be able to resolve its own address?
I would assume that in most setups every `t` would be able to resolve all of the others `t'` addresses and also its own. But this is a stronger assumption than saying every `t'` needs to be able to resolve 't' address and hence it breaks some setups (as people complained on the ML/SO).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395185040
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
##########
@@ -181,104 +169,115 @@ public static ActorSystem startActorSystem(
// if we come here, we have exhausted the port range
throw new BindException("Could not start actor system on any port in port range "
- + portRangeDefinition);
+ + externalPortRange);
}
/**
- * Starts an Actor System at a specific port.
- *
+ * Starts a remote Actor System at given address and specific port.
* @param configuration The Flink configuration.
- * @param listeningAddress The address to listen at.
- * @param listeningPort The port to listen at.
+ * @param actorSystemName Name of the started {@link ActorSystem}
+ * @param externalAddress The external address to access the ActorSystem.
+ * @param externalPort The external port to access the ActorSystem.
+ * @param bindAddress The local address to bind to.
+ * @param bindPort The local port to bind to.
* @param logger the logger to output log information.
+ * @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
+ * @param customConfig Custom Akka config to be combined with the config derived from Flink configuration.
* @return The ActorSystem which has been started.
* @throws Exception
*/
- public static ActorSystem startActorSystem(
+ private static ActorSystem startRemoteActorSystem(
Configuration configuration,
- String listeningAddress,
- int listeningPort,
- Logger logger) throws Exception {
- return startActorSystem(
- configuration,
- listeningAddress,
- listeningPort,
- logger,
- ForkJoinExecutorConfiguration.fromConfiguration(configuration));
- }
+ String actorSystemName,
+ String externalAddress,
+ int externalPort,
+ String bindAddress,
+ int bindPort,
+ Logger logger,
+ ActorSystemExecutorConfiguration actorSystemExecutorConfiguration,
+ Config customConfig) throws Exception {
- /**
- * Starts an Actor System at a specific port.
- * @param configuration The Flink configuration.
- * @param listeningAddress The address to listen at.
- * @param listeningPort The port to listen at.
- * @param logger the logger to output log information.
- * @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
- * @return The ActorSystem which has been started.
- * @throws Exception
- */
- public static ActorSystem startActorSystem(
- Configuration configuration,
- String listeningAddress,
- int listeningPort,
- Logger logger,
- ActorSystemExecutorConfiguration actorSystemExecutorConfiguration) throws Exception {
- return startActorSystem(
- configuration,
- AkkaUtils.getFlinkActorSystemName(),
- listeningAddress,
- listeningPort,
- logger,
- actorSystemExecutorConfiguration);
+ String externalHostPortUrl = NetUtils.unresolvedHostAndPortToNormalizedString(externalAddress, externalPort);
+ String bindHostPortUrl = NetUtils.unresolvedHostAndPortToNormalizedString(bindAddress, bindPort);
+ logger.info("Trying to start actor system, external address {}, bind address {}.", externalHostPortUrl, bindHostPortUrl);
+
+ try {
+ Config akkaConfig = AkkaUtils.getAkkaConfig(
+ configuration,
+ new Some<>(new Tuple2<>(externalAddress, externalPort)),
+ new Some<>(new Tuple2<>(bindAddress, bindPort)),
+ actorSystemExecutorConfiguration.getAkkaConfig());
+
+ if (customConfig != null) {
+ akkaConfig = customConfig.withFallback(akkaConfig);
+ }
+
+ return startActorSystem(akkaConfig, actorSystemName, logger);
+ }
+ catch (Throwable t) {
+ if (t instanceof ChannelException) {
+ Throwable cause = t.getCause();
+ if (cause != null && t.getCause() instanceof BindException) {
+ throw new IOException("Unable to create ActorSystem at address " + externalHostPortUrl +
+ " : " + cause.getMessage(), t);
+ }
+ }
+ throw new Exception("Could not create actor system", t);
+ }
}
/**
- * Starts an Actor System at a specific port.
+ * Starts a local Actor System.
* @param configuration The Flink configuration.
- * @param actorSystemName Name of the started {@link ActorSystem}
- * @param listeningAddress The address to listen at.
- * @param listeningPort The port to listen at.
- * @param logger the logger to output log information.
- * @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
+ * @param actorSystemName Name of the started ActorSystem.
+ * @param logger The logger to output log information.
+ * @param actorSystemExecutorConfiguration Configuration for the ActorSystem's underlying executor.
+ * @param customConfig Custom Akka config to be combined with the config derived from Flink configuration.
* @return The ActorSystem which has been started.
* @throws Exception
*/
- public static ActorSystem startActorSystem(
+ public static ActorSystem startLocalActorSystem(
Configuration configuration,
String actorSystemName,
- String listeningAddress,
- int listeningPort,
Logger logger,
- ActorSystemExecutorConfiguration actorSystemExecutorConfiguration) throws Exception {
+ ActorSystemExecutorConfiguration actorSystemExecutorConfiguration,
+ Config customConfig) throws Exception {
- String hostPortUrl = NetUtils.unresolvedHostAndPortToNormalizedString(listeningAddress, listeningPort);
- logger.info("Trying to start actor system at {}", hostPortUrl);
+ logger.info("Trying to start local actor");
Review comment:
```suggestion
logger.info("Trying to start local actor system");
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396559803
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/jobmaster/JobMaster.java
##########
@@ -571,9 +572,16 @@ private void releaseEmptyTaskManager(ResourceID resourceId) {
@Override
public CompletableFuture<RegistrationResponse> registerTaskManager(
final String taskManagerRpcAddress,
- final TaskManagerLocation taskManagerLocation,
+ final TaskManagerUnresolvedLocation taskManagerUnresolvedLocation,
final Time timeout) {
+ final TaskManagerLocation taskManagerLocation;
+ try {
+ taskManagerLocation = TaskManagerLocation.fromUnresolvedLocation(taskManagerUnresolvedLocation);
Review comment:
Technically speaking, the `JobMaster` would not have to resolve the TM's address iirc. I think only another TM would need to resolve the address. We could address this as a follow up. The current state is already much better than before.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395464715
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -351,21 +351,21 @@ public static TaskExecutor startTaskManager(
LOG.info("Starting TaskManager with ResourceID: {}", resourceID);
- InetAddress remoteAddress = InetAddress.getByName(rpcService.getAddress());
+ InetAddress externalAddress = InetAddress.getByName(rpcService.getAddress());
Review comment:
After looking more into this, I'm not entirely sure whether we should support unresolvable external addresses.
The prerequisites are unclear to me. If an external address cannot be resolved by the TaskManager itself, then where can it be resolved? Do we assume all other components (JobManager and other TaskManagers) except for itself can resolve the external address? That sounds like a strong assumption to me, which may not always be true. E.g., deploying a Flink cluster over machines from two different sub-network, each sub-network has multiple machines/TMs. TMs in the same sub-network might not be able to resolve each other's external address. Another example, multiple TMs on the same machine.
I don't see a good way to cover all such cases. If we can only support the special case where external addresses can be resolved by all other components, I'm not sure whether we should do it given the efforts needed. Alternatively, user might use IP addresses to workaround in such cases.
WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396889856
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniCluster.java
##########
@@ -261,26 +258,30 @@ public void start() throws Exception {
// bring up all the RPC services
LOG.info("Starting RPC Service(s)");
- AkkaRpcServiceConfiguration akkaRpcServiceConfig = AkkaRpcServiceConfiguration.fromConfiguration(configuration);
-
final RpcServiceFactory dispatcherResourceManagreComponentRpcServiceFactory;
if (useSingleRpcService) {
// we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, false, null);
+ commonRpcService = createLocalRpcService(configuration);
final CommonRpcServiceFactory commonRpcServiceFactory = new CommonRpcServiceFactory(commonRpcService);
taskManagerRpcServiceFactory = commonRpcServiceFactory;
dispatcherResourceManagreComponentRpcServiceFactory = commonRpcServiceFactory;
} else {
- // we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, true, null);
// start a new service per component, possibly with custom bind addresses
final String jobManagerBindAddress = miniClusterConfiguration.getJobManagerBindAddress();
final String taskManagerBindAddress = miniClusterConfiguration.getTaskManagerBindAddress();
+ final String jobManagerBindPort = miniClusterConfiguration.getJobManagerBindPortRange();
+ final String taskManagerBindPort = miniClusterConfiguration.getTaskManagerBindPortRange();
+
+ dispatcherResourceManagreComponentRpcServiceFactory =
+ new DedicatedRpcServiceFactory(configuration, jobManagerBindAddress, jobManagerBindPort);
Review comment:
I think you are right. A `DedicatedRpcServiceFactory` might be used to create multiple RPC services. In such cases, the current implementation will lead to conflicts on binding ports. To that end, I would agree with not supporting the binding port option and rely on the random port.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396570575
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/KvStateService.java
##########
@@ -170,7 +170,7 @@ public static KvStateService fromConfiguration(TaskManagerServicesConfiguration
int numProxyServerQueryThreads = qsConfig.numProxyQueryThreads() == 0 ?
taskManagerServicesConfiguration.getNumberOfSlots() : qsConfig.numProxyQueryThreads();
kvClientProxy = QueryableStateUtils.createKvStateClientProxy(
- taskManagerServicesConfiguration.getTaskManagerAddress(),
+ taskManagerServicesConfiguration.getExternalAddress(),
Review comment:
Not sure, whether this change is valid. The problem is that the underlying Netty server uses this address to bind against. If the external address is not resolvable, it might fail.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386810708
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/rpc/akka/AkkaRpcServiceBuilder.java
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.rpc.akka;
+
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.runtime.akka.AkkaUtils;
+import org.apache.flink.runtime.clusterframework.BootstrapTools;
+import org.apache.flink.util.NetUtils;
+import org.apache.flink.util.Preconditions;
+
+import akka.actor.ActorSystem;
+import com.typesafe.config.Config;
+import org.slf4j.Logger;
+
+import javax.annotation.Nullable;
+
+import java.net.InetAddress;
+
+/**
+ * Builder for {@link AkkaRpcService}.
+ */
+public class AkkaRpcServiceBuilder {
Review comment:
I think whether we make it an inner class or not should only depend on whether we need to leverage the characteristic of the inner class mechanism. It has nothing to do with how many tests we need to modify or how complicated the `AkkaRpcService` is.
In this case, we could get benefit from making it an inner class of `AkkaRpcService` is:
- For both AkkaRpcService and Builder, we could prevent unexpected access to their constructor.
- We could expose two static functions whose use case is well explained by the method name, instead of two constructors only different in param list.
The alternative, which annotates the constructor as `VisibleForTesting` and put the builder inside `AkkaRpcServiceUtils`, also LGTM. In this way, we could only build an `AkkaRpcService` through the utility class. If it needs lots of effort to change the test cases, I think it would be good.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-601536922
Thanks for the review @tillrohrmann.
> A quick question for my understanding. In the e2e test we give every container an external address which is equal to the hosts ip address, right? How can the docker container route the packages for this address if the docker network does not happen to use the same subnet as the host's IP address? Can docker container talk to services running in the host's network? I guess I don't understand Docker good enough...
As I understand, all the packages whose destination is not the docker network will be routed to the host. It depends on the host's setting whether and where to further route the package, which, without any special settings, should be simply forward the package like any other package sent from the host. Accessing other services running on the host should have no difference from accessing any public Internet address from inside a docker container.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396535358
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniCluster.java
##########
@@ -269,19 +269,33 @@ public void start() throws Exception {
} else {
// start a new service per component, possibly with custom bind addresses
+ final String jobManagerExternalAddress = miniClusterConfiguration.getJobManagerExternalAddress();
+ final String taskManagerExternalAddress = miniClusterConfiguration.getTaskManagerExternalAddress();
+ final String jobManagerExternalPortRange = miniClusterConfiguration.getJobManagerExternalPortRange();
+ final String taskManagerExternalPortRange = miniClusterConfiguration.getTaskManagerExternalPortRange();
final String jobManagerBindAddress = miniClusterConfiguration.getJobManagerBindAddress();
final String taskManagerBindAddress = miniClusterConfiguration.getTaskManagerBindAddress();
- final String jobManagerBindPort = miniClusterConfiguration.getJobManagerBindPortRange();
- final String taskManagerBindPort = miniClusterConfiguration.getTaskManagerBindPortRange();
+ final int jobManagerBindPort = miniClusterConfiguration.getJobManagerBindPort();
+ final int taskManagerBindPort = miniClusterConfiguration.getTaskManagerBindPort();
dispatcherResourceManagreComponentRpcServiceFactory =
- new DedicatedRpcServiceFactory(configuration, jobManagerBindAddress, jobManagerBindPort);
+ new DedicatedRpcServiceFactory(
+ configuration,
+ jobManagerExternalAddress,
+ jobManagerExternalPortRange,
Review comment:
Ok I see you have overridden the default `JobManagerOptions#PORT` value and set it to `0`. This should work.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #11284: [FLINK-15911][runtime]
Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot commented on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593402601
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 4683aaeae4a1ee29a403bf6495f2065c539b224f (Mon Mar 02 13:30:47 UTC 2020)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151875334",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154513428",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6492",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6507",
"triggerID" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"triggerType" : "PUSH"
}, {
"hash" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154538899",
"triggerID" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ff670f742762f45214ca32df966e02f4dbe6f758 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/154538899) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6507)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395184524
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
##########
@@ -181,104 +169,115 @@ public static ActorSystem startActorSystem(
// if we come here, we have exhausted the port range
throw new BindException("Could not start actor system on any port in port range "
- + portRangeDefinition);
+ + externalPortRange);
}
/**
- * Starts an Actor System at a specific port.
- *
+ * Starts a remote Actor System at given address and specific port.
* @param configuration The Flink configuration.
- * @param listeningAddress The address to listen at.
- * @param listeningPort The port to listen at.
+ * @param actorSystemName Name of the started {@link ActorSystem}
+ * @param externalAddress The external address to access the ActorSystem.
+ * @param externalPort The external port to access the ActorSystem.
+ * @param bindAddress The local address to bind to.
+ * @param bindPort The local port to bind to.
* @param logger the logger to output log information.
+ * @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
+ * @param customConfig Custom Akka config to be combined with the config derived from Flink configuration.
* @return The ActorSystem which has been started.
* @throws Exception
*/
- public static ActorSystem startActorSystem(
+ private static ActorSystem startRemoteActorSystem(
Configuration configuration,
- String listeningAddress,
- int listeningPort,
- Logger logger) throws Exception {
- return startActorSystem(
- configuration,
- listeningAddress,
- listeningPort,
- logger,
- ForkJoinExecutorConfiguration.fromConfiguration(configuration));
- }
+ String actorSystemName,
+ String externalAddress,
+ int externalPort,
+ String bindAddress,
+ int bindPort,
+ Logger logger,
+ ActorSystemExecutorConfiguration actorSystemExecutorConfiguration,
+ Config customConfig) throws Exception {
- /**
- * Starts an Actor System at a specific port.
- * @param configuration The Flink configuration.
- * @param listeningAddress The address to listen at.
- * @param listeningPort The port to listen at.
- * @param logger the logger to output log information.
- * @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
- * @return The ActorSystem which has been started.
- * @throws Exception
- */
- public static ActorSystem startActorSystem(
- Configuration configuration,
- String listeningAddress,
- int listeningPort,
- Logger logger,
- ActorSystemExecutorConfiguration actorSystemExecutorConfiguration) throws Exception {
- return startActorSystem(
- configuration,
- AkkaUtils.getFlinkActorSystemName(),
- listeningAddress,
- listeningPort,
- logger,
- actorSystemExecutorConfiguration);
+ String externalHostPortUrl = NetUtils.unresolvedHostAndPortToNormalizedString(externalAddress, externalPort);
+ String bindHostPortUrl = NetUtils.unresolvedHostAndPortToNormalizedString(bindAddress, bindPort);
+ logger.info("Trying to start actor system, external address {}, bind address {}.", externalHostPortUrl, bindHostPortUrl);
+
+ try {
+ Config akkaConfig = AkkaUtils.getAkkaConfig(
+ configuration,
+ new Some<>(new Tuple2<>(externalAddress, externalPort)),
+ new Some<>(new Tuple2<>(bindAddress, bindPort)),
+ actorSystemExecutorConfiguration.getAkkaConfig());
+
+ if (customConfig != null) {
+ akkaConfig = customConfig.withFallback(akkaConfig);
+ }
+
+ return startActorSystem(akkaConfig, actorSystemName, logger);
+ }
+ catch (Throwable t) {
+ if (t instanceof ChannelException) {
+ Throwable cause = t.getCause();
+ if (cause != null && t.getCause() instanceof BindException) {
+ throw new IOException("Unable to create ActorSystem at address " + externalHostPortUrl +
Review comment:
Maybe we should rather log the `bindHostPortUrl` here since we are looking for a `BindException`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395189461
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniCluster.java
##########
@@ -261,26 +258,30 @@ public void start() throws Exception {
// bring up all the RPC services
LOG.info("Starting RPC Service(s)");
- AkkaRpcServiceConfiguration akkaRpcServiceConfig = AkkaRpcServiceConfiguration.fromConfiguration(configuration);
-
final RpcServiceFactory dispatcherResourceManagreComponentRpcServiceFactory;
if (useSingleRpcService) {
// we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, false, null);
+ commonRpcService = createLocalRpcService(configuration);
final CommonRpcServiceFactory commonRpcServiceFactory = new CommonRpcServiceFactory(commonRpcService);
taskManagerRpcServiceFactory = commonRpcServiceFactory;
dispatcherResourceManagreComponentRpcServiceFactory = commonRpcServiceFactory;
} else {
- // we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, true, null);
// start a new service per component, possibly with custom bind addresses
final String jobManagerBindAddress = miniClusterConfiguration.getJobManagerBindAddress();
final String taskManagerBindAddress = miniClusterConfiguration.getTaskManagerBindAddress();
+ final String jobManagerBindPort = miniClusterConfiguration.getJobManagerBindPortRange();
Review comment:
Do we support ranges for the bind port?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396983930
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/KvStateService.java
##########
@@ -170,7 +170,7 @@ public static KvStateService fromConfiguration(TaskManagerServicesConfiguration
int numProxyServerQueryThreads = qsConfig.numProxyQueryThreads() == 0 ?
taskManagerServicesConfiguration.getNumberOfSlots() : qsConfig.numProxyQueryThreads();
kvClientProxy = QueryableStateUtils.createKvStateClientProxy(
- taskManagerServicesConfiguration.getTaskManagerAddress(),
+ taskManagerServicesConfiguration.getExternalAddress(),
Review comment:
Sounds good.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396523713
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniCluster.java
##########
@@ -261,26 +258,30 @@ public void start() throws Exception {
// bring up all the RPC services
LOG.info("Starting RPC Service(s)");
- AkkaRpcServiceConfiguration akkaRpcServiceConfig = AkkaRpcServiceConfiguration.fromConfiguration(configuration);
-
final RpcServiceFactory dispatcherResourceManagreComponentRpcServiceFactory;
if (useSingleRpcService) {
// we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, false, null);
+ commonRpcService = createLocalRpcService(configuration);
final CommonRpcServiceFactory commonRpcServiceFactory = new CommonRpcServiceFactory(commonRpcService);
taskManagerRpcServiceFactory = commonRpcServiceFactory;
dispatcherResourceManagreComponentRpcServiceFactory = commonRpcServiceFactory;
} else {
- // we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, true, null);
// start a new service per component, possibly with custom bind addresses
final String jobManagerBindAddress = miniClusterConfiguration.getJobManagerBindAddress();
final String taskManagerBindAddress = miniClusterConfiguration.getTaskManagerBindAddress();
+ final String jobManagerBindPort = miniClusterConfiguration.getJobManagerBindPortRange();
+ final String taskManagerBindPort = miniClusterConfiguration.getTaskManagerBindPortRange();
+
+ dispatcherResourceManagreComponentRpcServiceFactory =
+ new DedicatedRpcServiceFactory(configuration, jobManagerBindAddress, jobManagerBindPort);
Review comment:
In fact, I think this won't work with the `TestingMiniCluster` and having multiple `numberDispatcherResourceManagerComponents` configured because `JobManagerOptions#PORT` which is returned by calling `miniClusterConfiguration.getJobManagerBindPortRange` will only return a single port.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-602363516
@tillrohrmann, I have addressed your comments.
However, I'm still trying to understand how the following is a common need.
> What we actually need is that for a given TM `t` all other TMs `t'` can resolve the address of `t`.
I find it not natural to simulate the above scenario with Docker in the e2e test. For each container I have to manually add the hostname-ip entries of all other containers except for itself.
Say all the JM and TMs are running in Docker containers. If a given TM `t` cannot resolve its own external address from inside the container, then unless specifically configured like I did in the e2e test, how could the JM and all the other TMs `t'`, which are also inside containers, resolve the address of `t`? If `t'` can resolve the address of `t` by using the same DNS settings as its host, then shouldn't `t'` also use the same DNS settings as its host and be able to resolve its own address?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396521581
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniCluster.java
##########
@@ -261,26 +258,30 @@ public void start() throws Exception {
// bring up all the RPC services
LOG.info("Starting RPC Service(s)");
- AkkaRpcServiceConfiguration akkaRpcServiceConfig = AkkaRpcServiceConfiguration.fromConfiguration(configuration);
-
final RpcServiceFactory dispatcherResourceManagreComponentRpcServiceFactory;
if (useSingleRpcService) {
// we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, false, null);
+ commonRpcService = createLocalRpcService(configuration);
final CommonRpcServiceFactory commonRpcServiceFactory = new CommonRpcServiceFactory(commonRpcService);
taskManagerRpcServiceFactory = commonRpcServiceFactory;
dispatcherResourceManagreComponentRpcServiceFactory = commonRpcServiceFactory;
} else {
- // we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, true, null);
// start a new service per component, possibly with custom bind addresses
final String jobManagerBindAddress = miniClusterConfiguration.getJobManagerBindAddress();
final String taskManagerBindAddress = miniClusterConfiguration.getTaskManagerBindAddress();
+ final String jobManagerBindPort = miniClusterConfiguration.getJobManagerBindPortRange();
+ final String taskManagerBindPort = miniClusterConfiguration.getTaskManagerBindPortRange();
+
+ dispatcherResourceManagreComponentRpcServiceFactory =
+ new DedicatedRpcServiceFactory(configuration, jobManagerBindAddress, jobManagerBindPort);
Review comment:
Do we really want to support a bind port here? The problem is that we would have to support a bind port range because the factory can create multiple rpc services. Not sure whether it makes sense to support the bind port here. Maybe it is good enough to say that we don't support the bind options for the `MiniCluster`. WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151875334",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154513428",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6492",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0be3fd11e11d708b254036e9e91fc71f1fd02181 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/154513428) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6492)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386818991
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/rpc/akka/AkkaRpcServiceBuilder.java
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.rpc.akka;
+
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.runtime.akka.AkkaUtils;
+import org.apache.flink.runtime.clusterframework.BootstrapTools;
+import org.apache.flink.util.NetUtils;
+import org.apache.flink.util.Preconditions;
+
+import akka.actor.ActorSystem;
+import com.typesafe.config.Config;
+import org.slf4j.Logger;
+
+import javax.annotation.Nullable;
+
+import java.net.InetAddress;
+
+/**
+ * Builder for {@link AkkaRpcService}.
+ */
+public class AkkaRpcServiceBuilder {
Review comment:
I think whether we want to make a constructor private, depends on whether allowing it to be accessed from outside may cause problem. E.g., the none argument constructor of `ResourceProfile` must be private because the only case we need it is for constructing the singleton instance `UNKNOWN`. If we allow to create another instance of `ResourceProfile` with this none argument constructor, the created instance may not work properly.
In this case, I don't see such problem in allowing direct access to the constructors. That makes hiding the constructors a nice but not must to have approach, by which we suggest people to choose the more convenient builders.
On the other hand, it is not about how many efforts needed that we do not use builders in the test cases. Currently, the builder does two things: create a `ActorSystem`, and use it to create a `AkkaRpcService`. However, in many test cases, we need to use a custom testing actor system, which is different from the one we created for production. That's way we cannot easily use builders in the tests.
We could also make the builder to support custom actor system. But that complicates the builder logics (what should we do if both custom actor system and the fields for creating the production actor system are provided), and I think it doesn't make sense to do for pure testing purpose.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "PENDING",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* aa1386ad6c2ed2774d659263d7684e777eaccd2d Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/151359451) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395201803
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/ClusterEntrypoint.java
##########
@@ -248,10 +247,14 @@ protected void initializeServices(Configuration configuration) throws Exception
LOG.info("Initializing cluster services.");
synchronized (lock) {
- final String bindAddress = configuration.getString(JobManagerOptions.ADDRESS);
- final String portRange = getRPCPortRange(configuration);
-
- commonRpcService = createRpcService(configuration, bindAddress, portRange);
+ commonRpcService = AkkaRpcServiceUtils.createRemoteRpcService(
+ configuration,
+ configuration.getString(JobManagerOptions.ADDRESS),
+ getRPCPortRange(configuration),
+ configuration.getString(JobManagerOptions.BIND_HOST),
+ configuration.contains(JobManagerOptions.RPC_BIND_PORT) ?
+ configuration.getInteger(JobManagerOptions.RPC_BIND_PORT) :
+ null);
Review comment:
I'm not entirely sure whether a nullable `Integer` is the best way to encode that we have an optional bind port. Maybe consider using `Optional` to better express this contract.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396533105
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniCluster.java
##########
@@ -269,19 +269,33 @@ public void start() throws Exception {
} else {
// start a new service per component, possibly with custom bind addresses
+ final String jobManagerExternalAddress = miniClusterConfiguration.getJobManagerExternalAddress();
+ final String taskManagerExternalAddress = miniClusterConfiguration.getTaskManagerExternalAddress();
+ final String jobManagerExternalPortRange = miniClusterConfiguration.getJobManagerExternalPortRange();
+ final String taskManagerExternalPortRange = miniClusterConfiguration.getTaskManagerExternalPortRange();
final String jobManagerBindAddress = miniClusterConfiguration.getJobManagerBindAddress();
final String taskManagerBindAddress = miniClusterConfiguration.getTaskManagerBindAddress();
- final String jobManagerBindPort = miniClusterConfiguration.getJobManagerBindPortRange();
- final String taskManagerBindPort = miniClusterConfiguration.getTaskManagerBindPortRange();
+ final int jobManagerBindPort = miniClusterConfiguration.getJobManagerBindPort();
+ final int taskManagerBindPort = miniClusterConfiguration.getTaskManagerBindPort();
dispatcherResourceManagreComponentRpcServiceFactory =
- new DedicatedRpcServiceFactory(configuration, jobManagerBindAddress, jobManagerBindPort);
+ new DedicatedRpcServiceFactory(
+ configuration,
+ jobManagerExternalAddress,
+ jobManagerExternalPortRange,
Review comment:
I think `jobManagerExternalPortRange` will be `6123` in the default case, right? If this is the case, then this will break with multiple dispatcher resource manager components being configured for the the `TestingMiniCluster`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151875334",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154513428",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6492",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6507",
"triggerID" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"triggerType" : "PUSH"
}, {
"hash" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154538899",
"triggerID" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"triggerType" : "PUSH"
}, {
"hash" : "57914142eb72e078d48ee32c2b3f89de734fe72a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6555",
"triggerID" : "57914142eb72e078d48ee32c2b3f89de734fe72a",
"triggerType" : "PUSH"
}, {
"hash" : "57914142eb72e078d48ee32c2b3f89de734fe72a",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154724688",
"triggerID" : "57914142eb72e078d48ee32c2b3f89de734fe72a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 57914142eb72e078d48ee32c2b3f89de734fe72a Travis: [SUCCESS](https://travis-ci.com/github/flink-ci/flink/builds/154724688) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6555)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151875334",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154513428",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6492",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6507",
"triggerID" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"triggerType" : "PUSH"
}, {
"hash" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154538899",
"triggerID" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"triggerType" : "PUSH"
}, {
"hash" : "57914142eb72e078d48ee32c2b3f89de734fe72a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6555",
"triggerID" : "57914142eb72e078d48ee32c2b3f89de734fe72a",
"triggerType" : "PUSH"
}, {
"hash" : "57914142eb72e078d48ee32c2b3f89de734fe72a",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154724688",
"triggerID" : "57914142eb72e078d48ee32c2b3f89de734fe72a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 57914142eb72e078d48ee32c2b3f89de734fe72a Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/154724688) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6555)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593407935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5826",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151359451",
"triggerID" : "aa1386ad6c2ed2774d659263d7684e777eaccd2d",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151494400",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5853",
"triggerID" : "6e0a577481f9300c87299b0629fab6dc1b3bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151875334",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5942",
"triggerID" : "3e9df27458782312e53586e53f49cd55c11f4df5",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/154513428",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6492",
"triggerID" : "0be3fd11e11d708b254036e9e91fc71f1fd02181",
"triggerType" : "PUSH"
}, {
"hash" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ff670f742762f45214ca32df966e02f4dbe6f758",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0be3fd11e11d708b254036e9e91fc71f1fd02181 Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/154513428) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6492)
* ff670f742762f45214ca32df966e02f4dbe6f758 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386767581
##########
File path: flink-end-to-end-tests/test-scripts/test_nat.sh
##########
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+source "$(dirname "$0")"/common.sh
+
+DOCKER_MODULE_DIR=${END_TO_END_DIR}/../flink-container/docker
+DOCKER_SCRIPTS=${END_TO_END_DIR}/test-scripts/container-scripts
+DOCKER_IMAGE_BUILD_RETRIES=3
+BUILD_BACKOFF_TIME=5
+
+export FLINK_JOB=org.apache.flink.examples.java.wordcount.WordCount
+export FLINK_DOCKER_IMAGE_NAME=test_nat
+export INPUT_VOLUME=${END_TO_END_DIR}/test-scripts/test-data
+export OUTPUT_VOLUME=${TEST_DATA_DIR}/out
+export INPUT_PATH=/data/test/input
+export OUTPUT_PATH=/data/test/output
+
+export HOST_ADDR=$(get_node_ip | awk '{print $1}')
+export JM_RPC_EX_PORT=10000
+export JM_RPC_IN_PORT=10000
+export TM_1_RPC_EX_PORT=10001
+export TM_2_RPC_EX_PORT=10002
+export TM_RPC_IN_PORT=10000
+export TM_1_DATA_EX_PORT=11001
+export TM_2_DATA_EX_PORT=11002
+export TM_DATA_IN_PORT=11000
+
+RESULT_HASH="72a690412be8928ba239c2da967328a5"
+INPUT_ARGS="--input ${INPUT_PATH}/words"
+
+export FLINK_JOB_ARGUMENTS="${INPUT_ARGS} --output ${OUTPUT_PATH}/docker_wc_out"
+
+build_image() {
Review comment:
We need to avoid code duplication if possible. I think this could be moved to `common_docker.sh`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann edited a comment on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann edited a comment on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-602514974
> @tillrohrmann, I have addressed your comments.
>
> However, I'm still trying to understand how the following is a common need.
>
> > What we actually need is that for a given TM `t` all other TMs `t'` can resolve the address of `t`.
>
> I find it not natural to simulate the above scenario with Docker in the e2e test. For each container I have to manually add the hostname-ip entries of all other containers except for itself.
>
> Say all the JM and TMs are running in Docker containers. If a given TM `t` cannot resolve its own external address from inside the container, then unless specifically configured like I did in the e2e test, how could the JM and all the other TMs `t'`, which are also inside containers, resolve the address of `t`? If `t'` can resolve the address of `t` by using the same DNS settings as its host, then shouldn't `t'` also use the same DNS settings as its host and be able to resolve its own address?
I would assume that in most setups every `t` would be able to resolve all of the others `t'` addresses and also its own. But this is a stronger assumption than saying every `t'` needs to be able to resolve `t` address and hence it breaks some setups (as people complained on the ML/SO).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-603067473
@tillrohrmann Thanks for the review. I've addressed the comments.
Regarding queryable state, I did take a look but couldn't find an easy way to separate the external and bind address/port for it. It uses Netty `ServerBootstrap` (in `AbstractServerBase`), which seems not providing any API for separating external & bind. I guess it requires more background knowledge than I have to touch this feature.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395208308
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniClusterConfiguration.java
##########
@@ -102,21 +102,21 @@ public int getNumTaskManagers() {
public String getJobManagerBindAddress() {
return commonBindAddress != null ?
commonBindAddress :
- configuration.getString(JobManagerOptions.ADDRESS, "localhost");
+ configuration.getString(JobManagerOptions.BIND_HOST, "localhost");
}
public String getTaskManagerBindAddress() {
return commonBindAddress != null ?
commonBindAddress :
- configuration.getString(TaskManagerOptions.HOST, "localhost");
+ configuration.getString(TaskManagerOptions.BIND_HOST, "localhost");
}
- public String getJobManagerBindPortRange() {
- return String.valueOf(configuration.getInteger(JobManagerOptions.PORT, 0));
+ public int getJobManagerBindPort() {
+ return configuration.getInteger(JobManagerOptions.RPC_BIND_PORT, 0);
}
- public String getTaskManagerBindPortRange() {
- return configuration.getString(TaskManagerOptions.RPC_PORT);
+ public int getTaskManagerBindPort() {
+ return configuration.getInteger(TaskManagerOptions.RPC_BIND_PORT, 0);
Review comment:
With this change, the `MiniCluster` behaves a bit different from the rest of Flink because we ignore the external hostname and ports. This might not be a big problem because the `MiniCluster` is used for local processes but it could lead to confusion if `MiniCluster` user start configuring `JobManagerOptions.ADDRESS`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-595001635
@tillrohrmann
I checked the test failures.
The failures on Azure are all unrelated.
The NAT e2e test failure on Travis is cause by error message in logs.
- The job is executed successfully, with correct result outputted.
- The error message is about the RM RPC service not started yet when one of the TMs tries to connect to it.
- The following JM log shows that the message is successfully received before discarded, indicating the TM has no problem resolving the correct RM address and RPC port.
`The rpc endpoint org.apache.flink.runtime.resourcemanager.StandaloneResourceManager has not been started yet. Discarding message org.apache.flink.runtime.rpc.messages.RemoteFencedMessage until processing is started.`
- I think this is not a real problem, because TM will retry connecting to RM later.
I've set `skip_check_exceptions` for this test case. I think relying on the result hash check should be enough.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on issue #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on issue #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#issuecomment-593843946
Thanks @KarmaGYZ, I've addressed the comments, and travis has gave a green light. https://travis-ci.org/xintongsong/flink/builds/657623133
@tillrohrmann, @zhijiangW Could you please also help take a look at this PR? Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396539827
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/KvStateService.java
##########
@@ -181,7 +181,7 @@ public static KvStateService fromConfiguration(TaskManagerServicesConfiguration
int numStateServerQueryThreads = qsConfig.numStateQueryThreads() == 0 ?
taskManagerServicesConfiguration.getNumberOfSlots() : qsConfig.numStateQueryThreads();
kvStateServer = QueryableStateUtils.createKvStateServer(
Review comment:
Would we need for the `kvClientProxy` and the `kvStateServer` also a bind port? If yes, then this could be a follow up issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395563227
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniCluster.java
##########
@@ -261,26 +258,30 @@ public void start() throws Exception {
// bring up all the RPC services
LOG.info("Starting RPC Service(s)");
- AkkaRpcServiceConfiguration akkaRpcServiceConfig = AkkaRpcServiceConfiguration.fromConfiguration(configuration);
-
final RpcServiceFactory dispatcherResourceManagreComponentRpcServiceFactory;
if (useSingleRpcService) {
// we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, false, null);
+ commonRpcService = createLocalRpcService(configuration);
final CommonRpcServiceFactory commonRpcServiceFactory = new CommonRpcServiceFactory(commonRpcService);
taskManagerRpcServiceFactory = commonRpcServiceFactory;
dispatcherResourceManagreComponentRpcServiceFactory = commonRpcServiceFactory;
} else {
- // we always need the 'commonRpcService' for auxiliary calls
- commonRpcService = createRpcService(akkaRpcServiceConfig, true, null);
// start a new service per component, possibly with custom bind addresses
final String jobManagerBindAddress = miniClusterConfiguration.getJobManagerBindAddress();
final String taskManagerBindAddress = miniClusterConfiguration.getTaskManagerBindAddress();
+ final String jobManagerBindPort = miniClusterConfiguration.getJobManagerBindPortRange();
+ final String taskManagerBindPort = miniClusterConfiguration.getTaskManagerBindPortRange();
Review comment:
Same here: Please rename `getTaskManagerBindPortRange` into `getTaskManagerBindPort`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386811424
##########
File path: flink-end-to-end-tests/test-scripts/test_nat.sh
##########
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+source "$(dirname "$0")"/common.sh
+
+DOCKER_MODULE_DIR=${END_TO_END_DIR}/../flink-container/docker
+DOCKER_SCRIPTS=${END_TO_END_DIR}/test-scripts/container-scripts
+DOCKER_IMAGE_BUILD_RETRIES=3
+BUILD_BACKOFF_TIME=5
+
+export FLINK_JOB=org.apache.flink.examples.java.wordcount.WordCount
+export FLINK_DOCKER_IMAGE_NAME=test_nat
+export INPUT_VOLUME=${END_TO_END_DIR}/test-scripts/test-data
+export OUTPUT_VOLUME=${TEST_DATA_DIR}/out
+export INPUT_PATH=/data/test/input
+export OUTPUT_PATH=/data/test/output
+
+export HOST_ADDR=$(get_node_ip | awk '{print $1}')
+export JM_RPC_EX_PORT=10000
+export JM_RPC_IN_PORT=10000
+export TM_1_RPC_EX_PORT=10001
+export TM_2_RPC_EX_PORT=10002
+export TM_RPC_IN_PORT=10000
+export TM_1_DATA_EX_PORT=11001
+export TM_2_DATA_EX_PORT=11002
+export TM_DATA_IN_PORT=11000
+
+RESULT_HASH="72a690412be8928ba239c2da967328a5"
+INPUT_ARGS="--input ${INPUT_PATH}/words"
+
+export FLINK_JOB_ARGUMENTS="${INPUT_ARGS} --output ${OUTPUT_PATH}/docker_wc_out"
+
+build_image() {
+ ./build.sh --from-local-dist --job-artifacts ${FLINK_DIR}/examples/batch/WordCount.jar --image-name ${FLINK_DOCKER_IMAGE_NAME}
+}
+
+# user inside the container must be able to create files, this is a workaround in-container permissions
+mkdir -p $OUTPUT_VOLUME
+chmod 777 $OUTPUT_VOLUME
Review comment:
My bad. Let's keep it to 777.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396578616
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -351,21 +351,21 @@ public static TaskExecutor startTaskManager(
LOG.info("Starting TaskManager with ResourceID: {}", resourceID);
- InetAddress remoteAddress = InetAddress.getByName(rpcService.getAddress());
+ InetAddress externalAddress = InetAddress.getByName(rpcService.getAddress());
Review comment:
The problem with the external address is that it might resolve to an IP which is not known to the `TM`. This, of course, could also point towards a broken setup but if people want to run a `TM` behind a NAT, this could be a problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
xintongsong commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r386789347
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/rpc/akka/AkkaRpcServiceBuilder.java
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.rpc.akka;
+
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.runtime.akka.AkkaUtils;
+import org.apache.flink.runtime.clusterframework.BootstrapTools;
+import org.apache.flink.util.NetUtils;
+import org.apache.flink.util.Preconditions;
+
+import akka.actor.ActorSystem;
+import com.typesafe.config.Config;
+import org.slf4j.Logger;
+
+import javax.annotation.Nullable;
+
+import java.net.InetAddress;
+
+/**
+ * Builder for {@link AkkaRpcService}.
+ */
+public class AkkaRpcServiceBuilder {
Review comment:
I don't get it.
- Constructor of `AkkaRpcService` is used in various of test cases. Unless we want to modify them all, I don't think we can make the constructor private. Maybe it makes sense to annotate the constructor as `VisibleForTesting`, because without the test cases it can be package visibility.
- If we cannot make constructor of `AkkaRpcService` private anyway, then I would prefer to put `AkkaRpcServiceBuilder` in a separate file. The file `AkkaRpcService.java` is already quite long. No need to further complicate it.
- I don't see how `AkkaRpcService#local(remote)RpcServiceBuilder` is more readable compared to `AkkaRpcServiceUtils#local(remote)ServiceBuilder`.
- It probably make sense to make the constructor of the builder private, since we already have `AkkaRpcServiceUtils#local(remote)ServiceBuilder`. In that case, I would tend to put the builder inside `AkkaRpcServiceUtils` rather than `AkkaRpcService`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r396547487
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/jobmaster/JobMasterGateway.java
##########
@@ -167,13 +167,13 @@ void failSlot(final ResourceID taskManagerId,
* Registers the task manager at the job manager.
*
* @param taskManagerRpcAddress the rpc address of the task manager
- * @param taskManagerLocation location of the task manager
+ * @param taskManagerUnresolvedLocation location of the task manager
Review comment:
```suggestion
* @param taskManagerUnresolvedLocation unresolved location of the task manager
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #11284:
[FLINK-15911][runtime] Make Flink work with NAT.
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11284: [FLINK-15911][runtime] Make Flink work with NAT.
URL: https://github.com/apache/flink/pull/11284#discussion_r395182006
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
##########
@@ -82,92 +82,80 @@
.build();
/**
- * Starts an ActorSystem with the given configuration listening at the address/ports.
+ * Starts a remote ActorSystem at given address and specific port range.
* @param configuration The Flink configuration
- * @param listeningAddress The address to listen at.
- * @param portRangeDefinition The port range to choose a port from.
+ * @param externalAddress The external address to access the ActorSystem.
+ * @param externalPortRange The choosing range of the external port to access the ActorSystem.
* @param logger The logger to output log information.
* @return The ActorSystem which has been started
* @throws Exception Thrown when actor system cannot be started in specified port range
*/
- public static ActorSystem startActorSystem(
+ @VisibleForTesting
+ public static ActorSystem startRemoteActorSystem(
Configuration configuration,
- String listeningAddress,
- String portRangeDefinition,
+ String externalAddress,
+ String externalPortRange,
Logger logger) throws Exception {
- return startActorSystem(
- configuration,
- listeningAddress,
- portRangeDefinition,
- logger,
- ForkJoinExecutorConfiguration.fromConfiguration(configuration));
- }
-
- /**
- * Starts an ActorSystem with the given configuration listening at the address/ports.
- *
- * @param configuration The Flink configuration
- * @param listeningAddress The address to listen at.
- * @param portRangeDefinition The port range to choose a port from.
- * @param logger The logger to output log information.
- * @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
- * @return The ActorSystem which has been started
- * @throws Exception Thrown when actor system cannot be started in specified port range
- */
- public static ActorSystem startActorSystem(
- Configuration configuration,
- String listeningAddress,
- String portRangeDefinition,
- Logger logger,
- @Nonnull ActorSystemExecutorConfiguration actorSystemExecutorConfiguration) throws Exception {
- return startActorSystem(
+ return startRemoteActorSystem(
configuration,
AkkaUtils.getFlinkActorSystemName(),
- listeningAddress,
- portRangeDefinition,
+ externalAddress,
+ externalPortRange,
+ NetUtils.getWildcardIPAddress(),
+ -1,
logger,
- actorSystemExecutorConfiguration);
+ ForkJoinExecutorConfiguration.fromConfiguration(configuration),
+ null);
}
/**
- * Starts an ActorSystem with the given configuration listening at the address/ports.
+ * Starts a remote ActorSystem at given address and specific port range.
*
* @param configuration The Flink configuration
* @param actorSystemName Name of the started {@link ActorSystem}
- * @param listeningAddress The address to listen at.
- * @param portRangeDefinition The port range to choose a port from.
+ * @param externalAddress The external address to access the ActorSystem.
+ * @param externalPortRange The choosing range of the external port to access the ActorSystem.
+ * @param bindAddress The local address to bind to.
+ * @param bindPort The local port to bind to. If negative, external port will be used.
* @param logger The logger to output log information.
* @param actorSystemExecutorConfiguration configuration for the ActorSystem's underlying executor
+ * @param customConfig Custom Akka config to be combined with the config derived from Flink configuration.
* @return The ActorSystem which has been started
* @throws Exception Thrown when actor system cannot be started in specified port range
*/
- public static ActorSystem startActorSystem(
+ public static ActorSystem startRemoteActorSystem(
Configuration configuration,
String actorSystemName,
- String listeningAddress,
- String portRangeDefinition,
+ String externalAddress,
+ String externalPortRange,
+ String bindAddress,
+ int bindPort,
Review comment:
```suggestion
Optional<Int> bindPort,
```
might express the contract a bit better.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services