You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2021/08/05 08:36:00 UTC
[GitHub] [flink] dmvk opened a new pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
dmvk opened a new pull request #16720:
URL: https://github.com/apache/flink/pull/16720
## What is the purpose of the change
Currently, we've been using kubernetes api server url, when constructing the connection string for REST gateway. While this might work in some cases (eg. api is not behind loadbalancer), this is wrong. The correct approach is to construct address in form of `<nodeIp>:<nodePort>`, where nodeIp can be ip of any kubernetes node (that is running kube-proxy).
See [kubernetes docs](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport) for more details.
Unfortunately kubernetes nodes can have multiple IPs, with different type (external, internal), assigned to them and it's impossible to make this solution bullet proof without making assumptions about underlying infrastructure. Therefore I've introduced a new config option `kubernetes.rest-service.exposed.node-port-address-type`, that let's you choose which IP type you want to search for.
## Brief change log
- `Fabric8FlinkKubeClient#getLoadBalancerRestEndpoint` now searches for any node ip with matching address type, that is used for connection string construction
- nit: Fixed broken formatting due to deeply nested indentation
## Verifying this change
Added new unit tests for the change.
## Does this pull request potentially affect one of the following parts:
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes)
## Documentation
- Does this pull request introduce a new feature? (yes)
- If yes, how is the feature documented? (docs)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dmvk commented on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
dmvk commented on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-898297034
This is a breaking change (even thought it's fix), so I'm adding release notes for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-893276887
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 30c9b516b57b8e24851fe34cfd06d41e8a6ee288 (Thu Aug 05 08:39:17 UTC 2021)
✅no warnings
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-893298023
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9120cbffaf457c0b5bd7d973c085d28b6cc6be76 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-893298023
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597",
"triggerID" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"triggerType" : "PUSH"
}, {
"hash" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21703",
"triggerID" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"triggerType" : "PUSH"
}, {
"hash" : "0028013382d5c85307fab39ad5d463197c6b9a7b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21985",
"triggerID" : "0028013382d5c85307fab39ad5d463197c6b9a7b",
"triggerType" : "PUSH"
}, {
"hash" : "09ec90e7c7f7ac398e7f5771ea7f72459625e9a4",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "09ec90e7c7f7ac398e7f5771ea7f72459625e9a4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0028013382d5c85307fab39ad5d463197c6b9a7b Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21985)
* 09ec90e7c7f7ac398e7f5771ea7f72459625e9a4 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-893298023
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597",
"triggerID" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"triggerType" : "PUSH"
}, {
"hash" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21703",
"triggerID" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"triggerType" : "PUSH"
}, {
"hash" : "0028013382d5c85307fab39ad5d463197c6b9a7b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0028013382d5c85307fab39ad5d463197c6b9a7b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6637de3fb6b7ef24959c396d6db5dfed8ebb8551 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21703)
* 0028013382d5c85307fab39ad5d463197c6b9a7b UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] zentol commented on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
zentol commented on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-894021231
The Kubernetes E2E tests are failing. https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=21597&view=logs&j=c88eea3b-64a0-564d-0031-9fdcd7b8abee&t=070ff179-953e-5bda-71fa-d6599415701c
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-893298023
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597",
"triggerID" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"triggerType" : "PUSH"
}, {
"hash" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21703",
"triggerID" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"triggerType" : "PUSH"
}, {
"hash" : "0028013382d5c85307fab39ad5d463197c6b9a7b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21985",
"triggerID" : "0028013382d5c85307fab39ad5d463197c6b9a7b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0028013382d5c85307fab39ad5d463197c6b9a7b Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21985)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-893298023
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597",
"triggerID" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9120cbffaf457c0b5bd7d973c085d28b6cc6be76 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-893298023
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597",
"triggerID" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9120cbffaf457c0b5bd7d973c085d28b6cc6be76 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-893298023
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597",
"triggerID" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"triggerType" : "PUSH"
}, {
"hash" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21703",
"triggerID" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"triggerType" : "PUSH"
}, {
"hash" : "0028013382d5c85307fab39ad5d463197c6b9a7b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21985",
"triggerID" : "0028013382d5c85307fab39ad5d463197c6b9a7b",
"triggerType" : "PUSH"
}, {
"hash" : "09ec90e7c7f7ac398e7f5771ea7f72459625e9a4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=22073",
"triggerID" : "09ec90e7c7f7ac398e7f5771ea7f72459625e9a4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0028013382d5c85307fab39ad5d463197c6b9a7b Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21985)
* 09ec90e7c7f7ac398e7f5771ea7f72459625e9a4 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=22073)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-893298023
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597",
"triggerID" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"triggerType" : "PUSH"
}, {
"hash" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9120cbffaf457c0b5bd7d973c085d28b6cc6be76 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597)
* 6637de3fb6b7ef24959c396d6db5dfed8ebb8551 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] autophagy commented on a change in pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
autophagy commented on a change in pull request #16720:
URL: https://github.com/apache/flink/pull/16720#discussion_r685952953
##########
File path: flink-kubernetes/src/main/java/org/apache/flink/kubernetes/configuration/KubernetesConfigOptions.java
##########
@@ -59,6 +60,25 @@
"The exposed type of the rest service. "
+ "The exposed rest service could be used to access the Flink’s Web UI and REST endpoint.");
+ public static final ConfigOption<NodePortAddressType>
+ REST_SERVICE_EXPOSED_NODE_PORT_ADDRESS_TYPE =
+ key("kubernetes.rest-service.exposed.node-port-address-type")
+ .enumType(NodePortAddressType.class)
+ .defaultValue(NodePortAddressType.ExternalIP)
+ .withDescription(
+ Description.builder()
+ .text(
+ "The user-specified %s that is used for filtering node IPs when constructing %s connection string. This option is only considered when '%s' is set to '%s'.",
Review comment:
Tiny nit: I think "filtering node IPs when constructing **a** %s connection string" would fit better grammatically.
##########
File path: flink-kubernetes/src/main/java/org/apache/flink/kubernetes/kubeclient/Fabric8FlinkKubeClient.java
##########
@@ -441,8 +435,26 @@ private int getRestPortFromExternalService(Service externalService) {
address = loadBalancer.getIngress().get(0).getHostname();
}
} else {
- // Use node port
- address = this.internalClient.getMasterUrl().getHost();
+ // Use node port. Node port is accessible on any node within kubernetes cluster. We'll
+ // only consider IPs with the configured address type.
+ address =
+ internalClient.nodes().list().getItems().stream()
+ .flatMap(node -> node.getStatus().getAddresses().stream())
+ .filter(
+ nodeAddress ->
+ nodePortAddressType
+ .name()
+ .equals(nodeAddress.getType()))
+ .map(NodeAddress::getAddress)
+ .filter(externalIp -> !externalIp.isEmpty())
Review comment:
Small nit: Would it make more sense to name this `address`, rather than `externalIp`, since I assume the map above return an address? As I understand, the address we grab here might be external or internal depending on what the expoed node port address type is set to?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dmvk commented on a change in pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
dmvk commented on a change in pull request #16720:
URL: https://github.com/apache/flink/pull/16720#discussion_r687594396
##########
File path: flink-kubernetes/src/main/java/org/apache/flink/kubernetes/kubeclient/Fabric8FlinkKubeClient.java
##########
@@ -441,8 +435,26 @@ private int getRestPortFromExternalService(Service externalService) {
address = loadBalancer.getIngress().get(0).getHostname();
}
} else {
- // Use node port
- address = this.internalClient.getMasterUrl().getHost();
+ // Use node port. Node port is accessible on any node within kubernetes cluster. We'll
+ // only consider IPs with the configured address type.
+ address =
+ internalClient.nodes().list().getItems().stream()
+ .flatMap(node -> node.getStatus().getAddresses().stream())
+ .filter(
+ nodeAddress ->
+ nodePortAddressType
+ .name()
+ .equals(nodeAddress.getType()))
+ .map(NodeAddress::getAddress)
+ .filter(externalIp -> !externalIp.isEmpty())
Review comment:
address is already defined in the scope, I'll change this to `ip`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dmvk commented on a change in pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
dmvk commented on a change in pull request #16720:
URL: https://github.com/apache/flink/pull/16720#discussion_r687598249
##########
File path: flink-kubernetes/src/main/java/org/apache/flink/kubernetes/configuration/KubernetesConfigOptions.java
##########
@@ -59,6 +60,25 @@
"The exposed type of the rest service. "
+ "The exposed rest service could be used to access the Flink’s Web UI and REST endpoint.");
+ public static final ConfigOption<NodePortAddressType>
+ REST_SERVICE_EXPOSED_NODE_PORT_ADDRESS_TYPE =
+ key("kubernetes.rest-service.exposed.node-port-address-type")
+ .enumType(NodePortAddressType.class)
+ .defaultValue(NodePortAddressType.ExternalIP)
+ .withDescription(
+ Description.builder()
+ .text(
+ "The user-specified %s that is used for filtering node IPs when constructing %s connection string. This option is only considered when '%s' is set to '%s'.",
Review comment:
👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-893276887
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-893298023
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597",
"triggerID" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"triggerType" : "PUSH"
}, {
"hash" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21703",
"triggerID" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9120cbffaf457c0b5bd7d973c085d28b6cc6be76 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597)
* 6637de3fb6b7ef24959c396d6db5dfed8ebb8551 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21703)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-893298023
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597",
"triggerID" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9120cbffaf457c0b5bd7d973c085d28b6cc6be76 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dmvk commented on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
dmvk commented on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-898294308
After giving this some more thoughts, I'm switching default value to `InternalIP`. From a quick research:
- Minikube: Has only InternalIP assigned
- GKE: Has both external and internal, where external is public IP and internal is one from the VPC range. I guess defaulting to public IP make sense here as well.
- On premise kuberentes cluster, where all nodes are L3 routable from internal network and there is now access to nodes from public internet - only InternalIP is assigned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-893298023
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597",
"triggerID" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"triggerType" : "PUSH"
}, {
"hash" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21703",
"triggerID" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6637de3fb6b7ef24959c396d6db5dfed8ebb8551 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21703)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-893298023
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597",
"triggerID" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"triggerType" : "PUSH"
}, {
"hash" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21703",
"triggerID" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"triggerType" : "PUSH"
}, {
"hash" : "0028013382d5c85307fab39ad5d463197c6b9a7b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21985",
"triggerID" : "0028013382d5c85307fab39ad5d463197c6b9a7b",
"triggerType" : "PUSH"
}, {
"hash" : "09ec90e7c7f7ac398e7f5771ea7f72459625e9a4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=22073",
"triggerID" : "09ec90e7c7f7ac398e7f5771ea7f72459625e9a4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 09ec90e7c7f7ac398e7f5771ea7f72459625e9a4 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=22073)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] tillrohrmann closed pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
tillrohrmann closed pull request #16720:
URL: https://github.com/apache/flink/pull/16720
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dmvk edited a comment on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
dmvk edited a comment on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-898294308
After giving this some more thoughts, I'm switching default value to `InternalIP`. From a quick research:
- Minikube: Has only InternalIP assigned
- GKE: Has both external and internal, where external is public IP and internal is one from the VPC range. I guess defaulting to public IP make sense here as well.
- On premise kuberentes cluster, where all nodes are L3 routable from internal network and there is no access to nodes from public internet - only InternalIP is assigned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dmvk commented on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
dmvk commented on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-897538702
@autophagy Thanks for the review! I've addressed all your comments. Do you have any other concerns?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16720: [FLINK-23507] Use IP address of a kubernetes node when constructing node port connection string for the REST gateway.
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16720:
URL: https://github.com/apache/flink/pull/16720#issuecomment-893298023
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21597",
"triggerID" : "9120cbffaf457c0b5bd7d973c085d28b6cc6be76",
"triggerType" : "PUSH"
}, {
"hash" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21703",
"triggerID" : "6637de3fb6b7ef24959c396d6db5dfed8ebb8551",
"triggerType" : "PUSH"
}, {
"hash" : "0028013382d5c85307fab39ad5d463197c6b9a7b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21985",
"triggerID" : "0028013382d5c85307fab39ad5d463197c6b9a7b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6637de3fb6b7ef24959c396d6db5dfed8ebb8551 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21703)
* 0028013382d5c85307fab39ad5d463197c6b9a7b Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21985)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org