You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "elkhand (via GitHub)" <gi...@apache.org> on 2023/09/13 04:54:02 UTC
[GitHub] [flink] elkhand opened a new pull request, #23406: [FLINK-32884] [flink-clients] PyFlink remote execution should support URLs with paths and https scheme
elkhand opened a new pull request, #23406:
URL: https://github.com/apache/flink/pull/23406
[FLINK-32884](https://issues.apache.org/jira/browse/FLINK-32884) PyFlink remote execution should support URLs with paths and https scheme
## What is the purpose of the change
Currently, the `SUBMIT_ARGS=remote -m http://<hostname>:<port>` format. For local execution it works fine `SUBMIT_ARGS=remote -m http://localhost:8081/`, but it does not support the placement of the JobManager behind a proxy or using an Ingress for routing to a specific Flink cluster based on the URL path. In the current scenario, it expects JobManager to access PyFlink jobs at `http://<hostname>:<port>/v1/jobs` endpoint. Mapping to a non-root location, `https://<hostname>:<port>/flink-clusters/namespace/flink_job_deployment/v1/jobs` is not supported.
This will use changes from [FLINK-32885](https://issues.apache.org/jira/browse/FLINK-32885)(https://issues.apache.org/jira/browse/FLINK-32885)
Since RestClusterClient talks to the JobManager via its REST endpoint, the right format for `SUBMIT_ARGS` is a URL with a path (also support for https scheme).
## Brief change log
- Added 2 new options into RestOptions: RestOptions.PATH and RestOptions.PROTOCOL
- Added `https` schema support and jobManagerUrl with path support to RestClusterClient
- Added customHttpHeaders support to RestClusterClient (for example setting auth token from env variable)
- Updated `NetUtils.validateHostPortString()` to handle both `http` and `https` schema for hostPort
## Verifying this change
This change added tests and can be verified as follows:
- Extended flink-clients/src/test/java/org/apache/flink/client/cli/DefaultCLITest.java to cover `http` and `https` schemes
- Added unit test flink-core/src/test/java/org/apache/flink/util/NetUtilsTest.java to check `https` scheme and URL with path
- Extended test flink-clients/src/test/java/org/apache/flink/client/program/rest/RestClusterClientTest.java to validate customHttpHeaders and jobmanagerUrl with path and `https` scheme
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] gaborgsomogyi commented on pull request #23406: [FLINK-32884] [flink-clients] PyFlink remote execution should support URLs with paths and https scheme
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23406:
URL: https://github.com/apache/flink/pull/23406#issuecomment-1724924496
Not sure I understand the feature from user perspective🤔
The new URL is clear, the existing booleans for SSL are clear.
What I would like to ask is what happens when I set `SecurityOptions.SSL_REST_ENABLED = false` and `URL=https://...` and the other way around?
I mean which part of the code is configured by the new path var and which part by the old boolean?
I'm asking it because we're setting `SecurityOptions.SSL_REST_ENABLED` dynamically...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #23406: [FLINK-32884] [flink-clients] PyFlink remote execution should support URLs with paths and https scheme
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #23406:
URL: https://github.com/apache/flink/pull/23406#issuecomment-1716944631
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "291433141e0c930c751c3e990a159b1f1ab9823c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "291433141e0c930c751c3e990a159b1f1ab9823c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 291433141e0c930c751c3e990a159b1f1ab9823c UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] elkhand commented on a diff in pull request #23406: [FLINK-32884] [flink-clients] PyFlink remote execution should support URLs with paths and https scheme
Posted by "elkhand (via GitHub)" <gi...@apache.org>.
elkhand commented on code in PR #23406:
URL: https://github.com/apache/flink/pull/23406#discussion_r1329664714
##########
flink-core/src/main/java/org/apache/flink/configuration/RestOptions.java:
##########
@@ -80,6 +80,23 @@ public class RestOptions {
.withDescription(
"The address that should be used by clients to connect to the server. Attention: This option is respected only if the high-availability configuration is NONE.");
+ /** The path that should be used by clients to interact with the server. */
+ @Documentation.Section(Documentation.Sections.COMMON_HOST_PORT)
+ public static final ConfigOption<String> PATH =
+ key("rest.path")
+ .stringType()
+ .defaultValue("")
+ .withDescription(
+ "The path that should be used by clients to interact to the server which is accessible via URL.");
+
+ /** The protocol that should be used by clients to interact with the server. */
+ @Documentation.Section(Documentation.Sections.COMMON_HOST_PORT)
+ public static final ConfigOption<String> PROTOCOL =
+ key("rest.protocol")
+ .stringType()
+ .defaultValue("http")
+ .withDescription("The protocol to be used for rest endpoint");
Review Comment:
Thanks for the suggestion, makes sense. updated the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] mxm commented on pull request #23406: [FLINK-32884] [flink-clients] PyFlink remote execution should support URLs with paths and https scheme
Posted by "mxm (via GitHub)" <gi...@apache.org>.
mxm commented on PR #23406:
URL: https://github.com/apache/flink/pull/23406#issuecomment-1722527946
Tests are failing due to the new config options: https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=53293&view=logs&j=fc5181b0-e452-5c8f-68de-1097947f6483&t=995c650b-6573-581c-9ce6-7ad4cc038461
You need to follow the steps listed in the error message to generate the documentation for the config options. (line 32660)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] elkhand commented on pull request #23406: [FLINK-32884] [flink-clients] PyFlink remote execution should support URLs with paths and https scheme
Posted by "elkhand (via GitHub)" <gi...@apache.org>.
elkhand commented on PR #23406:
URL: https://github.com/apache/flink/pull/23406#issuecomment-1722651735
> Tests are failing due to the new config options: https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=53293&view=logs&j=fc5181b0-e452-5c8f-68de-1097947f6483&t=995c650b-6573-581c-9ce6-7ad4cc038461
>
> You need to follow the steps listed in the error message to generate the documentation for the config options. (line 32660)
Thank you, fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] mxm commented on a diff in pull request #23406: [FLINK-32884] [flink-clients] PyFlink remote execution should support URLs with paths and https scheme
Posted by "mxm (via GitHub)" <gi...@apache.org>.
mxm commented on code in PR #23406:
URL: https://github.com/apache/flink/pull/23406#discussion_r1324778392
##########
flink-core/src/main/java/org/apache/flink/configuration/RestOptions.java:
##########
@@ -80,6 +80,23 @@ public class RestOptions {
.withDescription(
"The address that should be used by clients to connect to the server. Attention: This option is respected only if the high-availability configuration is NONE.");
+ /** The address that should be used by clients to interact with the server. */
+ @Documentation.Section(Documentation.Sections.COMMON_HOST_PORT)
+ public static final ConfigOption<String> PATH =
+ key("rest.path")
+ .stringType()
+ .defaultValue("")
+ .withDescription(
+ "The path that should be used by clients to interact to the server which is accessible via URL.");
+
+ /** The address that should be used by clients to interact with the server. */
Review Comment:
```suggestion
/** The protocol that should be used by clients to interact with the server. */
```
##########
flink-clients/src/test/java/org/apache/flink/client/cli/DefaultCLITest.java:
##########
@@ -46,6 +46,28 @@ void testCommandLineMaterialization() throws Exception {
assertThat(configuration.get(RestOptions.ADDRESS)).isEqualTo(hostname);
assertThat(configuration.get(RestOptions.PORT)).isEqualTo(port);
+
+ final String httpProtocol = "http";
+ final String hostnameWithHttpScheme = httpProtocol + "://" + hostname;
+ final String[] httpArgs = {"-m", hostnameWithHttpScheme + ':' + port};
+ final CommandLine httpCommandLine = defaultCLI.parseCommandLineOptions(httpArgs, false);
+
+ Configuration newConfiguration = defaultCLI.toConfiguration(httpCommandLine);
+
+ assertThat(newConfiguration.get(RestOptions.ADDRESS)).isEqualTo(hostname);
+ assertThat(newConfiguration.get(RestOptions.PORT)).isEqualTo(port);
+ assertThat(newConfiguration.get(RestOptions.PROTOCOL)).isEqualTo(httpProtocol);
+
+ final String httpsProtocol = "https";
+ final String hostnameWithHttpsScheme = httpsProtocol + "://" + hostname;
+ final String[] httpsArgs = {"-m", hostnameWithHttpsScheme + ':' + port};
+ final CommandLine httpsCommandLine = defaultCLI.parseCommandLineOptions(httpsArgs, false);
+
+ Configuration httpsConfiguration = defaultCLI.toConfiguration(httpsCommandLine);
+
+ assertThat(httpsConfiguration.get(RestOptions.ADDRESS)).isEqualTo(hostname);
+ assertThat(httpsConfiguration.get(RestOptions.PORT)).isEqualTo(port);
+ assertThat(httpsConfiguration.get(RestOptions.PROTOCOL)).isEqualTo(httpsProtocol);
Review Comment:
Can we test the added `PATH` here as well?
##########
flink-core/src/main/java/org/apache/flink/configuration/RestOptions.java:
##########
@@ -80,6 +80,23 @@ public class RestOptions {
.withDescription(
"The address that should be used by clients to connect to the server. Attention: This option is respected only if the high-availability configuration is NONE.");
+ /** The address that should be used by clients to interact with the server. */
Review Comment:
```suggestion
/** The path that should be used by clients to interact with the server. */
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] tweise merged pull request #23406: [FLINK-32884] [flink-clients] PyFlink remote execution should support URLs with paths and https scheme
Posted by "tweise (via GitHub)" <gi...@apache.org>.
tweise merged PR #23406:
URL: https://github.com/apache/flink/pull/23406
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] elkhand commented on a diff in pull request #23406: [FLINK-32884] [flink-clients] PyFlink remote execution should support URLs with paths and https scheme
Posted by "elkhand (via GitHub)" <gi...@apache.org>.
elkhand commented on code in PR #23406:
URL: https://github.com/apache/flink/pull/23406#discussion_r1332375725
##########
flink-clients/src/main/java/org/apache/flink/client/program/rest/RestClusterClient.java:
##########
@@ -255,10 +266,22 @@ private RestClusterClient(
RestClusterClientConfiguration.fromConfiguration(configuration);
this.tempDir = tempDir;
+ this.customHttpHeaders =
+ ClientUtils.readHeadersFromEnvironmentVariable(
+ ConfigConstants.FLINK_REST_CLIENT_HEADERS);
+ jobmanagerUrl =
+ new URL(
+ configuration.getBoolean(SecurityOptions.SSL_REST_ENABLED)
+ ? "https"
+ : "http",
+ configuration.getString(JobManagerOptions.ADDRESS),
+ configuration.getInteger(JobManagerOptions.PORT),
+ configuration.getString(RestOptions.PATH));
+
if (restClient != null) {
this.restClient = restClient;
} else {
- this.restClient = new RestClient(configuration, executorService);
+ this.restClient = RestClient.forUrl(configuration, executorService, jobmanagerUrl);
Review Comment:
Thanks @gaborgsomogyi .
Added tests for jobmanagerUrl and customHeaders.
cc: @mxm .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] gaborgsomogyi commented on a diff in pull request #23406: [FLINK-32884] [flink-clients] PyFlink remote execution should support URLs with paths and https scheme
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on code in PR #23406:
URL: https://github.com/apache/flink/pull/23406#discussion_r1329728836
##########
flink-clients/src/main/java/org/apache/flink/client/cli/DefaultCLI.java:
##########
@@ -68,6 +76,12 @@ public Configuration toConfiguration(CommandLine commandLine) throws FlinkExcept
return resultingConfiguration;
}
+ private static boolean isHttpsProtocol(URL url) {
+ return url.getProtocol() == null
Review Comment:
Maybe: `return url.getProtocol() != null && (url.getProtocol().equalsIgnoreCase("https"));`
##########
flink-clients/src/main/java/org/apache/flink/client/program/rest/RestClusterClient.java:
##########
@@ -255,10 +266,22 @@ private RestClusterClient(
RestClusterClientConfiguration.fromConfiguration(configuration);
this.tempDir = tempDir;
+ this.customHttpHeaders =
+ ClientUtils.readHeadersFromEnvironmentVariable(
+ ConfigConstants.FLINK_REST_CLIENT_HEADERS);
+ jobmanagerUrl =
+ new URL(
+ configuration.getBoolean(SecurityOptions.SSL_REST_ENABLED)
Review Comment:
There is already a function to decide this: `SecurityOptions.isRestSSLEnabled(config)`
##########
flink-core/src/main/java/org/apache/flink/util/NetUtils.java:
##########
@@ -104,7 +104,10 @@ public static InetSocketAddress parseHostPortAddress(String hostPort) {
*/
private static URL validateHostPortString(String hostPort) {
try {
- URL u = new URL("http://" + hostPort);
+ URL u =
+ (hostPort.startsWith("http://") || hostPort.startsWith("https://"))
Review Comment:
Some places we use `equalsIgnoreCase` for protocol but here not allowed. It would be good to make it consistent. Either not allow or allow all the places.
##########
flink-clients/src/main/java/org/apache/flink/client/program/rest/RestClusterClient.java:
##########
@@ -255,10 +266,22 @@ private RestClusterClient(
RestClusterClientConfiguration.fromConfiguration(configuration);
this.tempDir = tempDir;
+ this.customHttpHeaders =
+ ClientUtils.readHeadersFromEnvironmentVariable(
+ ConfigConstants.FLINK_REST_CLIENT_HEADERS);
+ jobmanagerUrl =
+ new URL(
+ configuration.getBoolean(SecurityOptions.SSL_REST_ENABLED)
+ ? "https"
+ : "http",
+ configuration.getString(JobManagerOptions.ADDRESS),
+ configuration.getInteger(JobManagerOptions.PORT),
+ configuration.getString(RestOptions.PATH));
+
if (restClient != null) {
this.restClient = restClient;
} else {
- this.restClient = new RestClient(configuration, executorService);
+ this.restClient = RestClient.forUrl(configuration, executorService, jobmanagerUrl);
Review Comment:
This is never tested with `RestOptions.PATH`. Plz check `testRESTManualConfigurationOverride`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] mxm commented on a diff in pull request #23406: [FLINK-32884] [flink-clients] PyFlink remote execution should support URLs with paths and https scheme
Posted by "mxm (via GitHub)" <gi...@apache.org>.
mxm commented on code in PR #23406:
URL: https://github.com/apache/flink/pull/23406#discussion_r1328331551
##########
flink-core/src/main/java/org/apache/flink/configuration/RestOptions.java:
##########
@@ -80,6 +80,23 @@ public class RestOptions {
.withDescription(
"The address that should be used by clients to connect to the server. Attention: This option is respected only if the high-availability configuration is NONE.");
+ /** The path that should be used by clients to interact with the server. */
+ @Documentation.Section(Documentation.Sections.COMMON_HOST_PORT)
+ public static final ConfigOption<String> PATH =
+ key("rest.path")
+ .stringType()
+ .defaultValue("")
+ .withDescription(
+ "The path that should be used by clients to interact to the server which is accessible via URL.");
+
+ /** The protocol that should be used by clients to interact with the server. */
+ @Documentation.Section(Documentation.Sections.COMMON_HOST_PORT)
+ public static final ConfigOption<String> PROTOCOL =
Review Comment:
Can this ever be anything else than `http` or `https`? I'm thinking that a bool would suffice here.
##########
flink-core/src/main/java/org/apache/flink/configuration/RestOptions.java:
##########
@@ -80,6 +80,23 @@ public class RestOptions {
.withDescription(
"The address that should be used by clients to connect to the server. Attention: This option is respected only if the high-availability configuration is NONE.");
+ /** The path that should be used by clients to interact with the server. */
+ @Documentation.Section(Documentation.Sections.COMMON_HOST_PORT)
+ public static final ConfigOption<String> PATH =
+ key("rest.path")
+ .stringType()
+ .defaultValue("")
+ .withDescription(
+ "The path that should be used by clients to interact to the server which is accessible via URL.");
+
+ /** The protocol that should be used by clients to interact with the server. */
+ @Documentation.Section(Documentation.Sections.COMMON_HOST_PORT)
+ public static final ConfigOption<String> PROTOCOL =
+ key("rest.protocol")
+ .stringType()
+ .defaultValue("http")
+ .withDescription("The protocol to be used for rest endpoint");
Review Comment:
The general pattern we follow in the Flink configuration is to control the protocol via the `ssl.enabled` flag, both for server and client. Do you think it would suffice to use this flag instead of adding another flag?
- https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/security/security-ssl/#configuring-ssl
- https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#security-ssl-rest-enabled
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] mxm commented on pull request #23406: [FLINK-32884] [flink-clients] PyFlink remote execution should support URLs with paths and https scheme
Posted by "mxm (via GitHub)" <gi...@apache.org>.
mxm commented on PR #23406:
URL: https://github.com/apache/flink/pull/23406#issuecomment-1729080927
@elkhand Thanks for addressing our comments. We need to squash the commits prior to merging. If I merge using the Github UI it will use elkhand@users.noreply.github.com. This will still the associate the commit with your Github account. If you want to use a different email address, please squash and push the resulting commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] elkhand commented on a diff in pull request #23406: [FLINK-32884] [flink-clients] PyFlink remote execution should support URLs with paths and https scheme
Posted by "elkhand (via GitHub)" <gi...@apache.org>.
elkhand commented on code in PR #23406:
URL: https://github.com/apache/flink/pull/23406#discussion_r1329665960
##########
flink-core/src/main/java/org/apache/flink/configuration/RestOptions.java:
##########
@@ -80,6 +80,23 @@ public class RestOptions {
.withDescription(
"The address that should be used by clients to connect to the server. Attention: This option is respected only if the high-availability configuration is NONE.");
+ /** The path that should be used by clients to interact with the server. */
+ @Documentation.Section(Documentation.Sections.COMMON_HOST_PORT)
+ public static final ConfigOption<String> PATH =
+ key("rest.path")
+ .stringType()
+ .defaultValue("")
+ .withDescription(
+ "The path that should be used by clients to interact to the server which is accessible via URL.");
+
+ /** The protocol that should be used by clients to interact with the server. */
+ @Documentation.Section(Documentation.Sections.COMMON_HOST_PORT)
+ public static final ConfigOption<String> PROTOCOL =
Review Comment:
Removed this config option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] gaborgsomogyi commented on a diff in pull request #23406: [FLINK-32884] [flink-clients] PyFlink remote execution should support URLs with paths and https scheme
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on code in PR #23406:
URL: https://github.com/apache/flink/pull/23406#discussion_r1329637581
##########
flink-core/src/main/java/org/apache/flink/configuration/RestOptions.java:
##########
@@ -80,6 +80,23 @@ public class RestOptions {
.withDescription(
"The address that should be used by clients to connect to the server. Attention: This option is respected only if the high-availability configuration is NONE.");
+ /** The path that should be used by clients to interact with the server. */
+ @Documentation.Section(Documentation.Sections.COMMON_HOST_PORT)
+ public static final ConfigOption<String> PATH =
+ key("rest.path")
+ .stringType()
+ .defaultValue("")
+ .withDescription(
+ "The path that should be used by clients to interact to the server which is accessible via URL.");
+
+ /** The protocol that should be used by clients to interact with the server. */
+ @Documentation.Section(Documentation.Sections.COMMON_HOST_PORT)
+ public static final ConfigOption<String> PROTOCOL =
+ key("rest.protocol")
+ .stringType()
+ .defaultValue("http")
+ .withDescription("The protocol to be used for rest endpoint");
Review Comment:
+1 on using `SSL_REST_ENABLED`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] elkhand commented on a diff in pull request #23406: [FLINK-32884] [flink-clients] PyFlink remote execution should support URLs with paths and https scheme
Posted by "elkhand (via GitHub)" <gi...@apache.org>.
elkhand commented on code in PR #23406:
URL: https://github.com/apache/flink/pull/23406#discussion_r1326304575
##########
flink-clients/src/test/java/org/apache/flink/client/cli/DefaultCLITest.java:
##########
@@ -46,6 +46,28 @@ void testCommandLineMaterialization() throws Exception {
assertThat(configuration.get(RestOptions.ADDRESS)).isEqualTo(hostname);
assertThat(configuration.get(RestOptions.PORT)).isEqualTo(port);
+
+ final String httpProtocol = "http";
+ final String hostnameWithHttpScheme = httpProtocol + "://" + hostname;
+ final String[] httpArgs = {"-m", hostnameWithHttpScheme + ':' + port};
+ final CommandLine httpCommandLine = defaultCLI.parseCommandLineOptions(httpArgs, false);
+
+ Configuration newConfiguration = defaultCLI.toConfiguration(httpCommandLine);
+
+ assertThat(newConfiguration.get(RestOptions.ADDRESS)).isEqualTo(hostname);
+ assertThat(newConfiguration.get(RestOptions.PORT)).isEqualTo(port);
+ assertThat(newConfiguration.get(RestOptions.PROTOCOL)).isEqualTo(httpProtocol);
+
+ final String httpsProtocol = "https";
+ final String hostnameWithHttpsScheme = httpsProtocol + "://" + hostname;
+ final String[] httpsArgs = {"-m", hostnameWithHttpsScheme + ':' + port};
+ final CommandLine httpsCommandLine = defaultCLI.parseCommandLineOptions(httpsArgs, false);
+
+ Configuration httpsConfiguration = defaultCLI.toConfiguration(httpsCommandLine);
+
+ assertThat(httpsConfiguration.get(RestOptions.ADDRESS)).isEqualTo(hostname);
+ assertThat(httpsConfiguration.get(RestOptions.PORT)).isEqualTo(port);
+ assertThat(httpsConfiguration.get(RestOptions.PROTOCOL)).isEqualTo(httpsProtocol);
Review Comment:
Added, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] elkhand commented on pull request #23406: [FLINK-32884] [flink-clients] PyFlink remote execution should support URLs with paths and https scheme
Posted by "elkhand (via GitHub)" <gi...@apache.org>.
elkhand commented on PR #23406:
URL: https://github.com/apache/flink/pull/23406#issuecomment-1724945763
> Not sure I understand the feature from user perspective🤔 The new URL is clear, the existing booleans for SSL are clear. What I would like to ask is what happens when I set `SecurityOptions.SSL_REST_ENABLED = false` and `URL=https://...` and the other way around? I mean which part of the code is configured by the new path var and which part by the old boolean? I'm asking it because we're setting `SecurityOptions.SSL_REST_ENABLED` dynamically...
Updated code.
Now we set SSL_REST_ENABLED if URL starts with https only:
https://github.com/apache/flink/pull/23406/files#diff-2ebb5c60764d835a040c0e36aff8f42ae9a0239cb1186afd3d220b6e18d3dcb7R69
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org