You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/05/09 10:02:32 UTC
[GitHub] [flink] KarmaGYZ opened a new pull request #12054: [Flink 17579] Set the resource id of taskexecutor according to environment variable if exist in standalone mode
KarmaGYZ opened a new pull request #12054:
URL: https://github.com/apache/flink/pull/12054
## What is the purpose of the change
Set the resource id of taskexecutor according to environment variable if exist in standalone mode.
## Brief change log
Allow user to set the resource id of taskexecutor according to environment variable if exist in standalone mode.
The relevant env variable is "FLINK_STANDALONE_TASK_EXECUTOR_ID".
## Verifying this change
It could be verified manually.
- `bin/jobmanager.sh start`
- `FLINK_STANDALONE_TASK_EXECUTOR_ID=taskManager-1 bin/taskmanager.sh`
`2020-05-09 18:00:09,856 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner [] - Starting TaskManager with ResourceID: taskManager-1` occurred in the log file of task executor.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
}, {
"hash" : "311c21707305392d6bde18038bb1d8413867419b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "311c21707305392d6bde18038bb1d8413867419b",
"triggerType" : "PUSH"
}, {
"hash" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766",
"triggerID" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"triggerType" : "PUSH"
}, {
"hash" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3887",
"triggerID" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 311c21707305392d6bde18038bb1d8413867419b UNKNOWN
* e0c34c807adf768d904b5850c9ce4c8b4be4b70a Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3887)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
}, {
"hash" : "311c21707305392d6bde18038bb1d8413867419b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "311c21707305392d6bde18038bb1d8413867419b",
"triggerType" : "PUSH"
}, {
"hash" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766",
"triggerID" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"triggerType" : "PUSH"
}, {
"hash" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3887",
"triggerID" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"triggerType" : "PUSH"
}, {
"hash" : "e85a3c626720f035ff0379e849e36ea6e8db7249",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3938",
"triggerID" : "e85a3c626720f035ff0379e849e36ea6e8db7249",
"triggerType" : "PUSH"
}, {
"hash" : "e6338bad0847a8842e66a928e48f98083b63cff9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e6338bad0847a8842e66a928e48f98083b63cff9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 311c21707305392d6bde18038bb1d8413867419b UNKNOWN
* e85a3c626720f035ff0379e849e36ea6e8db7249 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3938)
* e6338bad0847a8842e66a928e48f98083b63cff9 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
}, {
"hash" : "311c21707305392d6bde18038bb1d8413867419b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "311c21707305392d6bde18038bb1d8413867419b",
"triggerType" : "PUSH"
}, {
"hash" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766",
"triggerID" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"triggerType" : "PUSH"
}, {
"hash" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3887",
"triggerID" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"triggerType" : "PUSH"
}, {
"hash" : "e85a3c626720f035ff0379e849e36ea6e8db7249",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3938",
"triggerID" : "e85a3c626720f035ff0379e849e36ea6e8db7249",
"triggerType" : "PUSH"
}, {
"hash" : "e6338bad0847a8842e66a928e48f98083b63cff9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3942",
"triggerID" : "e6338bad0847a8842e66a928e48f98083b63cff9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 311c21707305392d6bde18038bb1d8413867419b UNKNOWN
* e85a3c626720f035ff0379e849e36ea6e8db7249 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3938)
* e6338bad0847a8842e66a928e48f98083b63cff9 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3942)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KarmaGYZ commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r442707433
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##########
@@ -257,6 +257,18 @@
text("\"ip\" - uses host's ip address as binding address"))
.build());
+ /**
+ * The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix and a
+ * 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.
+ */
+ @Documentation.Section(Documentation.Sections.ALL_TASK_MANAGER)
+ public static final ConfigOption<String> TASK_MANAGER_RESOURCE_ID_PREFIX =
+ key("taskmanager.resource-id.prefix")
+ .stringType()
+ .noDefaultValue()
+ .withDescription("The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix "
+ + "and a 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.");
Review comment:
Thanks for the discussion. Add environment variable `TASK_MANAGER_RESOURCE_ID`
also sounds good. However, as @azagrebin said in FLINK-17579
> there are plans in the community to introduce a unified approach to set Flink options via environment variables.
Not sure is it ok to apply this approach. Should we move the discussion back to FLINK-17579?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #12054: [Flink 17579] Set the resource id of taskexecutor according to environment variable if exist in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
}, {
"hash" : "311c21707305392d6bde18038bb1d8413867419b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "311c21707305392d6bde18038bb1d8413867419b",
"triggerType" : "PUSH"
}, {
"hash" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766",
"triggerID" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"triggerType" : "PUSH"
}, {
"hash" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3887",
"triggerID" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"triggerType" : "PUSH"
}, {
"hash" : "e85a3c626720f035ff0379e849e36ea6e8db7249",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3938",
"triggerID" : "e85a3c626720f035ff0379e849e36ea6e8db7249",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 311c21707305392d6bde18038bb1d8413867419b UNKNOWN
* e0c34c807adf768d904b5850c9ce4c8b4be4b70a Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3887)
* e85a3c626720f035ff0379e849e36ea6e8db7249 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3938)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wangyang0918 commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r442032969
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -293,7 +295,15 @@ public static void main(String[] args) throws Exception {
LOG.info("Cannot determine the maximum number of open file descriptors");
}
- runTaskManagerSecurely(args, ResourceID.generate());
+ final Configuration config = loadConfiguration(args);
+ final String resourceIDPrefix = config.get(TaskManagerOptions.TASK_MANAGER_RESOURCE_ID_PREFIX);
+ ResourceID taskManagerResourceID;
+ if (resourceIDPrefix == null) {
+ taskManagerResourceID = new ResourceID(new AbstractID().toString());
Review comment:
Use `ResourceID.generate()`.
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -293,7 +295,15 @@ public static void main(String[] args) throws Exception {
LOG.info("Cannot determine the maximum number of open file descriptors");
}
- runTaskManagerSecurely(args, ResourceID.generate());
+ final Configuration config = loadConfiguration(args);
+ final String resourceIDPrefix = config.get(TaskManagerOptions.TASK_MANAGER_RESOURCE_ID_PREFIX);
+ ResourceID taskManagerResourceID;
+ if (resourceIDPrefix == null) {
+ taskManagerResourceID = new ResourceID(new AbstractID().toString());
+ } else {
+ taskManagerResourceID = new ResourceID(resourceIDPrefix + "-" + UUID.randomUUID().toString().substring(0, 6));
Review comment:
Could use `new AbstractID().toString().substring(0, 6)` here.
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -293,7 +295,15 @@ public static void main(String[] args) throws Exception {
LOG.info("Cannot determine the maximum number of open file descriptors");
}
- runTaskManagerSecurely(args, ResourceID.generate());
+ final Configuration config = loadConfiguration(args);
+ final String resourceIDPrefix = config.get(TaskManagerOptions.TASK_MANAGER_RESOURCE_ID_PREFIX);
+ ResourceID taskManagerResourceID;
Review comment:
Add `final` for `taskManagerResourceID` here.
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -293,7 +295,15 @@ public static void main(String[] args) throws Exception {
LOG.info("Cannot determine the maximum number of open file descriptors");
}
- runTaskManagerSecurely(args, ResourceID.generate());
+ final Configuration config = loadConfiguration(args);
+ final String resourceIDPrefix = config.get(TaskManagerOptions.TASK_MANAGER_RESOURCE_ID_PREFIX);
+ ResourceID taskManagerResourceID;
Review comment:
Could we wrap the logic to get the `resourceID` into a separate method? Also we could add some tests for it.
* Do not specify prefix
* Specify prefix
* get more than once and should have two different resource id
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##########
@@ -257,6 +257,18 @@
text("\"ip\" - uses host's ip address as binding address"))
.build());
+ /**
+ * The config options defined the prefix of the TaskManager's ResourceID. If configured, the ResourceID will be spliced
+ * with the prefix and an uuid. Otherwise, the ResourceID would be a random hex string. Only valid in standalone mode.
+ */
+ @Documentation.Section(Documentation.Sections.ALL_TASK_MANAGER)
+ public static final ConfigOption<String> TASK_MANAGER_RESOURCE_ID_PREFIX =
+ key("taskmanager.resource-id.prefix")
+ .stringType()
+ .noDefaultValue()
+ .withDescription("The config options defined the prefix of the TaskManager's ResourceID. If configured, the ResourceID "
Review comment:
"The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix and a 6-character random string. Otherwise, a 32-character random string will be used. Only valid in standalone mode."
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] tillrohrmann commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r442344479
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -413,9 +413,10 @@ static BackPressureSampleService createBackPressureSampleService(
* @param configuration The configuration for the TaskManager.
* @param haServices to use for the task manager hostname retrieval
*/
- public static RpcService createRpcService(
- final Configuration configuration,
- final HighAvailabilityServices haServices) throws Exception {
+ @VisibleForTesting
+ static RpcService createRpcService(
+ final Configuration configuration,
+ final HighAvailabilityServices haServices) throws Exception {
Review comment:
Why is this change necessary?
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -91,7 +93,8 @@
private static final int STARTUP_FAILURE_RETURN_CODE = 1;
- public static final int RUNTIME_FAILURE_RETURN_CODE = 2;
+ @VisibleForTesting
Review comment:
Why is this necessary?
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -306,10 +308,8 @@ public static void runTaskManager(Configuration configuration, ResourceID resour
taskManagerRunner.start();
}
- public static void runTaskManagerSecurely(String[] args, ResourceID resourceID) {
+ public static void runTaskManagerSecurely(Configuration configuration, ResourceID resourceID) {
try {
- final Configuration configuration = loadConfiguration(args);
Review comment:
I would put the `ResourceID` generation here instead of moving the configuration loading out of this method.
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##########
@@ -257,6 +257,18 @@
text("\"ip\" - uses host's ip address as binding address"))
.build());
+ /**
+ * The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix and a
+ * 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.
+ */
+ @Documentation.Section(Documentation.Sections.ALL_TASK_MANAGER)
+ public static final ConfigOption<String> TASK_MANAGER_RESOURCE_ID_PREFIX =
+ key("taskmanager.resource-id.prefix")
+ .stringType()
+ .noDefaultValue()
+ .withDescription("The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix "
+ + "and a 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.");
Review comment:
Why is this only valid in standalone mode? Shouldn't we have a general mean how to configure the resource id for `TaskExecutors` in general? I believe it should work across the different integrations (Yarn, Mesos, K8s, standalone) all the same. The only difference would be to what value the different implementations set it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] tillrohrmann commented on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-648683986
Thanks @KarmaGYZ. Merging this PR now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
}, {
"hash" : "311c21707305392d6bde18038bb1d8413867419b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "311c21707305392d6bde18038bb1d8413867419b",
"triggerType" : "PUSH"
}, {
"hash" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766",
"triggerID" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"triggerType" : "PUSH"
}, {
"hash" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3887",
"triggerID" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"triggerType" : "PUSH"
}, {
"hash" : "e85a3c626720f035ff0379e849e36ea6e8db7249",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e85a3c626720f035ff0379e849e36ea6e8db7249",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 311c21707305392d6bde18038bb1d8413867419b UNKNOWN
* e0c34c807adf768d904b5850c9ce4c8b4be4b70a Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3887)
* e85a3c626720f035ff0379e849e36ea6e8db7249 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KarmaGYZ commented on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-645883863
Thanks for the review! @wangyang0918 I've addressed your comments.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
}, {
"hash" : "311c21707305392d6bde18038bb1d8413867419b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "311c21707305392d6bde18038bb1d8413867419b",
"triggerType" : "PUSH"
}, {
"hash" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766",
"triggerID" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"triggerType" : "PUSH"
}, {
"hash" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 311c21707305392d6bde18038bb1d8413867419b UNKNOWN
* 0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766)
* e0c34c807adf768d904b5850c9ce4c8b4be4b70a UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
}, {
"hash" : "311c21707305392d6bde18038bb1d8413867419b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "311c21707305392d6bde18038bb1d8413867419b",
"triggerType" : "PUSH"
}, {
"hash" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766",
"triggerID" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"triggerType" : "PUSH"
}, {
"hash" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3887",
"triggerID" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 311c21707305392d6bde18038bb1d8413867419b UNKNOWN
* 0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766)
* e0c34c807adf768d904b5850c9ce4c8b4be4b70a Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3887)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] tillrohrmann commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r442831567
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##########
@@ -257,6 +257,18 @@
text("\"ip\" - uses host's ip address as binding address"))
.build());
+ /**
+ * The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix and a
+ * 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.
+ */
+ @Documentation.Section(Documentation.Sections.ALL_TASK_MANAGER)
+ public static final ConfigOption<String> TASK_MANAGER_RESOURCE_ID_PREFIX =
+ key("taskmanager.resource-id.prefix")
+ .stringType()
+ .noDefaultValue()
+ .withDescription("The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix "
+ + "and a 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.");
Review comment:
Why do you want to do it with an environment variable? Can't it be a configuration value? That way the user could also configure it simply in his `flink-conf.yaml`.
This would be a configuration value which is specific to the started process and which should not distributed to other processes, though. I think we already have similar cases in Flink. See `DeploymentOptionsInternal.CONF_DIR`, for example.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
}, {
"hash" : "311c21707305392d6bde18038bb1d8413867419b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "311c21707305392d6bde18038bb1d8413867419b",
"triggerType" : "PUSH"
}, {
"hash" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766",
"triggerID" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"triggerType" : "PUSH"
}, {
"hash" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3887",
"triggerID" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"triggerType" : "PUSH"
}, {
"hash" : "e85a3c626720f035ff0379e849e36ea6e8db7249",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3938",
"triggerID" : "e85a3c626720f035ff0379e849e36ea6e8db7249",
"triggerType" : "PUSH"
}, {
"hash" : "e6338bad0847a8842e66a928e48f98083b63cff9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3942",
"triggerID" : "e6338bad0847a8842e66a928e48f98083b63cff9",
"triggerType" : "PUSH"
}, {
"hash" : "24bf04500d3b02b90f6cd85201c12fd54f6304f0",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3959",
"triggerID" : "24bf04500d3b02b90f6cd85201c12fd54f6304f0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 311c21707305392d6bde18038bb1d8413867419b UNKNOWN
* 24bf04500d3b02b90f6cd85201c12fd54f6304f0 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3959)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wangyang0918 commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r442600774
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##########
@@ -257,6 +257,18 @@
text("\"ip\" - uses host's ip address as binding address"))
.build());
+ /**
+ * The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix and a
+ * 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.
+ */
+ @Documentation.Section(Documentation.Sections.ALL_TASK_MANAGER)
+ public static final ConfigOption<String> TASK_MANAGER_RESOURCE_ID_PREFIX =
+ key("taskmanager.resource-id.prefix")
+ .stringType()
+ .noDefaultValue()
+ .withDescription("The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix "
+ + "and a 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.");
Review comment:
Thanks for raising this valid concern. For standalone cluster, we could set different `TASK_MANAGER_RESOURCE_ID_PREFIX` per `TaskExecutor`. Currently, for active resourcemanager(Yarn, Mesos, K8s), we will generate a meaningful `ResourceID`(e.g. container id in Yarn, pod name in native K8s) per `TaskExecutor`. If we also want to respect the `TASK_MANAGER_RESOURCE_ID_PREFIX` config option, then it will be difficult to use the `ResourceID` to find the corresponding container/pod.
Given that `TASK_MANAGER_RESOURCE_ID_PREFIX` is configured to "my-yarn-cluster-tm", then all the `TaskExecutor` share the same schema "my-yarn-cluster-tm-xxxxxx". A corresponding container id could be "container_e01_1592277741400_0043_01_000002". It is really confusing.
If you just mean to override `TASK_MANAGER_RESOURCE_ID_PREFIX` option for active deployment with current meaningful name, i think it makes sense. We just use the config option to replace the environment to pass the `ResourceID` to `TaskExecutor`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #12054: [Flink 17579] Set the resource id of taskexecutor according to environment variable if exist in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626141677
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31 (Sat May 09 10:06:03 UTC 2020)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
* **Invalid pull request title: No valid Jira ID provided**
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
}, {
"hash" : "311c21707305392d6bde18038bb1d8413867419b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "311c21707305392d6bde18038bb1d8413867419b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e63ffe4889613b2b824b8d2d3159831a100199ef Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507)
* 311c21707305392d6bde18038bb1d8413867419b UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] tillrohrmann commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r444113829
##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunnerTest.java
##########
@@ -86,6 +88,40 @@ public void testShouldShutdownIfRegistrationWithJobManagerFails() throws Excepti
assertThat(statusCode, is(equalTo(TaskManagerRunner.RUNTIME_FAILURE_RETURN_CODE)));
}
+ @Test
+ public void testGenerateTaskManagerResourceIDWithConfig() throws Exception {
+ final Configuration configuration = createConfiguration();
+ final String resourceID = "test";
+ configuration.set(TaskManagerOptions.TASK_MANAGER_RESOURCE_ID, resourceID);
+ final String taskManagerResourceID = TaskManagerRunner.getTaskManagerResourceID(configuration, "", -1);
+
+ assertThat(taskManagerResourceID, equalTo(resourceID));
+ }
+
+ @Test
+ public void testGenerateTaskManagerResourceIDWithRemoteRpcService() throws Exception {
+ final Configuration configuration = createConfiguration();
+ final String rpcAddress = "flink";
+ final int rpcPort = 9090;
+ final String taskManagerResourceID = TaskManagerRunner.getTaskManagerResourceID(configuration, rpcAddress, rpcPort);
+
+ assertThat(taskManagerResourceID, notNullValue());
+ assertThat(taskManagerResourceID.substring(0, taskManagerResourceID.length() - 7),
+ equalTo(rpcAddress + ":" + rpcPort));
+ }
+
+ @Test
+ public void testGenerateTaskManagerResourceIDWithLocalRpcService() throws Exception {
+ final Configuration configuration = createConfiguration();
+ final String rpcAddress = "";
+ final int rpcPort = -1;
+ final String taskManagerResourceID = TaskManagerRunner.getTaskManagerResourceID(configuration, rpcAddress, rpcPort);
+
+ assertThat(taskManagerResourceID, notNullValue());
+ assertThat(taskManagerResourceID.substring(0, taskManagerResourceID.length() - 7),
Review comment:
Same here with the `containsString` matcher
##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunnerTest.java
##########
@@ -86,6 +88,40 @@ public void testShouldShutdownIfRegistrationWithJobManagerFails() throws Excepti
assertThat(statusCode, is(equalTo(TaskManagerRunner.RUNTIME_FAILURE_RETURN_CODE)));
}
+ @Test
+ public void testGenerateTaskManagerResourceIDWithConfig() throws Exception {
+ final Configuration configuration = createConfiguration();
+ final String resourceID = "test";
+ configuration.set(TaskManagerOptions.TASK_MANAGER_RESOURCE_ID, resourceID);
+ final String taskManagerResourceID = TaskManagerRunner.getTaskManagerResourceID(configuration, "", -1);
+
+ assertThat(taskManagerResourceID, equalTo(resourceID));
+ }
+
+ @Test
+ public void testGenerateTaskManagerResourceIDWithRemoteRpcService() throws Exception {
+ final Configuration configuration = createConfiguration();
+ final String rpcAddress = "flink";
+ final int rpcPort = 9090;
+ final String taskManagerResourceID = TaskManagerRunner.getTaskManagerResourceID(configuration, rpcAddress, rpcPort);
+
+ assertThat(taskManagerResourceID, notNullValue());
+ assertThat(taskManagerResourceID.substring(0, taskManagerResourceID.length() - 7),
Review comment:
maybe use `containsString` matcher
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] tillrohrmann commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r443620557
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -458,4 +462,11 @@ private static String determineTaskManagerBindAddressByConnectingToResourceManag
HostBindPolicy bindPolicy = HostBindPolicy.fromString(configuration.getString(TaskManagerOptions.HOST_BIND_POLICY));
return bindPolicy == HostBindPolicy.IP ? taskManagerAddress.getHostAddress() : taskManagerAddress.getHostName();
}
+
+ @VisibleForTesting
+ static ResourceID getTaskManagerResourceID(Configuration config) throws Exception {
+ final String resourceID = config.get(TaskManagerOptions.TASK_MANAGER_RESOURCE_ID);
+ return resourceID != null ?
+ new ResourceID(resourceID) : new ResourceID(InetAddress.getLocalHost().getHostName() + "-" + new AbstractID().toString().substring(0, 6));
Review comment:
By creating the `ResourceID` after the `RpcService` has been started, one could obtain this information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
}, {
"hash" : "311c21707305392d6bde18038bb1d8413867419b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "311c21707305392d6bde18038bb1d8413867419b",
"triggerType" : "PUSH"
}, {
"hash" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766",
"triggerID" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"triggerType" : "PUSH"
}, {
"hash" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3887",
"triggerID" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"triggerType" : "PUSH"
}, {
"hash" : "e85a3c626720f035ff0379e849e36ea6e8db7249",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3938",
"triggerID" : "e85a3c626720f035ff0379e849e36ea6e8db7249",
"triggerType" : "PUSH"
}, {
"hash" : "e6338bad0847a8842e66a928e48f98083b63cff9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3942",
"triggerID" : "e6338bad0847a8842e66a928e48f98083b63cff9",
"triggerType" : "PUSH"
}, {
"hash" : "24bf04500d3b02b90f6cd85201c12fd54f6304f0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "24bf04500d3b02b90f6cd85201c12fd54f6304f0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 311c21707305392d6bde18038bb1d8413867419b UNKNOWN
* e6338bad0847a8842e66a928e48f98083b63cff9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3942)
* 24bf04500d3b02b90f6cd85201c12fd54f6304f0 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
}, {
"hash" : "311c21707305392d6bde18038bb1d8413867419b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "311c21707305392d6bde18038bb1d8413867419b",
"triggerType" : "PUSH"
}, {
"hash" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e63ffe4889613b2b824b8d2d3159831a100199ef Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507)
* 311c21707305392d6bde18038bb1d8413867419b UNKNOWN
* 0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wangyang0918 commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r442712205
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##########
@@ -257,6 +257,18 @@
text("\"ip\" - uses host's ip address as binding address"))
.build());
+ /**
+ * The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix and a
+ * 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.
+ */
+ @Documentation.Section(Documentation.Sections.ALL_TASK_MANAGER)
+ public static final ConfigOption<String> TASK_MANAGER_RESOURCE_ID_PREFIX =
+ key("taskmanager.resource-id.prefix")
+ .stringType()
+ .noDefaultValue()
+ .withDescription("The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix "
+ + "and a 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.");
Review comment:
Yes, actually in FLINK-17579 we also want to start with environment instead of config option. And we could unify all the current environment name(e.g. `_FLINK_CONTAINER_ID`, `_FLINK_POD_NAME`) to `TASK_MANAGER_RESOURCE_ID`. For active resourcemanager, we generate with meaningful names and do not change the current behavior. And for standalone, it could be specified with an arbitrary string(e.g. hostname, podname, etc.).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
}, {
"hash" : "311c21707305392d6bde18038bb1d8413867419b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "311c21707305392d6bde18038bb1d8413867419b",
"triggerType" : "PUSH"
}, {
"hash" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766",
"triggerID" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e63ffe4889613b2b824b8d2d3159831a100199ef Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507)
* 311c21707305392d6bde18038bb1d8413867419b UNKNOWN
* 0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886)
* ccbbb809f8b8093ba54e93581018e8cf4acc682f Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501)
* e63ffe4889613b2b824b8d2d3159831a100199ef Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KarmaGYZ commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r442593692
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -413,9 +413,10 @@ static BackPressureSampleService createBackPressureSampleService(
* @param configuration The configuration for the TaskManager.
* @param haServices to use for the task manager hostname retrieval
*/
- public static RpcService createRpcService(
- final Configuration configuration,
- final HighAvailabilityServices haServices) throws Exception {
+ @VisibleForTesting
+ static RpcService createRpcService(
+ final Configuration configuration,
+ final HighAvailabilityServices haServices) throws Exception {
Review comment:
Same as above. I just think it would be good to limit accessibility.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] tillrohrmann commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r444097983
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -458,4 +462,11 @@ private static String determineTaskManagerBindAddressByConnectingToResourceManag
HostBindPolicy bindPolicy = HostBindPolicy.fromString(configuration.getString(TaskManagerOptions.HOST_BIND_POLICY));
return bindPolicy == HostBindPolicy.IP ? taskManagerAddress.getHostAddress() : taskManagerAddress.getHostName();
}
+
+ @VisibleForTesting
+ static ResourceID getTaskManagerResourceID(Configuration config) throws Exception {
+ final String resourceID = config.get(TaskManagerOptions.TASK_MANAGER_RESOURCE_ID);
+ return resourceID != null ?
+ new ResourceID(resourceID) : new ResourceID(InetAddress.getLocalHost().getHostName() + "-" + new AbstractID().toString().substring(0, 6));
Review comment:
I think this should be fine. The {{ResourceID}} is defined to contain a string field.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wangyang0918 commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r443533227
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -458,4 +462,11 @@ private static String determineTaskManagerBindAddressByConnectingToResourceManag
HostBindPolicy bindPolicy = HostBindPolicy.fromString(configuration.getString(TaskManagerOptions.HOST_BIND_POLICY));
return bindPolicy == HostBindPolicy.IP ? taskManagerAddress.getHostAddress() : taskManagerAddress.getHostName();
}
+
+ @VisibleForTesting
+ static ResourceID getTaskManagerResourceID(Configuration config) throws Exception {
+ final String resourceID = config.get(TaskManagerOptions.TASK_MANAGER_RESOURCE_ID);
+ return resourceID != null ?
+ new ResourceID(resourceID) : new ResourceID(InetAddress.getLocalHost().getHostName() + "-" + new AbstractID().toString().substring(0, 6));
Review comment:
How to set the `resourceID` when we use a random port for TaskManager rpc? Also the TaskManager could bind to an ip address. So i prefer a random uuid just as before when `TASK_MANAGER_RESOURCE_ID` is not configured.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] tillrohrmann commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r442690596
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -413,9 +413,10 @@ static BackPressureSampleService createBackPressureSampleService(
* @param configuration The configuration for the TaskManager.
* @param haServices to use for the task manager hostname retrieval
*/
- public static RpcService createRpcService(
- final Configuration configuration,
- final HighAvailabilityServices haServices) throws Exception {
+ @VisibleForTesting
+ static RpcService createRpcService(
+ final Configuration configuration,
+ final HighAvailabilityServices haServices) throws Exception {
Review comment:
Sounds good.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Set the resource id of taskexecutor according to environment variable if exist in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KarmaGYZ commented on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-648153154
Thanks for the review @tillrohrmann . PR updated and is rebased on the latest master.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] tillrohrmann commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r442690474
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -91,7 +93,8 @@
private static final int STARTUP_FAILURE_RETURN_CODE = 1;
- public static final int RUNTIME_FAILURE_RETURN_CODE = 2;
+ @VisibleForTesting
Review comment:
Ah ok. This makes sense. I did not see the hotfix commit.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
}, {
"hash" : "311c21707305392d6bde18038bb1d8413867419b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "311c21707305392d6bde18038bb1d8413867419b",
"triggerType" : "PUSH"
}, {
"hash" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766",
"triggerID" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"triggerType" : "PUSH"
}, {
"hash" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3887",
"triggerID" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"triggerType" : "PUSH"
}, {
"hash" : "e85a3c626720f035ff0379e849e36ea6e8db7249",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3938",
"triggerID" : "e85a3c626720f035ff0379e849e36ea6e8db7249",
"triggerType" : "PUSH"
}, {
"hash" : "e6338bad0847a8842e66a928e48f98083b63cff9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3942",
"triggerID" : "e6338bad0847a8842e66a928e48f98083b63cff9",
"triggerType" : "PUSH"
}, {
"hash" : "24bf04500d3b02b90f6cd85201c12fd54f6304f0",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3959",
"triggerID" : "24bf04500d3b02b90f6cd85201c12fd54f6304f0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 311c21707305392d6bde18038bb1d8413867419b UNKNOWN
* e6338bad0847a8842e66a928e48f98083b63cff9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3942)
* 24bf04500d3b02b90f6cd85201c12fd54f6304f0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3959)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ccbbb809f8b8093ba54e93581018e8cf4acc682f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501)
* e63ffe4889613b2b824b8d2d3159831a100199ef Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e63ffe4889613b2b824b8d2d3159831a100199ef Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KarmaGYZ commented on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-645898265
cc @tillrohrmann Would you like to take a look?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] tillrohrmann commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r442692589
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##########
@@ -257,6 +257,18 @@
text("\"ip\" - uses host's ip address as binding address"))
.build());
+ /**
+ * The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix and a
+ * 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.
+ */
+ @Documentation.Section(Documentation.Sections.ALL_TASK_MANAGER)
+ public static final ConfigOption<String> TASK_MANAGER_RESOURCE_ID_PREFIX =
+ key("taskmanager.resource-id.prefix")
+ .stringType()
+ .noDefaultValue()
+ .withDescription("The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix "
+ + "and a 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.");
Review comment:
Yes what I mean is to have a single mean how the `ResourceID` is configured across the different integrations (Yarn, Mesos, K8s). Maybe we should take a step back and not start with a prefix config option but simply with `TASK_MANAGER_RESOURCE_ID` which allows to set the `ResourceID`. That way it would be simple for the RM implementations to configure the `ResourceID`. Per default it is empty and then we could take the hostname and a random value, for example.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
}, {
"hash" : "311c21707305392d6bde18038bb1d8413867419b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "311c21707305392d6bde18038bb1d8413867419b",
"triggerType" : "PUSH"
}, {
"hash" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766",
"triggerID" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 311c21707305392d6bde18038bb1d8413867419b UNKNOWN
* 0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886)
* ccbbb809f8b8093ba54e93581018e8cf4acc682f UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] tillrohrmann commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r443420479
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -458,4 +462,11 @@ private static String determineTaskManagerBindAddressByConnectingToResourceManag
HostBindPolicy bindPolicy = HostBindPolicy.fromString(configuration.getString(TaskManagerOptions.HOST_BIND_POLICY));
return bindPolicy == HostBindPolicy.IP ? taskManagerAddress.getHostAddress() : taskManagerAddress.getHostName();
}
+
+ @VisibleForTesting
+ static ResourceID getTaskManagerResourceID(Configuration config) throws Exception {
+ final String resourceID = config.get(TaskManagerOptions.TASK_MANAGER_RESOURCE_ID);
+ return resourceID != null ?
+ new ResourceID(resourceID) : new ResourceID(InetAddress.getLocalHost().getHostName() + "-" + new AbstractID().toString().substring(0, 6));
Review comment:
I'm wondering whether the address the `RpcService` binds to would be more meaningful. Moreover, we could also add the port of the `RpcService`. But as a safety net we should still add a random suffix to filter out if a TM gets restarted on the same machine with the same port.
##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunnerTest.java
##########
@@ -86,6 +89,26 @@ public void testShouldShutdownIfRegistrationWithJobManagerFails() throws Excepti
assertThat(statusCode, is(equalTo(TaskManagerRunner.RUNTIME_FAILURE_RETURN_CODE)));
}
+ @Test
+ public void testGenerateTaskManagerResourceIDWithPrefixSpecified() throws Exception {
Review comment:
Method name seems out of date.
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -293,30 +294,32 @@ public static void main(String[] args) throws Exception {
LOG.info("Cannot determine the maximum number of open file descriptors");
}
- runTaskManagerSecurely(args, ResourceID.generate());
+ runTaskManagerSecurely(args, null);
}
public static Configuration loadConfiguration(String[] args) throws FlinkParseException {
return ConfigurationParserUtils.loadCommonConfiguration(args, TaskManagerRunner.class.getSimpleName());
}
- public static void runTaskManager(Configuration configuration, ResourceID resourceId, PluginManager pluginManager) throws Exception {
- final TaskManagerRunner taskManagerRunner = new TaskManagerRunner(configuration, resourceId, pluginManager);
+ public static void runTaskManager(Configuration configuration, PluginManager pluginManager) throws Exception {
+ final TaskManagerRunner taskManagerRunner = new TaskManagerRunner(configuration, getTaskManagerResourceID(configuration), pluginManager);
taskManagerRunner.start();
}
- public static void runTaskManagerSecurely(String[] args, ResourceID resourceID) {
+ public static void runTaskManagerSecurely(String[] args, String resourceID) {
try {
final Configuration configuration = loadConfiguration(args);
-
+ if (resourceID != null) {
+ configuration.set(TaskManagerOptions.TASK_MANAGER_RESOURCE_ID, resourceID);
+ }
Review comment:
I think it would be good if setting the `TASK_MANAGER_RESOURCE_ID` happens on the `ResourceManager` side.
##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunnerTest.java
##########
@@ -86,6 +89,26 @@ public void testShouldShutdownIfRegistrationWithJobManagerFails() throws Excepti
assertThat(statusCode, is(equalTo(TaskManagerRunner.RUNTIME_FAILURE_RETURN_CODE)));
}
+ @Test
+ public void testGenerateTaskManagerResourceIDWithPrefixSpecified() throws Exception {
+ final Configuration configuration = createConfiguration();
+ final String resourceID = "test";
+ configuration.set(TaskManagerOptions.TASK_MANAGER_RESOURCE_ID, resourceID);
+ final ResourceID taskManagerResourceID = TaskManagerRunner.getTaskManagerResourceID(configuration);
+
+ assertThat(taskManagerResourceID.toString(), equalTo(resourceID));
+ }
+
+ @Test
+ public void testGenerateTaskManagerResourceIDWithoutPrefixSpecified() throws Exception {
Review comment:
Same here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KarmaGYZ commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r443294370
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##########
@@ -257,6 +257,18 @@
text("\"ip\" - uses host's ip address as binding address"))
.build());
+ /**
+ * The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix and a
+ * 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.
+ */
+ @Documentation.Section(Documentation.Sections.ALL_TASK_MANAGER)
+ public static final ConfigOption<String> TASK_MANAGER_RESOURCE_ID_PREFIX =
+ key("taskmanager.resource-id.prefix")
+ .stringType()
+ .noDefaultValue()
+ .withDescription("The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix "
+ + "and a 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.");
Review comment:
Ahh, I think I misunderstand what you mean by "step back". You mean we could set the `ResourceID` to `host-uuid` by default. But for the RM implementations, they could override it to `containerID` through a configuration option `TASK_MANAGER_RESOURCE_ID `. Do I understand it correctly?
If that is the case, user should not set `TASK_MANAGER_RESOURCE_ID` in active mode (active RMs will override this option anyway). And in standalone mode, it is user's responsibility to ensure that these ids do not conflict.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886)
* ccbbb809f8b8093ba54e93581018e8cf4acc682f Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wangyang0918 commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r442088204
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##########
@@ -257,6 +257,18 @@
text("\"ip\" - uses host's ip address as binding address"))
.build());
+ /**
Review comment:
nit: Update the comments or remove them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [Flink 17579] Set the resource id of taskexecutor according to environment variable if exist in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wangyang0918 commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r443927993
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -458,4 +462,11 @@ private static String determineTaskManagerBindAddressByConnectingToResourceManag
HostBindPolicy bindPolicy = HostBindPolicy.fromString(configuration.getString(TaskManagerOptions.HOST_BIND_POLICY));
return bindPolicy == HostBindPolicy.IP ? taskManagerAddress.getHostAddress() : taskManagerAddress.getHostName();
}
+
+ @VisibleForTesting
+ static ResourceID getTaskManagerResourceID(Configuration config) throws Exception {
+ final String resourceID = config.get(TaskManagerOptions.TASK_MANAGER_RESOURCE_ID);
+ return resourceID != null ?
+ new ResourceID(resourceID) : new ResourceID(InetAddress.getLocalHost().getHostName() + "-" + new AbstractID().toString().substring(0, 6));
Review comment:
Well my only concern is whether it is safe to put some special characters(e.g. `:`) in the `ResourceID`. I am not aware of this could happen.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] tillrohrmann commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r442692589
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##########
@@ -257,6 +257,18 @@
text("\"ip\" - uses host's ip address as binding address"))
.build());
+ /**
+ * The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix and a
+ * 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.
+ */
+ @Documentation.Section(Documentation.Sections.ALL_TASK_MANAGER)
+ public static final ConfigOption<String> TASK_MANAGER_RESOURCE_ID_PREFIX =
+ key("taskmanager.resource-id.prefix")
+ .stringType()
+ .noDefaultValue()
+ .withDescription("The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix "
+ + "and a 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.");
Review comment:
Yes what I mean is to have a single mean how the `ResourceID` is configured for across the different integrations (Yarn, Mesos, K8s). Maybe we should take a step back and not start with a prefix config option but simply with `TASK_MANAGER_RESOURCE_ID` which allows to set the `ResourceID`. That way it would be simple for the RM implementations to configure the `ResourceID`. Per default it is empty and then we could take the hostname and a random value, for example.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KarmaGYZ commented on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-648031168
@flinkbot run azure
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] tillrohrmann commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r443415807
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##########
@@ -257,6 +257,18 @@
text("\"ip\" - uses host's ip address as binding address"))
.build());
+ /**
+ * The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix and a
+ * 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.
+ */
+ @Documentation.Section(Documentation.Sections.ALL_TASK_MANAGER)
+ public static final ConfigOption<String> TASK_MANAGER_RESOURCE_ID_PREFIX =
+ key("taskmanager.resource-id.prefix")
+ .stringType()
+ .noDefaultValue()
+ .withDescription("The prefix of the TaskManager's ResourceID. If configured, the ResourceID will be generated with the prefix "
+ + "and a 6-character random string. Otherwise, a random string will be used. Only valid in standalone mode.");
Review comment:
Yes, this was my idea. The down side would be that people could shoot themselves in the foot with it. But I believe that this should be fine given that it is an expert feature.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3507",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
}, {
"hash" : "311c21707305392d6bde18038bb1d8413867419b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "311c21707305392d6bde18038bb1d8413867419b",
"triggerType" : "PUSH"
}, {
"hash" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3766",
"triggerID" : "0ba310c8954ee4f4ea165e937f4e2b4f7c839fd6",
"triggerType" : "PUSH"
}, {
"hash" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3887",
"triggerID" : "e0c34c807adf768d904b5850c9ce4c8b4be4b70a",
"triggerType" : "PUSH"
}, {
"hash" : "e85a3c626720f035ff0379e849e36ea6e8db7249",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3938",
"triggerID" : "e85a3c626720f035ff0379e849e36ea6e8db7249",
"triggerType" : "PUSH"
}, {
"hash" : "e6338bad0847a8842e66a928e48f98083b63cff9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3942",
"triggerID" : "e6338bad0847a8842e66a928e48f98083b63cff9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 311c21707305392d6bde18038bb1d8413867419b UNKNOWN
* e6338bad0847a8842e66a928e48f98083b63cff9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3942)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12054:
URL: https://github.com/apache/flink/pull/12054#issuecomment-626143079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886",
"triggerID" : "c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31",
"triggerType" : "PUSH"
}, {
"hash" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501",
"triggerID" : "ccbbb809f8b8093ba54e93581018e8cf4acc682f",
"triggerType" : "PUSH"
}, {
"hash" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e63ffe4889613b2b824b8d2d3159831a100199ef",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c4140c3ce76c43d4b2d73c6c9aca85e65e7ebe31 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=886)
* ccbbb809f8b8093ba54e93581018e8cf4acc682f Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3501)
* e63ffe4889613b2b824b8d2d3159831a100199ef UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KarmaGYZ commented on a change in pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on a change in pull request #12054:
URL: https://github.com/apache/flink/pull/12054#discussion_r442592928
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java
##########
@@ -91,7 +93,8 @@
private static final int STARTUP_FAILURE_RETURN_CODE = 1;
- public static final int RUNTIME_FAILURE_RETURN_CODE = 2;
+ @VisibleForTesting
Review comment:
I saw this field only used in `TaskManagerRunner.java` and testing code. So I think it probably not makes sense to set the access modifier to `public`. However, we could not set it to `private` as it is used by testing code. So, I change it to `package-private` and annotate it with `VisibleForTesting`. It just a hotfix though.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] tillrohrmann closed pull request #12054: [FLINK-17579] Allow user to set the prefix of TaskManager's ResourceID in standalone mode
Posted by GitBox <gi...@apache.org>.
tillrohrmann closed pull request #12054:
URL: https://github.com/apache/flink/pull/12054
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org