You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/03/27 04:21:46 UTC
[GitHub] [flink] HuangZhenQiu opened a new pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
HuangZhenQiu opened a new pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541
## What is the purpose of the change
Add retry logic for netty client creation in PartitionRequestClientFactory. It is useful to make flink runtime tolerant physical link issue in the switch.
## Brief changelog
- Add a netty client retry config
- Add retry logic in PartitionRequestClientFactory for building new channel.
## Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no )
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (yes)
- If yes, how is the feature documented? (docs)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406609291
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -120,6 +127,40 @@ public void closeOpenChannelConnections(ConnectionID connectionId) {
}
}
+ void connectChannelWithRetry(ConnectingChannel connectingChannel, ConnectionID connectionId)
Review comment:
Agree. Updated according.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* 3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/158159031) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025)
* eeee4cdce1e85052f21aa8b70355516d8ed9b3e4 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/158917957) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404498256
##########
File path: docs/_includes/generated/netty_shuffle_environment_configuration.html
##########
@@ -57,17 +57,23 @@
<td>Number of extra network buffers to use for each outgoing/incoming gate (result partition/input gate). In credit-based flow control mode, this indicates how many floating credits are shared among all the input channels. The floating buffers are distributed based on backlog (real-time output buffers in the subpartition) feedback, and can help relieve back-pressure caused by unbalanced data distribution among the subpartitions. This value should be increased in case of higher round trip times between nodes and/or larger number of machines in the cluster.</td>
</tr>
<tr>
- <td><h5>taskmanager.network.netty.client.connectTimeoutSec</h5></td>
- <td style="word-wrap: break-word;">120</td>
+ <td><h5>taskmanager.network.retries</h5></td>
+ <td style="word-wrap: break-word;">0</td>
Review comment:
This should be consistent with default value in `NettyShuffleEnvironmentOptions`. What is the motivation for changing this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* 3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/158159031) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025)
* eeee4cdce1e85052f21aa8b70355516d8ed9b3e4 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159430095",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7241",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159477959",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159173715",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159737092",
"triggerID" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"triggerType" : "PUSH"
}, {
"hash" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7311",
"triggerID" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
* 8e2da05d8e34fe5f8d7b86e849b0df1275579242 Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/159737092) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7311)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159430095",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7241",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159477959",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159173715",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159737092",
"triggerID" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"triggerType" : "PUSH"
}, {
"hash" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7311",
"triggerID" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"triggerType" : "PUSH"
}, {
"hash" : "072a285e04f298ee3c6ff6c9062c3ffedae90a2f",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7422",
"triggerID" : "072a285e04f298ee3c6ff6c9062c3ffedae90a2f",
"triggerType" : "PUSH"
}, {
"hash" : "072a285e04f298ee3c6ff6c9062c3ffedae90a2f",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/160149187",
"triggerID" : "072a285e04f298ee3c6ff6c9062c3ffedae90a2f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
* 8e2da05d8e34fe5f8d7b86e849b0df1275579242 Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/159737092) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7311)
* 072a285e04f298ee3c6ff6c9062c3ffedae90a2f Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/160149187) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7422)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r402811540
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,14 +92,36 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
- nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
-
- client = connectingChannel.waitForChannel();
-
- clients.replace(connectionId, connectingChannel, client);
+ int count = 0;
+ ChannelException exception = null;
+ do {
+ try {
+ exception = null;
+ connectingChannel.error = null;
+ LOG.info("Connecting to {} at {} attempt", connectionId.getAddress(), count);
+ nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
+
+ client = connectingChannel.waitForChannel();
+
+ clients.replace(connectionId, connectingChannel, client);
+ } catch (ChannelException e) {
+ LOG.error("Failed {} times to connect to {}", count, connectionId.getAddress(), e);
+ exception = e;
+ }
+ count++;
+ } while (count <= retryNumber);
+
+ if (exception != null) {
+ throw exception;
+ }
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* 30370d7bb280c03c7b6767877f4405d4c06941ae Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/158923996) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404888547
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,14 +92,36 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
- nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
-
- client = connectingChannel.waitForChannel();
-
- clients.replace(connectionId, connectingChannel, client);
+ int count = 0;
+ ChannelException exception = null;
+ do {
+ try {
+ exception = null;
+ connectingChannel.error = null;
+ LOG.info("Connecting to {} at {} attempt", connectionId.getAddress(), count);
+ nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
+
+ client = connectingChannel.waitForChannel();
+
+ clients.replace(connectionId, connectingChannel, client);
+ } catch (ChannelException e) {
+ LOG.error("Failed {} times to connect to {}", count, connectionId.getAddress(), e);
+ exception = e;
+ }
+ count++;
+ } while (count <= retryNumber);
+
+ if (exception != null) {
+ throw exception;
+ }
}
else if (old instanceof ConnectingChannel) {
- client = ((ConnectingChannel) old).waitForChannel();
+
+ try {
+ client = ((ConnectingChannel) old).waitForChannel();
+ } catch (IOException e) {
+ LOG.info("Continue to waitForChannel until the original thread has completed or failed");
+ }
Review comment:
Ok. I see, I missed the outer `while (client == null)` loop.
But as it is now, isn't it possible that `B` will loop indefinitely? Shouldn't it also follow the max retry count logic and if exceeded, throw an exception?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* 8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/156746943) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406357349
##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactoryTest.java
##########
@@ -130,6 +130,23 @@ public void run() {
}
}
+ @Test
+ public void testNettyClientConnectRetry() throws Exception {
+ NettyClient nettyClient = mock(NettyClient.class);
Review comment:
Yes. I created a UnstableNettyClient to simulate the first n-1 connection failure and success in the last try.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3bc439b7c92f5e764033133845ac1ff2a2b14b91 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/155738776) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720)
* 0e228f36cdfc4610efd4da91d5964ffa05202c79 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r400973222
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,14 +92,36 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
- nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
-
- client = connectingChannel.waitForChannel();
-
- clients.replace(connectionId, connectingChannel, client);
+ int count = 0;
+ ChannelException exception = null;
+ do {
+ try {
+ exception = null;
+ connectingChannel.error = null;
Review comment:
Why don't you create a new `ConnectingChannel(...)`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3d6cba173780c51f34b577d8194343592fceacf5 Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/155930839) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772)
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* 8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/156746943) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0e228f36cdfc4610efd4da91d5964ffa05202c79 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/155743045) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404515695
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -120,6 +132,34 @@ public void closeOpenChannelConnections(ConnectionID connectionId) {
}
}
+ void connectChannelWithRetry(ConnectingChannel connectingChannel, ConnectionID connectionId)
Review comment:
It is updated each time there is an error happens.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159430095",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7241",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159477959",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159173715",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159737092",
"triggerID" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"triggerType" : "PUSH"
}, {
"hash" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7311",
"triggerID" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"triggerType" : "PUSH"
}, {
"hash" : "072a285e04f298ee3c6ff6c9062c3ffedae90a2f",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7422",
"triggerID" : "072a285e04f298ee3c6ff6c9062c3ffedae90a2f",
"triggerType" : "PUSH"
}, {
"hash" : "072a285e04f298ee3c6ff6c9062c3ffedae90a2f",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/160149187",
"triggerID" : "072a285e04f298ee3c6ff6c9062c3ffedae90a2f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
* 072a285e04f298ee3c6ff6c9062c3ffedae90a2f Travis: [SUCCESS](https://travis-ci.com/github/flink-ci/flink/builds/160149187) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7422)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404515481
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -120,6 +132,34 @@ public void closeOpenChannelConnections(ConnectionID connectionId) {
}
}
+ void connectChannelWithRetry(ConnectingChannel connectingChannel, ConnectionID connectionId)
+ throws IOException, InterruptedException {
+ int count = 0;
+ Exception exception = null;
+ do {
+ try {
+ connectingChannel = new ConnectingChannel(connectionId, this);
Review comment:
It is intentionally to create new instance each so that the old status such exception is cleaned.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404520411
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -120,6 +132,34 @@ public void closeOpenChannelConnections(ConnectionID connectionId) {
}
}
+ void connectChannelWithRetry(ConnectingChannel connectingChannel, ConnectionID connectionId)
+ throws IOException, InterruptedException {
+ int count = 0;
+ Exception exception = null;
+ do {
+ try {
+ connectingChannel = new ConnectingChannel(connectionId, this);
+ LOG.info("Connecting to {} at {} attempt", connectionId.getAddress(), count);
+ nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
+
+ NettyPartitionRequestClient client = connectingChannel.waitForChannel();
+
+ clients.replace(connectionId, connectingChannel, client);
+ } catch (IOException e) {
+ LOG.error("Failed {} times to connect to {}", count, connectionId.getAddress(), e);
+ exception = e;
+ } catch (ChannelException e) {
Review comment:
Good catch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r402807230
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -40,19 +43,27 @@
* instances.
*/
class PartitionRequestClientFactory {
+ private static final Logger LOG = LoggerFactory.getLogger(PartitionRequestClientFactory.class);
private final NettyClient nettyClient;
+ private final int retryNumber;
+
private final ConcurrentMap<ConnectionID, Object> clients = new ConcurrentHashMap<ConnectionID, Object>();
PartitionRequestClientFactory(NettyClient nettyClient) {
+ this(nettyClient, 0);
+ }
+
+ PartitionRequestClientFactory(NettyClient nettyClient, int retryNumber) {
this.nettyClient = nettyClient;
+ this.retryNumber = retryNumber;
}
- /**
- * Atomically establishes a TCP connection to the given remote address and
- * creates a {@link NettyPartitionRequestClient} instance for this connection.
- */
+ /**
+ * Atomically establishes a TCP connection to the given remote address and
+ * creates a {@link NettyPartitionRequestClient} instance for this connection.
+ */
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159430095",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7241",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159477959",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159173715",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
* 398866ade003725aad87bfadd508c7dcd61ef21d Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/159477959) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7241)
* 7346f3729147e2b94cbf145fbea2dcd3c3332e19 Travis: [SUCCESS](https://travis-ci.com/github/flink-ci/flink/builds/159173715) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159430095",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* 30370d7bb280c03c7b6767877f4405d4c06941ae Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/158923996) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134)
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
* e64d5ea150b48d976ec3b589144a29487ca20e33 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/159430095) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240)
* 398866ade003725aad87bfadd508c7dcd61ef21d UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159430095",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7241",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159477959",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159173715",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159737092",
"triggerID" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"triggerType" : "PUSH"
}, {
"hash" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7311",
"triggerID" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
* 8e2da05d8e34fe5f8d7b86e849b0df1275579242 Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/159737092) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7311)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* 8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761 Travis: [SUCCESS](https://travis-ci.com/github/flink-ci/flink/builds/156746943) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845)
* 3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/158159031) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404498987
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/NettyShuffleEnvironmentOptions.java
##########
@@ -234,6 +234,14 @@
.withDeprecatedKeys("taskmanager.net.client.connectTimeoutSec")
.withDescription("The Netty client connection timeout.");
+ @Documentation.Section(Documentation.Sections.ALL_TASK_MANAGER_NETWORK)
+ public static final ConfigOption<Integer> NETWORK_RETRIES =
+ key("taskmanager.network.retries")
Review comment:
better to be consistent with the same prefix as others. `taskmanager.network.netty.client.connectRetries`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3bc439b7c92f5e764033133845ac1ff2a2b14b91 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/155738776) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r400969700
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -40,19 +43,27 @@
* instances.
*/
class PartitionRequestClientFactory {
+ private static final Logger LOG = LoggerFactory.getLogger(PartitionRequestClientFactory.class);
private final NettyClient nettyClient;
+ private final int retryNumber;
+
private final ConcurrentMap<ConnectionID, Object> clients = new ConcurrentHashMap<ConnectionID, Object>();
PartitionRequestClientFactory(NettyClient nettyClient) {
+ this(nettyClient, 0);
+ }
+
+ PartitionRequestClientFactory(NettyClient nettyClient, int retryNumber) {
this.nettyClient = nettyClient;
+ this.retryNumber = retryNumber;
}
- /**
- * Atomically establishes a TCP connection to the given remote address and
- * creates a {@link NettyPartitionRequestClient} instance for this connection.
- */
+ /**
+ * Atomically establishes a TCP connection to the given remote address and
+ * creates a {@link NettyPartitionRequestClient} instance for this connection.
+ */
Review comment:
revert indentation?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159430095",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7241",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159477959",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159173715",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159737092",
"triggerID" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"triggerType" : "PUSH"
}, {
"hash" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7311",
"triggerID" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"triggerType" : "PUSH"
}, {
"hash" : "072a285e04f298ee3c6ff6c9062c3ffedae90a2f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7422",
"triggerID" : "072a285e04f298ee3c6ff6c9062c3ffedae90a2f",
"triggerType" : "PUSH"
}, {
"hash" : "072a285e04f298ee3c6ff6c9062c3ffedae90a2f",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/160149187",
"triggerID" : "072a285e04f298ee3c6ff6c9062c3ffedae90a2f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
* 072a285e04f298ee3c6ff6c9062c3ffedae90a2f Travis: [SUCCESS](https://travis-ci.com/github/flink-ci/flink/builds/160149187) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7422)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r402813477
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,14 +92,36 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
- nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
-
- client = connectingChannel.waitForChannel();
-
- clients.replace(connectionId, connectingChannel, client);
+ int count = 0;
+ ChannelException exception = null;
+ do {
+ try {
+ exception = null;
+ connectingChannel.error = null;
Review comment:
Yes, the code will be cleaner.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r402815267
##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactoryTest.java
##########
@@ -130,6 +130,23 @@ public void run() {
}
}
+ @Test
+ public void testNettyClientConnectRetry() throws Exception {
+ NettyClient nettyClient = mock(NettyClient.class);
Review comment:
I think it will be better. But when I try to implement the TestingNettyClient, I found the SucceededChannelFuture only visible to package io.netty.channel. Thus, I need to create a new class of SucceededChannelFuture in Flink to return a success ChannelFuture. Do you think it is a right way to go?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "CANCELED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3bc439b7c92f5e764033133845ac1ff2a2b14b91 Travis: [CANCELED](https://travis-ci.com/github/flink-ci/flink/builds/155738776) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720)
* 0e228f36cdfc4610efd4da91d5964ffa05202c79 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/155743045) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406592247
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -120,6 +127,40 @@ public void closeOpenChannelConnections(ConnectionID connectionId) {
}
}
+ void connectChannelWithRetry(ConnectingChannel connectingChannel, ConnectionID connectionId)
+ throws IOException, InterruptedException {
+ int count = 0;
+ Exception exception = null;
+ do {
+ try {
+ LOG.info("Connecting to {} at {} attempt", connectionId.getAddress(), count);
+ nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
+
+ NettyPartitionRequestClient client = connectingChannel.waitForChannel();
+
+ clients.replace(connectionId, connectingChannel, client);
+ exception = null;
Review comment:
we should exit the loop here to avoid entering the loop again if no exception
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* 30370d7bb280c03c7b6767877f4405d4c06941ae Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/158923996) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134)
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-608296022
Thanks for the suggestions. I updated the PR accordingly. My major concern is to add a TestingNettyClient. It is a little bit overhead to create a SucceededChannelFuture to simulate the success path. Do you think it is the right way to proceed or we can have a simpler way?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r400972963
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,14 +92,36 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
- nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
-
- client = connectingChannel.waitForChannel();
-
- clients.replace(connectionId, connectingChannel, client);
+ int count = 0;
+ ChannelException exception = null;
+ do {
+ try {
+ exception = null;
+ connectingChannel.error = null;
+ LOG.info("Connecting to {} at {} attempt", connectionId.getAddress(), count);
+ nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
+
+ client = connectingChannel.waitForChannel();
+
+ clients.replace(connectionId, connectingChannel, client);
+ } catch (ChannelException e) {
+ LOG.error("Failed {} times to connect to {}", count, connectionId.getAddress(), e);
+ exception = e;
+ }
+ count++;
+ } while (count <= retryNumber);
+
+ if (exception != null) {
+ throw exception;
+ }
Review comment:
Can you extract this into a separate method? This method is already a bit too big.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404518845
##########
File path: docs/_includes/generated/netty_shuffle_environment_configuration.html
##########
@@ -57,17 +57,23 @@
<td>Number of extra network buffers to use for each outgoing/incoming gate (result partition/input gate). In credit-based flow control mode, this indicates how many floating credits are shared among all the input channels. The floating buffers are distributed based on backlog (real-time output buffers in the subpartition) feedback, and can help relieve back-pressure caused by unbalanced data distribution among the subpartitions. This value should be increased in case of higher round trip times between nodes and/or larger number of machines in the cluster.</td>
</tr>
<tr>
- <td><h5>taskmanager.network.netty.client.connectTimeoutSec</h5></td>
- <td style="word-wrap: break-word;">120</td>
+ <td><h5>taskmanager.network.retries</h5></td>
+ <td style="word-wrap: break-word;">0</td>
<td>Integer</td>
- <td>The Netty client connection timeout.</td>
+ <td>The number of retry attempts for network communication. Currently it's only used for establishing input/output channel connections</td>
</tr>
<tr>
<td><h5>taskmanager.network.netty.client.numThreads</h5></td>
<td style="word-wrap: break-word;">-1</td>
<td>Integer</td>
<td>The number of Netty client threads.</td>
</tr>
+ <tr>
+ <td><h5>taskmanager.network.netty.clientRetry</h5></td>
Review comment:
It is a mistake. I manually changed the file. I will change it back.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406609362
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -120,6 +127,40 @@ public void closeOpenChannelConnections(ConnectionID connectionId) {
}
}
+ void connectChannelWithRetry(ConnectingChannel connectingChannel, ConnectionID connectionId)
+ throws IOException, InterruptedException {
+ int count = 0;
+ Exception exception = null;
+ do {
+ try {
+ LOG.info("Connecting to {} at {} attempt", connectionId.getAddress(), count);
+ nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
+
+ NettyPartitionRequestClient client = connectingChannel.waitForChannel();
+
+ clients.replace(connectionId, connectingChannel, client);
+ exception = null;
Review comment:
Yes, it should return directly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406591777
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,16 +92,12 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
- nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
-
- client = connectingChannel.waitForChannel();
-
- clients.replace(connectionId, connectingChannel, client);
+ connectChannelWithRetry(connectingChannel, connectionId);
+ client = (NettyPartitionRequestClient) clients.get(connectionId);
}
else if (old instanceof ConnectingChannel) {
- client = ((ConnectingChannel) old).waitForChannel();
-
- clients.replace(connectionId, old, client);
+ connectChannelWithRetry((ConnectingChannel) old, connectionId);
Review comment:
I have some concerns with this operation. `nettyClient.connect(connectionId.getAddress())` should be called only once if multiple threads invoking the `createPartitionRequestClient` at the same time. But now the `#connect` might be called multiple times by different threads to establish multiple physical connections.
E.g. when the first one `clients.putIfAbsent(connectionId, connectingChannel)` to enter `if (old == null)` path, then it would call `connectChannelWithRetry` to establish a connection.
The second one `clients.putIfAbsent(connectionId, connectingChannel)` to get `old instanceof ConnectingChannel` to enter this path, then it also calls `connectChannelWithRetry` to establish another connection.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406609157
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,16 +92,12 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
- nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
-
- client = connectingChannel.waitForChannel();
-
- clients.replace(connectionId, connectingChannel, client);
+ connectChannelWithRetry(connectingChannel, connectionId);
+ client = (NettyPartitionRequestClient) clients.get(connectionId);
}
else if (old instanceof ConnectingChannel) {
- client = ((ConnectingChannel) old).waitForChannel();
-
- clients.replace(connectionId, old, client);
+ connectChannelWithRetry((ConnectingChannel) old, connectionId);
Review comment:
Good catch. It is a bug. There should be a flag in parameter to check whether to connect or not.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406583546
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,16 +92,12 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
- nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
-
- client = connectingChannel.waitForChannel();
-
- clients.replace(connectionId, connectingChannel, client);
+ connectChannelWithRetry(connectingChannel, connectionId);
+ client = (NettyPartitionRequestClient) clients.get(connectionId);
}
else if (old instanceof ConnectingChannel) {
Review comment:
we can merge `if (old == null || old instanceof ConnectionChannel)` because the actions are actually the same.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3d6cba173780c51f34b577d8194343592fceacf5 Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/155930839) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772)
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* 8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #11541: [FLINK-15416][network]
add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot commented on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3bc439b7c92f5e764033133845ac1ff2a2b14b91 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* 30370d7bb280c03c7b6767877f4405d4c06941ae Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/158923996) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134)
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159430095",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7241",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159477959",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159173715",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
* 7346f3729147e2b94cbf145fbea2dcd3c3332e19 Travis: [SUCCESS](https://travis-ci.com/github/flink-ci/flink/builds/159173715) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177)
* 8e2da05d8e34fe5f8d7b86e849b0df1275579242 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404518565
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,14 +92,36 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
- nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
-
- client = connectingChannel.waitForChannel();
-
- clients.replace(connectionId, connectingChannel, client);
+ int count = 0;
+ ChannelException exception = null;
+ do {
+ try {
+ exception = null;
+ connectingChannel.error = null;
+ LOG.info("Connecting to {} at {} attempt", connectionId.getAddress(), count);
+ nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
+
+ client = connectingChannel.waitForChannel();
+
+ clients.replace(connectionId, connectingChannel, client);
+ } catch (ChannelException e) {
+ LOG.error("Failed {} times to connect to {}", count, connectionId.getAddress(), e);
+ exception = e;
+ }
+ count++;
+ } while (count <= retryNumber);
+
+ if (exception != null) {
+ throw exception;
+ }
}
else if (old instanceof ConnectingChannel) {
- client = ((ConnectingChannel) old).waitForChannel();
+
+ try {
+ client = ((ConnectingChannel) old).waitForChannel();
+ } catch (IOException e) {
+ LOG.info("Continue to waitForChannel until the original thread has completed or failed");
+ }
Review comment:
@pnowojski
The logic is always to let the first requester to retry. For the following requesters, will always go to wait until the first requester get the client or finished retry. Otherwise, there will be more retry times than expected.
@zhijiangW
I am not sure whether my understanding is correct or now. Once the ConnectingChannel is put into clients concurrentMap by the first requester, the object is never null after that. Thus, other requesters will go to line 97. So the process is what you described " let A retry connections and B is always waiting during A retries until final success or failure." Are you aligned?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159430095",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7241",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159477959",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159173715",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159737092",
"triggerID" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"triggerType" : "PUSH"
}, {
"hash" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7311",
"triggerID" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
* 7346f3729147e2b94cbf145fbea2dcd3c3332e19 Travis: [SUCCESS](https://travis-ci.com/github/flink-ci/flink/builds/159173715) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177)
* 8e2da05d8e34fe5f8d7b86e849b0df1275579242 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/159737092) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7311)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r403455187
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,14 +92,36 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
- nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
-
- client = connectingChannel.waitForChannel();
-
- clients.replace(connectionId, connectingChannel, client);
+ int count = 0;
+ ChannelException exception = null;
+ do {
+ try {
+ exception = null;
+ connectingChannel.error = null;
+ LOG.info("Connecting to {} at {} attempt", connectionId.getAddress(), count);
+ nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
+
+ client = connectingChannel.waitForChannel();
+
+ clients.replace(connectionId, connectingChannel, client);
+ } catch (ChannelException e) {
+ LOG.error("Failed {} times to connect to {}", count, connectionId.getAddress(), e);
+ exception = e;
+ }
+ count++;
+ } while (count <= retryNumber);
+
+ if (exception != null) {
+ throw exception;
+ }
}
else if (old instanceof ConnectingChannel) {
- client = ((ConnectingChannel) old).waitForChannel();
+
+ try {
+ client = ((ConnectingChannel) old).waitForChannel();
+ } catch (IOException e) {
+ LOG.info("Continue to waitForChannel until the original thread has completed or failed");
+ }
Review comment:
But shouldn't `B` implement the same retry logic? Otherwise we are swallowing exceptions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406585618
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -99,7 +106,7 @@ else if (old instanceof ConnectingChannel) {
// Make sure to increment the reference count before handing a client
// out to ensure correct bookkeeping for channel closing.
- if (!client.incrementReferenceCounter()) {
+ if (client != null && !client.incrementReferenceCounter()) {
Review comment:
Does the `client` have the possibility to be still `null` here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404512441
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,14 +92,36 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
- nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
-
- client = connectingChannel.waitForChannel();
-
- clients.replace(connectionId, connectingChannel, client);
+ int count = 0;
+ ChannelException exception = null;
+ do {
+ try {
+ exception = null;
+ connectingChannel.error = null;
+ LOG.info("Connecting to {} at {} attempt", connectionId.getAddress(), count);
+ nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
+
+ client = connectingChannel.waitForChannel();
+
+ clients.replace(connectionId, connectingChannel, client);
+ } catch (ChannelException e) {
+ LOG.error("Failed {} times to connect to {}", count, connectionId.getAddress(), e);
+ exception = e;
+ }
+ count++;
+ } while (count <= retryNumber);
+
+ if (exception != null) {
+ throw exception;
+ }
}
else if (old instanceof ConnectingChannel) {
- client = ((ConnectingChannel) old).waitForChannel();
+
+ try {
+ client = ((ConnectingChannel) old).waitForChannel();
+ } catch (IOException e) {
+ LOG.info("Continue to waitForChannel until the original thread has completed or failed");
+ }
Review comment:
The current logic seems have some problems. When it exists exception while waiting, then the client is still `null` which is replaced into the map via below `clients.replace(connectionId, old, client)`. In next loop, it would enter the above `connectChannelWithRetry` logic, that means the `B` might also encounter the same process as previous `A` to retry some times.
I guess the expectation we want is only let `A` retry connections and `B` is always waiting during `A` retries until final success or failure.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404499258
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/NettyConfig.java
##########
@@ -116,6 +116,10 @@ public int getClientConnectTimeoutSeconds() {
return config.getInteger(NettyShuffleEnvironmentOptions.CLIENT_CONNECT_TIMEOUT_SECONDS);
}
+ public int getNetworkRetries() {
Review comment:
getClientConnectRetries
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r400970930
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,14 +92,36 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
- nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
-
- client = connectingChannel.waitForChannel();
-
- clients.replace(connectionId, connectingChannel, client);
+ int count = 0;
+ ChannelException exception = null;
+ do {
+ try {
+ exception = null;
+ connectingChannel.error = null;
+ LOG.info("Connecting to {} at {} attempt", connectionId.getAddress(), count);
+ nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
+
+ client = connectingChannel.waitForChannel();
+
+ clients.replace(connectionId, connectingChannel, client);
+ } catch (ChannelException e) {
+ LOG.error("Failed {} times to connect to {}", count, connectionId.getAddress(), e);
+ exception = e;
+ }
+ count++;
+ } while (count <= retryNumber);
+
+ if (exception != null) {
+ throw exception;
+ }
}
else if (old instanceof ConnectingChannel) {
- client = ((ConnectingChannel) old).waitForChannel();
+
+ try {
+ client = ((ConnectingChannel) old).waitForChannel();
+ } catch (IOException e) {
+ LOG.info("Continue to waitForChannel until the original thread has completed or failed");
+ }
Review comment:
What do you mean by this change/log message?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #11541: [FLINK-15416][network]
add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot commented on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604807682
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 3bc439b7c92f5e764033133845ac1ff2a2b14b91 (Fri Mar 27 04:24:31 UTC 2020)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
* **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-15416).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work.
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159430095",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7241",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159477959",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159173715",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
* 7346f3729147e2b94cbf145fbea2dcd3c3332e19 Travis: [SUCCESS](https://travis-ci.com/github/flink-ci/flink/builds/159173715) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404509975
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -120,6 +132,34 @@ public void closeOpenChannelConnections(ConnectionID connectionId) {
}
}
+ void connectChannelWithRetry(ConnectingChannel connectingChannel, ConnectionID connectionId)
+ throws IOException, InterruptedException {
+ int count = 0;
+ Exception exception = null;
+ do {
+ try {
+ connectingChannel = new ConnectingChannel(connectionId, this);
+ LOG.info("Connecting to {} at {} attempt", connectionId.getAddress(), count);
+ nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
+
+ NettyPartitionRequestClient client = connectingChannel.waitForChannel();
+
+ clients.replace(connectionId, connectingChannel, client);
+ } catch (IOException e) {
+ LOG.error("Failed {} times to connect to {}", count, connectionId.getAddress(), e);
+ exception = e;
+ } catch (ChannelException e) {
Review comment:
These two exceptions {`IOException`, `ChannelException`} can be merged together because the handling logic is the same.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r402811264
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,14 +92,36 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
- nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
-
- client = connectingChannel.waitForChannel();
-
- clients.replace(connectionId, connectingChannel, client);
+ int count = 0;
+ ChannelException exception = null;
+ do {
+ try {
+ exception = null;
+ connectingChannel.error = null;
+ LOG.info("Connecting to {} at {} attempt", connectionId.getAddress(), count);
+ nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
+
+ client = connectingChannel.waitForChannel();
+
+ clients.replace(connectionId, connectingChannel, client);
+ } catch (ChannelException e) {
+ LOG.error("Failed {} times to connect to {}", count, connectionId.getAddress(), e);
+ exception = e;
+ }
+ count++;
+ } while (count <= retryNumber);
+
+ if (exception != null) {
+ throw exception;
+ }
}
else if (old instanceof ConnectingChannel) {
- client = ((ConnectingChannel) old).waitForChannel();
+
+ try {
+ client = ((ConnectingChannel) old).waitForChannel();
+ } catch (IOException e) {
+ LOG.info("Continue to waitForChannel until the original thread has completed or failed");
+ }
Review comment:
As multiple subpartition will share the same channel. Let's say subpartition a, b both needs the channel. A goes to wait first, B will still wait here. If exception happens, both of them will be notified. But we don't want the exception throw in B as A can still retry.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* 30370d7bb280c03c7b6767877f4405d4c06941ae Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/158923996) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134)
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
* e64d5ea150b48d976ec3b589144a29487ca20e33 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406589156
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,16 +92,12 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
Review comment:
TBH I do not like the previous way of implementing this logic via `ConcurrentHashMap`, because it results in five different `if..else` code paths together and some paths are almost very similar.
Instead if we realize it via an explicit `synchronized` lock outside, then the internal logic seems very simple, because we do not need to consider the race conditions of connect/wait among multiple threads. Only one thread would actually establish the connection and wait, all the other threads would be blocked outside of `synchronized` until the connection established or exception after retries.
It seems not get any benefits to have the small scope lock via `ConcurrentHashMap`, because this operation will be done only once, not frequent operations. But I am not forcing to refactor it in this PR since it can be considered as another issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404497951
##########
File path: docs/_includes/generated/netty_shuffle_environment_configuration.html
##########
@@ -57,17 +57,23 @@
<td>Number of extra network buffers to use for each outgoing/incoming gate (result partition/input gate). In credit-based flow control mode, this indicates how many floating credits are shared among all the input channels. The floating buffers are distributed based on backlog (real-time output buffers in the subpartition) feedback, and can help relieve back-pressure caused by unbalanced data distribution among the subpartitions. This value should be increased in case of higher round trip times between nodes and/or larger number of machines in the cluster.</td>
</tr>
<tr>
- <td><h5>taskmanager.network.netty.client.connectTimeoutSec</h5></td>
- <td style="word-wrap: break-word;">120</td>
+ <td><h5>taskmanager.network.retries</h5></td>
+ <td style="word-wrap: break-word;">0</td>
<td>Integer</td>
- <td>The Netty client connection timeout.</td>
+ <td>The number of retry attempts for network communication. Currently it's only used for establishing input/output channel connections</td>
</tr>
<tr>
<td><h5>taskmanager.network.netty.client.numThreads</h5></td>
<td style="word-wrap: break-word;">-1</td>
<td>Integer</td>
<td>The number of Netty client threads.</td>
</tr>
+ <tr>
+ <td><h5>taskmanager.network.netty.clientRetry</h5></td>
Review comment:
The generated key is not consistent with the new added "taskmanager.network.retries" in `NettyShuffleEnvironmentOptions`. Do you adjust these files manually or automatically?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-605382503
@pnowojski
Would you please review this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "CANCELED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159430095",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
* e64d5ea150b48d976ec3b589144a29487ca20e33 Travis: [CANCELED](https://travis-ci.com/github/flink-ci/flink/builds/159430095) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240)
* 398866ade003725aad87bfadd508c7dcd61ef21d UNKNOWN
* 7346f3729147e2b94cbf145fbea2dcd3c3332e19 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-606114299
@pnowojski
Thanks for the feedback. The network issue happens very often internally for us. Thus, we want to have a solution to mitigate at least internally. I will make sure to start to work after the ticket is assigned next week. For this PR, I will first add the test case for retry.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406586623
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -120,6 +132,34 @@ public void closeOpenChannelConnections(ConnectionID connectionId) {
}
}
+ void connectChannelWithRetry(ConnectingChannel connectingChannel, ConnectionID connectionId)
Review comment:
Yes, actually the intellij would give this warning for never used argument.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r402807556
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/NettyShuffleEnvironmentOptions.java
##########
@@ -234,6 +234,13 @@
.withDeprecatedKeys("taskmanager.net.client.connectTimeoutSec")
.withDescription("The Netty client connection timeout.");
+ @Documentation.Section(Documentation.Sections.ALL_TASK_MANAGER_NETWORK)
+ public static final ConfigOption<Integer> CLIENT_CONNECT_RETRY =
+ key("taskmanager.network.netty.clientRetry")
+ .defaultValue(0)
+ .withDeprecatedKeys("taskmanager.network.netty.clientRetry")
+ .withDescription("The retry times for building netty client connection.");
+
Review comment:
Yeah, it is a better config name
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3d6cba173780c51f34b577d8194343592fceacf5 Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/155930839) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772)
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "CANCELED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* eeee4cdce1e85052f21aa8b70355516d8ed9b3e4 Travis: [CANCELED](https://travis-ci.com/github/flink-ci/flink/builds/158917957) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132)
* 30370d7bb280c03c7b6767877f4405d4c06941ae Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/158923996) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* eeee4cdce1e85052f21aa8b70355516d8ed9b3e4 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/158917957) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132)
* 30370d7bb280c03c7b6767877f4405d4c06941ae UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404509648
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -120,6 +132,34 @@ public void closeOpenChannelConnections(ConnectionID connectionId) {
}
}
+ void connectChannelWithRetry(ConnectingChannel connectingChannel, ConnectionID connectionId)
Review comment:
it seems this argument is never used in below logics, maybe we need to refactor the logic
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406583546
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,16 +92,12 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
- nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
-
- client = connectingChannel.waitForChannel();
-
- clients.replace(connectionId, connectingChannel, client);
+ connectChannelWithRetry(connectingChannel, connectionId);
+ client = (NettyPartitionRequestClient) clients.get(connectionId);
}
else if (old instanceof ConnectingChannel) {
Review comment:
we can merge `if (old == null || old instanceof ConnectionChannel)` because the actions are actually the same.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-611885676
@zhijiangW Appreciate your kind review. It is really helpful to find out the bug and improvements.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404509648
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -120,6 +132,34 @@ public void closeOpenChannelConnections(ConnectionID connectionId) {
}
}
+ void connectChannelWithRetry(ConnectingChannel connectingChannel, ConnectionID connectionId)
Review comment:
it seems `connectingChannel` is never used in below logics, maybe we need to refactor the logic
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* 3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/158159031) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* 8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761 Travis: [SUCCESS](https://travis-ci.com/github/flink-ci/flink/builds/156746943) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845)
* 3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406609258
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -99,7 +106,7 @@ else if (old instanceof ConnectingChannel) {
// Make sure to increment the reference count before handing a client
// out to ensure correct bookkeeping for channel closing.
- if (!client.incrementReferenceCounter()) {
+ if (client != null && !client.incrementReferenceCounter()) {
Review comment:
No need. Mistakenly added.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406586376
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -120,6 +132,34 @@ public void closeOpenChannelConnections(ConnectionID connectionId) {
}
}
+ void connectChannelWithRetry(ConnectingChannel connectingChannel, ConnectionID connectionId)
+ throws IOException, InterruptedException {
+ int count = 0;
+ Exception exception = null;
+ do {
+ try {
+ connectingChannel = new ConnectingChannel(connectionId, this);
Review comment:
I see it is already out of loop in latest changes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "CANCELED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159430095",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
* e64d5ea150b48d976ec3b589144a29487ca20e33 Travis: [CANCELED](https://travis-ci.com/github/flink-ci/flink/builds/159430095) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240)
* 398866ade003725aad87bfadd508c7dcd61ef21d UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159430095",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7241",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159477959",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159173715",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
* 7346f3729147e2b94cbf145fbea2dcd3c3332e19 Travis: [SUCCESS](https://travis-ci.com/github/flink-ci/flink/builds/159173715) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404509975
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -120,6 +132,34 @@ public void closeOpenChannelConnections(ConnectionID connectionId) {
}
}
+ void connectChannelWithRetry(ConnectingChannel connectingChannel, ConnectionID connectionId)
+ throws IOException, InterruptedException {
+ int count = 0;
+ Exception exception = null;
+ do {
+ try {
+ connectingChannel = new ConnectingChannel(connectionId, this);
+ LOG.info("Connecting to {} at {} attempt", connectionId.getAddress(), count);
+ nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
+
+ NettyPartitionRequestClient client = connectingChannel.waitForChannel();
+
+ clients.replace(connectionId, connectingChannel, client);
+ } catch (IOException e) {
+ LOG.error("Failed {} times to connect to {}", count, connectionId.getAddress(), e);
+ exception = e;
+ } catch (ChannelException e) {
Review comment:
These two exceptions can be merged together because the handling logic is the same.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r403455029
##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactoryTest.java
##########
@@ -130,6 +130,23 @@ public void run() {
}
}
+ @Test
+ public void testNettyClientConnectRetry() throws Exception {
+ NettyClient nettyClient = mock(NettyClient.class);
Review comment:
Yes I noticed that and probably implementing `ChannelFuture` is not worth it, hence I suggested to use `NettyTestUtil`. It looks like you could just init a server (`NettyTestUtil#initServer()`) and `NettyClient` from `#initClient()` and you could use it directly, without a need to implement custom `ChannelFuture`? You can check usages of `NettyTestUtil`, like for example `NettyClientServerSslTest#testClientUntrustedCertificate`.
I might be missing something, but it looks like it could work quite easily.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404509453
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -120,6 +132,34 @@ public void closeOpenChannelConnections(ConnectionID connectionId) {
}
}
+ void connectChannelWithRetry(ConnectingChannel connectingChannel, ConnectionID connectionId)
+ throws IOException, InterruptedException {
+ int count = 0;
+ Exception exception = null;
+ do {
+ try {
+ connectingChannel = new ConnectingChannel(connectionId, this);
Review comment:
this can be done outside of loop, to not create it every retry.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404880461
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/NettyShuffleEnvironmentOptions.java
##########
@@ -234,6 +234,14 @@
.withDeprecatedKeys("taskmanager.net.client.connectTimeoutSec")
.withDescription("The Netty client connection timeout.");
+ @Documentation.Section(Documentation.Sections.ALL_TASK_MANAGER_NETWORK)
+ public static final ConfigOption<Integer> NETWORK_RETRIES =
+ key("taskmanager.network.retries")
Review comment:
@zhijiangW : https://github.com/apache/flink/pull/11541#discussion_r400969493
In the future we might want to re-use this property for other retries, not only client connection retries.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r400976516
##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactoryTest.java
##########
@@ -130,6 +130,23 @@ public void run() {
}
}
+ @Test
+ public void testNettyClientConnectRetry() throws Exception {
+ NettyClient nettyClient = mock(NettyClient.class);
Review comment:
We stopped using mockito for tests (for various of reasons), could you try to provide a proper mock of `NettyClient` class? Maybe if you used `NettyTestUtil` you could just implement `NettyClient#connect` method that just throws an exception on the first invocation, but succeeds (returns real/correct `ChannelFuture`) on the second attempt?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404519157
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/NettyShuffleEnvironmentOptions.java
##########
@@ -234,6 +234,14 @@
.withDeprecatedKeys("taskmanager.net.client.connectTimeoutSec")
.withDescription("The Netty client connection timeout.");
+ @Documentation.Section(Documentation.Sections.ALL_TASK_MANAGER_NETWORK)
+ public static final ConfigOption<Integer> NETWORK_RETRIES =
+ key("taskmanager.network.retries")
Review comment:
After discuss with @pnowojski. We aligned to use taskmanager.network.retries. This option will be used for other networking retries later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404518959
##########
File path: docs/_includes/generated/netty_shuffle_environment_configuration.html
##########
@@ -57,17 +57,23 @@
<td>Number of extra network buffers to use for each outgoing/incoming gate (result partition/input gate). In credit-based flow control mode, this indicates how many floating credits are shared among all the input channels. The floating buffers are distributed based on backlog (real-time output buffers in the subpartition) feedback, and can help relieve back-pressure caused by unbalanced data distribution among the subpartitions. This value should be increased in case of higher round trip times between nodes and/or larger number of machines in the cluster.</td>
</tr>
<tr>
- <td><h5>taskmanager.network.netty.client.connectTimeoutSec</h5></td>
- <td style="word-wrap: break-word;">120</td>
+ <td><h5>taskmanager.network.retries</h5></td>
+ <td style="word-wrap: break-word;">0</td>
Review comment:
It is a mistake.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406356843
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,14 +92,36 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
- nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
-
- client = connectingChannel.waitForChannel();
-
- clients.replace(connectionId, connectingChannel, client);
+ int count = 0;
+ ChannelException exception = null;
+ do {
+ try {
+ exception = null;
+ connectingChannel.error = null;
+ LOG.info("Connecting to {} at {} attempt", connectionId.getAddress(), count);
+ nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
+
+ client = connectingChannel.waitForChannel();
+
+ clients.replace(connectionId, connectingChannel, client);
+ } catch (ChannelException e) {
+ LOG.error("Failed {} times to connect to {}", count, connectionId.getAddress(), e);
+ exception = e;
+ }
+ count++;
+ } while (count <= retryNumber);
+
+ if (exception != null) {
+ throw exception;
+ }
}
else if (old instanceof ConnectingChannel) {
- client = ((ConnectingChannel) old).waitForChannel();
+
+ try {
+ client = ((ConnectingChannel) old).waitForChannel();
+ } catch (IOException e) {
+ LOG.info("Continue to waitForChannel until the original thread has completed or failed");
+ }
Review comment:
Agree. Apply the same retry logic for following channel request other than the first one.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159430095",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7241",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159477959",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159173715",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159737092",
"triggerID" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"triggerType" : "PUSH"
}, {
"hash" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7311",
"triggerID" : "8e2da05d8e34fe5f8d7b86e849b0df1275579242",
"triggerType" : "PUSH"
}, {
"hash" : "072a285e04f298ee3c6ff6c9062c3ffedae90a2f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "072a285e04f298ee3c6ff6c9062c3ffedae90a2f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
* 8e2da05d8e34fe5f8d7b86e849b0df1275579242 Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/159737092) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7311)
* 072a285e04f298ee3c6ff6c9062c3ffedae90a2f UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406583145
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -120,6 +127,40 @@ public void closeOpenChannelConnections(ConnectionID connectionId) {
}
}
+ void connectChannelWithRetry(ConnectingChannel connectingChannel, ConnectionID connectionId)
+ throws IOException, InterruptedException {
+ int count = 0;
+ Exception exception = null;
+ do {
+ try {
+ LOG.info("Connecting to {} at {} attempt", connectionId.getAddress(), count);
+ nettyClient.connect(connectionId.getAddress()).addListener(connectingChannel);
+
+ NettyPartitionRequestClient client = connectingChannel.waitForChannel();
+
+ clients.replace(connectionId, connectingChannel, client);
+ exception = null;
Review comment:
nit: this seems redundant, because the initialization of this variable is also `null`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* 8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761 Travis: [SUCCESS](https://travis-ci.com/github/flink-ci/flink/builds/156746943) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406583022
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -120,6 +127,40 @@ public void closeOpenChannelConnections(ConnectionID connectionId) {
}
}
+ void connectChannelWithRetry(ConnectingChannel connectingChannel, ConnectionID connectionId)
Review comment:
nit: can be private if we do not plan to make it access out of this class
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] pnowojski commented on issue #11541: [FLINK-15416][network]
add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
pnowojski commented on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-606051120
> This pull request references an unassigned Jira ticket. According to the code contribution guide, tickets need to be assigned before starting with the implementation work.
Please next time do not start work before being assigned to the ticket. Especially that there was my unresolved question in the Jira ticket whether we want to have such change or not.
In the end I think we can go in the direction of supporting retry mechanism in the network stack (including TCP connection loss in the middle of transfer), so we can just as well start with the reconnection retries as you are proposing.
I will give more detailed look at your PR later. For starter, could you try to provide unit tests for the `PartitionRequestClientFactory` class? For example by mocking `NettyClient`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0e228f36cdfc4610efd4da91d5964ffa05202c79 Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/155743045) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722)
* 3d6cba173780c51f34b577d8194343592fceacf5 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/155930839) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772)
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r404519839
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/NettyConfig.java
##########
@@ -116,6 +116,10 @@ public int getClientConnectTimeoutSeconds() {
return config.getInteger(NettyShuffleEnvironmentOptions.CLIENT_CONNECT_TIMEOUT_SECONDS);
}
+ public int getNetworkRetries() {
Review comment:
Explained Above.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-611648501
@pnowojski
Thanks for the feedback. Would you please take one more round of look?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
zhijiangW commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406582084
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/NettyShuffleEnvironmentOptions.java
##########
@@ -234,6 +234,14 @@
.withDeprecatedKeys("taskmanager.net.client.connectTimeoutSec")
.withDescription("The Netty client connection timeout.");
+ @Documentation.Section(Documentation.Sections.ALL_TASK_MANAGER_NETWORK)
+ public static final ConfigOption<Integer> NETWORK_RETRIES =
+ key("taskmanager.network.retries")
Review comment:
that is fine for me
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "baf2c55f7bb0b9df7bdef94083c947693b75eae2",
"triggerType" : "PUSH"
}, {
"hash" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5419f41a18eaae05814837d679f58480a2cb94c",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159430095",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7240",
"triggerID" : "e64d5ea150b48d976ec3b589144a29487ca20e33",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7241",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159477959",
"triggerID" : "398866ade003725aad87bfadd508c7dcd61ef21d",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
}, {
"hash" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/159173715",
"triggerID" : "7346f3729147e2b94cbf145fbea2dcd3c3332e19",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* baf2c55f7bb0b9df7bdef94083c947693b75eae2 UNKNOWN
* e5419f41a18eaae05814837d679f58480a2cb94c UNKNOWN
* 7346f3729147e2b94cbf145fbea2dcd3c3332e19 Travis: [SUCCESS](https://travis-ci.com/github/flink-ci/flink/builds/159173715) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7177)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* 3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/158159031) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
HuangZhenQiu commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r406609040
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##########
@@ -81,16 +92,12 @@ NettyPartitionRequestClient createPartitionRequestClient(ConnectionID connection
Object old = clients.putIfAbsent(connectionId, connectingChannel);
if (old == null) {
Review comment:
As the race condition only happens among the subpartitions for the same channel, the syncronized lock is probably too heavy. If considering a job with 256 tms, 255 connection needs to build. The lock will introduce a longer job initialization time.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0e228f36cdfc4610efd4da91d5964ffa05202c79 Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/155743045) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#discussion_r400969493
##########
File path: flink-core/src/main/java/org/apache/flink/configuration/NettyShuffleEnvironmentOptions.java
##########
@@ -234,6 +234,13 @@
.withDeprecatedKeys("taskmanager.net.client.connectTimeoutSec")
.withDescription("The Netty client connection timeout.");
+ @Documentation.Section(Documentation.Sections.ALL_TASK_MANAGER_NETWORK)
+ public static final ConfigOption<Integer> CLIENT_CONNECT_RETRY =
+ key("taskmanager.network.netty.clientRetry")
+ .defaultValue(0)
+ .withDeprecatedKeys("taskmanager.network.netty.clientRetry")
+ .withDescription("The retry times for building netty client connection.");
+
Review comment:
I think this option might be more general than just netty and not only for "client connect retries". Also I think `taskmanager.network.netty` are the properties that are passed directly to netty config, am I right?
So I would rename it to `taskmanager.network.retries`/`NETWORK_RETRIES`/`The number of retry attempts for network communication. Currently it's only used for establishing input/output channel connections.`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5ef19285000d692d6b1b8b0aa7a37ce35c216f5",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/156746943",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6845",
"triggerID" : "8b0f1772673f3e0fbc2d7ce5974f9d6f7a23a761",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158159031",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7025",
"triggerID" : "3c0e3fbee74abc99a22f70d2bfc4829c789fa9fb",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7132",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158917957",
"triggerID" : "eeee4cdce1e85052f21aa8b70355516d8ed9b3e4",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
}, {
"hash" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/158923996",
"triggerID" : "30370d7bb280c03c7b6767877f4405d4c06941ae",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b5ef19285000d692d6b1b8b0aa7a37ce35c216f5 UNKNOWN
* 30370d7bb280c03c7b6767877f4405d4c06941ae Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/158923996) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=7134)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0e228f36cdfc4610efd4da91d5964ffa05202c79 Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/155743045) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722)
* 3d6cba173780c51f34b577d8194343592fceacf5 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11541:
[FLINK-15416][network] add task manager netty client retry mechenism
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11541: [FLINK-15416][network] add task manager netty client retry mechenism
URL: https://github.com/apache/flink/pull/11541#issuecomment-604812212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6720",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"status" : "DELETED",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155738776",
"triggerID" : "3bc439b7c92f5e764033133845ac1ff2a2b14b91",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "FAILURE",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155743045",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722",
"triggerID" : "0e228f36cdfc4610efd4da91d5964ffa05202c79",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
}, {
"hash" : "3d6cba173780c51f34b577d8194343592fceacf5",
"status" : "PENDING",
"url" : "https://travis-ci.com/github/flink-ci/flink/builds/155930839",
"triggerID" : "3d6cba173780c51f34b577d8194343592fceacf5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0e228f36cdfc4610efd4da91d5964ffa05202c79 Travis: [FAILURE](https://travis-ci.com/github/flink-ci/flink/builds/155743045) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6722)
* 3d6cba173780c51f34b577d8194343592fceacf5 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/155930839) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6772)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services