You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@celeborn.apache.org by "RexXiong (via GitHub)" <gi...@apache.org> on 2023/04/28 08:56:37 UTC
[GitHub] [incubator-celeborn] RexXiong opened a new pull request, #1474: [CELEBORN-567] Timeout workers/app need consider long leader election period
RexXiong opened a new pull request, #1474:
URL: https://github.com/apache/incubator-celeborn/pull/1474
<!--
Thanks for sending a pull request! Here are some tips for you:
- Make sure the PR title start w/ a JIRA ticket, e.g. '[CELEBORN-XXXX] Your PR title ...'.
- Be sure to keep the PR description updated to reflect all changes.
- Please write your PR title to summarize what this PR proposes.
- If possible, provide a concise example to reproduce the issue for a faster review.
-->
### What changes were proposed in this pull request?
increase timeout deadline when leader changed to avoid app/worker status unexpected changes.
### Why are the changes needed?
Consider the situation in ha mode，if master election period exceeds the workerHeartbeatTimeout or appHeartbeatTimeout, all workers/apps may timeout immediately when new leader elected。And This will also cause every shuffle of the apps timeout, every register/every change partition fail and all data would be lost in worker. For Flink same system, It's also bad as the resource of Flink dataset would be timeout while even Flink app retry to connect the master. (Flink support long time retry strategy for one job, then this job would fail because of shuffle expired in Celeborn Cluster)
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Would test in k8s environment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] codecov[bot] commented on pull request #1474: [CELEBORN-567] Timeout workers/app need consider long leader election period
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #1474:
URL: https://github.com/apache/incubator-celeborn/pull/1474#issuecomment-1527238471
## [Codecov](https://codecov.io/gh/apache/incubator-celeborn/pull/1474?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1474](https://codecov.io/gh/apache/incubator-celeborn/pull/1474?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e4cfc71) into [main](https://codecov.io/gh/apache/incubator-celeborn/commit/8d933691ae0c8a9a032592fa668116a1dddd1a43?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8d93369) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## main #1474 +/- ##
==========================================
+ Coverage 44.81% 44.81% +0.01%
==========================================
Files 155 155
Lines 9589 9590 +1
Branches 955 955
==========================================
+ Hits 4296 4297 +1
+ Misses 5010 5009 -1
- Partials 283 284 +1
```
[see 2 files with indirect coverage changes](https://codecov.io/gh/apache/incubator-celeborn/pull/1474/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] RexXiong commented on pull request #1474: [CELEBORN-567] Timeout workers/app need consider long leader election period
Posted by "RexXiong (via GitHub)" <gi...@apache.org>.
RexXiong commented on PR #1474:
URL: https://github.com/apache/incubator-celeborn/pull/1474#issuecomment-1533993866
cc @AngersZhuuuu
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] RexXiong merged pull request #1474: [CELEBORN-567] Timeout workers/app need consider long leader election period
Posted by "RexXiong (via GitHub)" <gi...@apache.org>.
RexXiong merged PR #1474:
URL: https://github.com/apache/incubator-celeborn/pull/1474
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org