You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by "zhuyaogai (via GitHub)" <gi...@apache.org> on 2023/08/02 07:13:06 UTC
[GitHub] [incubator-uniffle] zhuyaogai opened a new pull request, #1069: Fail fast in WriteBufferManager when failed to send shuffle data to s…
zhuyaogai opened a new pull request, #1069:
URL: https://github.com/apache/incubator-uniffle/pull/1069
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
Examples:
- "[#123] feat(operator): support xxx"
- "[#233] fix: check null before access result in xxx"
- "[MINOR] refactor: fix typo in variable name"
- "[MINOR] docs: fix typo in README"
- "[#255] test: fix flaky test NameOfTheTest"
Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. Contributor guidelines:
https://github.com/apache/incubator-uniffle/blob/master/CONTRIBUTING.md
3. If the PR is unfinished, please mark this PR as draft.
-->
### What changes were proposed in this pull request?
Currently, it only checks for blocks that failed to send after all buffer data has been sent. This check also needs to be moved forward into the addRecord method, allowing it to fail fast.
### Why are the changes needed?
Fix: # ([1068](https://github.com/apache/incubator-uniffle/issues/1068))
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Add more test case in WriteBufferManager.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] zhengchenyu commented on pull request #1069: [#1068] improvement(tez): Fail fast in WriteBufferManager when failed to send shuffle data to shuffle sever.
Posted by "zhengchenyu (via GitHub)" <gi...@apache.org>.
zhengchenyu commented on PR #1069:
URL: https://github.com/apache/incubator-uniffle/pull/1069#issuecomment-1663345711
LGTM +1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] codecov-commenter commented on pull request #1069: Fail fast in WriteBufferManager when failed to send shuffle data to s…
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #1069:
URL: https://github.com/apache/incubator-uniffle/pull/1069#issuecomment-1661651300
## [Codecov](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1069?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
> Merging [#1069](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1069?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (e67ecb0) into [master](https://app.codecov.io/gh/apache/incubator-uniffle/commit/d149d5799a7e13bf5562e1a40354221cd505a826?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (d149d57) will **increase** coverage by `1.26%`.
> The diff coverage is `100.00%`.
> :exclamation: Current head e67ecb0 differs from pull request most recent head 86a3c2a. Consider uploading reports for the commit 86a3c2a to get more accurate results
```diff
@@ Coverage Diff @@
## master #1069 +/- ##
============================================
+ Coverage 54.20% 55.46% +1.26%
+ Complexity 2550 2542 -8
============================================
Files 386 365 -21
Lines 21867 19474 -2393
Branches 1813 1806 -7
============================================
- Hits 11852 10801 -1051
+ Misses 9315 8052 -1263
+ Partials 700 621 -79
```
| [Files Changed](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1069?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...library/common/sort/buffer/WriteBufferManager.java](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1069?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y2xpZW50LXRlei9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvdGV6L3J1bnRpbWUvbGlicmFyeS9jb21tb24vc29ydC9idWZmZXIvV3JpdGVCdWZmZXJNYW5hZ2VyLmphdmE=) | `89.35% <100.00%> (+2.49%)` | :arrow_up: |
... and [21 files with indirect coverage changes](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1069/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] zuston merged pull request #1069: [#1068] improvement(tez): Fail fast in WriteBufferManager when failed to send shuffle data to shuffle sever.
Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston merged PR #1069:
URL: https://github.com/apache/incubator-uniffle/pull/1069
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] jerqi commented on pull request #1069: [#1068] improvement(tez): Fail fast in WriteBufferManager when failed to send shuffle data to shuffle sever.
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on PR #1069:
URL: https://github.com/apache/incubator-uniffle/pull/1069#issuecomment-1661678267
@zhengchenyu Could you help me review this pr?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] zhuyaogai commented on pull request #1069: [#1068] improvement(tez): Fail fast in WriteBufferManager when failed to send shuffle data to shuffle sever.
Posted by "zhuyaogai (via GitHub)" <gi...@apache.org>.
zhuyaogai commented on PR #1069:
URL: https://github.com/apache/incubator-uniffle/pull/1069#issuecomment-1661660335
> Could you correct your title?
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] jerqi commented on pull request #1069: Fail fast in WriteBufferManager when failed to send shuffle data to s…
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on PR #1069:
URL: https://github.com/apache/incubator-uniffle/pull/1069#issuecomment-1661651807
Could you correct your title?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] jerqi commented on pull request #1069: [#1068] improvement(tez): Fail fast in WriteBufferManager when failed to send shuffle data to shuffle sever.
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on PR #1069:
URL: https://github.com/apache/incubator-uniffle/pull/1069#issuecomment-1666564078
@Lobo2008 MapReduce should need the similar ability. The code of Tez and MapReduce is same. If you have interest, you can raise a pr for MapReduce.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] zuston commented on pull request #1069: [#1068] improvement(tez): Fail fast in WriteBufferManager when failed to send shuffle data to shuffle sever.
Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston commented on PR #1069:
URL: https://github.com/apache/incubator-uniffle/pull/1069#issuecomment-1663366997
Thanks @zhuyaogai and @zhengchenyu review. Merged
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org