You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by "zuston (via GitHub)" <gi...@apache.org> on 2023/04/12 06:59:36 UTC
[GitHub] [incubator-uniffle] zuston opened a new pull request, #816: [#416] feat(hdfs): lazy initialization of hdfsShuffleWriteHandler when per-partition concurrent write is enabled
zuston opened a new pull request, #816:
URL: https://github.com/apache/incubator-uniffle/pull/816
### What changes were proposed in this pull request?
lazy initialization of `hdfsShuffleWriteHandler` when per-partition concurrent write is enabled
### Why are the changes needed?
Without this PR, it will create too much unnecessary, when enable the
concurrent write of per-partition and no-race condition.
This PR is to reduce unnecessary handler creation.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
1. UTs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] zuston commented on pull request #816: [#416] feat(hdfs): lazy initialization of hdfsShuffleWriteHandler when per-partition concurrent write is enabled
Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston commented on PR #816:
URL: https://github.com/apache/incubator-uniffle/pull/816#issuecomment-1510643893
> Could you raise a #773 backport pr to branch-0.7? [#773 (comment)](https://github.com/apache/incubator-uniffle/pull/773#issuecomment-1506232455)
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] codecov-commenter commented on pull request #816: [#416] feat(hdfs): lazy initialization of hdfsShuffleWriteHandler when per-partition concurrent write is enabled
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #816:
URL: https://github.com/apache/incubator-uniffle/pull/816#issuecomment-1504780746
## [Codecov](https://codecov.io/gh/apache/incubator-uniffle/pull/816?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#816](https://codecov.io/gh/apache/incubator-uniffle/pull/816?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (663349a) into [master](https://codecov.io/gh/apache/incubator-uniffle/commit/cae7cd9e379aedca95e2590eeedfcc27fd9f56b9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (cae7cd9) will **increase** coverage by `1.43%`.
> The diff coverage is `72.22%`.
```diff
@@ Coverage Diff @@
## master #816 +/- ##
============================================
+ Coverage 57.63% 59.06% +1.43%
+ Complexity 2058 2050 -8
============================================
Files 306 291 -15
Lines 14871 12877 -1994
Branches 1221 1216 -5
============================================
- Hits 8571 7606 -965
+ Misses 5808 4845 -963
+ Partials 492 426 -66
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-uniffle/pull/816?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ge/handler/impl/PooledHdfsShuffleWriteHandler.java](https://codecov.io/gh/apache/incubator-uniffle/pull/816?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmFnZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvdW5pZmZsZS9zdG9yYWdlL2hhbmRsZXIvaW1wbC9Qb29sZWRIZGZzU2h1ZmZsZVdyaXRlSGFuZGxlci5qYXZh) | `69.44% <72.22%> (+25.96%)` | :arrow_up: |
... and [16 files with indirect coverage changes](https://codecov.io/gh/apache/incubator-uniffle/pull/816/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] zuston merged pull request #816: [#416] feat(hdfs): lazy initialization of hdfsShuffleWriteHandler when per-partition concurrent write is enabled
Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston merged PR #816:
URL: https://github.com/apache/incubator-uniffle/pull/816
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] jerqi commented on pull request #816: [#416] feat(hdfs): lazy initialization of hdfsShuffleWriteHandler when per-partition concurrent write is enabled
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on PR #816:
URL: https://github.com/apache/incubator-uniffle/pull/816#issuecomment-1510652151
I have reverted your mistake commit. You should raise a new pr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] jerqi commented on pull request #816: [#416] feat(hdfs): lazy initialization of hdfsShuffleWriteHandler when per-partition concurrent write is enabled
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on PR #816:
URL: https://github.com/apache/incubator-uniffle/pull/816#issuecomment-1510603981
Could you cherry-pick this pr to branch-0.7? https://github.com/apache/incubator-uniffle/pull/773#issuecomment-1506232455
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] jerqi commented on pull request #816: [#416] feat(hdfs): lazy initialization of hdfsShuffleWriteHandler when per-partition concurrent write is enabled
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on PR #816:
URL: https://github.com/apache/incubator-uniffle/pull/816#issuecomment-1510650818
> > Could you raise a #773 backport pr to branch-0.7? [#773 (comment)](https://github.com/apache/incubator-uniffle/pull/773#issuecomment-1506232455)
>
> Done.
CI failed. You can't cherry-pick this pr. You should revert it and raise a new pr. There is conflict with branch 0.7.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org