You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@celeborn.apache.org by "lyy-pineapple (via GitHub)" <gi...@apache.org> on 2024/01/10 07:31:27 UTC
[PR] [CELEBORN-1218]Optimize dataPusher to get partitionLocationMap only once [incubator-celeborn]
lyy-pineapple opened a new pull request, #2220:
URL: https://github.com/apache/incubator-celeborn/pull/2220
### What changes were proposed in this pull request?
dataPusher fetches partitionLocationMap only once outside the loop.
### Why are the changes needed?
If an exception occurs while obtaining partitionLocationMap in dataPusher.takePushTasks, it will result in attempting to fetch partitionLocationMap for each iteration in the workingQueue traversal. This leads to an unusually prolonged job execution time.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Through existing uts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CELEBORN-1218]Optimize dataPusher to get partitionLocationMap only once [incubator-celeborn]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #2220:
URL: https://github.com/apache/incubator-celeborn/pull/2220#issuecomment-1884341229
## [Codecov](https://app.codecov.io/gh/apache/incubator-celeborn/pull/2220?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Comparison is base [(`daddd2d`)](https://app.codecov.io/gh/apache/incubator-celeborn/commit/daddd2d2c1e0485ab0ca6651fb6f77bf5f113089?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) 47.21% compared to head [(`d74db5c`)](https://app.codecov.io/gh/apache/incubator-celeborn/pull/2220?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) 47.21%.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #2220 +/- ##
=======================================
Coverage 47.21% 47.21%
=======================================
Files 192 192
Lines 11778 11778
Branches 1050 1050
=======================================
Hits 5560 5560
Misses 5844 5844
Partials 374 374
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/incubator-celeborn/pull/2220?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CELEBORN-1218] Optimize dataPusher to get partitionLocationMap only once [incubator-celeborn]
Posted by "FMX (via GitHub)" <gi...@apache.org>.
FMX commented on PR #2220:
URL: https://github.com/apache/incubator-celeborn/pull/2220#issuecomment-1893318354
@RexXiong How about your opinion?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CELEBORN-1218] Optimize dataPusher to get partitionLocationMap only once [incubator-celeborn]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 closed pull request #2220: [CELEBORN-1218] Optimize dataPusher to get partitionLocationMap only once
URL: https://github.com/apache/incubator-celeborn/pull/2220
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CELEBORN-1218] Optimize dataPusher to get partitionLocationMap only once [incubator-celeborn]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #2220:
URL: https://github.com/apache/incubator-celeborn/pull/2220#issuecomment-1895666643
Thanks, merged to main/0.4.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org